9. Evaluating project processes

9.1 Why evaluate processes?
9.2 What processes should be monitored/evaluated?
9.3 Who should be involved?
9.4 Timing of reports and reviews
9.5 The need for a reporting plan
9.6 Preparing status (progress) reports
9.7 Form and format of the final report
9.8 Include evaluation in the project budget

A project is a means to an end. Evaluating how the project has functioned as a means, as well as evaluating its outputs and outcomes, has value. Figure 2.1: ‘Project evaluation framework’ places the evaluation of project processes in the context of these other foci for evaluation. Project processes provide another focal point for evaluation in addition to foci outlined in Table 2.1, which are learning-centred foci.

9.1 Why evaluate processes?

Carrying out a project is usually a complex task involving a number of people, functioning as a project team. The success of the project hinges in large part on how well the team operates and is managed, and so a focus on this aspect of the project in planning an evaluation is important:

• Monitoring the processes during the formative stage of the project (i.e. during planning, design and development) can help in diagnosing problems and allow for suitable corrective action to be taken.
• Critically reflecting on processes upon completion of the project (to 'output' and/or implementation stage) can inform the management of future faculty/school projects.

These are evaluation activities that the competent project manager takes in their stride, and so are not ‘additional’ tasks to consider.

In the course of their duties, project managers will produce a range of status (or progress) reports and some form of final report, in addition to keeping a general finger on the pulse of the project. Evaluation is thus part and parcel of the normal reporting process.

9.2 What processes should be monitored/evaluated?

The planning process

For example:

• scoping of the project (goal/objectives setting; definition of 'deliverables')
• general work breakdown and task analysis
• time estimation/effort
• scheduling/setting of milestones
• project team selection and placement
• costing and resourcing (other than personnel)
• risk analysis/assessment; mitigation strategies
• specification of responsibilities of the various parties to the project
• assumptions made.

The development process

For example:

• meeting of milestones (or slippage)
• budget/cost control
• task management
• resources allocation
• personnel selection/mix and management
• quality assurance processes
• training of and support for team members.

Communication/teamwork processes

For example:

• effectiveness/efficiency of meetings, and other communication channels used (such as reports)
• functioning and commitment of the team.

Monitoring and control processes

For example:

• monitoring/reporting processes
• decision-making processes (e.g. in terms of ability to make timely adjustments to the project) and their effectiveness
• management/authority structures
• risk management.

Turning process concerns into questions for evaluation

As for other foci for evaluation, you need to change these concerns, or aspects, into questions for investigation. E.g. for ‘budget cost control’ — ‘Is budget expenditure on schedule?’, and so on.

9.3 Who should be involved?

It's most likely that the project manager has responsibility for producing ongoing status (or progress) reports, for distribution to the project leader and/or management committee (e.g. steering group, reference/advisory group, Faculty Teaching & Learning Committee) and other major stakeholders. A final report would also generally be expected. The role of the project team in assisting the project manager in this process will be for the project manager to determine. Clearly, team members will be able to provide much important data.

9.4 Timing of reports and reviews

Some monitoring will be more or less on a daily basis. Other reviews and reports may coincide with the completion of particular phases or milestones of the project, or be due according to some other agreed reporting schedule (e.g. monthly). A review in the form of a final report at the end of the project would be considered mandatory.

9.5 The need for a reporting plan

It's advisable to draw up some sort of report plan or schedule. Here is one Reporting Schedule [Word] that you can use.

9.6 Preparing status (progress) reports

What is reported will vary according to the phase(s) just completed. However, any progress report might include:

• progress made since previous report
• overall progress in relation to the project plan (reports of 'slippage' and so on)
• project 'successes'
• problems and 'hiccups' encountered
• actions taken to keep/bring project on track
• issues for resolution; issues resolved since last report
• modifications to plan that may be required (e.g. in terms of scoping, timeline etc.)
• changes to risk assessment and related contingency planning.

Form of the progress report

It may take the form of:

• a memorandum or letter
• a summary report associated with an oral presentation to a particular stakeholder or group of stakeholders
• a more formal comprehensive written report.

For a template for a written status or progress report, see Project Status Report [Word]— kindly provided by Beverly Goldfarb, Faculty of Arts, 2002.

9.7 Form and format of the final report

This will depend on whether the report on processes is for inclusion/incorporation into the final project report involving evaluation of outputs and outcomes, or to stand alone. See Section 11: Reporting the evaluation findings for guidance on form and format if to be an integrated report.

9.8 Include evaluation in the project budget

One of the best ways to ensure that rigorous evaluation of project processes occurs (as well as evaluation of outputs and outcomes) is to include evaluation as a budget item in the project plan. See Section 6: Costing an evaluation for advice on identifying and estimating the various costs involved.