14. Evaluating the evaluation

14.1 The same standard process
14.2 Foci for evaluation
14.3 Who should evaluate the evaluation?

14.1 The same standard process

As for the project itself, the evaluation can also be evaluated, using the same principles and procedures:

• Establish the evaluation criteria or standards
• Develop a plan that includes how and when data will be collected
• Analyse and report on the findings to relevant audience(s).

14.2 Foci for evaluation

As for the project, there are three key focal points for evaluation:

At the design stage

Evaluation here can check the soundness and worth of the evaluation plan:

• in its purpose, objectives, questions to pursue, stakeholders considered
• in the methodology chosen and data gathering techniques selected
• in the analysis strategies selected
• in the reporting strategy, format and identified target audiences
• with respect to the management plan for the evaluation.

During the evaluation

Evaluation can monitor the progress of the project evaluation and provide feedback for remedial or other action. Each phase or step in the evaluation process can be the subject of this (formative type of) evaluation.

At the end of the evaluation

There will always be room for improvement and the opportunity to learn from experience. Both the outcomes of the evaluation and its processes can be reviewed to inform future practice. Particular foci for evaluation could include:

• review of the range of, and worth, of the questions posed
• review of the design and its implementation
• assessment of the quality and usefulness of the data gathered and data gathering tools used
• review of the analysis techniques and validity of interpretation
• review of the reporting regime (frequency, format, contents etc.)
• review of the evaluation management structure and processes.

14.3 Who should evaluate the evaluation?

This is where an external consultant can be of great assistance. Although the internal evaluation team should always be critically reflecting on their evaluation plan and progress, having another perspective is of considerable value, particularly at the 'evaluation of design' stage.