5 Evaluation framework

The Hewlett Foundation Evaluation Principles and Practices states that ‘evaluation is an independent, systematic investigation into how, why, and to what extent objectives or goals are achieved’ (Twersky & Lindblom, 2012, p. 3) whilst the Kellogg Foundation (2012, p.2) Framework for Evaluation states that evaluation should provide ‘ongoing, systematic information that strengthens projects during their life cycle, and, whenever possible, outcome data to assess the extent of change’, adding that ‘the evaluation effort should leave an organization stronger and more able to use such an evaluation when outside support ends’.   The JISC Six Steps to Effective Evaluation handbook (Glenaffric Ltd, 2007, p.2) states that evaluation ‘can and should be viewed as an essential and important element to a well-managed project that recognises the value of timeous insights into the project’s progress and successful outcomes’ and ‘can be defined as any organised activity that helps draw out the value and prove the worth of development projects’.

Evaluating a project then is an important aspect of demonstrating that it has achieved what you initially set out to do, and checking the project whilst in operation. That might seem obvious but evaluation is often an afterthought, and by then the measurements and records that would have made for useful evaluation are not available. It is better then to build it in from the start. It is not just that this allows you to more easily and effectively perform the evaluation, it also helps structure the project. Asking questions such as “what do we ultimately want to achieve?” and “How will we know if the project has been successful?” will provide a clearer plan.

The objectives of project evaluation often include:

  1. Developing a body of knowledge about what works, what does not work, and why;
  2. Improving the quality of a project;
  3. Identifying successful processes and strategies for replication and expansion;
  4. Modifying unsuccessful processes and strategies;
  5. Measuring the impact of a project and the benefits for stakeholders;
  6. Giving stakeholders a say in the project output and quality;
  7. Demonstrating to funders the effectiveness of a project.

The OERRH evaluation framework is intended to achieve all of these objectives. You can see our evaluation framework here: http://oerhub.net/research-outputs/reports/. In it we sought to answer the following questions, which might form a useful basis for your own evaluation:

  • What aspect(s) of the project should be evaluated?
  • Who is the evaluation for?
  • What is it they want to find out?
  • What evaluation methods will be used?
  • What changes will be made when the results are gathered?
  • What are the evaluation criteria and what is their source?
  • When will the evaluation take place?
  • Who will be involved in the evaluation?
  • What constraints will be placed upon the evaluation?
  • How and when will the evaluation results be disseminated?