I recently took my comprehensive exams. If you pass them, it signals your completion of course work and you are qualified to move onto the dissertation phase of the Ph.D. I passed, WOOHOO! But, in preparation for that exam I studied and read a lot.

One of the things I spent time looking back at was the Joint Committee for Standards in Educational Evaluations (JCSEE) Program Evaluation Standards. The standards are in their third edition and are accredited by the American National Standards Institute (ANSI). Basically, this means they have gone through an extensive development process, and standards approved by ANSI become American National Standards. They are intended for program evaluation in the United States, but professional associations and organizations around the world have used them as a starting point for creating their own set of national standards for evaluation.

The Joint Committee is made up of representatives from a wide range of professional associations. A couple of very influential people in getting these standards developed and then maintaining and reviewing them are from Western Michigan University (WMU). So, these standards are in a way built into our (WMU trained evaluators) collective DNA in a sense. The standards have 5 main categories:

Utility: the extent to which program stakeholders find evaluation processes and products valuable in meeting their needs.
Feasibility: when evaluations can take place with an adequate degree of effectiveness and efficiency.
Propriety: what is proper, fair, legal, right, acceptable, and just in evaluations.
Accuracy: the truthfulness of evaluation representations, propositions, and findings, especially those that support judgements about the quality of program so program components.
Accountability: the responsible use of resources to produce value.

This is a blog about use, so I wanted to highlight the first main category, utility. Clearly, use is important to evaluators and evaluation users as represented by its place on the list. Within each main category there is a set of sub-standards.

With regards to the utility standards, there are a few I think are particularly important. The first is actually the first standard on the list, Evaluator Credibility. This refers to the evaluator being qualified to undertake a particular evaluation. Now, this comes at a time when there is much discussion on evaluator credentialing, and where the Canadian Evaluation Society has already implemented a credentialing process for evaluators. I’m not going to get into that, and you can read about that discussion on your own if you are interested. But, without a qualified individual to take on your evaluation, the whole process may as well stop. So, do your research on evaluator qualifications (the standards are a place to start, but also see Stevahn, King, Ghere & Minnema, 2006), consider your own needs, and be sure to obtain the appropriate evidence from your evaluator to be able to assess whether or not they are in fact able to take on the task you need them to.

The other utility standard I believe is particularly important is Negotiated Purposes. This basically describes the need to spend time on the front end to have a clear and honest discussion about the purpose for conducting the evaluation. This will help to ensure the evaluator understands what it is that needs to be done, and how to design an evaluation which will best serve the agreed upon purpose(s). This can help avoid situations where evaluations are conducted but end up being something completely different from what a client expected. This scenario is clearly not useful to anyone. An evaluation designed around the appropriate purpose is an evaluation which will be most useful once all is said and done.

COREY’S USEFUL TIP: Take a look at the JCSEE Program Evaluation Standards. This goes for evaluation users and evaluators (who should already have done this!). It is the most detailed and clear set of criteria we have for what makes evaluations good or bad. Knowing a little bit about this can help you procure better evaluators, and on the flipside for evaluators, do better evaluation.

Stevahn, L., King, J. A., Ghere, G., & Minnema, J. (2005). Establishing essential competencies for program evaluators. American Journal of Evaluation, 26(1), 43-59.