The idea of use is a good one, so good that we have dedicated this blog to the concept of using evaluation. All evaluators want their findings to be used, particularly for improving that which they evaluated. Evaluation is also used to make decisions about whether something, be it a program, product or policy, should continue, be modified, or be ended. However, we must also consider situations when it may be better if evaluation findings were not used or scenarios for how they could be misused.
What I am referring to is the misuse or non-use of evaluation findings, either intentionally or mistakenly. When an evaluator conducts an evaluation, they always run the risk of having their “words” taken out of context and used in an unintended way. This would be an example of misuse. An easy to conceptualize example of non-use would be an evaluation client hiding or ignoring any evaluation that portrays their program in a negative light.
When I first started my doctoral program, my professor described a situation he had been in. After conducting an evaluation of a curriculum program and submitting the final report, the client cherry picked a few words which were absolutely taken out of context and manipulated to sound like this particular evaluation had found wildly positive effects when the opposite was actually true.
This was about the time he introduced us to the figure you see below. This is taken from Brad Cousin’s (2004) paper, in which he describes a framework for the various ways that evaluation findings can be used or not used. This framework has served as a guide to me as I have thought about use in evaluations I have done.
Use and misuse of evaluation findings
The framework describes four quadrants. The axes are anchored by use vs. non-use and misuse vs. legitimate use. The story I shared about my professor’s experience would fall in misuse. Worse still, it would likely be labeled mischievous use, the worst kind of use, and not in the “haha” sort of way. This type of misuse by a client could potentially have a negative effect on the reputation of the evaluator; and with integrity being something that is critical to an evaluator’s success, there is real potential for harm here.
The framework is broad, and can be considered by evaluators and evaluation clients alike. In the coming weeks, my posts will refer back to this framework as I explore implications of it for evaluators and evaluation clients.
COREY'S USEFUL TIP: Evaluators, don’t lie or be dishonest when it comes time to submit your evaluation report. Listen to the angel on whichever shoulder they may rest and come clean about the limitations and weaknesses of an evaluation. Nobody is going to hate you for it, and you don’t want the blood of “horrific” decision-making on your hands (Cousins, 2004, p.393).
Cousins, J. B. (2004). Commentary: Minimizing evaluation misuse as principled practice. The American Journal of Evaluation, 25(3), 393-399.