Week 2: What are evaluation use, usefulness, utility, and utilization?


Stephen Hawking, in A Brief History of Time (1988), stated, “My publisher told me that the number of readers of this book will be inversely proportional to the number of equations it contains.” Do you know how many equations he then chose to include? One. E=MC2. In that same vein, we don’t want to go crazy with evaluation theories or technical jargon. But, we are going to share some key definitions as they’ve evolved over time and how we plan to use them in future postings.

The definitions of use, utility, and utilization have been debated in the evaluation literature for years:

  • Weiss (1979) did not like the term utilization because she felt it sounded too concrete, and the practice of using evaluation findings is more fluid than that.
  • Stevenson (1980) felt that too much credence was being given to utility and utilization is the more appropriate term. He thought utility only looked at the evaluators work while utilization was more suitable because it also focused on unintended evaluation consequences and multiple evaluation finding uses.
  • Alkin (1982) said that an evaluation has utility if it is presented in a way that makes it useful, and an evaluation is utilized if there are immediate and direct applications of the evaluation findings.
  • King and Pechman (1982) felt utilization was a term that could be easily misunderstood, while use is a term with few inappropriate connotations.
  • Daillak (1982) explained that utilization was simply use with the understanding that it has beneficial, profitable, or productive outcomes.
  • Cousins and Leithwood (1986) classified use as the processing of evaluation information, decision-making, and education.
  • King (1988) said if the definition of use was broader, basically including compliance with legislated evaluation requirements (specific for federally-funded program), then most evaluation findings are used.
  • An expanded explanation of use includes the following: basic knowledge and understanding of the evaluation plan and findings, possible justification for decisions already made, and the impetus for action (program improvement and/or decision-making) based on the evaluation findings (Owen & Rogers, 1999; Patton, 1997; Rich, 1977; Rossi, 1999).

For the purposes of this blog, here are the definitions we will use:

   DR. TACKETT'S USEFUL TIP: If evaluation findings have UTILITY, the client will understand the potential USE of the findings so they can be UTILIZED.

DR. TACKETT'S USEFUL TIP: If evaluation findings have UTILITY, the client will understand the potential USE of the findings so they can be UTILIZED.

  • UTILITY (interchangeable with USEFULNESS) will refer to the practical applicability of evaluation findings for the client’s use
  • USE will refer to the direct and immediate application of evaluation findings for program improvement or decision-making
  • UTILIZATION will refer to the fact if the evaluation findings were used to have intended or unintended impact

Clear as mud? Well, hopefully you at least get the idea, and we’ll be exploring ways to make these concepts come to life in future entries.

 

Alkin, M. C. (1982). Introduction: Parameters of evaluation utilization/use. Studies in Educational Evaluation, 8, 153-155.

Cousins, J. B., & Leithwood, K. A. (1986). Current empirical research on evaluation utilization. Review of Educational Research, 56(3), 331-364.

Daillak, R. H. (1982). What is evaluation utilization? Studies in Educational Evaluation, 8, 157-162.

King, J. A., & Pechman, E. M. (1982). The process of evaluation use in local school settings. Final report. National Institute of Education grant (81-0900). New Orleans: New Orleans Public Schools.

King, J. A. (1988). Research on evaluation use and its implications for evaluation research and practice. Studies in Educational Evaluation, 14, 285-299.

Owen, J. M. & Rogers, P. J. (1999). Program evaluation. Thousand Oaks, CA: Sage Publications.

Patton, M. Q. (1997). Utilization-focused evaluation (3rd ed.). Thousand Oaks, CA: Sage Publications.

Rich, T. (1977). Uses of social science information by federal bureaucrats: Knowledge for action versus knowledge for understanding. In C. Weiss (Ed.), Using social research in public policy making. Lexington, MA: Leggett.

Rossi, P. H., Freeman, H. E., & Lipsey, M. W. (1999). Evaluation (6th ed.). Thousand Oaks, CA: Sage Publications.

Stevenson, J. F. (1980). Assessing evaluation utilization in human service agencies. In J. A. Ciarlo (Ed.), Utilizing evaluation: Concepts and measurement techniques. Beverly Hills, CA: Sage Publications.

Weiss, C. H. (1979). Conceptual issues in measuring the utilization of research and information. Paper presented at the Annual Meeting of the Evaluation Research Society (Minneapolis, MN, 1979)