Week 13: Improving use by involving key stakeholders

The word “evaluation” can often be intimidating and nerve-racking for individuals and organizations. The fear of airing dirty laundry, examining outcomes that may or may not be there, and looking closely at what they do can bubble fears about job security, funding repercussions, and other uncomfortable indigestions. Evaluators can sometimes have a tough job of recruiting stakeholders to get on board with the work of conducting meaningful evaluation in their organizations. It can sometimes feel a bit like Sylvester trying to convince Tweety to jump in his mouth.

One way to ease the intimidation is to involve the key stakeholders at all levels of an organization/program throughout the entire evaluation process.

In a blog for the Harvard Business Review (January 23, 2012), Tony Schwartz writes, “Whatever else each of us derives from our work, there may be nothing more precious than the feeling that we truly matter—that we contribute unique value to the whole, and that we’re recognized for it.” He continues to explain that when people feel appreciated it lifts them up, makes them feel safe, and energizes them to do their best work.

These basic principles apply to evaluation. Engaging key stakeholders in an organization/program in the evaluation process creates buy-in, helps people see the value of their roles and work, and empowers participants to use the evaluation results to improve their organizations/programs. By involving key stakeholders in the planning, execution and analysis, they become a part of the process—adding unique value to the whole and feeling that they truly matter to the work.

So what does this look like in practice? How can key stakeholder be involved in the evaluation process? Below are some examples from our work at iEval that involved stakeholders—while there are certainly a myriad of ways to do this depending on the project, these examples will illustrate several methods with which we’ve had success.

1. Involvement in Evaluation Planning

  • In the evaluation of a county-wide cradle to career initiative, iEval took a developmental evaluation approach where the evaluation team spent the beginning phases of the project documenting the processes, systems, decisions, and tensions of the initiative to properly inform an evaluation plan for the initiative as the work moves forward. The documentation phase of this project involved extensive interviews, coordination, and relationship-building with each of the project leads. By involving the leader in each project area, we were able to build trusting relationships, listen to the stakeholders about what types of evaluation would be useful and strategize on how to implement those plans (e.g., social network analysis, GIS mapping, data collection analysis to assist with strategic planning).
  • In the starting phases of a funder-required evaluation of an afterschool program at a school district, iEval organized a meeting with the key contacts at the school district/buildings, the after school project manager/director, the site coordinators, and the data manager who we will be working with to gather the appropriate data. By bringing everyone together on day one, the key stakeholders are hearing the same message and we are often able to make an evaluation team for the project.

2. Involvement in Evaluation Implementation

  • The devil is in the details, right? And therefore, one of the strongest tools an evaluator can have is strong relationships with stakeholders on the ground where the evaluation is being conducted (e.g., teachers willing to administer surveys, site coordinators willing to box surveys up and mail them to the right place, a data contact who is willing to work hard to get you the right data). Involving and appreciating those on the ground level is key—we often do this by bringing snacks to meetings, offering incentive gift cards, sending emails or hand written notes to say thank you, following-through in a timely matter when we are asked questions, accommodating schedules when we can, etc.
  • Another way iEval involves key stakeholder in the evaluation implementation is by clearly outlining, at the beginning, what the necessary time involvement is of each of the stakeholders. Examples of this include: setting aside time to participate in interviews, coordinating staff members for focus groups, pulling data off the district server, or attending meetings. This provides an opportunity for those involved to allocate their time accordingly.

3. Involvement in Analysis

  • One of the most salient examples in our work has been the execution of Camp iEval. This working retreat takes place with a cohort of 21st CCLC after school program project directors across the state. At Camp iEval, we share each program’s annual evaluation report and review each of the key analysis elements, such as attendance trends and program outcomes. The most important aspect of Camp iEval, however, is not the discussion about their program findings, but the opportunity for project directors to engage in conversations with each other about program improvement and with iEval about what types of further analysis and digging deeper that would be meaningful for future evaluation work. The day-long session is instrumental in shaping the data analysis iEval will conduct in the coming year, which is largely based off of our discussions at this retreat.

These examples highlight some of the ways we work to involve and appreciate those we work with in evaluation. Please share some of your examples with us! 

 /* Style Definitions */
	{mso-style-name:"Table Normal";
	mso-padding-alt:0in 5.4pt 0in 5.4pt;
    KELLEY’S USEFUL TIP:  Remember to involve key stakeholders in all parts of the evaluation, from evaluation planning to implementation to analysis, which will encourage their ownership and use of the evaluation findings. 

KELLEY’S USEFUL TIP:  Remember to involve key stakeholders in all parts of the evaluation, from evaluation planning to implementation to analysis, which will encourage their ownership and use of the evaluation findings.