Week 66: Impact of Evaluation Focused on Use, a Client’s Perspective

Continuing our temporary monthly component of asking some of our clients to serve as guest bloggers, sharing their perspectives on how specific evaluation tools are useful or how they use evaluation findings in their jobs, next up is James Hissong. James was the quality and evaluation coordinator for Communities in Schools of Kalamazoo, working closely with their 21st Century Community Learning Centers programs. He has recently moved on to a position with St. Joseph County, where we wish him the best! Here are some of James’ thoughts on how he saw evaluation being useful…

Evaluation is a vital part of Communities In Schools of Kalamazoo’s (CIS) model of continuous quality improvement. iEval’s practices reinforce evaluation as an ongoing part of the organization’s culture as opposed to treating evaluation as an event. CIS uses the evaluation reports in a variety of ways.

At the programmatic level, our after school coordinators incorporate evaluation findings into annual program improvement plans, setting goals based on the data presented in the evaluation reports and using subsequent reports to monitor progress. This process has brought staff greater awareness of such issues as the impact that increased dosage (i.e., regular program attendance) has on student outcomes and has spurred staff to set goals using program attendance benchmarks. Using this method, CIS has had positive results in increasing regular program attendance from 63% of program participants attending 30 or more days in the 2011-12 school year to 77% in the 2012-13 school year.

At an administrative level, CIS has used evaluation findings to make changes to organizational policies and practices such as student enrollment procedures and staffing structures. For example, in an effort to improve student academic outcomes and increase ties to the school day curriculum, CIS employs school day teachers within our after school programs as instructional specialists. Additionally, feedback from iEval’s evaluations of like programs across the state prompted CIS to change staff recruitment strategies, targeting staff members with a stronger background in teaching and youth development.

We believe that these evaluations have made a positive impact on our programs because of our capacity, with assistance from iEval, to actually use the reports in our decision-making processes. Events such as Camp iEval days, are an added benefit, as they allow for CIS program directors to interact with other program directors with similar goals, facing similar issues. These interactions allow for genuine discussion across sites and are a great way to share best practices among colleagues that we do not normally interact with. iEval provides multiple mechanisms to prompt the use of data for healthy and continuous self-assessment. Among the other valued approaches of working with an external evaluation team is their responsiveness to questions, observations, and requests, but especially a joint commitment to making the evaluation as relevant as possible to the end users.

JAMES’ USEFUL TIP: It is important your organization to have built-in mechanisms and expectations for using the information provided by the external evaluation team.