The most exciting moments in evaluation work, for me, are when clients internalize the data and have that A-HA moment when they either truly understand the data or are able to apply the data/recommendations to make some meaningful changes. Finishing a dashboard or report may give me a sense of accomplishment, but it’s not until the client makes use of those things that I genuinely feel like I’ve made a difference. It really energizes me to see the light bulbs go on, facilitate the conversations around reports/data/findings/visualizations, and assist in the planning for program improvement.
Recently, I had a different type of A-HA moment, which was completely unexpected and still gives me goosebumps to talk about! My team was charged with determining the ideal staffing ratio to provide mandated services within domestic violence shelters in a state.
This was a pretty high-stakes evaluation, as any staffing ratio recommendation could result in the loss or addition of jobs across the state. It was a very complicated project, including implementation and analyses of program observations, interviews, surveys, document reviews, financial data, and service delivery data. After months of deep digging into the data, we were able to determine an ideal staffing ratio from two completely different perspectives (and sets of data)…and they matched! We found a staffing ratio that could be validated two completely different ways. We were so jazzed about this and couldn’t wait to share it with the client! The client was extremely excited by the findings, as well, and felt confident in our processes.
I always encourage the triangulation of multiple data sources to support findings. This, of course, encourages the use of the data, since the findings and recommendations are more robust. This recent experience, though, with the ideal staffing ratio, has reminded me just how critical the triangulation of data can be. How many evaluation reports have you read that make vast generalizations, only to find out that their n was 3 people? We, as evaluators, need to make sure we maintain high quality in our evaluation plans, implementation, analyses, and reporting…not only to encourage use, but also to improve client confidence in the work.
DR. TACKETT’S USEFUL TIP: Incorporate the triangulation of data in your key findings and recommendations, building confidence and encouraging use.