Week 64: Asking Your Evaluator Questions

A long, expensive car with darkly tinted windows pulls up…

A team of three individuals get out holding their leather briefcases, dressed in name brand tailored suits, and walking in union with dark sunglasses covering their eyes…

They walk on past you into your board room where you’re not even sure you belong anymore…

One removes her sunglasses, peers down at you, and says, “Now we’re going to tell you what your data mean, what you are going to do next, and when we expect payment.”

What a nightmare! Hopefully no evaluation team you’ve ever worked with operates like that, but sometimes the clients feel that way even if that’s the furthest thing from the truth. Data can be intimidating. Evaluation connotes judgment, which is always scary. As evaluators, we need to be aware of the image we portray, and do what we can to be friendly yet professional. As clients, you need to remember that you are paying for the evaluation service, and you have the right to question anything and everything your evaluation team is telling you. Just because the evaluators have fancy degrees or long reports does not mean that your questions are any less important to the process…just the opposite, in fact! The more questions the clients ask, the better the evaluation is understood and internalized, and the more likely the findings will be applied in the appropriate manner to improve programs and make important decisions.

This evaluation is FOR YOU…for the client! You need to be prepared to ask questions of the evaluation team. You may not feel ready to do that when you first see a report, which is why we typically share a report virtually first so the client has time to read through it and process it on your own time. That way you don’t feel pressured to ask brilliant questions on the spot. After the clients have had time to review the reports, we either have a phone call or meet in person to review the reports and answer any questions. Sometimes we are asked questions that we don’t have immediate answers for and need to go back and do some more digging to uncover the answers – those are the best kinds of questions because they push everyone to go further!

If you’re unsure of what to ask, here are some general ideas to get you started:

  • If a chart or table isn’t clear – ask for a step-by-step explanation of what it says or if it can be presented in a different visual or if there could be some narrative text inserted to explain it so you don’t forget later.
  • If there is a finding or recommendation you don’t quite understand or even disagree with – ask how that finding was validated…that is, were there multiple data sources that confirmed that finding or recommendation.
  • If a finding or recommendation seems unrelated to your specific project – ask how the evaluation team came up with that one (e.g., maybe it’s based on their knowledge of other similar projects and they’re trying to apply findings more universally).
  • If something you expected to see in the evaluation report is missing – ask the evaluation team where it is. It may be embedded in another component, it may have been purposefully excluded because of problems with the data, or it may have been forgotten…but you don’t know unless you ask.
  • If a finding interests you and you want to know more – do not be afraid to ask probing questions that may need some additional data analyses to uncover the answers. Most evaluators build in that expectation into their plan – do not assume that a report is final. We always present a final report more as a draft, with the understanding that there may be some additional work that needs to be done after it is shared.

DR. TACKETT’S USEFUL TIP: There are no dumb questions. In fact, some of the sillier questions have led to really useful opportunities to dig deeper into the data and develop meaningful shared understandings that could be immediately applied to programs!