This week iEval worked with a group that was considering a dataset that was actually quite simple but complicated by the fact that numbers in the data sets varied slightly between stakeholders. For example, while the researcher through there were X participants, the program staff through they had X + 2 participants. These nuances were hanging the group up–and as an evaluation firm that stands behind its analysis–they tripped us up too. How can we measure change when we don’t even really know where we are starting?

And then I had to remember the KISS principle–Keep it Simple, Stupid.

So in our case, the nuances of numbers being off by a few participants here and there really didn’t impact the overall picture of what the data were saying. So when reviewing the data with stakeholders, we decided to frame the data digging around two questions:

1. What are the data telling us?

What are the trends over the last year? What patterns emerge? How has the data changed since last year’s reporting? What surprises are there? What doesn’t the data account for? (e.g., forces of nature) Where are the bright spots–things that are going well despite difficult odds?

2. How can/should the data inform practice?

What strategies are working according to the data? What strategies aren’t working? How might we replicate successful strategies? What data do we need to tell a complete story that we aren’t yet tracking? What steps need to be in place to gather that data?

These two simple questions helped us get out of the weeds of sorting out the particulars when they didn’t really impact the overall picture of data trends, successes, and areas needing additional focus. Sometimes it is hard to step back, take a deep breath, and look holistically at what we do know instead of being bogged down in what isn’t quite right.

KELLEY’S USEFUL TIP: When working with a complicated data set or when conducting analysis, take a few minutes to step back and reflect on the overall project goals or research questions. Sometimes a group’s questions and energy redirect in a way that is counter-productive to larger goal. Don’t lose sight of the larger goal and questions as a project evolves.