In some of our recent work, we have been engaged in gathering, synthesizing, and presenting an array of data and information to a large, diverse group of stakeholders. Information has been presented to the group at large meetings, where smaller groups discuss information specific to a shared interests. The groups we have worked with have gone through a process whereby they interpret relationships in the data, which raises new questions. These questions often lead to new questions, and then new and different conclusions. By the end of this, we had provided the group with a vast array of information, and it got me thinking about what constitutes too much.
I believe as evaluators we have a responsibility to provide data, but also to help insure that information is used correctly. I started to consider what constituted too much information. We had provided so much data that 1) we had overwhelmed the group with information and 2) the group had come to conclusions which they believed to be true, but which were not fully supported by the data we shared.
Here are a few conclusions and thoughts I have about these types of situations:
- Be clear about the difference between correlation and causation: Groups who are unfamiliar with data and research methods may be quick to interpret information which looks correlational as evidence of a causal relationship. However, it is important to understand that correlations only suggest some relationship, not one that can be interpreted as X causes Y.
- Clarify limitations: Be sure that any limitations associated with specific data are made clear, so these are taken into account by the group as they dive into the sense making process.
- Consider the purpose: Not all data may be relevant to a particular issue, and presenting it may only confuse people or muddy the waters. Consider what the most important pieces of information are, how they connect to your overarching questions, and rely on prior research if available to guide you in the decision making process of what should or should not be used.
- Facilitate inclusion: As you facilitate the interpretation process, take steps to engage all members of the group. Even in smaller groups, people interpret things differently based on their experience with the issues at hand. It is critical to engage all people involved to get an inclusive array of conclusions.
COREY’S USEFUL TIP: Be prepared! We have spent many hours preparing for each of these meetings, including how we would present the information to the group. We developed guiding questions to keep the group focused, and made sure to limit the amount of time we spent on each question. By having a process, we were able to make sure that time was spent efficiently and effectively.