I presented at the American Evaluation Association annual conference in Denver last week to a standing-room only crowd. The topic was “Tips and Tricks for Conducting On-Site Observations.”
I geared the presentation for graduate students and new evaluators, but the topic hit a nerve with more than just the newest to our profession. One woman said she had been working in evaluation since 1999 and over half of the audience had already completed site visits.
Something about observations and site visits mystifies people. How do you do it? Here are three tips that generated the most conversation at my AEA presentation:
- Think about the image you want to portray and what is appropriate for the people you are observing. My experience with observations is in educational settings, mainly observing classrooms and professional development sessions. In those types of environments, the observers should wear business casual clothes. A business suit can be overkill for a kindergarten classroom. However, business casual may be too casual for the people you are observing. At my AEA session, one person said that within the cultures she works with, if you are observing and are not dressed in professional attire, they will not take you as seriously. Another person shared a story about worshipping in a mosque when a person collecting data came in without her head covered and people avoided her. I’m guessing she did not have very good response rates that day!
- Decide if you want to bring a computer, tablet or paper and pencil to take notes. I always stick to a paper and pencil when I conduct site visits. I find computers to be too intrusive, typing can be loud, and power sources are not always convenient. Again, know your audience. Maybe the people you are observing expect a computer. I used to be a classroom teacher and I put myself back in the teacher’s shoes when I conduct observations. If someone I didn’t know was sitting in the back of my classroom typing on a computer I would be even more nervous about the whole situation. A paper and pencil seem more kind. In addition to the paper and pencil, I bring copies of the protocols I am using and also take notes. Try to limit the amount of stuff you bring – don’t move in.
- Know your observer role. A lot of the conversation at AEA was around what is the appropriate role for an observer. I don’t think there is an easy answer. This is something to think about and discuss with your colleagues before you begin your observations. Gold’s Typology of Participant Observer Roles can help you identify your observer role.
Observer vs. Participant - Gold's (1958) Typology of the Participant Observer Roles
- The complete participant - takes an insider role, is fully part of the setting and often observes covertly.
- The participant as observer - the researcher gains access to a setting by virtue of having a natural and non-research reason for being part of the setting. As observers, they are part of the group being studied. This approach may be common in health care settings where members of the health care team are interested in observing operations in order to understand and improve care processes.
- The observer as participant - in this role, the researcher or observer has only minimal involvement in the social setting being studied. There is some connection to the setting but the observer is not naturally and normally part of the social setting.
- The complete observer - the researcher does not take part in the social setting at all. An example of complete observation might be watching children play from behind a two-way mirror.
Gold, R. (1958). "Roles in sociological field observation." Social Forces, 36, 217-213.
DR. EVERETT’S USEFUL TIP: Know your audience when you conduct a site visit. Will they expect you to wear a suit and bring a computer? Figure out what your participant observer role will be for the project before you begin your site visits. Site visits can be difficult but some pre-planning can help them run more smoothly.