Today’s Plan
- Surveys
- Focus Groups
- Interviews
Surveys
Surveys typically collect three kinds of information:
- Attitudes and Preferences
- Opinions and/or Reactions
- Demographic information
Generally, you measure attitudes and preferences using multiple choice, ranking (favorite to least favorite) or likert scales. While the term likert scale might be unfamiliar, I can almost guarantee you’ve encountered one before.
- It is very likely you have encountered a Likert scale
- It is likely you have encountered a Likert scale
- It is neither likely or unlikely you have encountered a Likert scale
- It is unlikely you have encountered a Likert scale
- It is very unlikely you have encountered a Likert scale
Note: social scientists and marketers often omit the middle option above. Doing so forces a respondent to make a decision (the middle option provides them an opt out).
Note: If you do a ranking scale, make sure you tell someone whether 1 is their favorite or 1 is their least favorite. That is, if you ask someone to rank their preferences from 1 to 5, then be sure to write something like: please rank the following options from 1 (least favorite) to 5 (favorite).
We can collect more information in surveys via open ended, free write questions. There’s a few issues with these though. One is that people are likely to skip them. If you have more than one of these in a survey, your response rate is likely to plummet. The other difficulty is that these require quite a bit of time to “code”: that is, to go through and synthesize responses. However, that time is usually rewarded; for instance, I’ve published several articles on experimental class projects and I often get the best data from open-ended student responses, but this data takes much more time to analyze than a likert question.
Collecting demographic information is tricky because (some) people are skeptical of surveys. People can become suspicious if they think they know what your survey is attempting to prove. This can, if they disagree with you, create animosity, and lead to survey trolling. This is one reason it is important to create neutral, objective, balanced questions that do not preference a particular response.
This skepticism often manifests itself as a resistance to supplying demographic information. However, sometimes demographic information is extremely important! You have to think about what information you need to meaningfully code and analyze your data. If you need to collect demographic data, particularly data relating to race, sex, gender, and/or sexuality, then you need to be careful, diligent, and respectful. I think gender and sexuality are particularly difficult these days, given the rapid reconceptualization of those concepts (which is a good thing). So we should spend some time investigating how to ask demographic questions, particularly about gender and sexuality.
There’s more information on question types and some tips in this article.
What Not to Do in a Survey
Some general tips (emphasis–avoid loaded words). Some more tips (emphasis–use audience’s language).
A classic example of how not to construct a survey.
Focus Groups
Focus Groups
Today I want to spend some time exploring focus groups. Focus groups can come in one of two generic flavors–either simple question/response (more common) or activity-based (less common). The value of a focus group lies in collecting a variety of perspectives *and* recording how people react to the ideas/opinions/experiences of other people. There is something unpredictably inventive, or choric, about the kind of dynamic conversations that focus groups can prompt.
- Group size: 3-5, 6-8
- Record discussion, assistant moderator takes notes
- Generate an outline and a “script” of questions
- Welcome
- Overview of Topic
- Ground Rules
- First Question [Build Trust]
- Follow Up Strategies
- Second Question [Probe Attitudes]
- Third Question [React to Commonplace / New Information]
- Sample questions / activities
- How to synthesize data
Let’s start by walking through this guide to focus groups by Richard Krueger.
Outside of academia, focus groups are generally used by marketing researchers.
You will also encounter focus groups in local journalism and political research
Interviews
Like focus groups, interviews are also constructed around recording responses to open-ended questions. Asking “yes” no questions, even with a “why?” follow up, tends to be counter productive.
For instance, don’t ask: “do you think gender portrayal in AAA video games is improving? … Why / not?” That might be an okay warm up question in a focus group (maybe), but if you’ve properly researched your interview subject, then you should be able to ask a more pointed, substantive question (move from attitudes to ideas): “What are some recent examples of games that are moving gender portrayal in more positive directions?” Or “Why, as the industry as a whole moves toward more positive portrayals, do you think games like Daddy’s Sugar Factory are still popular?” Good interviews will prompt experts to *think* rather than merely respond. And, like a good focus group leader, a good interviewer will be attuned to a subject’s comfort and mood–they can notice agitation and work to pull something more out of the subject (subtly re-asking a question).
So while you should enter an interview with a script of questions, you should also treat them as more free-flowing encounters. Great interviewers adapt their tone and approach to the specifics of the situation, paying close attention to their subject’s deportment and comfort level as the interview goes on.
Two resources on interviews:
Homework
I’d like you to sink at least one hour into your project between today and Friday: I leave it up to you on how to spend that hour. Except: if you are developing a research tool (survey, focus group, interview), then I’ll ask you go back into your proposal and revise it after today’s session.
I’ll review all of the proposals before Friday’s class. We will meet in the Ross 1240 computer lab Friday for an in-class writing day.
