Today’s Plan:
- Sharing the Proposal Template
- Let’s Talk Surveys (And Qualitative Research Principles)
Introducing the Proposal Template
Here’s a link to the proposal.
In Monday’s class, I’ll ask you to meet up with your folks and share initial project ideas. Wednesday will be a work day in the lab. On Wednesday I will try to meet with potential teams to discuss data collection and norming; I’ll ask folks doing a standard Sicart analysis to try and draft a clean heuristic to analyze a game. On Friday, we’ll cover characters and actions again in class (or I can give teams more time to work on the proposal sections–let’s check in at the end of Friday’s class).
Surveys
Surveys typically collect three kinds of information:
- Attitudes and Preferences [data]: Generally leans towards what we should do.
- Opinions and/or Reactions [data]: Generally measures thought about what we have done.
- Demographic information [What do I need to know about my audience to frame my data]
Generally, you measure attitudes and preferences using multiple choice, ranking (favorite to least favorite) or likert scales. While the term likert scale might be unfamiliar, I can almost guarantee you’ve encountered one before.
- It is very likely you have encountered a Likert scale
- It is likely you have encountered a Likert scale
- It is neither likely or unlikely you have encountered a Likert scale
- It is unlikely you have encountered a Likert scale
- It is very unlikely you have encountered a Likert scale
Note: social scientists and marketers often omit the middle option above. Doing so forces a respondent to make a decision (the middle option provides them an opt out).
Note: If you do a ranking scale, make sure you tell someone whether 1 is their favorite or 1 is their least favorite. That is, if you ask someone to rank their preferences from 1 to 5, then be sure to write something like: please rank the following options from 1 (least favorite) to 5 (favorite).
We can collect more information in surveys via open ended, free write questions. There’s a few issues with these though. One is that people are likely to skip them. If you have more than one of these in a survey, your response rate is likely to plummet. The other difficulty is that these require quite a bit of time to “code”: that is, to go through and synthesize responses. However, that time is usually rewarded; for instance, I’ve published several articles on experimental class projects and I often get the best data from open-ended student responses, but this data takes much more time to analyze than a likert question.
Collecting demographic information is tricky because (some) people are skeptical of surveys. People can become suspicious if they think they know what your survey is attempting to prove. This can, if they disagree with you, create animosity, and lead to survey trolling. This is one reason it is important to create neutral, objective, balanced questions that do not preference a particular response.
Survey skepticism can often surface as a resistance to supplying demographic information. You have to think about what information you need to meaningfully code and analyze your data. A lot of the time, I complete a survey and wonder why they need to know how old I am or my sexuality. However, sometimes demographic information is extremely important–especially if we hypothesize that subject position informs outlook. So–an important preliminary question is to ask whether I *really* need to know demographic data.
If you need to collect demographic data, particularly data relating to race, sex, gender, and/or sexuality, then you need to be careful, diligent, and respectful. I think gender and sexuality are particularly difficult these days, given the rapid reconceptualization of those concepts (which is a good thing). So we should spend some time investigating how to ask demographic questions, particularly about gender and sexuality.
There’s more information on question types and some tips in this article.
What Not to Do in a Survey
Some general tips (emphasis–avoid loaded words). Some more tips (emphasis–use audience’s language).
A classic example of how not to construct a survey.
Okay, now let’s try crafting a survey question.
