Working in Social Science/Public Opinion research has made me very sympathetic to anyone who makes their living talking on the telephone. Something about the anonymity of the phone seems to make people feel very safe in expressing their deepest (racist/sexist/classist) thoughts to the interviewer. Conversely, many people are certain that you are trying to steal their money, their opinions, nay, their very souls through your insidious questioning about their attitudes toward any number of innocuous things. I want to discuss some basics of survey research and explain some of the qualities of any kind of opinion or attitudinal research. These are often the surveys that form the basis of the “97% of Americans HATE BREATHING AIR” kinds of stories that we see on a daily basis. I feel very strongly that public opinion and social science research is incredibly important to understanding our society and our culture, and it is frustrating as all hell to see it done badly or reported in such a way as to obscure what was actually found.
Come with me, nerdlings, on a magical adventure in three parts that will answer all of your burning questions about survey research and statistics. Let’s start with the most important part of any survey: The Respondent. There are generally three types of survey respondents aside from the straight shooters who just answer the questions.
The suspicious respondent: These folks won’t answer your liberal conspiracy questions because they know that you are somehow recoding their DNA by asking them to strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree with the following statements. In the next breath these people will give out any number of intensely personal details about themselves and/or call Obama racial slurs. They will also wait until halfway through the survey to ask who’s funding the survey, what the results are used for, why you’re asking them x/y/z, and then refuse to answer all of the demographic questions. These respondents like to make snide jokes all the way through any survey that even comes near any kind of political topic. They expect the interviewer to laugh at their sallies about socialism, taxes, poor people, etc. (I’ll explain why the interviewer CANNOT DO THIS a bit later/in the next installment). When asked “What race or ethnicity best describes you?” they will often respond with “other” and specify things like “human” or “American” and act all offended that you are asking in the first place. They also almost always refuse to respond to the income question.
The crazy cat lady/man: Living alone in a house with 57 cats and stockpiles of Lipton Cup-a-Soup, these folks are desperate to talk to anyone who will listen and respond with something other than “meow.” Thus, each question the interviewer asks them becomes an occasion for them to tell various parts of their life story. While it is always interesting to hear what people think, in the context of a telephone survey the interviewer is just trying to get the responses, complete the survey, and move on to the next complete. The crazy cat person can turn a 5 minute survey into a 35 minute marathon.
The critic: These are the WORST. They will critique the wording of each question in pedantic language that they clearly don’t understand, refuse to choose a response from the list offered, and generally just make it difficult to finish the interview. They will ask numerous questions about why each question is being asked, and expect the interviewer (who often has NOTHING to do with the development of the actual survey) to answer intricate methodological questions.
Without these respondents (and hopefully a whole lot of non-problematic ones as well), no survey would ever get finished. In the next installment I’ll explain a bit about why we ask the questions we ask, what’s up with that whole “strongly agree to strongly disagree” scale, and how we go about measuring the things we try to measure in social science research. See you next time, nerdlings!
