So, you’re probably thinking, “That first post made no sense. Of course I’ll keep reading!” It’s okay. I never seem to be able to start at the beginning. SO. In the interest of explaining what survey research actually is, how it works in practice, and what it aims to accomplish, I’ll backtrack and give you just a little about the tools and methods that are commonly used. Get out your notebooks everyone, I’m going to get all teachery for a bit. If you’d rather be on Facebook, or sleeping, or texting, there’s the door.
Adjusts glasses.
Let’s start with the ‘Why?’ There are lots of reasons to do research that collects data by asking lots of people the same set of questions in order to make statistically supported inferences about a larger group. Political polls are one of the most common, but marketing surveys apply here as well. Academics, public agencies like HUD or DHS- particularly at the state and local level, policy-makers, watchdog groups, businesses; all of them need to know what people think, how they make decisions, how their behaviors are influenced, what they like, and what they dislike. The best way to know what lots of people think about something is to ask them, right? But unless you have godlike resources, it is impossible to ask every single member of the group you want to survey. This is where statistics comes in. (Just raise your hand if I lose you- I’ll double back, no problem). Statistics allows us to have a pretty good idea whether or not our results are due to more than chance in the numbers or effects from the interviewer or the questions. If the survey is carefully designed, and the methods are sound, AND the researcher is honest about assumptions and limitations (that is a big one. HUGE), then tentative conclusions can be drawn about patterns in the data based on statistical significance and effect size.
This is not the same as causal argument. Repetan, por favor: “Correlation is not causality.” Remember that sentence. You shouldn’t have to ask me if it’s going to be on the exam.
It’s extremely complicated to ask lots of people questions and be asking a representative sample at the same time. Representative samples are good. They ensure that you can make inferences about a larger group by sampling from within it. So, if I want to know what Minnesotans think about Michele Bachmann’s crazyeyes, I don’t have to call every single one of them and ask. I DO need to get a random, representative sample if I want to assert that 46.2% OF ALL MINNESOTANS said that they loved the crazyeyes. This would mean that the proportions of each demographic group in my sample (the group that I called on the phone) match the population of Minnesota. This is harder to do that it sounds. (There’s more to it than that, but you get the idea. More here).
“But,” you’re thinking “people lie their faces off.” There is that. I agree. However, there’s a much bigger problem. How do you measure how much people believe something? Take “religiosity,” or how religious one is (or is not). How do you measure this by asking a question? Do you go with “how often do you attend a public religious service?” Well, that measures one aspect of religiosity, but what about how close I feel to god? That’s an important part of religiosity too, right? And what about how I practice in everyday life? That’s important too, right? So you can see that religiosity cannot be measured using only one dimension of the concept. You have to ask lots of questions that get at the same concept in order to measure the concept you want to know about. (There are statistical tests that can tell you how well you’re measuring the concept, but that’s after you collect your data).
…you can see the problem. There are a lot of things that have to be accounted for before you even decide how you’re going to ask the questions (over the phone? In person? Via the web? Via mail)? We’ll talk about deciding on a data collection mode in part 2.
