Who Answers Polls? Do You?

Nate Silver recently gave a good description of something that’s been gnawing at me lately . . . do shifts in enthusiasm shift the response rates of partisans? 

Is part of Romney’s vote-preference bounce due to a shift in enthusiasm? And what does that mean for response bias in polls pre-debate? Here’s Silver:

Polling firms are hoping that the 10 percent of people that they do reach are representative of the 90 percent that they don’t, but who will nevertheless vote. But there are no guarantees of this, and it is really something of a leap of faith. The willingness to respond to surveys may depend in part on the enthusiasm that voters have about the election on any given day.

Aren’t adults who are more enthusiastic about politics also more likely to vote? Sure, but the polls have other ways of handling that problem. The platonic ideal is to reach a random sample of all American adults, and then to apply a likely voter method to eliminate those who are unregistered or unlikely to vote. If you have a sample that is biased toward enthusiastic respondents to begin with, and then apply a likely voter screen, it risks double-counting the enthusiasm factor, especially in cases like presidential general elections when overall turnout is quite high.

Here’s the added twist, though . . . If polling is susceptible to this kind of response bias, perhaps the pre-debate polls were over-estimating Obama’s advantage. On the Republican side, there is more than enthusiasm in the response equation. Going on anecdote, a fair chunk of conservatives won’t answer polls because they think they are biased against them and another group will answer with made-up responses for similar reasons.

When we’re talking about shifts in the vote of around 3 points or so, small shifts in response bias among the 90-95 percent of citizens who don’t answer surveys could make a significant difference.

Do you answer surveys? If not, why not? And if so, why? Do you give made-up answers?

  1. KarlUB

    I do not answer surveys as the calls come in on my land-line which I do not pick up, and does not have an answering machine. It only exists because in the event of a gigantic disaster I imagine the landlines will come back up faster than the cell towers.

    As yet pollsters do not have my cell phone.

    I do, though, answer internet polls on local elections, and on national issues involving the Campaign for Liberty.

  2. Mollie Hemingway

    I no longer answer phone calls from numbers I don’t recognize, so take that for what it’s worth, but I was once polled and answered the questions.

    I was just in Montana last weekend and the ladies I met with were regaling me with stories about how they have messed with pollsters. They were really pleased with themselves for answering in the most extreme — and wrong — way they could. I found it hilarious.

  3. At The Rubicon

    I do not respond to surveys. Period. They take too much time and are to intrusive. The call almost always comes in at a most inconvenient moment.

  4. SteveSc

    I hang up on most and don’t care who is on the other end.  Or I ask them “For the love of God, is this a political call?”  When they try to answer in an Obamian fashion, then I hang up.

  5. Misthiocracy

    I only answer polls if I’m pretty certain that the poll has been commissioned by one of the opposition parties, in which case I answer every question incorrectly in order to give the opposition party bad data.

    It’s the same reason I tell door-to-door campaigners that I love their candidate and I want all the lawn signs and bumper stickers they can give me. It costs ‘em.

  6. Foxman

    Do not ask for whom the poll calls.  It calls for you.

  7. Richard

    I have lived most of my adult life overseas, for the short period of time I lived in the US as an adult, I was only called by a pollster once, and it was to ask about whether or not I approve of Martha Stewart. It was during the time of insider trading issue that she was dealing with. I did answer the questions for the fun of it. I don’t remember if I said I approved of her or not, I don’t think I cared enough to have an opinion. 

  8. Adam Schaeffer
    C

    Ha . . . so, my base of anecdotes grows! 

    Molly, my father loves to give bizarre combinations of responses too. 

    This is a potentially very serious problem for the state of polling, and it’s very hard to think of a way to even get a handle on the scope of the problem, since non-respondents are not responding, so how do you know what they are thinking? 

    I’d imagine the overall impact is pretty small, just because very highly engaged and knowledgeable voters who I think we’re talking about are a pretty small group.

    But you don’t need very big numbers when what polls are looking for is a precise point estimate of a race that is, say, 48/45. Very small biases can make all the difference when an election is decided by a 3-point margin.

    Any more on how you respond, or don’t? I should throw up a survey to ask you all, but then how would I know you’re telling the truth? Man . . . the problems you all cause for poor old pollsters!

  9. Percival

    I’m pretty hostile to cold calls on a phone I’m paying for.  I think polls relying on phone contacts need to add another category: “Snarled and Hung Up.”

  10. Will Collier

    No.  I was one of those people who said (and meant it), “I’ve never been called by a pollster,” at least until this year.  In ’12, our phone rang off the hook with pollsters for most of the summer. Without exception (on the occasions I was home to receive the call), I hung up on all of them.

    I don’t have the time to answer inane questions for half an hour, and the odds that I can find out (a) who the pollster is and (b) who they’re working for are close to zero.  Sorry, not playing that game.

  11. Spin

    A pollster called my father in law yesterday and started asking about who he thought won the debate.  He said “I’ll tell you what I think of Obama!  He makes Nixon look like Billy Graham!”  The pollster hung up.

    To answer the question:  no, I do not.  I always hang up.  It will answer the final poll on that Tuesday in November.

  12. iWc

    I answer all calls. Then hang up on any polls or solicitations.

    My time is very valuable to me. When I want to waste it, there is always Ricochet!

  13. M1919A4

    I should answer the polls correctly, but we do not get called by respectable poll takers.  

    My wife checked directly with Rasmussen, whom we consider the most reliable one, and learnt that the identity that appears on the “tattletale” is their own name.  We’d respond to their questions.

     Any others don’t get answered.

  14. jarhead
    C

    I also don’t take calls on either landline or cellphone from numbers I don’t recognize, and I don’t use voicemail.  If I have time, I do look up the numbers on the Internet to see who’s calling.  As my wife and I tell people, most of you have either our email and/or snail mail address, and if it’s important enough, you know how to contact us.  For unwanted email, the Delete key works very well, and for unwanted snail mail, so does the paper shredder.

  15. Jimmy Carter

    I give made-up answers…. or am I making this up?

  16. Israel P.
    Misthiocracy: I only answer polls if I’m pretty certain that the poll has been commissioned by one of the opposition parties, in which case I answer every question incorrectly in order to give the opposition party bad data.

    Sooner or later someone will try to pass a law making that illegal. Like perjury.

  17. cdor

    My household of two people receives, on average, twelve calls per day from “out of area” or other such non-identifiable numbers. We receive at least three calls during dinner and three in the evening, the rest during the day. Sometimes I pick up and ask the supervisor to remove my number from their list. Mostly I take the phone off the hook and immediately hang up. If I let it ring thru to my answering machine, they hang up. The whole process is very irritating. If I accidentally answered a call and was not already aggravated to wits end, I would definitely answer the questions honestly. I have often wondered how much of the population no longer has a land line. Polling must be very difficult to do accurately.

  18. The Mugwump

    My general rule is that I don’t talk to strangers on the phone.  Period.  Not for any reason.  Why would I volunteer any personal information to someone I don’t know?

    RE:  Polling

    Every poll is based on a model, and every model is based on assumptions.  GIGO, right?  The second flaw is that most polling is designed to effect public opinion rather than provide an accurate picture of the race as it stands.  The internal polling by the campaigns is the best source of data, but that information is not shared with the public.  So, what are we left with?  

    It takes a bit of digging to get a true indication.  It’s far more meaningful to know, for example, that Candidate A has pulled all his money from State X.  We can also detect trends even when the data is faulty.  I can conclude that Romney has Florida sewn up and he’s surging nationwide based on the RCP averages even if much of the data is based on speculative modeling.  I also know that with 12 states rated toss-up, all of which went for Obama last election, that the Obama campaign is clearly in trouble.    

  19. At The Rubicon

    If I owned a polling company I would be very afraid for the future of my business. Polling is essentially a sales cold call. It is an interruption. Marketing gurus like Seth Godin have written many words about the hazards of interruption marketing.

    I sit here at my desk engaging in activities that have value for me. The phone rings. Dealing with a pollster has negative value: not only is the investment of my time answering the pollster’s questions returning zero value to me, it is preventing me from performing the activities that DO generate value.

    If the polling business is to survive, it will need to discover a way to gather data in a non-interruptive way, or to provide a perceived value to the person polled that exceeds the investment of time.

  20. Pseudodionysius

    I know several people who have worked for polling organizations and like knowing the inner workings of a sausage factory, the meat is not very appetizing. The last time I answered a poll in was in university, and Igave an insanely wrong answer to every question. Be on the lookout for a long haired twenty something investment banker with a summer home in the Hampton’s, several cars, and no discernible source of income except “other”. My main objection to polling is a moral one: they take my time for free and then sell the proceeds. If someone wants to pay me for my time that’s one thing, but I’m not giving away my opinions for free, and if I get asked a foolish or misleading question, I’d have no objection to doing some Newtjitsu on the questioner, which isn’t fair to the person simply reading a script.