Demosophia

Wishfully Thinking About the Polls (Updated)

This time it's Josh Marshall getting in synch with Ruy Teixeira. And the basic conjecture, is:

Gallup numbers are skewed because they include a substantially higher percentage of Republicans than have shown up to vote in the last several presidential elections.

Well, if we look at this carefully Teixeira is clearly not concerned about oversampling Republicans in the overall survey, because even a half-assed polling organization would take care to obtain roughly the same percentage of registered Republicans as there are in the population. (Although many states don't register party affiliation the proportion of registered voters who identify with a political party is usually known.) And even if they weren't able to do that they'd apply a weight when they conducted the analysis, to adjust for the oversample. What Ruy and Josh are talking about, I think, is the filter that's used to determine the likelihood of voting.

If the filter were more liberal for Republicans than it ought to be, the analysis would assume greater participation in the election by Republicans than will actually be the case. But that assumption, which allows Josh to be skeptical about the size of the Bush margin in the Gallup polls, is only justified if the assumption that Republicans will vote in this election in about the same proportion that they voted in 2000, is right. Well, why would it be?

Why would it be appropriate to assume Republicans will vote in the same proportions they were voting in the last handful of cycles? Republicans are far more likely to take the events of 9/11 seriously then are Democrats (although there are exceptions, like myself or Roger Simon, for instance). Indeed, one of the selling points of the Moven.org people is that there really is no terrorist threat. Now one has to assume that Moveon Democrats aren't voting just because there's nothing to worry about. To them, the threat is George W. Bush. But the Moveon folks aren't the majority of Democrats. At least, not yet.

They aren't the majority voice in the Democratic Party, although they may have influenced that majority significantly. But it's still unlikely that the majority is as motivated as the activists. And George W. is a relatively likable guy so the hard sell that he's Bushitler probably doesn't go over that well. It's more likely that the majority of Democrats have been influenced more by a soft sell that Bush just doesn't have the goods. Well, have the goods for what... if there's no terrorist threat? The problem is that the Moveon message is conflicting, even for Democrats. And a conflicted and conflicting message, especially when coupled to a less than inspiring candidate, is simply not going to translate into overwhelming voter turnout.

Well, that's the conjecture. Add to that the fact that many Republicans are inspired by George Bush, and are also concerned about certain below-the-bible-belt issues, and it really doesn't seem far-fetched to assume that Republican turnout will be up in 2004. Am I wrong?

However, that's just my argument that Josh's and Ruy's explanation for the disparity between the Pew and the Gallup doesn't invalidate the results of the Gallup. My argument doesn't verify (or deny) that the reason for the gap lies in the "likely voter filter" in the first place. But I'm still very skeptical that it does. After all, the Pew poll shows the same gap for the 9/11-14 wave whether you're looking at registered or likely voters. And for the 9/8-10 wave the gap is larger for likely voters! If Pew's methodological assumption is that Republican turnout will be lower than Gallup predicts, wouldn't you expect the likely voter gap to be smaller than the registered voter gap? (Results for all the recent polls are at the Polling Report.)

So I'm stickin' with my earlier speculation (as yet unproved) that the culprit messing up the polls is some unexpected manifestation of the ecological falacy, as applied to an electorate that's in flux. It may or may not have to do with the estimates about who will be voting, but it's more likely that the shakeup involves voter preferences and behavior in general. Not only are our estimates of who will probably vote subject to error, but so are our estimates about the way certain demographic groups (by age, party affiliation, race, etc.) will actually vote. And I would argue that letting more Republicans through the filter is probably appropriate, because they are more likely to vote than in the past. In fact, I'd even go so far as to say that the more seriously voters take the "terrorism issue" the more likely they are to vote in November, and the more likely they'll vote for Bush. In fact, that wisdom makes so much sense that you're probably undewhelmed by my brilliance.

Update: Having dispensed with the argument about the filter used to determine likely voters Ruy says the following about the Bush margin of 8% among registered voters. Frankly, it mystifies me a bit. I wish I had access to the internals to suss this stuff out, but anyhow he says:

But then there's this: the Gallup internals show Kerry with a 7 point lead among independent RVs. Huh? Kerry's losing by 8 points overall, yet leading among indenpedents by 7. How is that possible? Only if there are substantially more Republicans than Democrats in the sample.

That suggests that reweighting the sample to reflect the 2000 exit poll distribution (39D/35R/26I) would give a different result. It does: the race then becomes dead-even, instead of an 8 point Bush lead. (Note: Steve Soto of The Left Coaster got Gallup to give him their party ID distributions for this poll and confirms a 5 point Republican party ID advantage in their RV sample.)

Well, bear in mind that he's analyzing registered voters now, not likely voters. So why would he advise weighting the sample so as to conform to the proportions who voted in 2000 (exit polling) rather than the proportions who are registered now? Am I missing something?

And Ruy's analysis of the independents seems odd to me as well. The overall distribution isn't necessarily affected very much by the independent vote, unless there are a lot of independents. I don't really know how many independents there are in the distribution of registered voters, but if the marginals show, say, 20% then a 7% lead for Kerry means he has (ignoring the Nader vote for the moment) 53.5% of 20% while Bush has 46.5%. That works out to a difference in the overall vote (7% of 20%) of less than 2% of the vote! Even if we assume 26% are Independents (the 2000 exit poll proportion) that's still less than 3% of the vote. That's basically the margin of error, so how do you use a proportion amounting to the margin of error to make a point about the method used? I'll go out on a limb here, and say you can't.

But again, the biggest problem I have with his analysis is the logic behind using the 2000 voting proportions in order to analyze a sample of registered voters in 2004. That just seems, well, peculiar... and methodologically invalid... especially in view of the enormous changes that have happened in the political landscape since 9/11.

Posted by Demosophist at September 18, 2004 06:48 PM | TrackBack