Are Poll Sampling Complaints Legit?

By Sean Trende

Every election brings a raft of complaints about polls. I don’t expect this one to be any different — Republicans will bemoan the overabundance of Democrats in the samples, while Democrats will claim that the polls under-sample minorities.

What make these criticisms more salient this year, however, are signs that “working the refs” is having an effect on polling companies. In 2008, in response to complaints from many observers that minority and young voters were undercounted, Gallup actually produced two models: “traditional” and “expanded.”

The Battleground Poll likewise produced two different results, reflecting the differing assumptions of pollsters Celinda Lake and Lance Tarrance Jr. about the makeup of the electorate. In 2010, Gallup once again produced two separate samples after being continually harangued by Democratic observers.

The results of this “jawboning” of polling companies are mixed at best. In 2010, the criticism of Gallup was correct; in 2008, the traditional model had it right. So it was jarring to see the criticism taken to a higher level when Obama campaign strategist David Axelrod accused a recent Gallup poll of having “serious methodological problems.” He then directed anyone interested to this column from National Journal’s Ron Brownstein to explain why:

“The Gallup track, which is conducted among registered voters, has a sample that looks much more like the electorate in 2010 than the voting population that is likely to turn out in 2012: only 22 percent of the Gallup survey was non-white, according to figures the organization provided to Emory University political scientist Alan Abramowitz. That was close to the non-white share of the vote in 2010 (23 percent), but in 2008, minorities comprised 26 percent of all voters, according to exit polls; the Obama campaign, and other analysts, project the minority share of the vote will increase to 28 percent in 2012. In its survey, Pew, for instance, puts the non-white share at 25 percent.”

Obama advisers have likewise objected to Gallup’s “noisiness” — in other words, its tendency to show the president’s job approval shifting up and down without any intervening news story that would suggest that there should be movement (although this probably is simply a reflection of movement within the error margin from a pollster that does not weight its sample heavily).

Expect to hear a lot about this in the months ahead.

So, what to make of these criticisms from the left and from the right? I don’t make too much of the latter’s complaints regarding partisan weighting. Every election since 2004 — the first cycle I wrote about online — has featured debates among the pundits about whether pollsters should re-weight their samples by partisan identification. The problem with doing so — and I think the outcome from 2004 shows this — is that partisan ID is fluid; the partisan composition of the electorate really can change substantially in a relatively short period of time, as it did from 2000 to 2004, and again from 2004 to 2008.

More importantly, what we’re seeing right now is probably a function of the fact that pollsters are largely employing registered-voter screens, or no screens at all. Polls of adults and registered voters have tended to be more heavily Democratic than the actual electorate (Nate Silver estimated that in 2010 the difference was around six points), although this gap can disappear in a year like 2008, when enthusiasm is high on the Democratic side and low on the Republican side. (This year is currently looking a lot more like 2004 than 2008 in that regard.) I suspect these oversized Democratic samples will shrink once we get to the fall and likely-voter screens become ubiquitous.

The objections regarding the minority-vote share, and whether pollsters should aim for a sample that is somewhere over 25 percent minority, require a more in-depth discussion. There are two salient points involved:

1) The minority share of the electorate will probably stay flat, or even decrease, in 2012.

The reason for anticipating a 28 percent non-white electorate is pretty simple: That share of the electorate has been rising steadily since 1992. Since it grew from (using the final exit-poll data) 22.2 percent in 2004 to 25 percent in 2008, we might expect a similar rise this year, to 28 percent. In particular, given the well-documented, rapid growth of the Latino population in the United States, the non-white share of the vote might be expected to increase even further (you can see Abramowitz’s argumentation here).

But as I explain in my book, “The Lost Majority,” predictions about the ethnic composition of the electorate have been incorrect in the past, and are likely to be incorrect in the future. In late 2009, Abramowitz himself predicted that a GOP takeover of the House was highly unlikely because “[b]ased on the average rate of change in the racial composition of the electorate over the past two decades, by 2010 we can predict that no more than 76 percent of voters will be white while at least 11 percent will be African-American and at least 13 percent will be either Hispanic or members of other racial minority groups.” Due to this, Republicans would need to win around 58 percent of the white vote, something Abramowitz judged to be “exceedingly difficult.

Of course, Republicans won 60 percent of the white vote in 2010, probably the highest share of that vote for Congress won by either party since 1822 (assuming an almost entirely white electorate pre-1952). More importantly for our purposes, whites made up 78 percent of the electorate in that year, in excess of Abramowitz’s 76 percent ceiling.

Let’s analyze this a little closer. Why did the white share of the electorate decline from 2004 to 2008, and why might we not expect the feat to repeat in 2012? To answer this better, let’s break down the electorate from 1996 through 2010 by racial/ethnic group. Please note that I don’t have the final raw data for the 1996 or 2010 exit polls; without precise numbers, there may be rounding errors here:

Of the 2.8 percent decline in the white vote from 2004 to 2008, 1.2 percent is attributable to the growth of the African-American vote, about 0.5 percent to the growth of the Asian vote, and about one point to the growth of the “Other” vote. Note that the Latino vote increased by only 0.2 percent.

So the question is, where will the growth in the non-white electorate come from in 2008 through 2012? From the African-American population? This is possible, but not particularly likely. From 1996 through 2006, that population averaged a fairly consistent 10 percent of the vote, with a bit of a spike in 2004.

To get to 12.2 percent in 2008 required a herculean get-out-the-vote effort as well as the tremendous excitement surrounding the possible election of the first black president. In fact, the latter probably had quite a bit more of an impact than the former, as the decrease in the share of the white vote from 2004 to 2008 in uncontested red states and uncontested blue states was larger than in the contested purple states.

I can accept an argument that re-electing Obama will generate as much enthusiasm in the African-American community as electing him did, but I have a hard time seeing that turnout being appreciably better in 2012 than it was four years earlier. And there is, in fact, some evidence to suggest that African-American turnout will slip this year: Enthusiasm among non-white voters is down from 74 percent at this point in 2008 (vs. 58 percent for whites) to 48 percent today (the same goes for whites). And, indeed, in 2010, African-American turnout reverted to the mean. If this occurs in 2012, Democrats will need a massive surge in the minority population elsewhere to make up for this regression.

The most likely place for this to occur is within the Latino community. That population grew smartly over the 2000s. But — much less remarked upon — the Latino electorate did not. Indeed, since 2004, it has been almost perfectly flat, and it contributed only marginally to the decline of the white vote from 2004 to 2008. This has somewhat flummoxed analysts, who might expect it to follow suit with Latino population growth.

One possible explanation is that a large degree of Hispanic growth is among non-citizens, which prevents the population increase from translating to electorate increase. Another possibility is something of the reverse: Because a growing portion of the Latino population is second- and third-generation Latino-American, and those generations are much more likely to self-identify simply as “white,” exit pollsters get a different response than you might expect from the census (which asks a broader question about “origins”).

Regardless, while 2012 may be a year where we see an explosion in the Hispanic electorate, we should acknowledge that predicting this is to predict a deviation from the trend, rather than a continuation of the trend.

That leaves us with “Asian” and “Other” voters. Much like the Latino population, the growth here has been slight over the past decade, and uneven as well. Whether the sharp drop-off in 2010 is a reversion to the mean or an outlier remains to be seen. But these groups alone are unlikely to contribute more than a point or so to the decline of the white electorate.

Taken together, it is hard to see how the white vote will be depressed another three points from 2008. African-Americans are unlikely to be a substantially larger share of the electorate than they were in 2008, and may even shrink somewhat. Latinos show potential for growth, but it is a potential that has been completely unrealized over the past decade. Asians and “Others” are too small to contribute significantly to a reduction in the share of the white vote.

So it’s true that an electorate that looks like 2010 (78 percent white) isn’t particularly likely, absent a perfect demographic storm for Republicans in which African-Americans revert to mean turnout, Latino turnout continues to flat-line, and other groups record only marginal growth. At the same time, an electorate that becomes appreciably less white than the 2008 electorate seems similarly unlikely. Jawboning pollsters into trying to create such a sample is misguided.

2) Even if Gallup is undercounting minorities, its numbers might not be incorrect.

It’s also important not to overstate the impact of the difference between a 78 percent white electorate and a 72 percent white electorate. It isn’t enough to calculate the “non-white” vote and extrapolate from there, because the “non-white” vote is very uneven. If Gallup is undercounting African-Americans, for example, then their top lines are wildly off, as almost every vote undercounted is a vote for Obama that is lost. For example, if African-Americans are being undercounted by five percentage points, then Obama’s vote share should be about three points higher (you have to remember that Obama gets about 40 percent of white votes, so adjusting their share of the electorate downward hurts his vote percentage as well).

But Latinos have tended to give Republicans a third of their vote, and the Asian vote has typically been about 40 percent Republican. Similarly, with the exception of 2008, the “Other” category has been about 40 percent Republican.

If these are the categories that Gallup is undercounting, then the difference for Obama is only about one point. Of course even then there are variations between the Latino, Asian, and other communities and their voting habits to consider. But the bottom line is that this undercount, if it is occurring, probably isn’t resulting in a radical shift in Obama’s vote share.

At the end of the day, no one poll is the “correct” poll. The Gallup sample is certainly not unreasonable, and may even prove to be correct.

This article appeared on RCP on Friday 4/20/2012

follow us on facebook and twitter

Be the first to comment

Leave a Reply

Your email address will not be published.