Are Republican pollsters “flooding the zone?”

And what would our forecast look like with only high-quality surveys?

The presidential race is really close. You know it, I know it, and pretty much everyone but the most motivated partisans know it. As of Tuesday, our forecast gives Kamala Harris a 50 percent chance of winning the Electoral College … and Donald Trump has a 50 percent chance too. That’s obviously about as close to a pure tossup as you can get.

If you squint at the polling averages, things also getting closer. Trump has gained 0.3 points in our national polling average since last week and the polls have also tightened in some of the swing states.¹ Harris leads by 0.6 points in Pennsylvania (down from 1.3 points last week), 1 point in Michigan (down from 1.9), and 0.8 points in Wisconsin (down from 1.9).

There are a few ways to explain Trump’s improvement in the polls. For example, we might be at the tail end of a post-debate bounce for Harris. Alternatively, this could be the start of a bigger surge for Trump as his — not particularly informative — internal polls suggest. Or the movement could just be random variation in the polls — if Harris really is ahead nationally by 3 points and in the Blue Wall states by about 1 point we’d expect her to have better and worse weeks that vary around that average. With three weeks left, there could also still be another momentum swing back toward Harris.

But there’s another explanation popular among certain segments of Democrats: Republican pollsters are “flooding the zone” by releasing a bunch of polls that are overly favorable to Trump in an effort to move polling averages rightward and make the race look closer than it actually is. Here’s Simon Rosenberg describing the theory:

“As I wrote in my last look at this rancid project, it is time for those who analyze polls to start acknowledging that there is now a third type of poll - the red wave, right-aligned narrative polling that only exist for a single purpose - to move the polling averages to the right. They are exploiting the “toss it in the averages and everything will work out philosophy” of these sites to once again launder these polls and game the averages - and thus our understanding of the election. Party leaders should expect them to keep these polls coming, and keep working the averages until it looks like Trump is winning in all polling averages.”

But is this a real problem? Is the supposed deluge of Republican-aligned polls meaningfully moving the Silver Bulletin polling averages and forecast?

Does “flooding the zone” really matter?

Republican-leaning firms have released a lot of polls this cycle. Rosenberg lists 27 Republican-aligned pollsters. Some are clearly pulling for Trump — Rasmussen Reports is credibly alleged to be giving the Trump campaign previews of its results, for example, something that no nonpartisan pollster should ever do. But other alleged zone-flooders, like Echelon Insights, are high-quality polling firms that get good marks in both the Silver Bulletin and 538 pollster ratings. 

Rather than speculating about the motivations of every firm, it’s more useful to ask whether a flooding of the proverbial zone could meaningfully move the polling averages. Yes, in theory it could if two things were true. First, polls from Republican-leaning firms are more favorable to Trump than nonpartisan polls. And second, the polling averages don’t adjust these polls based on pollster quality and house effects, allowing them to push the averages right. So the important question here is: are both of these things actually happening? 

First, are polls from Republican-aligned firms more favorable to Trump this cycle? Yes, but not by that much. Here’s a simple average of all national presidential polls conducted in October, and separate averages of polls from Republican-aligned and non-Republican-aligned polling firms according to Rosenberg’s list. 

Harris is ahead by 3.0 points nationally in this simple average. But when you look at only Republican-aligned firms, she’s up by only 2.0 points. Removing those polls from the average brings Harris up to a 3.4 point national lead. These aren’t huge differences, but they’re not nothing. Combined with a similar pattern in state-level averages, polls from Republican-leaning firms could push polling averages — and by extension forecasts — rightward. 

But that’s only one piece of the puzzle. These polls might be more favorable to Republicans than nonpartisan polls, but that isn’t a good way to tell whether they’re moving polling averages. For the flooding-the-zone theory to hold water (pun somewhat intended), polling averages and forecasts would have to just toss these polls in the average without any adjustment. But that isn’t happening. Here at Silver Bulletin, for example, we weight polls based on pollster quality and adjust them based on pollsters’ house effects. And every other high-quality polling average does something similar.

What’s the result? The polling averages say pretty much the same thing, regardless of which polls they choose to include or exclude. Nationally and in the battleground states, the biggest difference in Harris-Trump margin between the Silver Bulletin average and averages from 538, Split Ticket, The New York Times, and VoteHub is 0.5 points. In Pennsylvania — the likeliest tipping point state — our average is Harris +0.6. Split Ticket has the race as Harris +1, 538 has it as Harris +0.7, VoteHub has it as Harris +0.7, and The New York Times has Harris up by less than 1 point.

The important thing here is that these averages have somewhat different philosophies on which polls they use. For example, Split Ticket excludes Rasmussen Reports and Trafalgar; we include them, but automatically designate them as Republican partisan polls. 538 uses polls from Big Data Poll, Quantus and SoCal Data and ActiVote but we don’t.² And VoteHub only uses high-quality nonpartisan polls. But because we’re all weighting and adjusting the polls in reasonable ways, we all end up in about the same place.

So what happens if we go all the way and run our model with the same rules as VoteHub? Only polls they designate as high-quality, and no officially partisan polls?