If there's an anti-Harris bias resulting from the herding, I really hope we don't get an ugly November 6th, with Trump supporters screaming that there must be fraud because Harris won Michigan by 8 points or something.
I'm not worried about Rudy Giuliani and Trump's diehards. I'm worried about Joe Saltofearth, who doesn't closely follow politics but has "common sense". The kind of guy that thinks Trump's usually full of hot air, but *8* points in Michigan? That's clearly fake and he needs to do something about it.
There aren’t enough of those people to matter. If they do something dumb and get arrested then it ends up in the pile with the J6/2020 crew. If Trump loses then it’s not like there’s anyone who can save people who riot in his defense.
It’s not a big enough group to actually cause more than a days worth of disturbance. Even J6 only slowed things down by half a day.
Maga media is bigger than msm media. Do not underestimate what maga media can do. They will convince even die hard kamala voters that the election was rigged. This is bush v gore all over again.
Although, to this point my feeling has been Harris had a really uphill climb, female being her biggest issue, I'm hoping common sense will prevail and this country will elect someone who is decent and sound of mind. And like you, I've been very concerned about physical violence should Trump lose - which for the sake of our country I hope he does.
It won't be the Democrats storming the bastions if Harris loses, but it will certainly be January 6th all over again, and much worse, if Trump loses.
Huh. Over at the rightwing Townhall complex of forums, everyone thinks the exact opposite. That if Trump wins, the left will riot all over Washington and maybe other big cities.
This is bound to happen. And this time they'll do much worse than screaming. They are now organized to block county-level certifications. Expect a messy election this year.
It doesn't matter what happens. Trump will claim it was stolen. And his die hards will be with him. But the normie Republicans won't go for this twice.
Let them bitch. If Harris takes Michigan by eight points, she'll have won decisively. Also, while I deplore violence, if Trump's supporters go crazy it's only going to hurt Republicans. I'll go to my grave believing Trump *got very lucky* things didn't spiral out of control more spectacularly on 6 January 2021. Imagine if hundreds had died (including some members of Congress). Impeachment might well have gotten a conviction super majority.
The less accurate pollsters, less reliable pollsters started pulling in from the Harris +5 and +6 numbers they've been publishing for two months, but to be frank, it's not because those were their real results and now they want to publishing something fake for Trump's sake. They just don't want their final numbers to be recorded as 5 or 6 points off the mark.
Harris has a decent shot at winning, but if you're expecting her to greatly exceed Biden's winning percentages, let alone eclipse them with an 8 point win in a Rust Belt state, well there's about a 1 in 9.5 trillion chance of that happening, so I wouldn't get worried about that.
Polling firms remind me of financial analysts in that there’s never any incentive to go against (what is typically a bullish) consensus. When the inevitable bear market hits, there’s a lot of security in being able to say, “Welp, everyone else was bullish too!” Similarly, pollsters are finding emotional safety by glancing at one another’s test answers and adjusting accordingly—in this case, to repeated, statistically improbable ties.
The pollsters, many of them, are lying like carpets: you don't see this as a problem? I do. GIGO --- what's the point in averaging polls with a fancy statistical model if the data is rotten? I can see Nate would be annoyed.
This post is precisely why I subscribe to this newsletter. What’s interesting is that Nate himself may have helped cause the phenomenon by making poll aggregation such a focal point.
Honestly, Nate put a level of polish that was needed on something existing.
I can't fault him for doing so, and having it in public means that there's someone to call them out in public. If anything, he's adding a lens of discussion about it that need to be.
This was always going to become an issue,
as the pollsters started to be directly compared to each other once every 4 years like the world series of poker, winner gets the big fat bonus contracts and name recognition for national polling and research contracts.
But if you can't be confident of being first, or have a fall back in case you do badly, the polls have a fiscal responsibility to not be the least accurate, dooming them to 4 years of bad business.
Which, ultimately, has led us here.
I'm going to find the post season for polls very interesting, and given what I'm seeing elsewhere I think I know what's going to come out of the wood work, but I'd rather not spend the next 5 days arguing about my analysis with the political faithful online, and will instead share in the aftermath for those who are actually interested.
I REALLY wonder if this race ends up being far less of a "close race" than expected. It wouldn't shock me if pollsters more and more tweak their numbers to always show it as a close race, because there is just too much downside risk to them getting it "wrong."
If so, this could be a very different systemic bias in the polls than in previous cycles, where instead of tilting the poll to one party or the other they are simply tilting the poll to show something close to a 50/50 split.
I think it's possible Harris will win handily in the Blue Wall states and lose handily in the other swing states. That's 270-268, a closer PV, but a smaller margin in the tipping point state.
I didn’t say that’s what the model said, I said that I’m betting on the scenario that the New York Times (one of the few to not weight by recalled vote and not be herding now) is indicating - Harris does better in the rust belt (and NC, maybe) than the sun belt.
Yep, that's where it's likely to end up. Harris with 270. Nevada, Arizona, Georgia, and North Carolina are unlikely.
I have hope for her in the rust belt, primarily because of
Anne Selzer's comments earlier that Iowa took an unexpected shift to the left when she showed Harris behind only four points.
Another positive sign for Harris is the female turnout, which should be voting predominantly Harris except in some of these Southern States. And the disaffected Republicans, we have quite a few of those here in Wisconsin.
The combination of the women, the young people and the disaffected Republicans should do it.
I think the young males will gravitate to Trump. The Macho thing. Tough guy and all that crap. They're not looking at his associations with Victor Orban and his complete dismantling of democracy in Hungary, which, of course, is what Trump would just love to do here. Doubt they have a clue on what has happened in that country.
With such a narrow EV, of course, there will be huge battles, all the way up to the Supreme court.
Don't read too much into the Selzer +4. While Anne is an honest pollster, and a good one, the inherent limitations of polling can mean she isn't going to hit the nail on the head every time. Her September poll in 2020 showed a tied race, where the actual result was +8.
This worries me. Makes me sad we didn't fix the Blue Dot in Nebraska... Imagine if Trump loses because one state representative didn't vote to get rid of it!
You've said this before and I like the idea. I used to say that when people say what they'll DO, believe them. (People never do; they say, Oh, he didn't really mean that ----) But your slant on it is good, too, IMO.
Although it seems to me important to make sure it's real: who people are and what they want to do are often enemy propaganda and not true at all. I think that's a good principle, too, that before we act or come to a conclusion about something, first ask, Is it real?
I think anyone would be against an isolated increase in democratic representation that caused their preferred candidate to lose. For instance, I'd imagine anyone who is for Kamala would be against more democratic representation in the way California allocates its electoral votes.
You make a fair point. But you are not talking about an isolated increase. You are talking about an isolated decrease, i.e., taking away the voting power that people currently have.
Pollster degrees of freedom sound like a thing. Maybe polling models should be preregistered before the election cycle? Including decision trees for weird circumstances.
Academia at least gives people incentives to buck the trends. People who go against the conventional wisdom get attention, and in academia attention is good. People count your citation numbers, not whether those citations agree with you or disagree with you.
One of the pieces of evidence I find personally convincing for "why is climate change probably not a hoax" given that I don't have the science background to personally muck around in too fancy of climate models is "Mannnn, anything suggesting that climate change wasn't happening or wasn't manmade which could hold any sort of water in peer review would get any fresh young academic tenure SO fast. All the citations. And somehow those papers have not come out. Hmm, I wonder why."
Academia gives people incentives to buck the trends? Try writing a thesis critical of climate change, gender dysphoria or any other controversial popular topic and watch your academic career get destroyed.
If you make a point that is at all controversial, you’ll have a million people citing your work to say why it’s wrong. Sure you’ll have a mob after you, but your name will instantly be big, and some place will hire you (though maybe not in the field you thought you were working in). A department of climate science might not hire you, but a department of geology or petroleum engineering or earth science will, especially at a second tier university that wants to get attention. And having a guaranteed job at a second tier university is great, given the risks and uncertainties of academia generally (unless you’re already at a top tier place with security).
She seems like someone who had a full career, became both chair of department, and was given the title emerita on retirement. Sounds like she did just fine in academia.
But then she bucked the trend and was forced out. So you think that shows she had academic freedom? And that it's ok to fire her if she only had a few more years to work anyway? Your point was academia gives people incentive to buck the trend and I think it's one of the worst for forcing conformity.
She seems to have retired to concentrate more on commercial interests. These include being part of a company to better predict local weather for commerce. This includes improving the predictions for, and maximising the income of, wind farms.
More akin to a cook repeatedly overcooking a steak to avoid 'food poisoning',
And therefore committing atrocities to good food in the name of 'food safety'.
They know some of the polls are bad, and if they are really off from the right number, they could end up accidentally ruining the companies reputation and get all their friends fired as no *corporate client* wants to work with them.
I've seen and tracked the same instances in every workplace on the planet,
everyone who makes a stern declaration and weak data because 'a decision has to be made', and then wonders why they didn't get the results projected because reality doesn't care about your massaged numbers.
And that *corporate client* angle is something everyone should consider about these polls. These polls do not earn the real dollars for the company. Breaking even on them is considered good in the market. It's the connections and the advertising and the follow up contracts that matter. Being 'close enough' in 'the right regions' are what matter.
But if your focus isn't Pennsylvania, you are not going to spin up a division to focus on Pennsylvania for the election. You'll take the draw nationally and point to your actual good market and say 'See how good we were where it matters *for you?* And we weren't that far off nationally...'
Do any political polling groups pre-register their study designs, covariate decisions, and results publications regardless of their findings? (Apologies if this is an uninformed or naive question.) It seems like political polling would especially benefit from pre-registration, because they don't even have a favored hypothesis to defend. (Pre-reg has become standard and expected practice in behavioral science, where I work.)
For instance, Pollster Z says "In October, we will poll 10 samples in PA, aiming for at least 600 LVs, using these and only these controls for education, prior voting,... (etc etc), and publish the results regardless of what these findings show."
If only a few % of the pollsters are actually polling, and aggregators and modelers are adjusting and correcting as well, we might as well be using a Ouija board.
Nate, you're going to need to expose and exclude these not-pollsters if you want us to keep caring about these simulations.
I know, I'm advocating for excluding them entirely and making a show of it. Because if 95% of pollsters are herding, but are still weighted even 10%, the models are useless.
Intereting, pollster herding, reminds me of so called “active” managers hugging the benchmark in the asset management world. They charge much higher fees than passive ETFs, but their portfolios make only the tiniest deviation from the benchmark (Say S&P500), to keep their pathetic little comfy jobs. Their after-fee performance is usually worse than the benchmark.
hes actually always right in this scenario because forecasters' description of the odds is somewhat counterintuitive to our understanding of odds. a 50% chance of rain in the area does not mean you flip a coin and see if it rains. theres a 100% chance of rain, but that area only covers 50% of the area in question.
Your voting for Trump in Florida today doesn't change the forecast.
In different states, different sets of supporters, each set with similar characteristics (age, race, registered party (or not), educational qualifications) participate in early voting in different proportions, which also vary from one election to another. So it is impossible to make any predictions based on early voting figures. Nate explained that in a post here.
Not that everyone understands that, so some analysts might try to draw conclusions from any early voting data which becomes available. But that is very dangerous as early voting data has often shown very different outcomes from the eventual total vote outcome.
Even if it did make a difference, it's one vote out of 11,000,000 (the number of total votes in Florida in 2020) in a state that has only a 2% chance of being the tipping point.
With this analysis + the Nevada early voting data from Jon Ralston, I can't help but fear this likely points to Trump's numbers being meaningfully underbaked in polling. It seems logical that the meaningful shift to Trump nationally is also happening at the state level (albeit less consistently), and that we're in for a rough ride next Tuesday.
Instinctually gravitating to the following outcome: Georgia, Nevada, Arizona are all going Trump with margin to spare, with the big question remaining if Pennsylvania has enough buffer for Harris to pull it out, with the added uncertainty of North Carolina given all the hurricane mess there.
It means the polling is completely meaningless, and we have no idea where the election is going to land. Remember yesterday's article - Nevada isn't really correlated with the rest of the swing states.
In theory, if there is less herding nationally than the state side, wouldn't it indicate that the national polling is more reliable? Therefore it seems reasonable that the state polls are underbaked on Trump, no? (just WHERE its underbaked is hard to predict)
This is addressed in the article. If you look at the high quality pollsters who aren't herding, you see Harris doing well in the rust belt and Trump doing well in the sun belt. So one strong possibility is that Trump wins NV, GA, AZ, NC but Harris wins MI, PA, and WI - which would give Harris the win (albeit barely).
Right, and I've seen that map on various Internet sites today. Yow, that would be a very close election, if it really divides that way. Not good for the rioting in the streets issue.
I could be reading it wrong but I thought Nate was saying there IS herding going on at the national level which is the reason for the tightening numbers. The pollsters are herding the national numbers into alignment with the swing state numbers.
>The pollsters are herding the national numbers into alignment with the swing state numbers.<
I hadn't thought of that, but it has the air of plausibility. Swing states wouldn't be swing states absent a tendency to be close, after all. So, yeah, maybe one or the other candidates in reality is doing better overall than the national polls are showing, because they're being massaged to fit the tight nature of the state level polls.
Whatever the national numbers really will be, Harris needs to get more than 4.5% to win comfortably, because of the Dem vote wastage. If there still is so much wastage: Trump may have broken into some of that with his appearences in lost states. But right now, if we believe the unbelievable national polls, Harris only has -0.3%, which won't do the job. I think this issue is fascinating, how much extra national popular vote Dems need to overcome the wastage. Trump, of course, doesn't need any, except that it would be much better for peace in the streets if he won the national vote comfortably.
Plausible theory, but the NYTimes polls also point to a demographic realignment that could explain the tightening national numbers of he’s running up the score in places like Florida Texas etc
Alternatively, it's possible that herding is what led to the national polls moving more for Trump than the states did. The states were already close, so herding around a tie wouldn't move them a ton. The national polls were further apart, so hearing around a tie would create more movement.
I never really saw an argument for why the polls started moving towards Trump in October. Nothing of note happened in the race. Before that, the only real movement we saw was after bad debate performances or when Harris replaced Biden.
Per my count, the polls have only swung toward Trump by about two points since the beginning of October (from Harris+3 to Harris+1). That seems like fairly typical tightening of the kind that...might not require a big event? I think some of the tightening may reflect simple "waning excitement" as the burst of enthusiasm centered on the Brand New Candidacy faded. Trump possibly got a tiny boost from the VP debate, too.
All that said, I do think it's possible that herding may be having the effect you state (puffing up Trump's numbers). I wouldn't be even mildly surprised if Harris wins, and more handily than most deem likely. I also wouldn't be surprised if Trump wins pretty easily.
This election is by far the most "unknowable" since 2000 as far as I'm concerned. (I include 2016 in that: we all "knew" Clinton was going to prevail; we were just wrong, Nate excepted of course).
fwiw, NYT is the poll that had Nevada/Arizona going the opposite direction of PA/MI/WI. So, if Ralston is right and Nevada goes, say, Trump +3, that may just be more evidence of further realignment and a likely Harris win.
imo (not worth much), that's what will happen. Everywhere where pollsters feel free to publish outlier results, we're seeing movement toward Harris among white voters and movement away among POC, and that's the result bleeding out from all the noise and herding in aggregate. PA/MI/WI are much whiter than GA/AZ/NC/NV and don't rely on running up huge margins with non-white voters nearly as much.
Though not my preferred outcome, I have a strong feeling we will see Harris hold the blue wall and Trump win the sunbelt, resulting in a Harris victory with the point from Nebraska. I do feel like pollsters are herding to suppress Harris’s numbers because they don’t want to be embarrassed by another Trump over performance.
Yep. What are the chances of a three- peat? Trump overperforming. It's more likely pollsters are holding down Harris's numbers. Which would be my preferred outcome. LOL
I'm not saying we have Nevada in the bag, but Ralston always does this. Wait until the final Clark tally before freaking out. 25 percent of the electorate are not registered with either party.
As someone who's taught statistics courses in the past: thanks to Nate for this post. It's a very nice meta-analysis of what polling firms do with the statistics they produce, and almost something I would like in some way to mention to students if I teach statistics again.
> This uses a binomial distribution, which assumes that all polls are independent of one another, which theoretically they should be
This seems obviously wrong. A simple binomial of sample size alone assumes polsters take random samples and report raw percentage. In reality, they *weight* by various demographic info (including, sometimes, who they voted for last election). This dramatically reduces variance!
In a simplified example, if there were 3 demographics truly voting 0%, 50%, and 100% for Harris, we would expect random unweighted polls of 900 people to vary with a standard deviation of 1.67% (sqrt(1/(4*900))). We would expect random demographic-weighted polls of 900 people to vary with a standard deviation of 0.96% (1/3*sqrt(1/(4*300))), much less.
I'm other words, demographic weighting looks like herding.
It would be an enormous task to try to find or model each polsters demographics and the variance you should expect given that weighting. I can see why Nate didn't attempt that... But as it stands, the extremely small probabilities in that chart are borderline misinformation. Unless I'm completely misunderstanding, it shows the odds that the poll numbers we see could have been unweighted polls. We already know they're not unweighted polls!
Dividing the *odds* by 10 (or any constant) isn't charitable, since they scale exponentially with the number of polls. With enough data, any bad theory will be disproven with astronomical odds. The bad theory of unweighted AND unbiased pollsters is here disproven... But since we already know they're not unweighted, the odds don't really say anything.
Your reaction is frankly the reason I said the table is misinformation. Presenting those odds uncritically is bound to be misinterpreted.
If we were to divide anything by a charitable factor, it would be the variance. Ballparking again: If we assume all 249 of his polls were the same size, expecting a 55% to lie within 2.5% has a s.d. of 3.22%. The actual 78% that lie within 2.5% would happen with a s.d. of 2.05%. That's a big difference, but not so huge a difference that it's anywhere close to obvious that demographic weighting alone couldn't explain it.
It's pretty surprising there's zero discussion of this from Nate. Figuring out what distribution to expect from pollsters is exactly the kind of analysis I'd expect to see from an election modeler.
Great work Nate. This is premium analysis. I have to wonder if polling aggregation is dead. You've got your work cut out for you making sense of this mess.
Pro-herd, anti-herding.
If there's an anti-Harris bias resulting from the herding, I really hope we don't get an ugly November 6th, with Trump supporters screaming that there must be fraud because Harris won Michigan by 8 points or something.
They claimed fraud in 2020 when Biden was projected to romp. They really don’t care about reality, they either win or they were cheated
I'm not worried about Rudy Giuliani and Trump's diehards. I'm worried about Joe Saltofearth, who doesn't closely follow politics but has "common sense". The kind of guy that thinks Trump's usually full of hot air, but *8* points in Michigan? That's clearly fake and he needs to do something about it.
There aren’t enough of those people to matter. If they do something dumb and get arrested then it ends up in the pile with the J6/2020 crew. If Trump loses then it’s not like there’s anyone who can save people who riot in his defense.
It’s not a big enough group to actually cause more than a days worth of disturbance. Even J6 only slowed things down by half a day.
They won’t wait for Jan 6 this time, it'll get “wild” sooner at state level.
Maga media is bigger than msm media. Do not underestimate what maga media can do. They will convince even die hard kamala voters that the election was rigged. This is bush v gore all over again.
Trump claimed fraud when he won. Remember? He claimed he’d have won the popular vote if not for the millions of illegal votes.
That's kinda the entire point of these R-aligned polls flooding the market
They'd just point to the D leaning pollsters that are also herding to the closer race. "Even the libs didn't think you would cheat this bad" etc
Although, to this point my feeling has been Harris had a really uphill climb, female being her biggest issue, I'm hoping common sense will prevail and this country will elect someone who is decent and sound of mind. And like you, I've been very concerned about physical violence should Trump lose - which for the sake of our country I hope he does.
It won't be the Democrats storming the bastions if Harris loses, but it will certainly be January 6th all over again, and much worse, if Trump loses.
Huh. Over at the rightwing Townhall complex of forums, everyone thinks the exact opposite. That if Trump wins, the left will riot all over Washington and maybe other big cities.
Well, I hope nobody riots.
I also hope nobody riots and will leave it plainly at that.
This is bound to happen. And this time they'll do much worse than screaming. They are now organized to block county-level certifications. Expect a messy election this year.
It doesn't matter what happens. Trump will claim it was stolen. And his die hards will be with him. But the normie Republicans won't go for this twice.
Let them bitch. If Harris takes Michigan by eight points, she'll have won decisively. Also, while I deplore violence, if Trump's supporters go crazy it's only going to hurt Republicans. I'll go to my grave believing Trump *got very lucky* things didn't spiral out of control more spectacularly on 6 January 2021. Imagine if hundreds had died (including some members of Congress). Impeachment might well have gotten a conviction super majority.
The less accurate pollsters, less reliable pollsters started pulling in from the Harris +5 and +6 numbers they've been publishing for two months, but to be frank, it's not because those were their real results and now they want to publishing something fake for Trump's sake. They just don't want their final numbers to be recorded as 5 or 6 points off the mark.
Harris has a decent shot at winning, but if you're expecting her to greatly exceed Biden's winning percentages, let alone eclipse them with an 8 point win in a Rust Belt state, well there's about a 1 in 9.5 trillion chance of that happening, so I wouldn't get worried about that.
Polling firms remind me of financial analysts in that there’s never any incentive to go against (what is typically a bullish) consensus. When the inevitable bear market hits, there’s a lot of security in being able to say, “Welp, everyone else was bullish too!” Similarly, pollsters are finding emotional safety by glancing at one another’s test answers and adjusting accordingly—in this case, to repeated, statistically improbable ties.
The pollsters, many of them, are lying like carpets: you don't see this as a problem? I do. GIGO --- what's the point in averaging polls with a fancy statistical model if the data is rotten? I can see Nate would be annoyed.
Not a problem for the pollsters, seems like they’ve solved the problem of being scandalously wrong.
I guess you are right, but color me shocked!
This post is precisely why I subscribe to this newsletter. What’s interesting is that Nate himself may have helped cause the phenomenon by making poll aggregation such a focal point.
Honestly, Nate put a level of polish that was needed on something existing.
I can't fault him for doing so, and having it in public means that there's someone to call them out in public. If anything, he's adding a lens of discussion about it that need to be.
This was always going to become an issue,
as the pollsters started to be directly compared to each other once every 4 years like the world series of poker, winner gets the big fat bonus contracts and name recognition for national polling and research contracts.
But if you can't be confident of being first, or have a fall back in case you do badly, the polls have a fiscal responsibility to not be the least accurate, dooming them to 4 years of bad business.
Which, ultimately, has led us here.
I'm going to find the post season for polls very interesting, and given what I'm seeing elsewhere I think I know what's going to come out of the wood work, but I'd rather not spend the next 5 days arguing about my analysis with the political faithful online, and will instead share in the aftermath for those who are actually interested.
I REALLY wonder if this race ends up being far less of a "close race" than expected. It wouldn't shock me if pollsters more and more tweak their numbers to always show it as a close race, because there is just too much downside risk to them getting it "wrong."
If so, this could be a very different systemic bias in the polls than in previous cycles, where instead of tilting the poll to one party or the other they are simply tilting the poll to show something close to a 50/50 split.
I think it's possible Harris will win handily in the Blue Wall states and lose handily in the other swing states. That's 270-268, a closer PV, but a smaller margin in the tipping point state.
This seems like a really likely scenario too
Not according to Nate. He says the most likely two scenarios are that either Trump or Harris sweeps the seven swing states.
The Blue Wall to 270 EV is the #4 most likely scenario right now.
3.5% in the Monte Carlo.
But note that the combined probability of either of those outcomes was still less than 50%.
I didn’t say that’s what the model said, I said that I’m betting on the scenario that the New York Times (one of the few to not weight by recalled vote and not be herding now) is indicating - Harris does better in the rust belt (and NC, maybe) than the sun belt.
All the polling averages are showing her better in the rust belt too though. But “better” is still just even with Trump.
By a hair, I think the gap will be 2-4 points, maybe 5
That article is more than a week or two old now, maybe its now longer accurate?
Yep, that's where it's likely to end up. Harris with 270. Nevada, Arizona, Georgia, and North Carolina are unlikely.
I have hope for her in the rust belt, primarily because of
Anne Selzer's comments earlier that Iowa took an unexpected shift to the left when she showed Harris behind only four points.
Another positive sign for Harris is the female turnout, which should be voting predominantly Harris except in some of these Southern States. And the disaffected Republicans, we have quite a few of those here in Wisconsin.
The combination of the women, the young people and the disaffected Republicans should do it.
I think the young males will gravitate to Trump. The Macho thing. Tough guy and all that crap. They're not looking at his associations with Victor Orban and his complete dismantling of democracy in Hungary, which, of course, is what Trump would just love to do here. Doubt they have a clue on what has happened in that country.
With such a narrow EV, of course, there will be huge battles, all the way up to the Supreme court.
Don't read too much into the Selzer +4. While Anne is an honest pollster, and a good one, the inherent limitations of polling can mean she isn't going to hit the nail on the head every time. Her September poll in 2020 showed a tied race, where the actual result was +8.
I think Harris has a chance in 1 or 2 of AZ NV NC GA due to abortion bans and abortion ballot measures.
This worries me. Makes me sad we didn't fix the Blue Dot in Nebraska... Imagine if Trump loses because one state representative didn't vote to get rid of it!
Praise the Lord and pass the champagne. God is still looking out for us.
You want to go to a Victor Orban Hungary, have at it. Read and weep:
https://www.google.com/search?q=call+Victor+Orban+dismantled+democracy+in+Hungary&oq=call+Victor+Orban+dismantled+democracy+in+Hungary&gs_lcrp=EgZjaHJvbWUyBggAEEUYOTIJCAEQIRgKGKABMgkIAhAhGAoYoAEyCggDEAAYgAQYogQyCggEEAAYgAQYogTSAQg1Mjk4ajFqN6gCFLACAQ&client=ms-android-americamovil-us-revc&sourceid=chrome-mobile&ie=UTF-8
Thoroughly disgusting. And this is Trump's BFF. Does no one pay attention?
When people tell you who they are - believe them
You've said this before and I like the idea. I used to say that when people say what they'll DO, believe them. (People never do; they say, Oh, he didn't really mean that ----) But your slant on it is good, too, IMO.
Although it seems to me important to make sure it's real: who people are and what they want to do are often enemy propaganda and not true at all. I think that's a good principle, too, that before we act or come to a conclusion about something, first ask, Is it real?
Why would you be against more democratic representation?
I think anyone would be against an isolated increase in democratic representation that caused their preferred candidate to lose. For instance, I'd imagine anyone who is for Kamala would be against more democratic representation in the way California allocates its electoral votes.
If they want to do it, go ahead -- just not weeks before the election when they know Maine can't do the same thing.
sadly, they don't. I wish they would now.
Why does it need to be tied to Maine? States can allocate however they want.
You make a fair point. But you are not talking about an isolated increase. You are talking about an isolated decrease, i.e., taking away the voting power that people currently have.
it's not taken away. They could still vote in NE. It's just that the electoral votes would be determined by state rather than by district.
I wonder if that would be so bad.
It would definitely be good. Regardless of which candidate wins, a definitive victory is MUCH better for the country than a nailbiter
No, I mean if polls sapped their credibility by hewing to "close" regardless of the truth. We could stop obsessing about them and just vote.
Gotcha, I misunderstood.
They had Biden by +8 in 2020, trust me, they're not that worried about it.
Is this substantially different from fraud?
It seems worse than p hacking in academia.
Pollster degrees of freedom sound like a thing. Maybe polling models should be preregistered before the election cycle? Including decision trees for weird circumstances.
That would be good. An form of honesty and accountability would be good.
Academia at least gives people incentives to buck the trends. People who go against the conventional wisdom get attention, and in academia attention is good. People count your citation numbers, not whether those citations agree with you or disagree with you.
One of the pieces of evidence I find personally convincing for "why is climate change probably not a hoax" given that I don't have the science background to personally muck around in too fancy of climate models is "Mannnn, anything suggesting that climate change wasn't happening or wasn't manmade which could hold any sort of water in peer review would get any fresh young academic tenure SO fast. All the citations. And somehow those papers have not come out. Hmm, I wonder why."
Academia gives people incentives to buck the trends? Try writing a thesis critical of climate change, gender dysphoria or any other controversial popular topic and watch your academic career get destroyed.
If you make a point that is at all controversial, you’ll have a million people citing your work to say why it’s wrong. Sure you’ll have a mob after you, but your name will instantly be big, and some place will hire you (though maybe not in the field you thought you were working in). A department of climate science might not hire you, but a department of geology or petroleum engineering or earth science will, especially at a second tier university that wants to get attention. And having a guaranteed job at a second tier university is great, given the risks and uncertainties of academia generally (unless you’re already at a top tier place with security).
Google Judith Curry
She seems like someone who had a full career, became both chair of department, and was given the title emerita on retirement. Sounds like she did just fine in academia.
But then she bucked the trend and was forced out. So you think that shows she had academic freedom? And that it's ok to fire her if she only had a few more years to work anyway? Your point was academia gives people incentive to buck the trend and I think it's one of the worst for forcing conformity.
She seems to have retired to concentrate more on commercial interests. These include being part of a company to better predict local weather for commerce. This includes improving the predictions for, and maximising the income of, wind farms.
Of course it's fraud!
It's not fraud,
That would require malice, and the like.
It's human nature,
More akin to a cook repeatedly overcooking a steak to avoid 'food poisoning',
And therefore committing atrocities to good food in the name of 'food safety'.
They know some of the polls are bad, and if they are really off from the right number, they could end up accidentally ruining the companies reputation and get all their friends fired as no *corporate client* wants to work with them.
I've seen and tracked the same instances in every workplace on the planet,
everyone who makes a stern declaration and weak data because 'a decision has to be made', and then wonders why they didn't get the results projected because reality doesn't care about your massaged numbers.
And that *corporate client* angle is something everyone should consider about these polls. These polls do not earn the real dollars for the company. Breaking even on them is considered good in the market. It's the connections and the advertising and the follow up contracts that matter. Being 'close enough' in 'the right regions' are what matter.
But if your focus isn't Pennsylvania, you are not going to spin up a division to focus on Pennsylvania for the election. You'll take the draw nationally and point to your actual good market and say 'See how good we were where it matters *for you?* And we weren't that far off nationally...'
and then it goes from there.
I'm like Eloise:
I say it's spinach, and I say the hell with it.
Do any political polling groups pre-register their study designs, covariate decisions, and results publications regardless of their findings? (Apologies if this is an uninformed or naive question.) It seems like political polling would especially benefit from pre-registration, because they don't even have a favored hypothesis to defend. (Pre-reg has become standard and expected practice in behavioral science, where I work.)
For instance, Pollster Z says "In October, we will poll 10 samples in PA, aiming for at least 600 LVs, using these and only these controls for education, prior voting,... (etc etc), and publish the results regardless of what these findings show."
This would be a great idea if implemented.
If only a few % of the pollsters are actually polling, and aggregators and modelers are adjusting and correcting as well, we might as well be using a Ouija board.
Nate, you're going to need to expose and exclude these not-pollsters if you want us to keep caring about these simulations.
He says he downgrades frequent herders in his pollster ratings.
I know, I'm advocating for excluding them entirely and making a show of it. Because if 95% of pollsters are herding, but are still weighted even 10%, the models are useless.
I just used a Ouija board - it picked against your candidate.
Intereting, pollster herding, reminds me of so called “active” managers hugging the benchmark in the asset management world. They charge much higher fees than passive ETFs, but their portfolios make only the tiniest deviation from the benchmark (Say S&P500), to keep their pathetic little comfy jobs. Their after-fee performance is usually worse than the benchmark.
This reminds me of the joke about the weather man who predicts a 50% chance of rain every day so that he’s never wrong.
But in the long run, he is. Unless there really was rain on ca. 50 % of the days.
This was such a satisfying exchange to read.
hes actually always right in this scenario because forecasters' description of the odds is somewhat counterintuitive to our understanding of odds. a 50% chance of rain in the area does not mean you flip a coin and see if it rains. theres a 100% chance of rain, but that area only covers 50% of the area in question.
What if it doesn't rain though? Seems like you need to consider both variables.
I voted early for Trump today in Florida. How does that change the forecast? according to thomas bayes
Statistically, it means there’s a 100% of Carlos Corredor being okay with a president that wishes he were a dictator.
Your voting for Trump in Florida today doesn't change the forecast.
In different states, different sets of supporters, each set with similar characteristics (age, race, registered party (or not), educational qualifications) participate in early voting in different proportions, which also vary from one election to another. So it is impossible to make any predictions based on early voting figures. Nate explained that in a post here.
Not that everyone understands that, so some analysts might try to draw conclusions from any early voting data which becomes available. But that is very dangerous as early voting data has often shown very different outcomes from the eventual total vote outcome.
Even if it did make a difference, it's one vote out of 11,000,000 (the number of total votes in Florida in 2020) in a state that has only a 2% chance of being the tipping point.
Wow Peter and Calvin you guys must be fun at parties
There are at least 4 big reasons why it doesn't change it at all:
(1) Florida is already known to be a Trump win.
(2) The forecast doesn't look at early votes.
(3) Your 1 vote is an immeasurably small portion of the Florida vote.
(4) Bayes passed away recently*, so it's too late for him to weigh in.
* Recently as in 263 years ago.
To me this is a masterpiece. Your best article this election cycle.
With this analysis + the Nevada early voting data from Jon Ralston, I can't help but fear this likely points to Trump's numbers being meaningfully underbaked in polling. It seems logical that the meaningful shift to Trump nationally is also happening at the state level (albeit less consistently), and that we're in for a rough ride next Tuesday.
Instinctually gravitating to the following outcome: Georgia, Nevada, Arizona are all going Trump with margin to spare, with the big question remaining if Pennsylvania has enough buffer for Harris to pull it out, with the added uncertainty of North Carolina given all the hurricane mess there.
It means the polling is completely meaningless, and we have no idea where the election is going to land. Remember yesterday's article - Nevada isn't really correlated with the rest of the swing states.
In theory, if there is less herding nationally than the state side, wouldn't it indicate that the national polling is more reliable? Therefore it seems reasonable that the state polls are underbaked on Trump, no? (just WHERE its underbaked is hard to predict)
This is addressed in the article. If you look at the high quality pollsters who aren't herding, you see Harris doing well in the rust belt and Trump doing well in the sun belt. So one strong possibility is that Trump wins NV, GA, AZ, NC but Harris wins MI, PA, and WI - which would give Harris the win (albeit barely).
Right, and I've seen that map on various Internet sites today. Yow, that would be a very close election, if it really divides that way. Not good for the rioting in the streets issue.
I could be reading it wrong but I thought Nate was saying there IS herding going on at the national level which is the reason for the tightening numbers. The pollsters are herding the national numbers into alignment with the swing state numbers.
>The pollsters are herding the national numbers into alignment with the swing state numbers.<
I hadn't thought of that, but it has the air of plausibility. Swing states wouldn't be swing states absent a tendency to be close, after all. So, yeah, maybe one or the other candidates in reality is doing better overall than the national polls are showing, because they're being massaged to fit the tight nature of the state level polls.
Could be!
Whatever the national numbers really will be, Harris needs to get more than 4.5% to win comfortably, because of the Dem vote wastage. If there still is so much wastage: Trump may have broken into some of that with his appearences in lost states. But right now, if we believe the unbelievable national polls, Harris only has -0.3%, which won't do the job. I think this issue is fascinating, how much extra national popular vote Dems need to overcome the wastage. Trump, of course, doesn't need any, except that it would be much better for peace in the streets if he won the national vote comfortably.
Plausible theory, but the NYTimes polls also point to a demographic realignment that could explain the tightening national numbers of he’s running up the score in places like Florida Texas etc
Alternatively, it's possible that herding is what led to the national polls moving more for Trump than the states did. The states were already close, so herding around a tie wouldn't move them a ton. The national polls were further apart, so hearing around a tie would create more movement.
I never really saw an argument for why the polls started moving towards Trump in October. Nothing of note happened in the race. Before that, the only real movement we saw was after bad debate performances or when Harris replaced Biden.
Per my count, the polls have only swung toward Trump by about two points since the beginning of October (from Harris+3 to Harris+1). That seems like fairly typical tightening of the kind that...might not require a big event? I think some of the tightening may reflect simple "waning excitement" as the burst of enthusiasm centered on the Brand New Candidacy faded. Trump possibly got a tiny boost from the VP debate, too.
All that said, I do think it's possible that herding may be having the effect you state (puffing up Trump's numbers). I wouldn't be even mildly surprised if Harris wins, and more handily than most deem likely. I also wouldn't be surprised if Trump wins pretty easily.
This election is by far the most "unknowable" since 2000 as far as I'm concerned. (I include 2016 in that: we all "knew" Clinton was going to prevail; we were just wrong, Nate excepted of course).
fwiw, NYT is the poll that had Nevada/Arizona going the opposite direction of PA/MI/WI. So, if Ralston is right and Nevada goes, say, Trump +3, that may just be more evidence of further realignment and a likely Harris win.
imo (not worth much), that's what will happen. Everywhere where pollsters feel free to publish outlier results, we're seeing movement toward Harris among white voters and movement away among POC, and that's the result bleeding out from all the noise and herding in aggregate. PA/MI/WI are much whiter than GA/AZ/NC/NV and don't rely on running up huge margins with non-white voters nearly as much.
Though not my preferred outcome, I have a strong feeling we will see Harris hold the blue wall and Trump win the sunbelt, resulting in a Harris victory with the point from Nebraska. I do feel like pollsters are herding to suppress Harris’s numbers because they don’t want to be embarrassed by another Trump over performance.
Yep. What are the chances of a three- peat? Trump overperforming. It's more likely pollsters are holding down Harris's numbers. Which would be my preferred outcome. LOL
I'm not saying we have Nevada in the bag, but Ralston always does this. Wait until the final Clark tally before freaking out. 25 percent of the electorate are not registered with either party.
Slick analysis by Nate as always. Nate’s like “tell me you’re cheating without telling me you’re cheating”
As someone who's taught statistics courses in the past: thanks to Nate for this post. It's a very nice meta-analysis of what polling firms do with the statistics they produce, and almost something I would like in some way to mention to students if I teach statistics again.
> This uses a binomial distribution, which assumes that all polls are independent of one another, which theoretically they should be
This seems obviously wrong. A simple binomial of sample size alone assumes polsters take random samples and report raw percentage. In reality, they *weight* by various demographic info (including, sometimes, who they voted for last election). This dramatically reduces variance!
In a simplified example, if there were 3 demographics truly voting 0%, 50%, and 100% for Harris, we would expect random unweighted polls of 900 people to vary with a standard deviation of 1.67% (sqrt(1/(4*900))). We would expect random demographic-weighted polls of 900 people to vary with a standard deviation of 0.96% (1/3*sqrt(1/(4*300))), much less.
I'm other words, demographic weighting looks like herding.
It would be an enormous task to try to find or model each polsters demographics and the variance you should expect given that weighting. I can see why Nate didn't attempt that... But as it stands, the extremely small probabilities in that chart are borderline misinformation. Unless I'm completely misunderstanding, it shows the odds that the poll numbers we see could have been unweighted polls. We already know they're not unweighted polls!
This is one of the most cogent arguments against Nate's.
And yet, if you're extra charitable, and cut those anomalous odds in half (or even by ten) and you'll still see the same story.
Dividing the *odds* by 10 (or any constant) isn't charitable, since they scale exponentially with the number of polls. With enough data, any bad theory will be disproven with astronomical odds. The bad theory of unweighted AND unbiased pollsters is here disproven... But since we already know they're not unweighted, the odds don't really say anything.
Your reaction is frankly the reason I said the table is misinformation. Presenting those odds uncritically is bound to be misinterpreted.
If we were to divide anything by a charitable factor, it would be the variance. Ballparking again: If we assume all 249 of his polls were the same size, expecting a 55% to lie within 2.5% has a s.d. of 3.22%. The actual 78% that lie within 2.5% would happen with a s.d. of 2.05%. That's a big difference, but not so huge a difference that it's anywhere close to obvious that demographic weighting alone couldn't explain it.
It's pretty surprising there's zero discussion of this from Nate. Figuring out what distribution to expect from pollsters is exactly the kind of analysis I'd expect to see from an election modeler.
Great work Nate. This is premium analysis. I have to wonder if polling aggregation is dead. You've got your work cut out for you making sense of this mess.