The biggest issue with the changes is that while the backend process for 538 has been totally overhauled, the front end report is very similar to how it always was. This can easily lead to people misinterpreting the report as comparable to the last few elections. Even if the new 538 models were sound, presenting the findings so consistently with such a different process wouldn't be much better.
One point that has been interesting to me is that the Economists model in 2020 (when Morris was in charge there) was much more confident… it had a higher percentage chance of Biden winning much earlier in the race, which would have had to have been based on the polls , because the fundamentals pointed toward Trump or a close race… so it seems like his attitude to fundamentals vs polls has flipped in the last 4 years… I hesitate to suggest motivated reasoning, but it’s hard to avoid that the methodological choices in both 2020 and 2024 lead to suggesting the democrat is has a higher probability of winning
Yeah, the inconsistency between the Morris 2020 model and the Morris 2024 model is hard to explain (and Morris certainly hasn't explained it) by anything other than motivated reasoning.
Doesn’t this just show a pro-Democratic bias in the model though? If it’s overconfident when the Democrat is ahead, and is 50-50 when the Republican is, I think that tells us that whatever assumptions Morris makes helps the Democrat, intentionally or not.
I word for a big publicly-listed company. Believe me - if you think things are that well secretly coordinated at Disney you’re almost certainly wrong. It’s just too hard to do.
Absolutely agree. Risk-averse, consensus-by-committee, decisions based on avoiding UX disruption, hahahaha. Vomit in mouth. Doesn't take much more than one mid level exec and a couple "team players" to keep the model output a toss-up. Being not wrong is safer than not being right. Probably something folks "don't disagree with?" Ha! Vomit in mouth.
If you were right, Disney would be a lot less political than it's gotten and would be making a lot more money. Their recent series of flops is because it's NOT that hard to do when political correctness trumps (no pun intended) corporate profitability. At the company you work for, I'm sure the priorities are 1. risk aversion; 2. profitability; 3. predictability. That's not Disney. It was Disney when Walt was alive, two generations ago. Now it's 1. political progressivism; 2. favorable treatment by the film community and the critics; 3. profits and risk aversion.
How do you publish a model that has not one single poll showing Biden leading and then say “Biden is leading.”
And I am not sure how you model for impact of Biden’s debate (and the Trump assassination for that matter) without seeing how the polls move. Those events are so extraordinary that you have to account for them somehow...and that is the polls absent any other data.
Because it is more priors than data, it’s more premature than broken; given its assumptions, there’s just too little data for it to be publishing a forecast. Come mid-September, the results likely will seem more sensible, even if not impressive.
I think Morris has explained what's happening with Wisconsin, fwiw. G Elliott Morris dislikes me and blocked me on Twitter, and I think he is a bad analyst and that his model is wrong. However, I feel he's explained the specific way in which his model is wrong (which he doesn't accept is wrong).
He wrote: “As a starting point I would try to get people to think of the model as working on vote shares for candidates and not on margins. It’s multivariate not univariate. The confusion w adjusting to more Dem-leaning numbers than the fundamentals comes partly from the huge drop in Biden % in close states without the corresponding increase in Trump or undecided share that is implied by the correlations”
So that final row of the 538 model is doing two things. One, the 538 model treats Biden's and Trump's vote shares (and independents' vote shares and undecideds) as independent. The model thinks they're all going to fall, until in the last row the model forces them back together so they add up, which amplifies the relative margins by multiplication. If you have a 1-point gap that adds up to 75%, it'll become a 1.33% gap when you multiply everything up to 100%.
Two, and more important, the 538 model has extremely strong uplifts for similar states in the correlation matrix, and this is also only incorporated in the final row. If Biden is doing better than expected in Minnesota, the model will give him a huge boost in Wisconsin too. The model simply refuses to believe the Wisconsin polls if they are outliers from similar states, because as Nate stated Elliott pays as little attention to the polls as he can.
This is all very bad practice and it's because G Eliott Morris is a very bad analyst and a pompous ass. But he is a pompous bad analyst who has explained his bad practice.
Agreed. It seems from the explanations from Morris that most of the weight here is in correlations between states mostly based on historical data, not current data. He said that turning off the correlation weighting today gives Trump 67% to win.
So, we’re talking a 20 percentage point swing because the model sees the election as fixed relative vote shares across states modulated mostly by fundamentals! If Biden for example is underperforming in NY according to live polls, the 538 model says “that can’t be right, and if that’s wrong, there’s a systemic polling error nationwide”, so let’s bump up the numbers everywhere. Since NY has such a tight CI, it tries really hard to revert to historical data.
Another thing we don’t know is how it uses demographic data to alter the correlation weighting. If e.g. TX has a demographic shift toward minorities, the model may reduce the correlation with other historically red states, or one could even become negatively correlated so that as Trump does better in AL, TX becomes more blue!!
This guy needs to look at the CarFax, IMO. After these comments it’s not about “changing the language on the chart.” They need to show the correlation matrices, provide raw data, or develop dynamic param edits in the UI so we can do a “what if” analysis on the model. Definitely feels like they’re using obfuscation and branding to keep people from asking questions at this point.
I had no opinion of Morris until the model started performing its head-in-the-sand act after the debate.
He tweeted that he was working 12 hour days (!que lastima!) and was being "honest and transparent" that he could not analyze the model's results. This was in response to Nate Cohn, so Nate S. isn't the only analyst befuddled by 538's performance.
Thing is, the model certainly is more crucial to his work output than whatever else he's doing in those 60+ hour weeks. He's avoiding the topic because he knows he can't explain it in any way that is defensible.
So we can rest assured that Morris will honestly & transparently avoid the shit out of this issue because it's beyond his reckoning.
Right, but GEM thinks the model is totally irrelevant until Labor Day! He's consistent in thinking it. He's sincere in believing the model isn't yet crucial in any way. He's also wrong, but hey-ho.
That couldn’t explain the 538 result. Kennedy gets less than 10% of the vote basically everywhere, so you are multiplying by 1.11 or less. But you are multiplying everyone’s numbers by the same coefficient, so it’s really not important at all until third party abc undecided gets much bigger than it is.
Yeah, the multiplier bit is by far the smaller element. But it's not about re-allocating RFK Jr. It's that the model is badly designed so it is making everyone's vote share fall. Then at the end they ad hoc it back to 100%.
The main bit is the correlation matrix against other states. His model thinks Biden is doing well in states like Minnesota, so it simply rejects the Wisconsin polling. It's ludicrously bad but it's clear enough why it's happening.
strikes me the wisconsin polling says more about minnesota than the minnesota polling does about wisconsin. there’s alot more wisconsin than minnesota polling.
"For instance, they say that Biden has a 14 percent chance of winning the national popular vote by double digits."
Honestly, that does it right there. Pelosi, Schumer et al are begging him to drop out and 538 says there's a 1 in 7 chance he's en route to the biggest landslide in 40 years.
I’m amazed ABC has not taken down the 538 model or at least updated it. It’s hilarious to see it say Biden 53% chance then immediately below that show a list of 10 polls with Trump leading in all 10. If July polls are meaningless why are they the first thing shown on the front page of their model??
Also, I think keeping your Twitter handle as “@natesilver538” is contributing to the brand confusion.
The 538 model obviously is broken and it has caused a lot of damage, because BlueMaga continues to point to it and you can bet safely Team Biden are selling it to Biden as well.
Facially I think the logic is okay for a model to deeply discount summer polls. As Nate said, tho, if you don’t believe in polls then you don’t have a model. Can’t rely on fundamentals to this extent.
Yes, it will be deeply ironic if Morris has tilted the model blue because he prefers blue, and then his blue-favoring forecast is what keeps Biden from dropping out, which then results in a Trump landslide.
I'm not surprised 538 is creating poor-quality contrarian content now. Disney is trying to consume the brand for the last few dollars of clicks before it sunsets the site. It's part of the broader trend towards enshittification (in the sense of Doctorow; see https://statmodeling.stat.columbia.edu/2024/07/16/google-is-violating-the-first-law-of-robotics/) in media properties these days.
Hey, at least they had the courtesy to fuck up the branding* on your way out, so there's a clear scorch mark in the geologic strata for future generations to measure from.
* 538 vs FiveThirtyEight, for those who haven't noticed
They should let Fivey decide who he wants to live with.
The biggest issue with the changes is that while the backend process for 538 has been totally overhauled, the front end report is very similar to how it always was. This can easily lead to people misinterpreting the report as comparable to the last few elections. Even if the new 538 models were sound, presenting the findings so consistently with such a different process wouldn't be much better.
Indeed. Have they even made clear anywhere other than the methodology page that nobody reads that this is an entirely new model?
One point that has been interesting to me is that the Economists model in 2020 (when Morris was in charge there) was much more confident… it had a higher percentage chance of Biden winning much earlier in the race, which would have had to have been based on the polls , because the fundamentals pointed toward Trump or a close race… so it seems like his attitude to fundamentals vs polls has flipped in the last 4 years… I hesitate to suggest motivated reasoning, but it’s hard to avoid that the methodological choices in both 2020 and 2024 lead to suggesting the democrat is has a higher probability of winning
Yeah, the inconsistency between the Morris 2020 model and the Morris 2024 model is hard to explain (and Morris certainly hasn't explained it) by anything other than motivated reasoning.
Perfectly consistently pointing to his desired outcome.
Doesn’t this just show a pro-Democratic bias in the model though? If it’s overconfident when the Democrat is ahead, and is 50-50 when the Republican is, I think that tells us that whatever assumptions Morris makes helps the Democrat, intentionally or not.
Begs the question, what is motivating some of the choices over at Disney?
I word for a big publicly-listed company. Believe me - if you think things are that well secretly coordinated at Disney you’re almost certainly wrong. It’s just too hard to do.
Absolutely agree. Risk-averse, consensus-by-committee, decisions based on avoiding UX disruption, hahahaha. Vomit in mouth. Doesn't take much more than one mid level exec and a couple "team players" to keep the model output a toss-up. Being not wrong is safer than not being right. Probably something folks "don't disagree with?" Ha! Vomit in mouth.
If you were right, Disney would be a lot less political than it's gotten and would be making a lot more money. Their recent series of flops is because it's NOT that hard to do when political correctness trumps (no pun intended) corporate profitability. At the company you work for, I'm sure the priorities are 1. risk aversion; 2. profitability; 3. predictability. That's not Disney. It was Disney when Walt was alive, two generations ago. Now it's 1. political progressivism; 2. favorable treatment by the film community and the critics; 3. profits and risk aversion.
The 538 model is broken. Full Stop.
How do you publish a model that has not one single poll showing Biden leading and then say “Biden is leading.”
And I am not sure how you model for impact of Biden’s debate (and the Trump assassination for that matter) without seeing how the polls move. Those events are so extraordinary that you have to account for them somehow...and that is the polls absent any other data.
PS - this post got me across the line to a paid subsription!
Me too!
Me too.
Because it is more priors than data, it’s more premature than broken; given its assumptions, there’s just too little data for it to be publishing a forecast. Come mid-September, the results likely will seem more sensible, even if not impressive.
That 95% confidence interval is just fucking WILD. +20! +30!
Either candidate could drop dead tomorrow and the survivor wouldn't pull those numbers.
More likely that a candidate drops dead and *his own side* pulls those numbers!
Let me hear you say "OV-hoe"
"I'm not a fan of Elliot Morris"
literally starts the article by making clear he's the biggest hater.
Good. Transparency.
I think Morris has explained what's happening with Wisconsin, fwiw. G Elliott Morris dislikes me and blocked me on Twitter, and I think he is a bad analyst and that his model is wrong. However, I feel he's explained the specific way in which his model is wrong (which he doesn't accept is wrong).
He wrote: “As a starting point I would try to get people to think of the model as working on vote shares for candidates and not on margins. It’s multivariate not univariate. The confusion w adjusting to more Dem-leaning numbers than the fundamentals comes partly from the huge drop in Biden % in close states without the corresponding increase in Trump or undecided share that is implied by the correlations”
So that final row of the 538 model is doing two things. One, the 538 model treats Biden's and Trump's vote shares (and independents' vote shares and undecideds) as independent. The model thinks they're all going to fall, until in the last row the model forces them back together so they add up, which amplifies the relative margins by multiplication. If you have a 1-point gap that adds up to 75%, it'll become a 1.33% gap when you multiply everything up to 100%.
Two, and more important, the 538 model has extremely strong uplifts for similar states in the correlation matrix, and this is also only incorporated in the final row. If Biden is doing better than expected in Minnesota, the model will give him a huge boost in Wisconsin too. The model simply refuses to believe the Wisconsin polls if they are outliers from similar states, because as Nate stated Elliott pays as little attention to the polls as he can.
This is all very bad practice and it's because G Eliott Morris is a very bad analyst and a pompous ass. But he is a pompous bad analyst who has explained his bad practice.
Agreed. It seems from the explanations from Morris that most of the weight here is in correlations between states mostly based on historical data, not current data. He said that turning off the correlation weighting today gives Trump 67% to win.
So, we’re talking a 20 percentage point swing because the model sees the election as fixed relative vote shares across states modulated mostly by fundamentals! If Biden for example is underperforming in NY according to live polls, the 538 model says “that can’t be right, and if that’s wrong, there’s a systemic polling error nationwide”, so let’s bump up the numbers everywhere. Since NY has such a tight CI, it tries really hard to revert to historical data.
Another thing we don’t know is how it uses demographic data to alter the correlation weighting. If e.g. TX has a demographic shift toward minorities, the model may reduce the correlation with other historically red states, or one could even become negatively correlated so that as Trump does better in AL, TX becomes more blue!!
This guy needs to look at the CarFax, IMO. After these comments it’s not about “changing the language on the chart.” They need to show the correlation matrices, provide raw data, or develop dynamic param edits in the UI so we can do a “what if” analysis on the model. Definitely feels like they’re using obfuscation and branding to keep people from asking questions at this point.
I had no opinion of Morris until the model started performing its head-in-the-sand act after the debate.
He tweeted that he was working 12 hour days (!que lastima!) and was being "honest and transparent" that he could not analyze the model's results. This was in response to Nate Cohn, so Nate S. isn't the only analyst befuddled by 538's performance.
Thing is, the model certainly is more crucial to his work output than whatever else he's doing in those 60+ hour weeks. He's avoiding the topic because he knows he can't explain it in any way that is defensible.
So we can rest assured that Morris will honestly & transparently avoid the shit out of this issue because it's beyond his reckoning.
Right, but GEM thinks the model is totally irrelevant until Labor Day! He's consistent in thinking it. He's sincere in believing the model isn't yet crucial in any way. He's also wrong, but hey-ho.
Publishing the model before labor day isn't consistent with that position, though.
Indeed. My sense is that ABC/Disney ordered him to release it now for the clicks, and he is resentful of that because he thinks it's a waste of time.
It’s not just G Elliot who dislikes you, anyone who reads enough of your comments here at the Silver Bulletin starts to dislike you as well 🫶
That couldn’t explain the 538 result. Kennedy gets less than 10% of the vote basically everywhere, so you are multiplying by 1.11 or less. But you are multiplying everyone’s numbers by the same coefficient, so it’s really not important at all until third party abc undecided gets much bigger than it is.
Morris is spouting mumbo jumbo
Yeah, the multiplier bit is by far the smaller element. But it's not about re-allocating RFK Jr. It's that the model is badly designed so it is making everyone's vote share fall. Then at the end they ad hoc it back to 100%.
The main bit is the correlation matrix against other states. His model thinks Biden is doing well in states like Minnesota, so it simply rejects the Wisconsin polling. It's ludicrously bad but it's clear enough why it's happening.
strikes me the wisconsin polling says more about minnesota than the minnesota polling does about wisconsin. there’s alot more wisconsin than minnesota polling.
Yep! The new 538 model is so bad lmao. GEM is a hack.
"For instance, they say that Biden has a 14 percent chance of winning the national popular vote by double digits."
Honestly, that does it right there. Pelosi, Schumer et al are begging him to drop out and 538 says there's a 1 in 7 chance he's en route to the biggest landslide in 40 years.
Saturday when Biden drops out, you can give us a paid article.
Monday. Biden doesn’t work weekends
Saw the title of this article and immediately grabbed the popcorn.
As somebody who's not a statistician but has been pretty confused by the output of the 538 model, this is really good to know.
I’m amazed ABC has not taken down the 538 model or at least updated it. It’s hilarious to see it say Biden 53% chance then immediately below that show a list of 10 polls with Trump leading in all 10. If July polls are meaningless why are they the first thing shown on the front page of their model??
Also, I think keeping your Twitter handle as “@natesilver538” is contributing to the brand confusion.
The 538 model obviously is broken and it has caused a lot of damage, because BlueMaga continues to point to it and you can bet safely Team Biden are selling it to Biden as well.
Facially I think the logic is okay for a model to deeply discount summer polls. As Nate said, tho, if you don’t believe in polls then you don’t have a model. Can’t rely on fundamentals to this extent.
Yes, it will be deeply ironic if Morris has tilted the model blue because he prefers blue, and then his blue-favoring forecast is what keeps Biden from dropping out, which then results in a Trump landslide.
Which is why the model is better described as premature rather than broken. If it wasn’t released until September, it would look more sensible.
I'm not surprised 538 is creating poor-quality contrarian content now. Disney is trying to consume the brand for the last few dollars of clicks before it sunsets the site. It's part of the broader trend towards enshittification (in the sense of Doctorow; see https://statmodeling.stat.columbia.edu/2024/07/16/google-is-violating-the-first-law-of-robotics/) in media properties these days.
Stunt on these hoes
Hey, at least they had the courtesy to fuck up the branding* on your way out, so there's a clear scorch mark in the geologic strata for future generations to measure from.
* 538 vs FiveThirtyEight, for those who haven't noticed
Nate never should have sold that trademark.
And Martin B for POTUS!
Seconded! Haha.