369 Comments

Death rates diverged at the point in time at which the vax was introduced. Isn’t that itself strong evidence? Unlikely that the confounders changed substantially at that exact time.

Expand full comment

I find this analysis all pretty compelling. Thanks for the follow up, Nate. One item that does catch my eye, though, is this:

>>Are all states counting COVID deaths in the same way? (Probably not.)<<

It probably would be interesting to look at excess mortality instead of COVID deaths. I do recall seeing nation-to-nation comparisons that appears to differ quite a bit when using the former rather than the latter. But I haven't looked into whether the US (or individual states) compile state-level excess mortality numbers.

Expand full comment

Wouldn't kulldorff's argument also imply that red states should have had higher fatality rates pre-vaccine? So it fails that test too.

Expand full comment

It's good for trusted sources to occasionally demonstrate their rigor on takes to show that they should continue to be trusted. You're right that doing that every time is overkill, but never doing that also allows for take artists to conform their takes to their priors if they never have to go up against strict scrutiny.

Expand full comment

Glad you at least briefly addressed the first claim in Friday's post, and admitted it might not be as robust.

First, I'll note your brief summary today is technically stronger than your original assertion: here you say NPIs had no effect, whereas Friday your explicit claim was simply that there was little difference in death rates between red and blue states. But the summary, that NPIs had *no* effect, while not explicitly stated Friday, I think reasonably reflects the sense readers would take away from the post.

I think your hostility towards NPIs remains overstated and I would strongly urge you to back off any implication that all NPIs were entirely useless. It is reasonable to at least point out other costs of NPIs, and to question whether the benefits outweigh them, but categorically rejecting them is counterproductive.

First, we know now they vary a lot in effectiveness. Masking is a good example, as N95 masks are far more effective than cloth or lower grade paper masks at reducing COVID spread, but also a poorly worn mask is much less effective than one that fits well. Also even a good, well fitting mask does more to prevent you from spreading COVID to others than it does to reduce your risk of catching it. Masking also reduces the odds of spreading other illnesses, which is why for years mask wearing was non-controversially routine for health care workers in closer contact settings, like dentistry or surgery.

Reducing people gathering also varied in effectiveness. On the one hand, outdoor gatherings such as spring break beach trips or public protests after George Floyd's death did not result in surges of COVID cases many people feared. The better ventilation of being outdoors was sufficient to keep risk of transmission in those settings low. On the other hand, early contact tracing often found links to unmasked indoor gatherings, especially of people from very different areas, like a wedding or conference.

I appreciate the attempt to quantify and simplify analysis by focusing on state-level death rates, but I'd note that especially early in the pandemic, prevalence varied widely across states, while if anything government mandates about NPIs was more uniform. Most of the country put in place mask mandates and bans on large gatherings (even outdoors). We don't have a good control group to compare states with high prevalence that did *not* have lockdowns or mask mandates before vaccination was available, so it is harder to quantify how effective those NPIs were at reducing spread and death.

I am comfortable saying NPIs were less effective than vaccines, even way less effective, but I'd note they were pushed most strongly before vaccines were available, and when less was known about COVID spread. We can and should be smarter about when and where to use, and in particular, mandate them.

At a minimum, I'd say encouraging people to stay home when possible if sick, or at least mask when they have to go out when sick, would be a positive development, not simply with respect to COVID, but infectious disease in general. What helped COVID spread widely was that people were often infectious before becoming symptomatic, and also that it spread primarily by air and not necessarily direct contact.

The next pandemic, however, may spread differently, in which case some NPIs that were largely ineffective for COVID may be more valuable.

Finally, I suggest you not frame things as simply vaccines good, NPIs bad. While I'd agree vaccines have been more effective, NPIs covered a broad range of things, and well targeted NPIs may still have value, especially before vaccines are developed. Stigmatizing NPIs as being inherently ineffective and not worth their costs risks overfitting your model. It is far easier to find in hindsight which interventions were ineffective against this pandemic than it is in the face of a new threat to figure out what works.

Expand full comment

I'm fine with the second claim but am still not convinced of the first claim. I don't really buy that masks and distancing had zero benefits. I see two potential confounding factors:

1. Possibly blue states had equal deaths pre-vaccine because they got hit earlier. NY and NJ being high in the first list lends to this.

2. Even though the restrictions between states were different, many people in red states may have still been masking and distancing (not sure what the data says in this).

So while some of the blue state restrictions may have been unnecessary, I am nervous people are interpreting this data as saying in the next pandemic we shouldn't bother with masking or distancing.

Expand full comment

Your posts are really well thought out and refreshing to read. I think you admirably try to produce fragments of truth despite the maelstrom of emotional bickering that surrounds politics and COVID.

I am sure you already know this, but I hope to remind you that the insane leftists who have hounded you on COVID are self-selected, very online leftists and surely don't represent the progressive movement. The more rational a person is, the less likely they are to waste their time tweeting etc at prominent people (I'm drunk-writing this right now).

I can imagine it feels like the whole world is ganging up on you when you post on twitter, even if you know that's not the case. So being told this may have little effect, but please let your thoughts and writing be affected by online morons as little as possible.

To be clear I am not criticising this post where you address points made by a person regarded as an expert. I just get the impression it weighs on you at times. I know it can't be helped because you're human, and I wouldn't be able to stand it as well as you can, but I want to remind you that the sane, busy (and sober) majority just reads and doesn't post.

And please keep posting on twitter, and ignore the fickle mob!

Expand full comment

I appreciate this analysis, but I think one pretty major issue in it is that vaccine uptake is strongly correlated with taking additional precautions - particularly in the 2021-2022 time frame. Amongst people who were getting vaccinated early, they didn’t just decide to go live life like normal right away - there were still restrictions in place. The anti vax crowd living in these red states had very little of that.

So there’s a very strong latent variable here - which admittedly has waned over time - so I’d like to see how this analysis looks since some time in 2022, basically when other preventative measures were eliminated and masking stopped at a broad level in this country.

Expand full comment

https://www.scientificamerican.com/article/people-in-republican-counties-have-higher-death-rates-than-those-in-democratic-counties/#:~:text=Sehgal%20and%20his%20colleagues%20found,relative%20to%20majority%2DDemocratic%20counties.

Republican counties have worse health outcomes than Democratic counties in general.

Disclaimer: I am posting this link as a jumping off point. Personally, my guess is that most of the disparity is probably due to poverty, lower number of health care workers per capita in rural counties, etc.

Expand full comment

A few follow up questions:

Why is the US the only country to recommend another round of boosters to everyone over 6 months old, regardless of previous COVID vaccination or health status?

Why has booster uptake declined every round?

How many boosters have you gotten?

The CDC and Pfizer claimed that the COVID jabs prevent transmission and infection, but don’t affect menstrual cycles and breast milk. Since real world data proves all of those claims false, were they spreading lies and misinformation?

Why did Dr. Aseem Malhotra change his mind about the COVID jabs?

How much money have Pfizer and Moderna made on COVID jabs?

Expand full comment

Your piece was posted on MarginalRevolution.com and the top comment right now is this. I am not statistically literate enough to know if he is right so I'd like if someone (preferrably Nate Silver himself) who is, to respond to this (https://marginalrevolution.com/marginalrevolution/2023/10/sunday-assorted-links-438.html?commentID=160661089)

"Silver isn’t quite as convincing as he thinks.

What he is positing is a two-stage model where partisanship explains vax rates and vax rates and age explain death rates, but he is actually only running a one-stage regression. So in his model including Biden vote share, age, and vax rate, Biden’s vote share coefficient is near zero and statistically insignificant. If you are taking his model seriously, then you should think partisanship is not a good explanation.

That there is a partisan divide in both vax rates (which Silver actually only takes for granted but there is evidence for… just google it) and mortality means there’s is probably something there, but in his previous post on the topic, his own analysis shows an inflection point for difference in mortality rates started in September 2020. That makes me think it’s vax uptake plus behavior.

Instead of Nate Silver nailing it, I’d say he whiffs it"

Also ". Let's abstract away a bit. Suppose you have an outcome variable y and 3 explanatory variables: x1, x2, and x3.

Then you run the model y = a*x1 + b*x2 + c*x3, and you find that the coefficient on x1 is near zero and statistically insignificant and the coefficients on x2 and x3 are large and statistically significant. Additionally, when you drop x1, you find r-squared stays the same."

Expand full comment

It's not just age. It's also obesity, poverty, worse health care in rural counties, etc. We know (or strongly suspect) that those other factors have a significant effect on health outcomes so they need to be included in any analysis.

As a simple check, run the regression for cancer diagnoses and see what the result is.

Expand full comment

Can you do this analysis with excess deaths instead of COVID deaths to see if counting COVID death statistics makes any difference? Though I suspect the states with the higher rates are actually the ones more likely to undercount.

Expand full comment
Oct 1, 2023·edited Oct 1, 2023

This is a useful analysis, but “vaccines really did work” probably won’t be particularly surprising for people without their heads in the sand.

I feel like there is a lot more room for argument on the “did anything else work?” side. Certainly nothing worked in the way vaccines did, but I would have hoped that was uncontroversial. The underlying patterns are messy before schools reopened. One could suggest that the data are consistent with the idea that dems were more susceptible (e.g. due to community density or living condition density) but sometimes took precautions that neutralized that difference.

Expand full comment

Nate, this isn't a Presidential election with an electoral college. States are arbitrary units of observation for this outcome variable – such analyses can only lose information, or distort reality for dumb people.

States vary enormously in population, and in countless other ways, and there are more "red" states than "blue". You're giving a bunch of weight to Southern states – you're basically running up a score when you count very similar Southern states over and over and over.

It would be just as valid to group states together into Silver Regions 1 through 11, and run your silly model. It adds no info to compare state-level variables. If you want to say something about vax efficacy, just cut straight to the science at the human individual and population levels, ideally with a good cost-benefit grouped by age (by decade), sex, and health status.

As always, it helps to zoom out and be a good philosopher. COVID carries no mortality risk for most humans. It's been very difficult to get any details or cost-benefit analyses from US government agencies, but it looks like it doesn't kill healthy people under 50. (I'm being conservative on the age. Obviously what "healthy" comes down to will matter. Again, details are scarce.)

This means that your outcome variable is irrelevant to most people's decisions to take the shots. Now, if we're talking something like hospitalization or a severe symptoms outcome, then we can address those outcomes with solid data – maybe the same pattern will hold, or maybe not.

You're not even scratching the surface here in terms of getting at whether it's rational for someone to take the shots re: their own mortality risk (or others' – they don't prevent transmission; it's possible they reduce it – I've seen estimates of 20% reduction, but *how* it would do this is unexplained in the CDC's junk comms, and I'm not clear on the validity of the measures).

IFR for most is well below 0.1%. Say the shots knock that down to, what, 0.01%? At this point, it probably doesn't matter what the specific value is because humans can't necessarily do anything with differences in probabilities like 0.0001 vs. 0.00008 given all the other factors that we'd think about. Plug in other expected benefits. Then plug in that it's brand new, experimental pharma, maybe existing for six months on this planet at rollout. Humans don't normally take experimental pharma, any pharma that has only existed for six months, or even five years. We've never done this before, and it would be a disastrous habit to normalize the way you seem to want. So against the shaving down of a 0.1% or 0.03% IFR, a person needs to weigh that this is experimental pharma whose effects are largely unknown. (We don't have any new scientific methods or epistemics that can tell us the long-term effects of brand new pharma, nothing like the accelerated aging methods in materials science, structural engineering, etc. We're centuries away from that kind of scientific ability.)

Almost everyone who passes on the shots lives, and avoids any of the known and unknown side effects of them. Maybe 0.1% (probably less) end up dying. It looks like most of those would've been saved by the shots, but a good chuck would've died anyway (I saw a 25% figure recently). But for those who died, they chose to pass on experimental pharma in the face of a 0.1% IFR, weighing the unknown risks heavier. They made their choice, and were unlucky. It looks like a reasonable choice, arguably the most rational choice (and we'll always need some people to make that choice – we couldn't possibly allow 100% of humans or of a country to take experimental pharma). If they were informed, they knew the odds, made their choice. They might make the same choice again, or in any future similar scenario. Nothing about those odds and weights changes if you end up being in the 0.1%, right? People make their choices, and live with them, or not. Counting up deaths in a massive country and presenting relative stats without even discussing the fact that this doesn't apply to most people is just too neurotic – other people's choices are theirs. We don't know anything here about contexts or nature of the deaths. You might be talking about a year of life in many cases, or less, and I'm not sure what high-risk people's contexts might be, and how they're different from my own.

In any case, it's incredibly dangerous to normalize most humans, or even 5%, taking experimental pharma in the dark. This is a "Do NOT do this again" situation. There's no reason to settle for the lack of rigor, the stunning recommendations for young, healthy people, even infants, when there's no mortality risk there (possibly not much severity risk either). Humans need to be a lot more conscious and scientifically serious than this if we want to survive the century. These are the worst-performing vaccines most of us have ever taken, with the possible exception of the flu shots – they don't stop infection, or transmission, they don't last, and their long-term risks are unknown. It's crazy to promote them as the choice of the enlightened. Politics is just so toxic right now, especially leftist/Dem politics and prejudice.

It's also very early. This whole discussion is bogus without accurate COVID deaths data, which you don't have. Ideally we'd have the discipline and rigor to say "We don't have accurate deaths data" and walk away, tell you come back when you do. But I saw the other points as important enough to entertain your bad data. (Localities have warped financial incentives to code deaths as COVID deaths. It's the with/from problem. And the CDC seems to overcount COVID deaths every few months, sometimes children specifically, sometimes total. They've been embarrassingly corrected by stay-at-home moms – true story – which also led to the revision of a peer-reviewed journal article.) This is the kind of thing we don't sort out until years later, so it's more interesting to me to see what we've got in 2027 or so, assuming we don't have a White House that would censor scientific discourse and rigor (the way Biden did on things like side effects, the superior protection of natural immunity, even policy views, etc.)

p.s. Black population is the strongest demographic predictor of your bogus mortality data. I ran it and GOP registration. It's crazy that you didn't control for race or population, just nuts. Blacks are much less vaccinated than Republicans, something like 44% vs 63%. Your pattern of "red" states is strikingly similar to Newsom-style scams on gun violence, homicides, homicide-by-gun on weekends, etc. where he tries to paint "red" states as stained with blood. Much or most of that is driven by blacks in leftist-run Southern cities within "red" states, another example of the arbitrariness of the states comparison.

p.s. 2: You falsely made NH and VA blue in your table, and NC red. This subtly affects the appearance of your ridiculous eyeballing method. You also have Florida, Kentucky, and Texas as red, and Colorado as blue, when they're purple by registration. This wouldn't apply to your regression of fascist Biden voters, but it definitely weakens your color method.

p.s. 3: Most people couldn't get the vax by Feb 1, 2021, so I have no idea why you started there. This is an extremely stupid analysis – starting at some date, sacrificing three bats, etc – but if you're going to do this, you should choose a much later date after it was widely available. I wasn't eligible until the summer. Ideally, you'd run it for several different start dates, say each month of 2021, and see what's what.

Expand full comment

Maybe I missed it, but I would be curious to see the same regression analysis run for pre-COVID times (and after the first wave, although it might be considered cherry-picking data).

Also, I would be curious as to what geographic level - county, city, state, etc. - yields more statistically significant results.

Expand full comment