I think my two favorite posts for the year were the one on using Chat GPT for poker, and the airport statistics one. The airport statistics one because it's exactly the type of awesome data that I love, and has some practical applications as well. For the GPT poker one, it marked a good example of you critically challenging your prior position as previously demonstrated in "On The Edge" and your prior posts, and updating your opinions as more data came in (how very Bayesian of you!).
In particular, you seemed to come to the same conclusion many of us did that as soon as you ask LLMs about something that you are an expert in, it fails in surprising - and sometimes hard to detect - ways. While they are still good for many purposes, and in particular for things that you can easily check, this fundamental issue makes them dangerous to use for topics you know nothing about. More relevant to the topic is that this limitation means the exponential growth and use to replace human jobs entirely is stunted, if it is indeed still possible.
Also, as someone who's been hands on with many of these successful projects there's a layer the layman needs to understand that isn't clear in the implementation phase.
Good AI implementation often uses it to bridge specific parts is traditional automation, by 'sandwhiching' it between automation tasks. An example I can site, reading and identifying systems affected by security advisories, but it's only triggered when the traditional automation fails, and it alerts the maintainer that 'system has failed.'
This is something the developer MAY need to fix, but traditional this means that the work would fall back to being hand written for a time. However, if the AI is successful it can be left to run with basic monitoring for several months allowing the developer to note if it was a one off, a consistent reoccurring error, or a new standard that triggered the automation.
Given the scope of millions of individual analysis-es a year, this allows for a .1% to be reduced to a 'review and approve' for 90% of those cases, 'minor fixes of AI work' for about 8% of cases, and only 2% requiring hand tooling. Those are rough roundings since I'm remembering the numbers off the top of my head, but we're talking about reducing work hours from impossible to only mildly impossible here.
Thus the reason I say the correct implimentation of AI is key.
Also, many AI success stories are of more traditional automation or machine learning that just tacked on AI elements to get management behind larger initiatives, or to provide a way to simplify complex data for 'quick reports'.
The concept that this tool can just replace programmers outright is a folly, one recently admitted to by Sales Force and Microsoft.
As a member of the minority here (conservative Trump voter), just a note to say that this is one of my top sites for content and thoughtful reading/discourse. I find it all incredibly interesting and devour every article, whether I know anything about it or not. I consider myself enriched by what I read here, including the reader comments, and fully appreciate the different and varying perspectives. Thank you, Nate, for your efforts on this!
Love the summaries, and mostly agree about the fun ones over the year. Thanks for all the interesting topics, looking forward to more in 2026!
For what it’s worth, I may be in the minority that thinks footnote #11 is a shame to have to say out loud. Personally, I love the nuanced approach to SB posts (it’s the primary reason I’ve enjoyed the writing over the years). I would think that it’s a big draw for at least some paid subscribers, since Nate et all do it better than most. Maybe it’s the policy nerd in me, but I enjoy those “you’re mostly right, but let’s talk about the 15% you got wrong” critiques.
Doesn’t scratch that partisan itch on political posts for lots of people, so I guess that’s the world we all have to live in.
I agree with this perspective. One thing that Nate taught me, back in the 538 days, was to be more critical of my (and other's) statistical intuition. I was one of those people who was guilty of treating something that had a 75% chance to occur like it was as good as 100% likely, and I was not good at really understanding that difference.
As I've grown, I've continued to apply that mindset, appreciating that teasing out magnitude errors (like bad vs really bad) is a much more difficult skill than teasing out outcome errors (bad vs good). So I always appreciate nuanced conversations that help better hone those skills and develop better instincts.
Nate did some good Substack Live appearances this year. I think at least a couple of them came from the SB feed. Hope to see more of those in 2026. Also love it when the chat function is up and running during high profile events. Lastly, don’t sleep on Winter Olympics coverage. Best-on-Best hockey combined with fraying international relations makes for traffic boffo!
When you do your qb rankings how do you account for the drastic change in the rules especially the rules which allow the offensive players vast amounts of freedom and limits the hits on both receivers and QBs?
Congratulations to Nate, Eli, and Joseph on the continued success of the newsletter. The remarkable post-election statistics are a testament to the value of this style of “data journalism,” cringe as that term may have become.
Thanks also for the end-of-year recap. Fun to review the year, and the “tick-tock” style seemed to flow a little better than the top/bottom N of last year.
I have never been very confident I know what "meta" means, but I think this comment may be an example. Online endnotes, such on this Bulletin, drive me crazy. In books where there are actual pages, they are tolerable. But online, toggling from the text to an endnote and back to the text seems impossible. Am I missing something?
If you read the newsletter in the browser, rather than in the email, you can just hover the mouse over the footnote number and it shows the footnote contents where you are.
Nate says, "I have a considerable backlog of AI-related ideas" --- great, go for it, I'd say! I was dismayed to see that there already was a post that I missed about AI Jan. 27 ----- before I got clued in there was such a thing. I guess I figured it out in July, and dove straight in at the deep end and do daily work on learning AI and prompts and reading about it. This is HUGE. And political? Last week some woman here said no one should use AI because it would "kill the planet," and I quote. I had a few things to say about that, never mind, I'm better now ---- and then the very next day Politico had an article saying Americans all hate AI!!!!!!! Meaning "decent" people, presumably, that is, leftists. So now the situation seems to be the Left wants us all to hate AI and the Right --- is for AI? Those of us who have heard of it, anyway. Darn, that was quick polarization! I think the pro-AI side is the better one to be on, and if leftists are eager to be anti-AI, it's remarkably anti-intellectual of them. Weren't they supposed to be the elite? I am so confused.
My wife and I lived in the West Village until recently and our daughter lives in the East Village. East or west preference may be age dependent (mid 60s vs 30). Either way let’s all agree never ( or at least only rarely) go above 20th street.
The reason for the asymmetry is because Biden was running for office *again*, and Nate was saying it was a bad choice (something that, in my opinion, was obvious to everyone except for deep partisan hacks). For Trump, I don't see much reason to talk about his mental decline at this point, since his own voters don't care, and he's not up for reelection. Only way he's leaving that position is when his time is up - one way or another.
For Biden, I don’t see much reason to talk about his mental decline at this point, since his own voters don’t care, and he’s not up for reelection. (He’s also not the current president)
Nate has not talked about Biden's decline qua Biden's decline in a long time that I have noticed. Biden's decline has been mentioned in the context of discussing the forces within the Democratic party that had their heads buried in the sand and made extremely poor strategic choices as those people are still there, their heads seem to still be buried in the sand, etc, and that has ongoing relevance for election analyses. (This is perhaps a bit like the use-mention distinction in philosophy, if you wanted to read up on that.)
That said, I do think it would be worth Nate doing a post on Trump's decline at some point. I suspect Nate's concern is what he stated in this post: that there are already many such takes and he doesn't think he has something distinctive to add. I certainly don't think Nate is biased in favor of Trump, or that he thinks Trump is as sharp as a tack, or that his decline is irrelevant for politics or policy.
Wow, you didn't read (or didn't comprehend) my comment or the comment I was responding to, and then have the lack of self-awareness to end your comment with a criticism of how people don't read things.
I think my two favorite posts for the year were the one on using Chat GPT for poker, and the airport statistics one. The airport statistics one because it's exactly the type of awesome data that I love, and has some practical applications as well. For the GPT poker one, it marked a good example of you critically challenging your prior position as previously demonstrated in "On The Edge" and your prior posts, and updating your opinions as more data came in (how very Bayesian of you!).
In particular, you seemed to come to the same conclusion many of us did that as soon as you ask LLMs about something that you are an expert in, it fails in surprising - and sometimes hard to detect - ways. While they are still good for many purposes, and in particular for things that you can easily check, this fundamental issue makes them dangerous to use for topics you know nothing about. More relevant to the topic is that this limitation means the exponential growth and use to replace human jobs entirely is stunted, if it is indeed still possible.
100% agree about Nate's ChatGPT / Poker article.
Another interesting point on this front is the MIT article that was widely cited in the press as "95% of generative AI pilots fail".
The actual article is more of a compare and contrast of what works and what doesn't, in terms of activities, processes and metrics.
https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Business_2025_Report.pdf
As of right now, AI is more analogous to upgrading from a hammer to a nailgun, rather than automating a carpenter out of a job.
Bingo.
Also, as someone who's been hands on with many of these successful projects there's a layer the layman needs to understand that isn't clear in the implementation phase.
Good AI implementation often uses it to bridge specific parts is traditional automation, by 'sandwhiching' it between automation tasks. An example I can site, reading and identifying systems affected by security advisories, but it's only triggered when the traditional automation fails, and it alerts the maintainer that 'system has failed.'
This is something the developer MAY need to fix, but traditional this means that the work would fall back to being hand written for a time. However, if the AI is successful it can be left to run with basic monitoring for several months allowing the developer to note if it was a one off, a consistent reoccurring error, or a new standard that triggered the automation.
Given the scope of millions of individual analysis-es a year, this allows for a .1% to be reduced to a 'review and approve' for 90% of those cases, 'minor fixes of AI work' for about 8% of cases, and only 2% requiring hand tooling. Those are rough roundings since I'm remembering the numbers off the top of my head, but we're talking about reducing work hours from impossible to only mildly impossible here.
Thus the reason I say the correct implimentation of AI is key.
Also, many AI success stories are of more traditional automation or machine learning that just tacked on AI elements to get management behind larger initiatives, or to provide a way to simplify complex data for 'quick reports'.
The concept that this tool can just replace programmers outright is a folly, one recently admitted to by Sales Force and Microsoft.
It's got the rough skill level of a overcaffinated sycophantic intern looking to impress you.
Remember that, treat it with the respect, care, and massive distrust of its 'judgement' that it deserves, and you'll be fine.
Signed,
The cyber security expert who's spent the past three years cleaning up behind AI.
As a member of the minority here (conservative Trump voter), just a note to say that this is one of my top sites for content and thoughtful reading/discourse. I find it all incredibly interesting and devour every article, whether I know anything about it or not. I consider myself enriched by what I read here, including the reader comments, and fully appreciate the different and varying perspectives. Thank you, Nate, for your efforts on this!
Agreed, and as another Trumpy, I too am having a good time here. Smart people on this forum, fun topics.
Love the summaries, and mostly agree about the fun ones over the year. Thanks for all the interesting topics, looking forward to more in 2026!
For what it’s worth, I may be in the minority that thinks footnote #11 is a shame to have to say out loud. Personally, I love the nuanced approach to SB posts (it’s the primary reason I’ve enjoyed the writing over the years). I would think that it’s a big draw for at least some paid subscribers, since Nate et all do it better than most. Maybe it’s the policy nerd in me, but I enjoy those “you’re mostly right, but let’s talk about the 15% you got wrong” critiques.
Doesn’t scratch that partisan itch on political posts for lots of people, so I guess that’s the world we all have to live in.
I agree with this perspective. One thing that Nate taught me, back in the 538 days, was to be more critical of my (and other's) statistical intuition. I was one of those people who was guilty of treating something that had a 75% chance to occur like it was as good as 100% likely, and I was not good at really understanding that difference.
As I've grown, I've continued to apply that mindset, appreciating that teasing out magnitude errors (like bad vs really bad) is a much more difficult skill than teasing out outcome errors (bad vs good). So I always appreciate nuanced conversations that help better hone those skills and develop better instincts.
Hi, Nate. I've read you other places but began subscribing to the newsletter in 2025. I've enjoyed it immensely. Happy New Year!
Nate did some good Substack Live appearances this year. I think at least a couple of them came from the SB feed. Hope to see more of those in 2026. Also love it when the chat function is up and running during high profile events. Lastly, don’t sleep on Winter Olympics coverage. Best-on-Best hockey combined with fraying international relations makes for traffic boffo!
When you do your qb rankings how do you account for the drastic change in the rules especially the rules which allow the offensive players vast amounts of freedom and limits the hits on both receivers and QBs?
Congratulations to Nate, Eli, and Joseph on the continued success of the newsletter. The remarkable post-election statistics are a testament to the value of this style of “data journalism,” cringe as that term may have become.
Thanks also for the end-of-year recap. Fun to review the year, and the “tick-tock” style seemed to flow a little better than the top/bottom N of last year.
Critical question for SBSQ: Once you reach 100 months, how will you name the SBSQ episodes?
How successful/what will be the impact of the 3 downstate gambling licenses? Which facility will be the most successful?
I have never been very confident I know what "meta" means, but I think this comment may be an example. Online endnotes, such on this Bulletin, drive me crazy. In books where there are actual pages, they are tolerable. But online, toggling from the text to an endnote and back to the text seems impossible. Am I missing something?
If you read the newsletter in the browser, rather than in the email, you can just hover the mouse over the footnote number and it shows the footnote contents where you are.
Thank you! I thought I might be missing something,
SBSQ: Can you explain in a game theory way why you voted for a communist to run the city you live in?
Nate says, "I have a considerable backlog of AI-related ideas" --- great, go for it, I'd say! I was dismayed to see that there already was a post that I missed about AI Jan. 27 ----- before I got clued in there was such a thing. I guess I figured it out in July, and dove straight in at the deep end and do daily work on learning AI and prompts and reading about it. This is HUGE. And political? Last week some woman here said no one should use AI because it would "kill the planet," and I quote. I had a few things to say about that, never mind, I'm better now ---- and then the very next day Politico had an article saying Americans all hate AI!!!!!!! Meaning "decent" people, presumably, that is, leftists. So now the situation seems to be the Left wants us all to hate AI and the Right --- is for AI? Those of us who have heard of it, anyway. Darn, that was quick polarization! I think the pro-AI side is the better one to be on, and if leftists are eager to be anti-AI, it's remarkably anti-intellectual of them. Weren't they supposed to be the elite? I am so confused.
My wife and I lived in the West Village until recently and our daughter lives in the East Village. East or west preference may be age dependent (mid 60s vs 30). Either way let’s all agree never ( or at least only rarely) go above 20th street.
The reason for the asymmetry is because Biden was running for office *again*, and Nate was saying it was a bad choice (something that, in my opinion, was obvious to everyone except for deep partisan hacks). For Trump, I don't see much reason to talk about his mental decline at this point, since his own voters don't care, and he's not up for reelection. Only way he's leaving that position is when his time is up - one way or another.
For Biden, I don’t see much reason to talk about his mental decline at this point, since his own voters don’t care, and he’s not up for reelection. (He’s also not the current president)
Nate has not talked about Biden's decline qua Biden's decline in a long time that I have noticed. Biden's decline has been mentioned in the context of discussing the forces within the Democratic party that had their heads buried in the sand and made extremely poor strategic choices as those people are still there, their heads seem to still be buried in the sand, etc, and that has ongoing relevance for election analyses. (This is perhaps a bit like the use-mention distinction in philosophy, if you wanted to read up on that.)
That said, I do think it would be worth Nate doing a post on Trump's decline at some point. I suspect Nate's concern is what he stated in this post: that there are already many such takes and he doesn't think he has something distinctive to add. I certainly don't think Nate is biased in favor of Trump, or that he thinks Trump is as sharp as a tack, or that his decline is irrelevant for politics or policy.
Wow, you didn't read (or didn't comprehend) my comment or the comment I was responding to, and then have the lack of self-awareness to end your comment with a criticism of how people don't read things.
Pretty embarrassing for you...
Yeah, we should keep an eye on that. The Biden decline has put us on notice that we are not supposed to be in deep denial anymore.