Ep #42 Recursive Self-Improvement? End of the World? Or an Opportunity…

Kevin and Matt work through the week’s AI safety headlines, including an Anthropic researcher putting a 10% chance on AI ending humanity and Dario Amodei’s follow-up essay calling for slowing down. Then they get into recursive self-improvement, why its slope breaks the assumption of gradual competitive change, and what that means for the tools businesses are buying today.

An Anthropic researcher went public with a 10% chance that AI does away with humanity, implied it’s a common feeling inside the company, and other researchers backed him up. Dario Amodei followed with an essay saying it’s time to talk seriously about slowing down. Sam Altman chimed in, Bernie Sanders wants it banned outright, the administration is calling it a hoax, and there’s an election in a few weeks.

Kevin Williams and Matt skip the politics and go after the technology behind the panic: recursive self-improvement. The slope is the part nobody is planning around. Once a model can pull its own levers, improvement stops being a steady climb and becomes a rock tipping from potential energy to kinetic. Astra 6 to Astra 20 to Astra 2000 in hours, days, minutes. The 2015 prisoner’s dilemma argument holds that the first lab there wins the whole game.

Then they get practical. What an RSI-style loop looks like inside a business today, why constantly shifting models make solutions hard to build against, treating context as code, sharing reusable skills, and Kevin’s cost work: a resources tool that ran $40 per update until he rebuilt it around an MCP, and thumbnail images at seven and a half cents each.

In this episode:

– The 10% claim, who made it, and why the regulatory capture read is tempting

– Why RSI improvement is exponential and what tipping over looks like

– The national security version: what if China’s model goes to Astra 2000 overnight

– Building solutions on a model that quietly becomes Opus 17

– Self-improvement loops applied to your own processes, not just the model

– Why Kevin downgraded Claude Max from $200 to $100