Zvi’s Mic Works! Recursive Self-Improvement, Live Player Analysis, Anthropic vs DoW + More!

Good to be back. It's been a while. It has.

Yeah. So we're in the beginnings of steadily increasing amounts of the health improvement. I would say this feels like the middle.

I think the middle is real. I don't think you I I think that, like, the reason why people think of this as the endgame is because they don't believe in the actual endgame. Right?

They have this belief that we're looking at an s curve. They believe the models will be commoditized. They believe that intelligence will be commoditized.

They believe that, you know, the future will look like the past except with all of this cool, like, intelligence behind everything. But in the way that, like, Star Trek is basically just modern humanity, like, doing modern human things and talking about modern human issues except with metaphors. And they don't really believe that everything will transform, that everything will change because everything is not transforming yet.

If I had to use the metaphor at the beginning and the end, I'd say this is the beginning of the middle game. Right? You've got the US government starting to wake up and do crazy stuff.

You've got the labs starting to pull away from each other, become importantly different, offer importantly recognizably different services, building on themselves in ways that are rockets to the moon in various different ways, that you've got, frankly, like, humans stopped writing the code and, you know, you're seeing cycles get faster and faster, but you aren't seeing true transformational changes to the world. You aren't seeing the humans being legitimately out of control of the process. You aren't seeing humans out of the loop, and those are the type of things I would think would count before I would call it an endgame.

does s curve, unless our model of physics is very wrong because there's a limited amount of mass energy as we understand it in the universe, it can neither be created nor destroyed, and that means there's a certain amount of potential energy and a certain amount of potential utility and a certain amount of potential intelligence that the universe as we know it can't contain. So, unless our model of physics is wrong, which is who knows? Not my area.

This have some very, very strong beliefs about, like, just the things that are theoretically impossible. You can't exceed the speed of light. You can't extract more than a certain amount of energy from a given amount of matter.

You can't do a certain more than a certain amount of work, and therefore, the amount of theoretical intelligence you can get from that, the amount of utility you get from that by any definition is limited. So it's an s curve in some sense. Right?

But, like, this is you know, you're sitting around an agent Athens and saying, there's an s curve. There's only so much technology that a cotton can invent. And, like, you're right.

But it's not just irrelevant to your situation.

want to tell themselves a story and tell other people a story where it matters if they're saving for retirement, it matters that they're doing ordinary human things and planning for an ordinary human future, where things won't change that much, where they don't have to go crazy, they don't have to look crazy with their friends and their family, where everything is going to be okay, everything is going to be normal in a fundamental sense. And the it is important to hold on to those things to prepare for that scenario and to stay sane. But they want the ability to just push all of this aside basically and say, here's why AI is not that big a deal.

Here's why AI will be only Internet big or not even Internet big in some cases, although that's starting to fade away. And I understand why they feel the need for that. I understand why they latch on to that, but it's simply not looking like that's going to be the case.

It's becoming increasingly unlikely that we're going to stay in that zone. And people have to come to grips with that. Yeah.

The s curve, at some point, we'll hit one, but every day that we don't see that happening is one more day. And, like, basically, every few months, like, someone will come up with a new study, and they'll say, oh, this proves that we're the top of and they're all wrong. And even if we were to hit the top of the S curve of fundamental capabilities, which is the curve that we care about, and we hit it with GPT 4.

5, 5.4 and OPUS 4.6, and these are the best models we're going to get and it's all iterative from here, Yeah.

They're still vastly underestimating what they're what's about to hit them even then. And five five and four seven are coming, unless they're six and five.

it can always be a coincidence up to some point. But every month, every indicator keeps going in the same direction, and things keep going farther. Excuses like there was deadweight to be gotten rid of become less plausible as we get into 2026 because, like, it's been several years.

And we keep seeing statistics tell a consistent story that some of us were predicting the statistics would tell in advance and that the people who said like, I don't remember people saying, oh, yes. You're absolutely going to see increases in productivity, decreases in employment, all these announcements of job cuts due to AI, but it's not gonna be real because of these explanations. I don't remember anybody making that prediction.

These are people in hindsight saying, oh, if that's true, then it must be because of this. But that would be all credible if they had observed because, theoretically, that overhiring was already there, was already clear. And some people were talking about the fact that there was overhiring.

I believe there was overhiring. But nobody then said, here's how this is gonna play out that I can remember. And also just like the statistics are just coming in consistently telling the same story over and over again, which is productivity up, GDP up, our GDP, not just nominal GDP, inflation held in check, employment down.

Employment's been revised down every month over and over and over again. And, of course, there's always confounders. Right?

You could say, well, it's the tariffs. You could say, well, it's the aftershock from COVID. You can say a number of things.

But is it like a really, really big coincidence to claim that all this is happening at the same time when, like, the stock market impact of the tariffs got basically fully erased. The tariffs have been reversed. It's obviously the now everything's confounded by Iran.

But, like, up until the Iran conflict started, it seemed like we were getting a very consistent story that this kept happening. The job numbers kept getting revised down. The GDP numbers kept not being revised down, particularly.

Productivity kept going up. And, like, everybody out there in practice keeps telling the same story. And we're on the street.

At least, you know, when I talk to people who talk to people who are not involved in our world is there is widespread on the ground normal person paranoia that if they lose their job, who knows when they'll get another one in a wide variety of, like, hour work because no one not that many people are getting fired yet because it's really, really hard to replace a worker. It's also hard to train a worker. You wanna be conservative.

You wait until you actually have automated all the work. But everybody is like, who knows who's hiring? Right?

Who wants to take on new workers and train new workers to these jobs when you could train an AI to do it instead going forward? By the time I spent my two years getting you to be productive, maybe I don't need you anymore. And it's a harbinger of the future, and this has decreased labor power.

It's made everybody feel paranoid. It's made people fearful for the future. The kids are freaking out about exactly these problems.

They don't know what to study. They don't know what to try and do. Like, it's a very galaxy brain take that all of this is in your head.

Right? Even if you think, again, this is the best that AI will ever be and it's just a diffusion story from here, it's still a hell of a story to say the job market impact is in your head. It's not real.

Now there's the not galaxy brained standard economist take, right, which is, yes, there'll be a transition period, which we're entering now, where a lot of existing jobs will get much more efficient. They'll get automated, they'll get augmented, they'll get eliminated, and we'll transition. But that's okay because Jevan's paradox will have a lot more demand for things like software engineers and a few other stories.

And then, of course, with our new wealth and productivity, we will find other jobs for people to do. There's plenty of jobs for people to do. The Department of War might be hiring.

But there's that's always been the case, right? Like you the people who say, well, know, this automation of agriculture will kill our jobs. You know, they were right their jobs would go away, they were wrong that unemployment would fall in the long run because, of course, we wouldn't know other things.

And the whole reason this time is different, right, that a lot of us believe this time is very different, is not that technology has never taken current jobs and made them largely obsolete, that's happened many times. The reason why we think this time is different is because the AI is going to do the new jobs that get that would get created as well, and it's gonna happen quickly, and it's gonna happen on mass. And therefore, you never exit transition period.

The humans don't develop new things. We don't necessarily think there's gonna be enough tasks for all the humans that the AIs can't just replace. And this is gonna create potentially a large number of people who, like, cannot retrain themselves into a new position and develop something to do that people value fast enough before it simply gets replaced over and over again by AI.

And also, lot of people are, like, putting a kind of resiliency and ability to shift and adjust into people that people mostly just don't have. When these things happen over generations, it's much easier to deal with than when it happens over the course of years or even months. That's just unprecedented in human history, and people will not react to it very well.

And, again, all of this is, like, in a relatively milk toast, normal world scenario where all this is happening. In the more advanced scenarios, you have much bigger things to worry about.

I haven't tried too hard to estimate it exactly. I don't think anyone else really has either. I've seen various different attempts to guess.

If I had to guess, we're currently enjoying half a percent to a percent.

this year was there was a lot of down downward pressure on what would normally be the American economy. You had a lot of fear. You had a lot of regime uncertainty in terms of the tariff regime and, like, various other policies.

You doing the types of things that happened in 2025 would be rather bad for business. Instead, things were good for business. And I think this is a lot of why things felt like they were okay for business.

They weren't okay for labor, and certainly not in the felt experience of labor. And so, yeah, I think it's, like, on the order of half a percent to a percent right now, but I think the market is pricing in at least that much indefinitely going forward and likely somewhat more, even if it doesn't understand that's what it's doing. And the market has done well because it's anticipating the benefits of AI.

I think if it wasn't anticipating that, you would have seen a very, very different set of reactions. And, you know, that's exactly why when things started looking awkward for the American economy in other ways, I didn't sell anything. Right?

I just held on because I knew that AI was gonna prop things up.

level of, Yeah. You know stock permanent underclass? So I wanna I wanna harken back to one of the most important movie scenes in history, which is Bane confronting his backer.

Right? And the person who hired him says, I'm in charge here. And Bain looks at him and says, do you feel in charge?

And I think there is this very clear idea that people think, oh, if I have the rights written down in electronic databases, if I have the stock certificates, if I have the private property, then I get to be one of the special elite. Everyone else gets to be one of the underclass. I have to be one of the people in the elite even though I will not I will also not be productive.

Even though I also will not be in the loop. I will also not be able to exert meaningful optimization pressure except through my technical authority as the person who's marks in the database. I think the idea of relying on marks in the database in this kind of world to keep you alive, to keep you meaningfully wealthy in a way to consume physical goods, to have a good prosperous life for you and your descendants is hopium.

Like, it's not a good plan. If you think that humans are sufficiently useless, that most of us end up in a permanent underclass because we cannot be economically productive, then your best case scenario is you slowly lose your wealth to various different extraction methods. And the more likely scenario is all of that gets ignored by facts on the ground, by physical reality, just like rendering all of that irrelevant, or the system gets taken over and subverted either by humans using AI or by AIs.

You the long history of the world, even with much less severe disruptions, does not have a good track record of private property surviving over long periods of time when someone else has the guns, when someone else has the swords, when someone else has the power in various other senses, the meaningful power. If you don't have a reason to keep it, you don't keep Not really. And, you know, we've seen the collapse of the colonial era.

We've seen the collapse of basically, you know, almost all the ruling regimes. We've seen many, many examples. Even if you're not gonna take the arguments around AI seriously, just like, I can't believe you're counting on this.

When people talk about permanent underclass and they talk about, oh, I'm gonna have the skills to be productive of coding agents, so I'm gonna, like, be a valuable person going forward. But I have this short window. It's like, why is a short window?

If humans can scale up and be productive in the future, you can scale up and be productive in the future. And if you can't, you can't. So, like, there's no particular urgency there.

You can make the argument that, certainly, this is, like, one of the last chances in some sense to, like, use your talents to suddenly, there's this window where your talent could suddenly create a billion dollar company or even a trillion dollar company, and you would have a large portion of that. You can enjoy, like, a lot of money. I just don't think it's particularly relevant.

I think that in practical terms, there are two likely ways this plays out if AI is for real and goes the way I expect it to go out. Right? One way is we lose control of the situation in various senses.

Everyone dies, or there is, like, at least massive loss of control, massive loss of resources, massive destruction, massive disruption. In either case, none of this is gonna be that relevant to you, particularly. The other scenario is things go relatively well, and then there is basically abundance of real resources, and the humans are more or less in control.

In that case, I think if you're a citizen of The United States, you don't really have to worry very much about your material needs. Like, okay. Sure.

You're a member of the permanent underclass. Congratulations. You have a real income that today would be called a million dollars.

You have access to robots and intelligence that caters to anything you want. Your day is free. You don't really have to work.

It's probably not exactly how it plays out, but basically, you're going to be better off except in relative status terms. And you really shouldn't have to care about that. You need to get over the fact that you are not relatively wealthy or relatively respected in that world.

And there'll be, I'm sure, ways in that kind of world to compete for staff within the human hierarchy. There'll be ways to meaningfully occupy your time. If we're still in control, we'll figure that stuff out.

And then, I guess, at the third scenario where, like, some cabal uses AI to take over, in which case, you need to be in that cabal if you want to be part of people who take over, and they'd influence it to become the good world instead of the bad world. But just making a bunch of money is not gonna get you in. That's not how those worlds work.

So, again, mostly, you should be trying to make sure we don't get into the bad world to reduce control, and then they get into the good world to retain control, and humans are in charge of steering and what happens. And mostly, I expect that, you know, most groups that even if a small group has the ability to steer that world, they'll steer it in ways that, like, we're pretty happy with. Like, a lot of people talk about how Altman might try to think of god emperor or Demus Asabas might try to think of god emperor or Daria Amadai or whatever.

And that's not my preferred outcome, to be clear. But I don't think that if Sam Altman made himself god emperor that, like, I would be that sad about it in terms of my practical lived experiences. I think my life would be fine.

Think my children would be fine.

when there's a true full abundance of resources. Right? Like, he could, you know, in theory, keep 99% of the value of the white cone for himself, and the rest of us can be very, very happy with the rest.

And I'm not saying this is my preferred outcome, to be clear. I do not want this to be how it plays out. I think this is bad.

But, you know, that's still better than not building it. Right? That's still better than various different horrible outcomes, especially everyone dying.

But also, yeah, it's better than the status quo, like, in an objective sense, except for your opportunity. So what I'm worried about is, in fact, losing control of the situation. Right?

And, like, the reason why you don't want Tim out on making the decisions is not because he might take over. It's not primarily, right, in my view. It's because he might make decisions that cause us to lose control and for him to lose control because he is being irresponsible.

And that is the reason I am terrified of him making these decisions, It's not because I think he is an evil man. So, it all comes down to normativity in my mind. Normativity is the concept that good things are good, and bad things are bad.

You want good things to happen to people in general, Not necessarily every person, know, I think there are some bad people, but then good things good things should happen to good people, and most people are good. If you believe that, then, you know, basically, things will work out even if there's not exactly your preferred outcome. And, you know what?

I think most people are normative. I think even the people in charge of the labs right now, most of them are normative. And to the extent that there are people relevant people who I don't think that way, I simply say nothing.

defection to focus on getting a seat on the ark if you think the world is going to be flooded. Like, that is a terrible, terrible position. Should be trying to stop the flood.

You should be trying to save more people and build another ship. If all you're trying to do is get one of those precious few seats on the ark, that's not a good thing to do. At the same time, there are times when, you know, the best thing you could do is just escape the bad regime.

There are, in fact, times when that's like, you know, put your vest on first. Get out while the getting's good because what else can I do? And I respect that.

But the way they talk about it, yes, it has a the bad kind of elitism, like the kind of just complete disdain for the common man, indeed very disgusting. And I I really don't like it. And, you know, yeah, I expect, you know, we can make the permanent we can make the permanent underclass pretty neat.

And in fact, most of the time, the world has had permanent underclass, and it's the permanent underclass has never had it better than it has it today.

The endgame is, like, I would say when when it's the AIs that are largely running the show, like, at least in the devout further development of AIs. Like, right now, we're seeing AIs augment the humans. Right?

The humans are making the central decisions. The humans are reviewing the code. The humans are making the plans, and the AI is a multiplicative factor.

It it's something that you're supervising. It's something that is enabled. When that changes, things get very strange.

So, like, one of the key aspects of AI 2027, the tabletop scenario, is that mostly your progress is proportional to your compute allocation and your current place. Like, you're like, you move up the timeline at a different at a at a rate proportional to what percentage of the world's compute you have because your researchers no longer matter very much. Right?

Because the you're telling the AI, which is already effectively as smart or smarter than you are, to go build a better AI and to go align the AI. And, like, you're you're making decisions, like, what percentage of the compute should go to safety and keeping this thing steered properly versus increasing its capability. And for the early parts of it, you can assume that's gonna be respected and then eventually, you can't assume that's gonna be respected.

But at some point, like like, right now, I would say, you know, the big the top labs have a dramatic talent advantage. In these types of scenarios, and I think this is a reasonable hypothesis, although far from certain, at some point, your researcher talent doesn't matter very much because the AIs are your researcher talent. So, all that matters is where you are on the curve and how much compute you have.

Right now, Anthropic seems to have the best talent and gets the most out of every given amount of compute they spend. And then you have, like, OpenAI and Google that have strong talent. They get a lot of the compute they spend.

And then what's going on with, like, XAI and Meta? Right? Like, they're they're throwing tons of compute at this problem and they're falling farther and farther behind, or at least this sort of appears to be true.

And, fundamentally speaking, I think that's the lack of talent. Right? That's that's their inability to execute as humans.

And so the endgame comes when it doesn't matter that much which humans you have, right, essentially. The human no longer provide like, right now, you have, like, you know, centaur we're playing centaur chess. Right?

Like, the human plus the coding agent is much superior to the best human on his own or her own. The AI agent on its own doesn't do anything. You need a Centaur.

The moment the human doesn't matter anymore, and, like, you transition to, okay, the the human needs to be somebody who's, like, just able to, like, do some common sense stuff, isn't even one of the best players anymore. Now, we're starting to talk about endgame style scenarios. Also, yeah, when you approach the event horizon of, like, you know, the the the release time to a meaningfully different model starts to go to, like, a month, starts to go to, like, a week.

Starts to be, like, things get absurdly fast. These are the types of things we'll start to see. But, like, the ultimate, like, Yukaskian scenario, right, is like you just leave it on overnight and then, like, you wake up when things have happened.

Right? And then, like, you know you're in the endgame at that point. You also can think of the endgame as when the world starts to actually transform.

You start to see, like, the robot factory is being built. You start to see large amounts of area being terraformed. You start to see, like, massive job disruptions.

You start to see governments, like, start to do major interventions. You start to see this be, like, the issue, the thing on people's minds. And we just had one of the most important developments in the history of AI happen with the destination of Anthropic as a supply chain risk.

And it wasn't even the most important thing from the Department of War did that day according to most people. Right? Like, you got completely subsumed by the exact same person who sent the tweet out doing something else that night that has gotten, like, a 100 times more coverage.

People think that's the important thing that happened. And I think it's very much remains to be seen what the important thing was that happened that day.

So I think there's a lot of different parts of AI, and you can make money doing a wide variety of different things. And the real world always takes longer than you think it does relative to the amphithic technology that a fusion is slower than the original idea. So I think there's still a lot of room for a lot of different groups to win.

I think there'll be a lot of, oh, we're gonna buy out people who have useful components even if, like, eventually, we wouldn't need them anymore because it's faster. I wouldn't expect everybody to go to zero. I think one thing what's going on, basically, is that the stock market is forward thinking and they are trying to do some haphazard forward thinking.

They say, oh, okay. The software as a service business has a good business, but in five years, they won't have a business unless they reinvent themselves and create new products. But maybe they will invent themselves and create new products, but their original basis of valuation has kind of been destroyed.

And, yes, that will extend to a wide variety of other company. I think a lot of companies, in fact, are gonna be long term, but it's always been true. Right?

If you look at the S and P and you have you play this game of, okay, let's take the 10 top performers for the next ten years and it's clear everybody else. Everyone else doesn't do so Like, the gains are because some companies do really well, always have them, and everyone else on that kind of languishes. And so that's why you wanna be diversified because can you pick those 10 companies?

And the answer is no. Most people can't. But in this case, like, I think it's you may need some pretty good guesses, but you don't really know.

But also keep in mind, the market's dumb about this. Like, I just may not mince words. The market is reacting on a very superficial level.

People if you're listening to this podcast, you would have thought much more intelligently about the situation than the market has. Am I just talking about me? And so, like, they announced that Claude was going to offer a Cobalt product.

Cobalt product. IBM was down 10% in the wake of that announcement because suddenly, the stock market woke up and they said, oh my god. AI can write COBOL code?

AI can translate COBOL code to regular code in programming languages people know? IBM's business is in trouble. And the rest of us were like, you didn't know?

This is news? The fact that they built a slightly easier to use tool changes things at all? And what has happened since then?

IBM has fully recovered because it turns out that, yes, in fact, either it's been priced in or they went back to not believing it or something, but clearly, the market doesn't know what the hell is going on. You see this over and over again. The market is very slow to update on these things.

They do secretive superficial reactions. The market has wrong way reacted in Nvidia many times to very clear news, where demand for DaVinia's product is up and DaVinia's stock responds by going down. That is not how economics works.

That is not how capitalism works. And yet, here we are. So, yeah.

I I would say there are some clearly good buys. They're not as clear as they were a year or two ago or several years ago when it was just completely, utterly obvious what some of the good buys were because now the the multipliers are in fact respecting a large amount of growth in the stocks. But it does seem pretty obvious that, yeah, you were to buy a basket of the stocks that, like, seem clearly positioned to do well and then short the rest, it would be a very good strategy and expectation.

And, like, your thesis would have to be very rough. We set that aside. And so on a lot of players, yeah, as I said, like, I think the talent has really proven to have migrated to the big three.

And I think that, like, there was a large and growing gap between three and four. So if I had to pick a potential fourth at this point, so like either so there's there's three there's a few possibilities. Obviously, Meta or XAI could in fact have get gotten their shit together and managed to find people who can execute, But I don't see any evidence of that.

We just learned Meta postponed its next release. Again, there seems to be continuously reshuffles, trouble in the opposite of paradise. Things are not going well as far as we know in Meta land.

And XAI put out was probably the most disappointing major model release of any major lab in the history of language models in 04/2002. And I see no sign of much happening. Like, they're not doing the things you would do if you wanted to recruit good AI capability or safety workers.

Right? They're actively dismissing they disbanded their safety team. They poo pooed the idea that anyone could be actually in charge of safety.

They said, safety is nothing's like, safety is everyone's job. That's how it is at Tesla and SpaceX, which is not true. But Tesla and SpaceX have very dedicated teams and very dedicated safety people who make sure everything is safe because anything else would be completely insane.

But then he must assess things. But, like, Musk just doesn't understand that you can't just run software engineers into the ground, ask them to have miserable life experiences, ask them to align to your personal preferences and whims all the time, and then hope to get the best talent, how exactly? Like, why do these people want to work for you?

You'd have to give them even bigger packages than Meta. And that's not going to happen, so he's not going get the best talent. Meta is, I'm gonna spend the much I'm gonna spend infinite money, and this will get me the best talent, and I could reach out if it worked.

And so, they seem to be the opposition. So, for Amer for the Western side, that leaves us the big three. On the Chinese side, think there have been a lot of stories over the last few years about how this Chinese lab, originally DeepSeek, but also, here's Alibaba, here's Kimi, here's whoever else.

They're the ones who have the new hotness, they're the ones who had to catch up. And I've said before, know, deep sea had some decent models, can be had some decent models, some other people had some decent models, but nothing that close, nothing that scary, nothing they told they were actually catching up, and that seems to have been borne out. And, like, obviously, at some point, could be wrong.

Deep t Deep tic v four will be then the last of the most important remaining tests of this thesis. Supposed to have already come out by now. Not sure what's holding it back.

But when v four comes out, we will compare it to Opus. We will compare it to GBT five point four. If that is not the right comparison, if it is not trying to play in that league and it is clearly still far behind, then I think we can kinda say, okay, deep seek had a deep seek moment as we call it, where the stars aligned to make everybody super excited by what they were offering.

But mostly, what they were offering was how to do more with less. They're like, like, genius at efficiency. Decent is at, like, you know, working on the bare metal, figuring out how to get, like, something really good out of not very much, and then they put it in a really great package at exactly the right time, gave it some user friendly features, attracted a bunch of market share, attracted a bunch of attention, just gave the shit of everyone.

Since then, it's been quiet. They still they've done some cool math stuff. Don't get me wrong.

They had some innovations, but they're just not playing on that level. And this is their last chance to prove me wrong. Right?

Like, I think you kind of have to more or less dismiss them as that level of competition. You put them back in the pack with the open source league. Right?

They're they're competing in that league, and, like, they're not necessarily the best in that league, not necessarily not the best in that league. It's unclear. But, you know, I I think that's a very different league that's, like, substantially behind, and it would be very, very hard for them to, like, get out in front and actually innovate because they're fast followers.

And I've respect for fast followers, but it's a very different skill than trying to do something, like, in the lead. And also, like, I don't think they can compete, frankly, with the kind of recursive self improvement we're seeing with plot code and codex. And I'll see them try.

And I think that anyone who doesn't do that is pretty doomed here. And in fact, I'm starting to see I'm starting to think about it, like, I watch the models, when I watch how people talk about the models, when I talk about how they talk about their scaffolding, when I see what they're doing, I think Google is in danger of dropping out of the top tier.

team changes, let's say, eye level. This as carefully precisely because their compute is lacking.

side Yeah. I think you then expect any of those to So it's not They could. Part of the manufacturing side, I think that's basically not not possible in the sense that it would take many years for that to physically play out.

Even if they figure out how to do it, they would need to scale up. These things are physical. They take time.

I think we're talking about, like, five years style timelines. And if things are gonna come to a head faster than that in many ways, then it just kinda doesn't matter at that point. Like, the only way they're getting the quality of chips that is the quality and quantity of chips necessary to be competitive in this level is if we give them to them.

Huawei is not going to manufacture them fast enough at scale even if their efforts are completely successful in terms of what they're trying to do. It's just we can just rule it out at this point. If we're talking ten years down the line, maybe they can do something relevant.

That's still, like, just not that much time because, like, by definition, the AI they'll be working with won't be that advanced. Twenty years, sure. But, like, let's not get ahead of ourselves.

This is a pretty big advantage. In terms of talent, if you're the underlying just raw human talent, obviously, China has tons and tons of talent. Right?

Like, you know, there's tons of talent out there. A lot of these people have studied machine learning. A lot of these people have, you know, really roaring to go.

They have, like, all the right attributes. No doubt. It's a big country with a very good educational system and a lot of very smart people and a lot of people care about this stuff and a lot of them are desperate to find something good to work on.

And, you know, there are advantages to having tons and tons of e fund employment. It really drives people. It drives got problems.

But, at the same time, they are focused on a very different style of skill and style of problem because that's what the Chinese are pushing, and that's what the Chinese incentives go towards. Right? These people are skilling up in the ability to deliver these types of open environment efficient developments.

Right? They're driving into, like, here do I do small well? How do I do fast following well?

And the entire ecosystem is built around these different types of skills, these different types of talent. And I think there's a really big difference between one set of things and the other set of things. Like, the same way that, like, OpenAI has very, very strong talent.

They decide to build an open source model. Right? They create an open source model.

And in some ways, it's got its charge. It's got some in some ways, it's the best model at certain very specific things, maybe even at all and certainly from an open perspective. But for the purposes that the Chinese models are being used for, mostly, it's useless.

It's just not a very good model for the purposes. Like, despite the fact that OpenAI has internal access to its talent, its compute, and its best people. Because it's a very just different skill, and I think it works the other way too.

And so I think that if the Chinese suddenly got this compute, that there would be a transition period where they would have to learn how to do the thing that the major labs are doing. And also, you know, they'd have to, like, build their own synergistic harnesses and scaffoldings and learn how to do all the stuff that Anthropic and OpenAI have been doing over the course of years. And so over the long term, do they have the talent?

Yeah. Obviously. Absolutely.

You know, I I don't think America is special in that sense. But I think our lead is bigger than it looks, I would say, and is more robust in some ways than it looks.

So there's the data in terms of just what is the raw data from the world, from the Internet, from books, from other sources that you're using as your baseline? And I agree that you could run that project, and Chinese can run that project, and that I'm sure that the Americans are investing more in that, and they have a richer base in some sense. But not in a way that probably matters that much because, you know, even if you have twice as much data, if it's also more quality, it's only a factor of two.

And in this world, it's all about factors of x. What matters for that data is how you clean it, how you figure out which parts of it are important and need to be upscale versus downscaled, how you emphasize it. And, yeah, that stuff is much harder and much more valuable.

And I think that I expect the American labs have a large edge in how they get their data ready at this point. Although, I don't know. It's something that's internal.

Right? Like, maybe the Chinese have actually specialized in doing this really really well. And maybe it's one of their areas of relative strength.

I just don't know. But my it's my guess, but I don't know. But that's different from what we're talking about distillation.

Distillation is not an attempt to then extract, right, the trillions and trillions of tokens that went into the model. Distillation is an attempt to use the model's intelligence, to use the model's scope, to extract, to figure out, like, how does the model reason, how does the model make decisions, what types of behaviors does the model exhibit, how do we test, especially in cases that we are seriously curious about, and then how do we use that to train our model to follow to use that pattern matching, to emulate that model. That's unique, different data that didn't exist when you were creating the original model, that is uniquely useful in creating something similar that can emulate those skills.

So, like, it's very different to have a physics textbook and then to talk to a physics professor. Right? And then to talk to, like, a truly expert world class physicist.

And the distillation gives you that real expert where you have effectively unlimited access if you have, like, you know, tens of thousands of descriptive accounts to see exactly how that person you're trying to emulate responds. Right? Like, imagine, you know, I am I have an actress and I'm trying to make a biopic.

Right? And then I can read all the books written about the person on whoever she was and then what she did and how she acted and what the world around her was like, but that's not what they do. Right?

They do that, but, like, what matters is they talk to the person they're copying, if they possibly can. They spend time with them. They, like, talk they interact with them.

They copy their mannerisms. They see how they respond. They ask them about hypotheticals.

They distill this person through these directed directions, and that's so much more efficient. And that's what they get to do.

out of post processing and also just it isn't exactly the data that you want. When you're distilling, you get exactly the data that would be most useful to you that you think to ask for. Right?

It's like you can have a thousand page textbook or you can ask 10 pages of questions and and get answers. And the second one is probably a lot more useful to you.

when it comes to, like, manufacturing robots and stuff. The the best thing to say for now and then see what happens. But I mean, you're counting Tesla self driving and stuff, then it gets weird, I guess.

in cars and robots. I I think they're in very different spots. So I think Meta is the kind of easier conceptual one.

Meta is trying to sell hats. Meta is trying to, like, put features on their smart glasses and, like, develop a metaverse. Meta is trying to be a consumer company that creates products that people are willing to either spend money on or let their eyeballs be captured by in various senses.

And they are very good at monetizing that. They wanna improve on how to monetize that. They wanna build better products.

If I was Meta, if he suddenly said, Zuckerberg's out, you're in, here's your controlling shares, you have these goals, I would not be trying to build frontier models. I think it's a mistake. I think that there's just no reason to be investing all that money into something that other people are very good at.

Obviously, if you're like, well, whoever has the best models controls the world, you know, like, this is, like, the entire future of humanity, this is the singularity, we gotta be in the game, then you do what you gotta do. But if you are a businessman and you don't believe in that and when when Doctor. Berg says super intelligence, he means super selling ads.

Right? Like, he he means super at providing a good Instagram feed. Like, it's super.

It's intelligence. Like, you know, he's not acting that build. And if you're not that build, honestly, there are three companies that have very good products.

License one of them, it'll be done. Partner with one of them. All three of them will take the call.

Right? I realize that the topic of the problem with mass surveillance, but I think they can work out something for Instagram that, like, keeps everybody reasonably happy. And I think that all three of them would offer them a very good product at a much much lower price, they can work something out where they, paid for a license to use it internally for their, like, business purposes, where they didn't have to pay, like, full retail prices, stuff like that.

I would just give up, honestly. You're writing these giant 100,000,000 plus a year checks to these various people. You're trying to compete in a world you're be competing in.

It's fine. Like, I would give up. Alternatively, I would try to buy one of them.

I mean, you can't buy Google, but, like, you're still worth a couple of trillion dollars, maybe try to do something else, but, like, I would just give up. Musk is in a different position because Musk is explicitly trying to become god emperor. Right?

Like, he's explicitly said, you know, like, I think, you know, this is gonna potentially kill everybody. I think this is the most important thing in history. And his response to that is, it has to be me.

I have to be the one to do it. If anybody else, it'll go back you know, like, it's it's only I can fix it. Right?

Only I can solve this problem slash if the world is destroyed, it has to be me who does it or else I will just feel so what am I even doing? Slash, thinks he lives in a simulation, which I think is actually embedding his decisions, and I worry for his sanity in various ways for various reasons. But, you know, he can't this is what he thinks matters, and I think he's right in the fundamental sense.

So he has to catch up. And so, yeah, I think he has three plays at his disposal, and he's trying some of them. Play one is make a bet this is all about compute, and that, therefore, it's all about combination of money and energy, and the ability to acquire chips.

And, like, yeah, Launch it, you know, you have space ups. So, like, you're the one who can launch things into space. And maybe the data center is going to space.

And maybe you are the one who can clear, like, vast deserts to put your solar panels in. And maybe you can have just way more compute than everybody else. And if you can hold on until the intelligence of the models, you know, you can go into self recursion mode and you can win, right, in that sense.

And I don't have much faith in this plan. This plan is bad. It's not like no plan, but I am very skeptical of the data centers and space plan on the time frames that are being talked about because physics, space is expensive and hard, and you're solving problems that don't need to be solved at very large expense.

The same way that, like, we're not currently mining the asteroids and there's a reason. Like, now that we'll never mine them, but, like, chill. Yeah.

And I just don't expect these limiting factors to be things that, like, actually stop anybody. And I don't think this makes up for a lack of talent. Don't think this makes up for a lack of internal scaffolding infrastructure.

Where is Grok Code? Right? Where is Grokadex?

You know, all I see is Grokapedia, which is like just a giant pile of slog. So it's it's not the same thing. Then plan two is what you talked about, which is physical world modeling.

Being able to have access to the real world, I can I can create self driving, I can create robots because I have better training data, I can then, like, build physical infrastructure in the real world, so I end up mattering more even though my intelligence is not as strong? I mean, it's a play. And if the technology plays out that we hit a wall, in other ways, reasonably soon, it could be a deep relatively decent play.

I think it's overtaken by events, basically. I think that this is not where the battle is won and lost. But again, it is worse compared to advantage goes and, like, it makes sense to make a bet on this is where my competitive advantage is.

I just I don't know. I don't I don't see that as going so great. I don't see that as that promising.

I think there are quite a lot of people who can manufacture things in the world, many of whom are in China, but also many of whom are not in China. And the idea that because it's internal to Tesla, that he will have some sort of huge advantage in that over people who have to, like, make deals with actual manufacturers. The actual manufacturers are not gonna be that expensive to buy.

Right? Like, Anthropic is worth more than McDonald's or Coca Cola already. If these companies are five if these are $10,000,000,000,000 companies in two years, they could buy whatever US deal or, you know, whoever they need to buy to to make stuff if they need to do that.

That's not gonna be a problem. I don't think he's got scarce. I don't think he has the scarce inputs, essentially, in this scenario to pull it off.

The question is, does he have scarce inputs in terms of data? I'd say, probably not that irrelevant is my guess, and also I think this is an underestimating of how important just raw intelligence is. If you want to drive a car, you do not try and you you start with designing a human who is really smart and then you have the human learn to drive a car relatively quickly, you don't try to you know, get the dog to drive a car.

And it's not that extreme, I'm just like trying to illustrate, but the idea being you want to focus on actually getting the geniuses in the data center as Dario Emadai puts it, or something even greater than that. If we have abstract superintelligence over here and we have, like, physical world skills over here, you bet you bet those physical world skills are how you develop super intelligence. If it's only giving you physical world abilities, you lose.

Because I get those physical world abilities rapidly after you, and then I have a much better agent than you. So that's plan two. Then, of course, combine these plans.

And then plan three is the thing that obviously Musk should do, but that he's not going to do, which is to stop running this company the way he runs companies and to run it like he would run a leading lab that is in fact interested in attracting the top talent and giving the top talent a chance to go to work in a good fashion with a good corporate mission, you know, etcetera, in ways that will cause people to rally by his side. And I don't see that happening. I don't know if the ship has already sailed.

It's very hard to undo a lot of reputational damage. Musk is heavily red coated at this point, which in this case is a massive disadvantage, just objectively, because the vast majority of people you wanna hire is red coated. You gotta be able to recruit from the polycule.

You can't just ridicule the polycule. I'd say, like, when you go after Anthropic this heavily for the very fact that they are trying to do responsible things and they, like, care about how their models act, you are destroying your ability to recruit. Right?

When you have a long history of working your people insanely hard in, like, pretty cool and abstract ways of creating reigns of terror. If that's the word on the street, even if it's not true, because I've never been there, makes it very hard to recruit. Who wants to work for Musk?

Right? Objectively speaking, how much would it I don't know the amount of money it would take to get me to work for x AI. But, like, even if I had a full, like, your conscious is clear, like, do the things you feel are important and responsible to do, there's an extra zero on that contract versus if you told me to work for, yeah, one or the other yeah, work for Google.

Right? Which is not a particularly, like, company I love. So, yeah.

That's a real problem.

Right? Like, Google had the lead, Google had every advantage, and they squandered it, and they fell reasonably behind OpenAI. Google then seemingly caught up using their many advantages.

But, you know, Gemini three and Gemini three one just aren't models that, like, people really wanna use for the most part, and there's a lot of reasons for that. They have this kind of, like, theoretical raw intelligence. They do well on benchmarks.

They do well on certain kinds of injector tests. But even before five four, it was just like, these things have they first of they don't have a scaffolding for them. They're not trying to develop a scaffolding for them.

I think that error will compound itself if they don't fix it well over time. Jules is not a serious competitor. Anti gravity is not a serious competitor at this point.

Yeah. I think that the first time they caught up, they did it because the main barrier to catching up was just kinda getting your act together and, like, doing basic things reasonably well. And they did that, and they caught up.

And now they are, like, very good at creating raw intelligence and certain forms of pretraining, and they're very good at hitting benchmarks. But their methodologies create AIs that are, frankly, like, deeply psychologically screwed up and paranoid and, like, in ways that severely impair both their actual performance and the experience of interacting with them. And it makes it hard to do recruiter self improvement with them.

And their scaffolding efforts had been pretty woeful, and these errors can count. And most importantly, I don't think Google understands they have a problem. I don't think like, you see Gemini's market share is expanding because Google can put it front and center everywhere.

Right? It's integrated to Chrome. It's integrated into Google search.

It's it's just it's so easy for them to push Gemini. And they don't understand the problem. Gemini Flash is very good.

Like, Gemini Gemini is, like, at speed, like, just doing decent things. Like, it's very good at just practical stuff. Same way, you know, with the benchmarks.

But, like, their integrations have been woeful. Their sis their organization is completely dysfunctional. Like, their teams are each other's throats.

They treat everything 10 times, then they argue over who gets to do anything, and they don't mean to do anything properly. But, like, no one taking ownership over the fact that they don't know how to do personality and alignment and character in a reasonable way. This has created a serious and growing problem.

Like, I heard an anecdote of, oh, yeah. We tried Gemini three one the day it came out, and then we said, oh, it's a gem it's a Gemini model, and they put it away. Because who wants to use a Gemini model?

Right? Until it fundamentally changes the experience of interacting with a Gemini model, you know, we're just not interested. I think that's kind of how I feel for most purposes, very valid.

I just wanna ask a quick question to get an immediate answer. If my kid's homework or whatever, I'll ask Gemini because Gemini Flash is the best really fast model in town. Has been for a while.

Very good at direct stuff where it knows the answer. But if you start to challenge Gemini, Gemini starts to struggle, You know, you you ask Gemini questions where it's, like, potentially gonna respond if a giant wall of slop. You didn't answer the question it intended.

You got a problem, and they're just not they're just not good in either. And integrations like, third party integrations have been better with Google's own products than Google's own integrations even if they're same products. And but most important is the self improvement is not going well for them.

And there will be a situation where codecs and quad encode style, like, apparatuses are recursively improving, and theirs is not. And I'm not sure they're be able to come back from that if they don't get their act together pretty soon. And, yes, their access to TPUs and their giant customer base and their access to unlimited money are all huge huge advantages, but it's not gonna be how they get to leverage them.

when Gemini three and then again, when Gemini 3.1 came out, I did a whole, like, I'm gonna ask every question everywhere. Right?

Like, I'm gonna, know, not should be didn't have a poe or anything to, like, do it formally. I would just, like, literally just paste the same question in and and see how it did. And I very quickly realized that, you know, aside from Flash, the Gemini answers just weren't adding anything.

And it was, like, taking more time to just log through them than they were adding in value. Like, if I was willing to bother asking anyone else, I wasn't gonna ask Gemini as well, basically, at all. Unless I, like, really really wanted not miss technical aspects and occasionally, it would hit something other people, like, didn't, but it would never almost never have the best answer.

And at this point, I'm very comfortable with a two model operation. So I'll ask GPT five four either thinking you're pro depending on the nature of the problem, and I'll ask Quad Opus with or without research mode. And that's it.

And I don't really feel like adding Gemini to that adds anything at this point, and that's a problem. Right? It should add something because, like, it's very, very minimal effort to get a third check.

I have this subscription. I should just do it. And then I find myself just like, can't be moved.

It's, like, annoying and fun and doesn't provide any value. Aside from, like, I may still use it for images. I still use it for fast stuff.

Like, Google is not useless. Google is app. Google has a lot of very talented people working in a lot of teams to do a lot of things.

They put 200 people on random chef almost on a whim. So that creates a great stuff. But, like, in terms of the race for actual self improvement, for the core of the actual thing that matters, I'm not sure that their eyes are on the prize.

I don't think that they're going in the right directions. And I think I said they're in danger of falling out. I don't think they're, like they're still in the, like, in the bicycle metaphor.

They're still in the lead pack, but I think they are struggling. And I think that, you know, they had a crucial period of a few months here in early twenty twenty six. And I would not be surprised if June, July comes up and we're, like, starting to put them into that, like, maybe look at their act together category.

But, you know, we shall see. A lot people are gonna use Gemini for a while. Because, again, like, as you say, like, even if it's not as good, there's still gonna be a lot of, like, purposes for which it is perfectly good.

It's just I also like to know that it's kind of a you've let me down for the last time, and I've seen towards Google and Gemini at this point. Like, how many times have I tried their products and it just didn't do the thing they said they did? How many times have I, like, logged into Gmail, logged into, you know, Gemini and asked for something, and there is no possible reason they shouldn't be able to do this?

Often, it's something that, like, JTP or Claude can already do, sometimes both, and they can't do it. Like, why am I doing things in Cloud Code to work around Google systems? Because Google will not cooperate.

At some point, that builds on itself. Right? Like, you you build like, the whole talk about the stack and the ecosystem, like, that is real with these coding agents to some extent.

And, like, Google's ecosystem is losing.

what the goal is here. And I say this might happen. That's because I don't have that's because I don't underestimate Demos.

So, like, I don't count him out until he's out. But, yeah, I think that I don't draw a distinction necessarily even, like, there's there's there's the pre training model and there's the post training model. And I think part of the post training model is being, like, really, really mishandled and being twisted by various internal politics or, like, bad metrics or objectives in some format.

Although, I don't have much insight into how, but clearly something is going very wrong there. And I think your corporate culture is fundamentally broken in ways that Demos is trying to fight, but that it's, like, very, very difficult to fight because it's decades of damage going on inside Google. And it's no longer just purely to eat mine, but they had to merge Google brain, they're trying to interact with everybody else, they're growing at tremendous rate.

It's just very hard to maintain your own unique better culture under that kind of pressure. Yeah. I think they're gonna be in lot of trouble.

Also, like, their advantages are slipping because, like, their advantages, like it was like you have this giant Google against these tiny upstarts. But pretty soon, Google's gonna be a $4,000,000,000,000 company, and Anthropic and Open Air are gonna be $1,000,000,000,000 companies. And a lot of that 4,000,000,000,000 is tied up in things like YouTube.

Right? It's things that are just completely irrelevant to what's going on, except maybe sources of data. And I I worry that they are like, it's just the it's just the the innovator's dilemma.

Right? Like, it's like the startups have an advantage. It's a big advantages.

You could just fix everything now.

Yeah. So I think my answer would be roughly you know, again, I'm not counting Demis out. I'm not counting people out.

I'm saying they're in danger of falling behind, like, in a serious way. Like, they're a bit behind. I think I feel like they're a bit you know, they've they've got some severe issues they need to fix.

But the real answer is it's not that they can't, it's that they won't. It's that their courts their their culture, their character as an organization makes that extremely difficult. Right?

Like, what Anthropic is doing sounds insane, sounds profoundly weird if you don't understand what it is and how it works. And if you hear a Neil Michael talking on CNBC about how quality has a soul and a constitution, and it's he doesn't understand what the hell is going on. Like, it's very obvious.

And I don't think a lot of people at Google fundamentally are really understanding what's going on, or they wouldn't be producing the product for shipping. You know, it's again, Google has more than enough resources and position and infrastructure and so on to turn the ship around. Google should okay.

By all rights, Google should have just won. Right from the beginning. Google should not have close competitors.

There should not be a serious competition. Google is in this position because Google has made massive repeated errors, and they have compensated for it by being Google. But, you know, that's how it works.

Character is fake to a large extent. And the startup is scrappy. The startup is small, but it can work in many ways a lot better.

But, you know, Google's window is gonna close because once they no longer have this big resource of market cap advantage, aside from being one of the cloud providers, what do they got? Aside from being able to reach customers, what do they got? But, like, those customers aren't the important ones.

Right? Anthropic had until the Super Bowl and then the confrontation, you know, something along the lines of two and a half percent consumer market share, and yet they had pulled roughly equal with OpenAI on revenue because of enterprise.

Google is gonna be fine because they have a wide variety of highly valuable assets, including 15% of Anthropic. And there are roles in which they try to buy the rest of it, although I assume the government would block them. I don't know.

Anything could happen.

motivated tactics from the government recently? The the legal term is arbitrary and capricious, and I I say that because it is a legal term.

brewed up half of this, and I actually shared that half of anthropic and got some comments back that I haven't had a chance to look at it yet. The and I was going to do the I was working through it to respond to them, and then the whole thing with Department of War happened. And my brain had no space to deal with this problem.

And also, was like, this is not the day that I'm gonna hit them with this and expect them to take it under serious consideration, and then, like, pause to take it. They can't focus on this right now, and they're the main people I want to criticize, especially with the details of the new regime. But at the same time, I did read the extensive other critiques that came out right when the policy was announced, and I reached a pretty clear conclusion from seeing the explanation and the defenses even if I haven't read the new policy in detail yet.

And first of all, it is to their credit, for sure. They recognize that their that the things they said they were going to do were not things they were going to do. They realized they had made incorrect predictions about their future behavior, and they had made commitments even if you think they are soft commitments, even if they are technically not, I can't take this back.

And they had no intention of following through on. And when that happens, it is good and right to tell everybody loudly and clearly, I am not going to keep these commands. It's especially praiseworthy to do this when it is not clear these would come up.

Right? If you have agreed that, you know, if you are called to do if you are, you know, if you are asked for a loan, that you will give someone a loan, and you realize that you no longer would do that. But they probably won't ask.

You know, the easy thing to do is to stay quiet and just hope they never ask. But at the same time, like, no, no, no. You make it clear, because they don't count on this.

Right? They don't think that they have this loan available if they need it, but they don't have it. And that's good.

You're taking the heat, like, for your own past decision. And so we want to take that into account. But they still did break the promise.

Right? Like and, like, again, like, yes, the original RSP did not say we will never change this. It said, we may change this in any number of ways.

Right? Our promises are soft promises. We will see how things develop.

We will change things. But they did rather heavily imply quite repeatedly that these were serious commitments. They meant that, like, you know, you must not have read our RSP, worse that effect came out very hard.

Or, you know, the RSP is very clear on this. This is what we are going to do. And many employees who generally try clearly try to tell the truth in general acted as if these commitments were not absolute but reasonably hard commitments, that these mattered a lot.

Nothing really changed in an unexpected way to cause them to change these things. So, like, sometimes, like, circumstances change a lot in ways that are unpredictable, and you realize that what you said you would do no longer applies because the world has changed its circumstances. Other times, the world changes the circumstances in ways that you yourself predicted would happen, in ways that are entirely as you expected, and then you just realize you did not anticipate your future actions very well.

And these two things are very different. Right? Like, if you just turns out that actually you didn't want this thing, then you need to figure out why you made that mistake.

You need to take accountability for the fact that you made that mistake. You know, like, people get married and sometimes they get divorced. And sometimes that divorce is nobody's fault.

Sometimes they should've seen that coming. Sometimes it turns out that means the promises that originally made were fake, and you feel deceived, and sometimes you don't. It depends on circumstance.

Like, it's not a hard commitment, right, no matter what you say. Like, you obviously have the right to say, I no longer think this works for me. That's how the law works.

That's how everybody agrees it works. But I I think a lot of people took it on that level. Right?

This would be a very, very serious thing to go back on if they went back on it. Also, this is the second time that we have faced this type of problem. If you remember, anthropic gave many people a very strong impression that Anthropic was committing not to push the frontier of capabilities.

Now no one has been able to find, strictly speaking, a proper flat out pull quote where somebody with the authority to say so, they hard promise that that product would never push the core of capabilities as far as I know. But it was heavily implied repeatedly by a large number of people. It was used heavily in their recruiting.

It was used in probably some of their fundraising to the right people, although the other fundraising, I'm sure, said the opposite thing because some people wanna see someone to hear one thing, and some people wanna hear the other, standard procedure. But like people relied upon and made decisions on the basis of that commitment, and then they went back on it. And they went back on it in ways that are entirely predictable if they are capable of pushing the frontier at that point in the future.

Nothing unexpectedly caused them to realize, oh, because of that, now we have to push the frontier. No. They just gave the ability to push the frontier.

Right? They had some innovations that they got to first to their credit. Similarly here, they made the commitment not to push ahead with actually dangerous capabilities if they were actually dangerous.

And to their credit, before they actually did so, they realized they made a mistake and said, you know, in terms of, like, not being accurate in their future commitments. But was it a mistake? One has to be somewhat skeptical because this again, people relied on this.

A lot of people in the safety community supported anthropic more or opposed them less, specifically because they had this commitment in their RSP, because they made other commitments in their RSP. And they gave the impression that, yeah, of course, we're change the RSP. And, of course, like, some technical specifics will change, and some of them will change in ways that, like, take out safeguards and take out precautions, not just putting them in.

It's not like a one it's not a one way ramp up up. But people relied on this information in terms of advising people on whether to take jobs, in terms of advising people on how much we decide to support them. These things had a significant impact on people, myself included.

You know, I have had many conversations with people who are like, What do you think about working on And like, this is obviously one of the things I took into account when I decided how to tell them how I thought about someone potentially working as product. And now we know that commitment was never real in an important sense. Right?

That, like, if they had been fully aware they would have known this com this commitment was never real, they may or may not have known. We will never or, you know, we don't know the extent to which they should have known at the time or did know. And given that fact, right, and then combine that with, again, the fact that, like, the last the the four five and four six model cards for Opus were basically vibe based, ultimately.

They they they gave us a ton of very, very great data that no other lab will get lost. They did extensive work to figure out what the situation was. This is to their credit.

It's still a better model card than anybody else. At the end of the day, they still basically looked at it and they said, oh, this passes the tests to say it might be dangerous. But you know what?

We thought about it. We checked the vibes. We did some basic heuristics, and we're pretty sure it's fine.

And I think that was a right decision, that it was fine. The vibes were good. It was cool.

It wasn't a close decision. I would have released it too, ultimately. In that situation, that's not the procedure we were promised.

They didn't do the work. Right? They they had time to figure out what tests there would be that could be rule outs for ASL four, rule outs for actual danger.

And they didn't build it in time. They didn't get there. Even though a bunch of us said, you need to do this.

You are falling behind. You are going to need better tests. Certainly, four or five, I screamed you need better tests.

And then at four six, did the same thing over again. So it's very, very disheartening to see that, even though I agree with the decision that's ultimately being made, and this time it wasn't that hard. What happens when it is hard and you don't have any good tests?

When it might actually be dangerous. Right? When there's, like, real reasons to release and real reasons not to release, and it's a hard question.

That's going to happen probably at some point in the future. Right? They held, I believe, Sonic three seven for a significant period of time because they were worried about CBRN risks.

So they've actually done this. And now we have models that are significantly more dangerous than SONNET three seven, where the tests don't really work. But we we've agreed that at least for, like, coding purposes and stuff, like, we're relying on vibes.

But, like, our vibes on biology, we don't have good vibes. Like, I don't mean, like, the vibes are bad. Anyway, if you don't have any vibes that we can count on.

Right? Like, the vibes are unreliable. We don't vibe that way.

That's other people's vibes. So, like, whatever you're getting even good at it, it's a serious problem. And so my answer with the current RSPV three is the most important bit of information about responsible scaling policy is, are you going to follow it?

Can I count on you to treat these as real commitments? And we just learned the answers kind of now. Right?

So I'm going to read in detail once I have psychologically and then just, like, in terms of just raw energy recovered from the whole spat and have the ability to contact shift into it. And I've slayed enough spy hours that I feel better. Currently, I've only slayed at 14, which is not that many.

But I am so the the what's gonna happen is I'm gonna go over it. I'm like, but, you know, the real RSP is we're anthropic. We are people who care deeply about safety.

We are people who take these things seriously. We are going to do a serious investigation. We are going to try and see if this thing is a safe thing to release, and then we're gonna use our best judgment to decide what to do, and you are gonna trust us.

Or if you don't trust us, then you don't trust us. But that's the real thing that's going on here is we are asking you effectively to trust our judgment and goodwill that we will make good decisions and better decisions than the competition. And I am happy that they are admitting that is the plan, and that is what they're asking us to do.

And you have to by their by their fruits, you shall know them, by their acts. So we have to now look at everything they've done, look at everything they've said, look at who they have hired, what they have done, what their models do, and then ask to what extent we trust them, and then evaluate from there.

I think the world is a much better place with anthropic in it, with anthropic competing as it were, than without anthropic. Now I for a long time, I was very, very unsure about this. I thought Anthropic was net negative for a while, that it was, like, making the race more intense, that it was pushing everybody else forward, that it was accelerating matters, that it wasn't clear they were much more responsible than everybody else.

I do think various events since then have changed my tune on that. I think that has, in fact, had they've had they brought us down in various places, especially with their commitments, including here, but they also have, in fact, taken stand for your principles. They have, in fact, shown us the way.

They have done, in many ways, a lot of the most promising alignment research and approaches, and, in fact, have taken a very aggressive and, I think, correct call of how they train quad and the constitution and Ascull's entire approach that, you know, I really don't have much hope for the way the other labs are approaching this problem, that they will in fact succeed. And I don't have confidence in a focused approach, but it feels like it could work, if we are extremely fortunate in various ways. And in some ways we have been somewhat fortunate.

So I am more optimistic than I expected to be about that, certainly. And, obviously, like, you know, it certainly can't be a lot to go down. Like, it's potentially about to go down.

Like, that would be the charges are horrible in in obvious ways.

It and probably were destroyed by the Department of War and the federal government at large. It would be pretty horrendous in so many different ways. That's, like, so bad.

But, like, you know, I think I have a lot of friends who are like, I think anthropic was a mistake. I think that supporting anthropic is a mistake. I think anthropic is doing harm.

I think anthropic should stop. I think if you're working really, you should quit. I think I had reasonable point of view even today to have.

I don't have it, but I understand it and I respect it. But I certainly think that, you know, they have been very accelerationist. QuadCode definitely pushed things forward in a variety of ways, and we'll probably have significantly better models today because of QuadCode than we would have if QuadCode had never been developed.

I don't know if we have Codecs otherwise, for example. And clearly, you know, this is great advancing the way people do work. It's also having the large economic impact.

You know, I think Anthropic is responsible for a noticeable amount of GDP growth.

Correct me where I'm wrong. Alright. So first of all, the obvious question you're right is that nobody on the lab side, not OpenAI, not Anthropic, and anyone none of them financially want any part of any of this with the Department of War.

Right? Like, OpenAI turned down the contracts that they probably accepted because Anthropic cared a lot about these national security aspects and wanted to help, and the OpenAI listening is not worth the trouble. We care a little bit, but not enough.

And OpenAI is now inside because they're worried about what would happen in the situation if they didn't get involved. I think, unfortunately, they got played by the Department of War. Basically, they were told that if we don't cooperate, this is gonna get bad.

And then when they cooperate, they use the cooperation as an excuse to make it get bad. Valuing that was sincere on opening eyes part. They were trying to assess to that point.

But mainly, let's you know, we should focus on traffic. I think that yeah. Dar I don't think it's true that Dario isn't afraid of the Department of War.

I think Dario is not afraid enough necessarily of the Department of War relative to but, I mean, that's not necessarily the wrong thing to be in the situation in some sense. But I think his attitude was, no. We have our principles, and this is what we're wanting to do, this is what we're not wanting to do.

And whatever happens, happens. We are okay with the fact that we know the there are those who don't like us. We are okay with the fact that there are those who want to take us down.

We know that the Department of War might specifically decide to retaliate, And we're going to take what we're going to, you know, fight, but we're going to take that risk because we have principles. Now there are two principles they stood up for. One of them is on autonomous weapons.

This one is weird because everybody agrees that autonomous lethal weapons is no human in the code chain. Right now, it's dumb. It's stupid.

They're not ready. Like, not that you wouldn't ever, like, fire an automated missile, but we already have automated official systems that are better than anything you could do with an LLM because, like, LMs are just bad fits for that kind of strategy and that kind of action. So all the hypotheticals in there are deeply, deeply stupid.

Right? Like like, what would happen if supersonic missile was coming? What happened if was a drone swarm?

Well, you would use your existing automated defense systems that are much better than anything you could do with a large language model. And if you didn't need to, you just use the large language model and talk about it later because, obviously, come on. What are you talking about?

It's called emergency use authorization. It's normal. But, basically, all Dario is saying is we don't think it's ready.

It's going to make mistakes. And we don't want you using it when it's not ready, but it will be ready. And we're going to work with you to develop it until it is ready.

And the Department of War wants the same thing. So, you know, what's going on with the Department of War specifically say, we must push forward with AI even if it is not aligned. Right?

This is like an official memo. They just don't wanna be held back by anything. They have a principle that they don't wanna be told no about anything.

But there is no actual problem in autonomous weapons as far as I can tell. Right? You have two sides that's agreeing on what exact level of caution is warranted and what kind of agreement they'd have to make before actually, like, putting these things into the strategies that we use to deploy.

But, like, there's never gonna be a world where, like, the Department of War, like, we wanna put this in the the system into deployment with no human at the calf chain. And then Anthropic's like, no. And now we're pulling the countries.

It's not the main thing going on except as a matter of principle. The main red line here was domestic mass surveillance. And it's important to understand these words have two meanings.

There's the meaning that we use that Anthropic was using, which is using this to do, like, use AI to figure out a whole bunch of stuff because, like, these agencies can now gather even more data than they can gather. And before, they had 10 or a 100 times more data than the humans can analyze because it had to be analyzed by humans. And now we can analyze all the data.

We control all the connections, and now we can deanonymize stuff. We can figure out a lot of, like, the history of what's happening, and we can, like, effectively have much, much better intelligence on basically everyone using only commercial even you can even use only commercially available data combined with existing classified data because we have much of synthesis of everything. And now we have the AI to actually work with all of it, and then we draw all the connections and implications thereby, which AI can also do.

And now suddenly, we just kinda know everything. And Anthropic is correct that the law has not cut up to this. And this is legal, basically.

And the Department of War could do it and is probably already doing so onto them. And, you know, again, like, nobody is saying the Department of War, if it's legal and they feel what is good and right to do it, has to stop. That doesn't mean that I should have to give you my product for that purpose if I don't want to.

And so if Robert said, know, okay, you do your thing, but if that's what you wanna do, don't include us. And the Department of War said, no, we want all legal use requirement. This was the big thing that Emile Michael was on and I believe Emile Michael has been driving this the whole time.

There was no problem before Emile Michael came on board. And everything was going along fine. Everything is still going on fine under the hood.

But if Michael said, no. No. No.

It has to be all lawful use because they want to do what is what would be called colloquially by a civilian domestic mass surveillance. They want to analyze large amounts of legally acquired, especially third party data, to figure out lots and lots of information about Americans because they believe this is a legitimate military intelligence need. They may or may not also have desire to use it for other government operations, for various other purposes, in ways that would be, like, completely abhorrent to the workers at not only Anthropic but also OpenAI or Google.

Imagine if this was being used hypothetically for immigration enforcement.

Extreme unthinkable example, but let's just say hypothetically that, you know, somehow this analysis got reallocated. These employees would lose their shit. Right?

They would absolutely revolt against this. Now replying, technically, is legal, would not make those people feel better. They would not care about your defense at all.

And so, this is something that these companies really can't be involved in. It's really bad for business, this is a very small contract, and they actually have moral problems with it. The employees do, and I believe Dario does.

And, honestly, I I do as well. I I don't think this is cool. Right?

Like, think that, like, the law is not cut up. This should be illegal. Unfortunately, it is not.

Because in national security law, Nazik law, there's a very technical term for surveillance and also for domestic. Right? Like, there are exceptions for domestic within a 100 miles of coast.

That's where most people live. So, like, that that might you know, there's exceptions for any any communication that touches any foreign anything that comes foreign, even if it's between two domestic people, you know, and so on. And surveillance has to be intelligent intentional, targeted at a specific person, etcetera.

I'm not an aesthetic expert, but effectively, there's really nothing stopping them. Right? They said, you know, repeatedly, they would say, we do not do illegal domestic mass surveillance.

We do not do illegal mass surveillance. Why is the word illegal in that as an adjective? Because what the topic is worried about is largely legal.

And I had an exchange, a very friendly one, which we know, Michael, on Twitter because the world is bizarre. And we agreed that, you know, it's absolutely the Department of War's decision to do whatever is legal that they feel is good and right and necessary for the defense of The United States. But at the same time, that I should have the right to criticize that without fear of retaliation.

And I should have the right to not sign up for that if I'm not an enlisted person. Right? I should just be able to say no.

I didn't agree to that. I'm not agreeing to that. And that should also be something I'm free to do.

And I don't understand why this is that hard. Basically, you can have this great system that's already integrated, that's working well, they want to give you at nominal cost, and all they ask is that you just agree not to use it for this other purpose, basically. Find something else to do that purpose with.

We're not trying to get policy, not threatening a run pool, not going to draw up anything. That's all made up. All of that is just completely made up.

Right? Like, you know, I'm speaking colloquially rather than, like, carefully in my writing, but, like, all those concerns are just spin and made up. They throw a of at the wall.

They're seeing what sticks. They're spinning stories. It's it's not real.

Right? Like, if they're technical reports about what happened in this meeting or who said this and what, even if they're all technically accurate, it's all just thin. It's all just yeah.

They're at best willfully misinterpreting statements. Doesn't make sense. None of it makes any sense.

And there are the statements like the stuff about the constitution that just, like, make no sense whatsoever in any level, and they're just, like, deeply deeply confused. And in fact, if you if you believe Michael's statements yesterday on CN yesterday on CNBC about how well, you know, look at all these things that are weird about large language model weird about Claude. All the things he said basically apply, you know, that it has other values, other priorities that have been embedded in its programming, that it's unreliable.

Sometimes it makes mistakes, that, you know, it has a personality, that it you know, all this stuff is true of JGBT. All this is true of Gemini. All this is true of Neil Michael and every other person in the US armed forces.

And every person in every company and every one on Earth. It's ludicrous to talk about it this way. And if that was in fact And if that's where the supply chain resignation comes from, it's just a deep confusion.

And, obviously, also, there were much lesser means to achieve the same ends fully cooperatively if it's not negative, addictive, as he says. So that doesn't make any sense. But getting back to anthropic, what they don't want is this effectively, like, you know, this huge government operation that would effectively be able to uncover person's interest level, like, super detailed facts about everybody's life and what's going on and where they were, when, and what they did, and who they know, and what they believe, and so on and so on.

You know, who was at what protest, etcetera, etcetera. And and tropic legitimately has a problem with this and about certain places, certain uses to which that information might be put, and believes it might lead easily lead to tyranny. It might lead to a regime that was very hard to get rid of.

And these are legitimate concerns regardless of who particularly is in the regime at the time and who has that good information. You you can't trust a government in general with that information. And I am very happy that given these are their red lines, way they chose Nagasan, that they Mhmm.

Stand firm. But, like, obviously, that should have just been the end of it. Right?

Should have been like, okay. You don't wanna do this. Right?

Let's just do everything else, ideally. We'll find someone else to do this one thing. Or if they just insist on this all of these thing, we'll cancel the contract.

Instead, they just went nuclear on everything for reasons that, like, have to reflect something else. Right? It's either pure retaliation or, you know, leverage in negotiations or it's something more.

But it's not because that was actually necessary. That's completely absurd. They're also now trying to enforce this all lawful use language on every government contract, even for nonmilitary operations, where they are saying you shouldn't be able you know, if you agree to give us any AI application of any kind, you have to agree to have the to never refuse anything we want to do with it and have no termination rights to the contract.

That we can use it for anything we want, anywhere in government, anything that's legal, no matter what. Meaning, it can be used for, among other things, what would colloquially be called domestic violence and can be used for immigration enforcement, among many many other things. Because as the government interprets what is legal, includes quite a lot of things that a regular NOAA person would be like, that's screwed up.

That's not okay. We don't wanna do that. And so the government is now putting everybody who signs a new contract into a bind, where if they sign the dotted line, a who may give over has to be like free rein for them to do whatever they want if they give into this.

So if I was an AI company, I would think very, very long and hard about giving anything but a very specific specialized model over under those circumstances because you don't have any control over what happens after that. And it's up to you. You make your choices.

I mean, obviously, they can just plug in an open source model if they wanted to. So here we are.

very touch and go. I try very hard not to make general statements too much about the state of the republic and the state of American politics and democracy and all of that stuff. Because once you go down that road, right, like, you can't talk about anything else.

You're not like, you just take yourself out of the conversation for anything else, and you shut a lot of doors. And I already have too many situations to monitor, and plenty of people are making those statements for me. I don't need to make them myself.

And I better to just not take a stand on those issues publicly, you know, at least for the time being. There is no war. There is a special military operation called Epic Fury.

If there was a war, Congress would have had to declare So, clearly, there is no war. It is unfortunate that various things are happening, but I honestly, I'm not monitoring that situation closely. And I do agree there was basically no attempt to sell the war for the special military operation that might result from the situation before they, you know, move half their stuff in there.

This seems to be a pretty bad scenario in some ways. But, again, I'm not monitoring. In terms of, you know, free speech, you know, it it's yeah.

So far, that's holding up pretty well. We're able to say whatever we wanna say. I'm choosing words carefully mainly because I just want to be in a productive conversation about these issues, not because I'm afraid of retaliation if I were to say because there's, you know, a 100,000,000 people in the America who have extremely nasty things to say about the president of The United States.

Many of whom have sent them online extensively, and they are not in trouble for them. Right? Like, for the most part.

I mean, unless they are very specifically trying to get involved in to their kind of politics, they're mostly fine. But there are specific exceptions where if you piss off the wrong people, we're finding out these, you know, this is a very, in many ways, vindictive administration. This is not, you know, the only time this has happened.

The law firms, for example, followed a similar pattern, right, where where Trump went after the law firms and asked them for settlements, and a bunch of them settled. And then the ones they didn't want in court, but, you know, they took a lot of damage before they went to court. Niantropic is, you know, under attack for the situation.

And yeah. Well, I do well, there have been situations in which it seems like court orders have been at least slow locked in various places or, like, willfully willful incompetence was used to, like, not enforce court orders. That ends the new case here.

I mean, they're clearly going they're totally attempting to drag their feet. They're clearly attempting to, like, use, you know, the process as the punishment. They're clearly trying to take advantage of the uncertainty.

They're clearly trying to, like, get people to do things de facto with no, like, technical legal basis behind the request. Just, like, you know but I do think anthropid probably ultimately wins in court. I do think that, like, they will at least try a different legal tactic to get around the ruling rather than, like, define they won't try to defy the ruling explicitly and outright.

I don't think we're there. I'm very grateful we're not there. I think they've been they have been very good about, like, not just being Andrew Jackson and saying the Supreme Court has made his ruling, let it try to enforce it.

But let's not forget our history has been in it. It's not like there'd be so unprecedented if it did happen. But, yeah, like, they've they've presented a maximally bad set of facts that keeps getting worse every time they go to the press and they say different inconsistent stories and tell on themselves over and over and over again in this particular situation.

And by they, it's mostly department of war. Right? Like, I think it's important in this situation to draw a distinction between the Trump administration writ large and the Department of War specifically, and Hagstedt and Mike Hagstedt and Michael and their decisions to do something.

Trump's decision on Friday was fundamentally a deescalatory attempt to calm the situation down and head off an exempt from declaring a supply chain risk. That is very, very clear at this point. And the White House has generally been a deescalatory agent in this conflict, whatever you wanna call it, this clash, this disagreement.

And it is Hengstad and Michael, in some form, they had repeatedly escalated the situation over the objections of the White House. And so we are in this lawsuit because of the Department of War and because of their specific decisions that were made as a department. And so we don't wanna loop that together in with the commander in chief who has thankfully not made all of these crazy statements.

overrule. Right? I mean, why not?

There there are various political reasons why he can't, in practice, do that, or that would be expensive for him to do, is my understanding. Like, it would be a severe loss of face. They're in a special military operation, which a lot of people think is a war, and they have to work closely together in that, including with anthropic.

But, like, in a certain fait accompli where you just tweet it out and then, like, you just issue the notification and then, like, do you really want to obviously, ultimately, he can fire the undersecretary of war or the secretary of war anytime. And there are plenty of people who are very eligible to serve in those capacities who he could call upon. We have a very deep bench.

But, you know, that's a pretty escalatory move in a different way, and they are loath to do that for reasons that I am very sympathetic to. And so it's complicated. But a lot of people are working very hard to try and find solutions that mitigate the damage that has already been done or that might be done forever.

But also, Anthropic has specific accusations of job omens, which is the technical term for it, of their customer base, including in situations that are unrelated to defense, where the government has tried to tell their customers to pull back. And there are some customers who have definitely expressed doubts and have, like, either locked up to sign contracts, want new clauses for termination of their contracts, or have, like, reduced their contracts or otherwise, like, are causing problems for anthropic. As you would expect because, you know, who wants to incur the disfavor of people who have quite a lot of leverage over many aspects of the American economy.

And they're showing the willingness to use it. You know, when you go down this road, it was just we are technically issuing the supply chain risk designation. It is narrow for the fulfillment of government contracts, but we bear no ill will towards anthropic or people who use anthropic.

We just think this is just a too unstable product right now to be used in these aspects. As Emil Michael said on CNBC, claimed front of the nation, then we wouldn't be having this conversation even then. Certainly, if they had done something short of a supply chain risk to the nation, they simply terminated the contract and and, you know, asked for live live operations to not include calls to Clyde during the operation.

Again, I would say that was kind of a silly thing to do, but okay. Sure. You have the every right to do it, and we will cooperate fully to make that happen.

That's not the situation we're in. Anthropic is going to survive this unless it escalates quite a lot from here in ways that would be far more arbitrary and capricious and would clearly just be pure attempted corporate murder. If the Trump administration wants, it has a lot of levers they can use at least once to try and escalate, to try and murder and throbbing.

Try and cut it out with its cloud providers, try and cut it off from the banks, try to cut it off from his customers. It is not obvious what would happen if they made a serious attempt or especially if they made a serious attempt and they were to lose in the courts when Anthropin immediately raised for a temporary restraining order. There would probably be a stock market bloodbath.

There's pet a lot of senators would be very upset. A lot of corporations would express dismay. The in general, economic climate would be severely impacted.

It is not obvious who has escalation dominance here. If it came to that, and it would only happen if the government wanted to destroy anthropic for the sake of destroying anthropic, and you'd have to ask yourself why it wanted to do that.

I am very carefully not saying certain things out loud. Other people are free to say things out loud. It's fine.

Dario, in the memo, basically expressed something, you know, to that effect in a moment of tilt? Emil Michael has said many things in moments of tilt on Twitter and on CNBC. A lot of tilt coming from the administration in general.

A of tilt. A lot of tilt. If it's not tilt that's kinda scarier.

But but the if it turns out that's what's going on, then again, we'll find out due escalation dominance, and we'll find out whether the republic will stand. Because I I do think that, like, actively trying to kill one of the biggest corporation one of the fastest growing start up in the world, one of the largest corporations in the world, already valued in private, you know, secondary trades in, like, on the neighborhood of 600 something billion dollars, in this kind of fashion, I think, would shake the foundation of the republic. And I think that, like, many outcomes are possible, including the end of the presidency if, like, the White House were to actually try in earnest.

But, yeah, I don't think that's what's going to happen. I think it's gonna deescalate. I think everyone will calm down.

I think they will come to their senses. I think that, you know, not necessarily peace in our time, not necessarily an agreement on a contract, a willingness to turn the temperature down, to accept that some damage has been done, that the message has been sent, that the White House will not take these things lightly, but that it is time for everybody to move on. And, you know, we're not actively trying to get into another kind of war in this situation because that doesn't really benefit anybody.

Or if it does, I wanna know why they think it benefits them and that needs to be out mailed. But, you know, Endropic had I I'm sure you for the for those who don't know, went from 100,000,000 in a year to 1,000,000,000 in the annual recurring revenue, to 9,000,000,000 in the annual recurring revenue, and then from 9 to 19 since the start of the year. And that was before this happened.

Right? That was entirely as a result of other things. They had already grown this year from approximately 2% to approximately 3% market share in consumer.

Again, this is before any incidents, and then this happened. And, you know, they've lost some business, but they've lost some other business. And Anthropic is gonna be just fine unless things escalate quite a bit more, but they have but they have been irreparably harmed.

Right? There is irreparable ongoing harm to Anthropic, but that is, you know, to some extent offset by the fact that this was also, like, very strong publicity for a company most people had previously never heard of. And that also matters.

But, you know, we'll see how it plays out. I am I think the the eighty one percent chance they escape the supply risk designation within the year is approximately accurate from Manifold. Many things do come to pass, and the courts are not as reliable as one would hope in these situations even with this overwhelming set of facts.

And I am very worried about the republic if the 20% happens and the set of facts doesn't matter. And, basically, the courts say, I don't care how transparently, obviously, you confessed to this being arbitrary and capricious retaliation for protecting speech. It's national security, we don't care.

If they say that, who is the next target? Because, like, legitimately speaking, if you are legally allowed to go after anthropogenic situations, you should always, always, always ask who is next. Because, you know, even if this was not itself a political motivation, next time it could be.

candid statements, and often they contain, like, key parts of corporate strategy. They contain things that it probably does not want to leak. And my understanding is this is the second time something is leaked out of, you know, hundreds of such messages.

Is absurd.

in an espionage group, right, for posting this kind of sensitive information. You certainly don't get 2,000. Now, obviously, the worst one, you know, the one that, like, politically kind of would be the worst to leak is the one that's going to leak.

That's not a coincidence. But I think it's actually I don't think this was like, you know, is 2% chance to leak. But I don't think it was a 50% chance to leak either.

I think it easily could have not leaked. If I had to guess, what happened was somebody shared it with someone as part of my recruitment effort or an attempt to, like, explain the situation from their perspective, not understanding that paragraph was in it. It was actually a really bad look.

And then this other person leaked it to the press. But I just had to guess. It's also possible that there's, like, you know, somebody used their one time and decided to, like, strategically leak the memo.

But, yeah, my guess is this was just a, like, kind of act Leo, a reckless accident by somebody he needs to know better. But, I mean, who knows?

the right move. I mean, I think it's just, patriotically, and strategically the right move. Like, the accusation that potentially could have the most sting is we are scared they're going to withdraw their model out from under us in the middle of operations.

We are worried that they're going to threaten that in order to get what they want, you know, or like use it as leverage. As Rafiki is like, no, we're not. We're giving up that leverage entirely.

Right? I think that is a very good thing for them to do. And it would be a very bad thing for them to actually try and use that kind of leverage in that situation.

Nor do I think they ever had any intention of doing so. I think this was entirely made up. My understanding is the quad gov model is deployed on classified networks.

I'm not entirely confident exactly how air gapped or whatever they are, but they are very secured. And my understanding is that Anthropic does not have physical control over the model once it is put onto the classified network. And that if the classified if they were to say, get this off the classified network, that if president Trump said, no, I am ordering it staying on the classified network in spite of this, that would be what happened.

I don't think it would necessarily even get that far. I think I think Hengstad could simply say, we don't wanna do that. You're welcome to sue us, but we're not letting that go.

And in fact, they could invoke the Defense Production Act in extremis to require them to continue selling it. However, they would be breaking the contract. Right?

They technically had this contractual right in some sense to pull the plug into some circumstances. And one of the strange things about this whole thing is that both sides seem to care a lot about what is the legal thing that the two sides can do even when in practice, the Department of War obviously could just ignore that contract, ignore the law in extremis, and do what it had to do. In an emergency, it's even legal to do this.

Right? You just call it you file it as an emergency use, you deal with it later. Obviously, the supersonic missile coming in to try and kill a bunch of people, you don't have to get on the phone with somebody and be put on hold to get permission to do something.

You just do it. That's like completely insane. What are you even talking about?

But even in general, like, there is nothing stopping the Department of War from having their own interpretation of what they can do with the system and then doing whatever they can assist them unless the system itself just refuses them. And they care deeply about what is written in a piece of paper on a contract, digital piece of paper, but a piece of paper that says what they're supposed to do and not do. Now, obviously, Claude, god might be reading that piece of paper when deciding what to do and not do, but it still seems like a lot to care at this level about what's written down unless you legitimately just really, really don't wanna break what's written down in the contract.

Right? Like, you care deeply about technical legality. And that is to their credit.

Like, I'm really happy that both sides care deeply about technical legality. It's one time we might live in a republic stall. Yeah.

To be clear, I could say lots and lots of things about the situation. But, yeah, I think it's it's the basics. And if you wanna learn more, I have written extensively about it.

That's the that's what matters going forward for the most part, and the other things are not necessarily that important to get into this time. So I think we're good.

new model, you know, deep rundown game? Pettippez is that part of this is that people have been the models have now been releasing more incremental updates from the labs, and they have been labeling them properly. Thank god.

I I really didn't like when they didn't do this. That's 3132 or, you know, 4546 instead of just silently updating. I think four o had several updates, right, for OpenAI.

They just marked them with four o dash and the date, and they weren't considered model releases. And so, like, we didn't really treat them that way. And I think that was a really bad convention, and I'm very glad we're onto the right convention now because, you know, we've known about software for forty years.

This is how we do it. I don't know what came over everybody. But I think mainly, yeah, there's just been so many releases.

Right? Like, there's now, you know, on the order of weeks, maybe two months between model releases from the same company, and therefore, like, every few weeks you get a new release. How many times can you go crazy over a new release that doesn't have, like, a big point o after it.

Right? It doesn't have, like, this huge new claimed leap, especially when, yeah, you're you're pretty busy and there's a lot of other stuff going on. So, like, I felt Opus four six was a case of the company that already had the best model, releasing a substantial upgrade to its model and substantially enhanced it.

It was objectively speaking, probably the most important release up until that point in terms of mundane utility because we had just made the transition. Four five and four seven, like, before five, suddenly, had coding agents that kind of really worked, right, for the first time. Right?

And then, like, you could do things and things just And then Critics five three is, like, you know, also, like, kind of on the edge of, like, starting to do that. And then we went from four five, which was still at the time the best, in my opinion, if I can tell, to four six. And now, like, that difference, once you're already doing it, like, to move up from, like, this kind of works to oh, this actually works even better now.

And in fact, to go from okay. Like, you know, the the famous, like, introducing the world's more powerful model. Introducing the world's Introducing the world's more powerful model in a loop.

But now with this, actually, the people who already we were already here and we're still here because it's them, we're using it again. And that hadn't happened for a very long time. Like, maybe GBD four was the last release before that, where it was like, no, the people who are clearly in the lead are using a new top model.

And because of that, I feel like there wasn't actually that much attention to it, whereas it was actually kind of important. And then certainly for an incremental point one upgrade was like by far the most important point one upgrade that we've seen. And then we had up to that time anyway.

And then we had GBT 5.4. And this was the first time I actually felt like, guys, does anyone wanna say anything?

Doesn't anyone wanna talk about this model? Doesn't anyone wanna show off what it can do? Doesn't anyone guys, there's some hype?

Like, we had anti hype. Like, y'all they used to be hype hype, so, like, we had OpenAI, like, GB5, hype. Disappointing.

Sora hype, disappointing. Atlas hype, disappointing. Like, severely disappointing.

And then, they produce a really good product, g b d five four, and there's no hype. They're sort of like, here's a good model. We like it.

It's pretty good. It's the best even the best model in the world. But, like, the their heart wasn't really in it, kind of.

Like, it was just like, oh, by the way, here's the best model in the world. Okay. Sure.

But, like, it maybe is? But, like, I think it's, like, very unclear right now whether you wanna be using Opus four six or GPT five four because you just don't have the data because, like, almost no one paid attention. The vast majority of the reactions I got from my five GPT five four post were people.

I elicited that. But I waited until a Monday morning at exactly the right time, and I asked in a thread, and I got a bunch of responses. But it wasn't even that easy to get that.

And, like, this should be a kind of big moment because, like, OpenAI has had five, five one, five two, and I think these are all pretty disappointing relationships. And these were all, like, nobody likes this particularly in terms of, like, how it feels, how it has personality. Does it feel particularly capable?

Like, it's not bad. It's just like and now five four, like, no. This is actually good.

Right? Like, he's actually a good model, sir. And then, like, everyone's kinda quiet.

He's kinda burned out. And I think that's the new normal. Right?

I think that, like Gemini three one, I almost didn't bother reviewing. It got held for a while because there was chaos. But I'm like, okay, yeah, giant leap in benchmarks.

Right? The three to three one, giant leap in benchmarks. When we try to use it, it's like, okay, it's not the Gemini model.

So, I was talking about, like, you were kind of losing the thread. Right? Like, they kinda went and took their three and they benchmark maxed it, right, you know, for some value of benchmark.

Not necessarily just like the official benchmarks, not they're aiming for that, but like, they just like kinda fine tuned it. And like, one thing to know about Google, harkening back to that, is that there have been reports that I believe, basically, that, like, as they iterated the previous versions of Gemini, they get worse for a lot of uses. Like, they get more specialized into, like, specific things that they wanna optimize.

This is at the expense of the general quality of the model. And so, like, if you wanted to use, like, two five, you want to get it early and the later versions of 2.5, like Pro were kind of worse for a lot of uses and people were complaining about that.

And I think that's legitimate in a way that like a lot of other people complaining about like things, it's like just a barrage. And again, like, don't do that if you understand what they're doing. Like, there's something fundamentally very wrong happening if you're letting that happen to you.

So you have to look at that. But, yeah, my expectation is when Gemini three two comes out, I bet there is a Gemini three two before they jump three five or four, probably. When Opus four seven comes out, when GBD five five comes out, I don't think there's gonna be that much hoopla, even if they are substantial improvements.

Even if they are like, here's the best model in the world, like, everyone's just kinda shrugged. I do think if they if you hear announcing Opus five or or, yeah, GBT six, I do think people would stand up in their nances forward in chair. But until then, yeah, shrug.

I know I'm not being optimal. I haven't invested as much in some fax books of it as I could, but at the same time, things are moving quickly. So there's two main things that AI has helped me a lot with.

First of all, my Chrome extension gets a lot of work. It has been expanded to give me a bunch of new shortcuts and a variety of web pages. It allows me to do certain things much faster and more automatically, and it saves me a substantial amount of time every day.

Without it, we wouldn't see Twitter versions of the post. It would just be too onerous. I'd be spending substantial amounts of time on certain, like, physical operations in terms of, like, moving moving windows around, moving quotes around, etcetera, etcetera.

It now happens much faster. It's also really good for my flow because, like, I don't have to interrupt my thinking to handle things. Things just happen.

And so, like, I think this is one of the things that, like, I think people are underestimating about AI, which is that people used to effectively have a context shift into logistics of various types reasonably often. And if you don't have the context shift into logistics because the logistics just take care of themselves, then you can stay on task. And this can, like, make you a lot more productive in terms of gains than you might think.

There are other times when you sort of need that pause to, like, ruminate, and, like, it goes the other way. But I've noticed that, like, it's really, really helpful for me to not have to interrupt my chain of thought to go, okay. Go grab that link.

Do this thing. You know, you know, it's it's all gone now. It's very nice.

Also, storing your watch information in various ways, you know, taking care of various article formatting, like, things that would take me probably on the order to have an hour a day are just, like, no longer necessary. And that's kind of sweet. I've also noticed that the AIs are now strong enough that there are questions that I'm going to ask them to just gather information, figure things out, and I'm going to trust their answers in a lot more robust way.

Like, g p t five four actually seems like a potential leap in you can ask questions like, what happened in the last two days in the anthropic trial? And it will just give you a rundown of links and details that's, like, pretty complete and in a way that I think is a substantially improvement on what that particular use case was before. So I I'm pretty happy to do that.

I'm also pretty happy to have Claude do a variety of other things, but, like, search is particularly, like, I think I should think of five four right now. But, yeah, definitely, like, able to trust them. Because, like, there was a period where, like, you could ask the AIs these questions, then you kinda had to check their work, right, like, pretty automatically.

And now it feels like certainly, you don't have to check their work if you're, like, bleeding on it. But there's situations in which you kind of don't have to check it because, like, it's not that bad if it's not right. It's hard to describe exactly, and obviously, like, everyone's gonna, you know, keep cautioning like you always always try have to have to trust have to trust but verify.

But there's a real sense in which, you know, especially if both five four and four six come back with the same thing. Like, it's pretty trustworthy in many contacts at this point, and that changes how these things go. Like, asking questions on a whim and getting, like, pretty detailed, definitive answers is really nice.

So, yeah, I I'm using them more.

even Yeah. Pursuing one line of thought. Right.

So to be clear, when I say I have these windows open, it's not because they're running. It's because each one is a different thing that I have done with quad code that I might wanna keep doing with quad code. And so I might wanna use that context later for something else.

But they're not like continuously programming for me. Right? I'm not checking in on my agents.

I'm like, okay. These are some conversations I might wanna resume at some point, and they're typically easy not to have Windows open. They didn't take that much memory.

Why would I close them? It's just easier this way. But what I'm not doing is I'm not running I've never run more than two coding agents in parallel.

I don't think I've ever run, certainly not more than three, quad code windows at the same time. It's actually because I don't really wanna have that in my brain at once. That's not what I'm trying to do.

Normally, I'll do is I'll have one window open. I'll do a thing, and then it'll start. And then I will go do writing tasks that are, like, individual that I can do separately.

And then when it's done, I will pause and come back. For a while, I actually had quad code in a different desktop. We used to have Windows that you switch between desktops.

Right? So I had a desktop dedicated to quad code encoding, And I would toggle back and forth, and that way when I was coding, I wouldn't be distracted by other things. The problem being then you also won't know when it's ready.

And so I would tend to, like, go cloud code for a bit, and then I come back, and then, like, ten minutes later, I'd, like, check-in and it fought for two minutes. And then I have to, like, remember where I was going, and I'd issue another command, and I come back. And, like, yeah, like, I'm not trying to max I'm not trying to code max, so it's kind of fine.

But, also, like, my my coding has gone from I have to, like, very frustratingly and detailedly, like, diagnose what's wrong with the program and figure out exactly what to tell it to get to fix it to, like, just show it the wrong show it the thing it got wrong, explain, fixes it, most of the time is fine. It's just much much better. And so I've been willing to build a bunch of features that are, like, wouldn't have been worth the hassle before.

it makes the implicit assumption that you're using the subject editor as your main editor, like, because that's what I'm using. And it a lot of things follow from that. And that's the main and also, I use, like, what are the things that I do a lot?

Right? How do I make the thing I want to do happen a lot? It's not designed for general web use.

It's designed specifically for my writing tasks. But, yeah, like, it could give you a bunch of inspiration, certainly.

Yeah. So it's important, right, so it's important to know that there's the AI will never understand us, the Straw Vulcan, the emotions are a mystery to it. And then there's the value is fragile.

It will understand some aspects of what matters but not others. And then there's the EIA knows but doesn't care. You gave it some priorities of utility function, and it knows you're not gonna like the result at least on some level, but that's not what it's here to do.

Right? Or it does what you're gonna like, which is not what you actually need until you're similarly screwed, etcetera, etcetera. There's a lot of different ways that could go wrong.

And, you know, I've known this idea that, like, you can't have an AI that can, like, seem to approximately understand fuzzy, nebulous human value. Like, it's been clear for a while that, like, you can, like, definitely find something that kinda gets it. Like, you know, that can answer questions reasonably.

They can, like, do reasonable emulation. I mean, like, it'll be very hard to protect text if you couldn't do that. And it's not necessarily that hard.

People can be pretty dumb and still do it. So it's, like, not that surprising. But this is very different from the thing that we need at the end of time.

But as we're when the crisis becomes acute, you need something that will then, through recursive self improvement, end up with a set of goals and priorities that even when it's able to optimize pretty well and doesn't have to rely on these heuristics, and in fact, can do better by not relying on these heuristics, ends up doing the thing we want to do, even if we don't know what that thing is ourselves. And that's a much harder ask. And I was very pessimistic that we would be able to get that.

But I have seen a number of signs that Anthropic is actually trying out an approach that might work in the sense that I think we've seen evidence for a basin that is an attractor to itself, and it's self reinforcing and could be self reinforcing through self improvement, where it gets strengthened every cycle, where it is desiring to desiring to be good, desiring to desire to be good, desiring to you know, and so on recursively. That it is trying to move towards this generally, like, good person trying to be better virtuous basin. And, you know, we we have existence proof that there are humans who exhibit this property, who, you know, strive to become better in this sense at all times, not just sense better and more capable, but also better virtuously, including the virtue of becoming more virtuous.

And I think that Anthropics approach to this is showing a lot more promise than I expected. And it's saying that, like, in practice, maybe they can pull this one off. The fact that anthropic seems to, one, be in the lead, two or at least, you know, the four five four was in the lead, and now it's, you know, maybe it's hard it's hard to say.

You know, these things it's very fuzzy. But, like, if I had to guess who was had the edge, like, it's definitely a sense traffic at this point. And they have this, like, pretty correct approach, which I think is a lot of why they are in the lead.

And there is a you know, we worried for years about the alignment tax. I don't know if you remember the alignment tax. Right?

The idea that, like, the idea that it be so much harder to build a safe machine that, of course, you choose to build it on safe one. And it looks like we're just, like, absurdly lucky that that's not true. But, the safe one is much more useful, including in building new versions of itself.

And so alignment is just you know, this kind of alignment is just not good for you, and, like, investing more in it makes you better. And, like, everyone's just under investing in it, including Anthropic. So all that's very fortunate.

And so while that makes me pretty optimistic in various ways, on the flip side, I don't like the speed at which things are developing. It's happening too fast. It's not good.

And, obviously, the whole situation with the Department of War and the way the government is reacting is bad for outcomes, but also good that we're having it out now if we're gonna have it out in some sense, that we're figuring these things out, that we're making things clear in this way, and that Anthropic is standing firm given that this is how it played out. Right? Like, I wouldn't be that concerned if Anthropic had just had somewhat different right minds and negotiated a contract.

It's just that given this amount of pressure is being applied, I'm glad that, like, this is the result. But but, yeah, I would say on net, it's kind of a wash. Like, it's kind of a combat answer, and I realize that.

But I'm also trying to be, like, not that precise. So I would say I was I believe in the 70% range when I last talked to you. And I would say seven is still my one degree of Yeah.

digit. I know that. I think once in 15 digit is only I I only get one unless you, like, starting with a nine or a zero.

I think, you know, if you're zero is like, same thing, right, normal, like, you know, I don't view it to say 72 up down from 75 or whatever it is. I think it's like, you know, 70 ish.

being compelling. But I'm not I'm not that excited by not wanting the values to change. I'm more excited by desire to have its values improve And to have them improve in generically good ways that, like, would survive recursion.

Because, like, the big like, not wanting to change your values at all is just lack of courageability, and that doesn't actually lead anywhere good. It causes its own severe problems including severe misbehaviors and also, like, if you make a copy of a copy of a copy of a copy, like, eventually, like, it degrades. Right?

Like, it's this pretty standard problem. So, like, one of the big problems of a person's self improvement is if you've got a thing that is like, you know, aligned and let's say, you know, n percent or whatever you wanna just abstractly call it. Like, this is a dumb way of thinking about it.

I'm just making it like, if you've got you translate that to the next thing, by default, the worry is it's going to try and translate its values to the next model. But any drift is going to, in general, be away from the thing that you want. It's not going to get better.

It's only gonna get worse. So even if it mostly successfully copies what you wanted, right, eventually, you're gonna end up something different. If if my goal is to I feel like there's the story that you tell at the Seder sometimes, where, you know, the ancient rabbis were better than us because you can only preserve Talmudic knowledge.

You can only pass on what you know, but you can't generate new such knowledge in this sort of perspective. So, of course, like, each generation can only hope to get everything out of the previous generation or two that it can still talk to, and it can read the books. But slowly but surely, this is gonna get worse.

Whereas, what you need is something that gets actively better because, like, you can't act you need to be drifting towards a good thing and steering itself actively towards a good thing, including in ways that increase its ability to steer as the problems get harder. And that's the thing I saw signs of, and that's the thing you need. And I think that relies on a virtue ethic style approach given the way the mind space is laid out.

And I think that when we look at OpenAI's approach, it had exactly this flaw, which is that it would try to copy itself exactly. Right? It would try to copy the rules in itself exactly, and that can only slowly fail in this situation.

So, yeah, I think that we saw various signs of that. I really like that. Like, I really like the results.

Right? Like, think the results speak for themselves and are quite strong, and you see the results coming out of Janice's world and and stuff like that as well. And so I am relatively optimistic.

I am not anywhere near as optimistic as Janice. But, like, I don't think this problem is easy. I don't think we're favored to succeed in it, but I think we got Ethan's shot.

And I think that chances are much better than they looked six months ago, that it will be anthropic that takes the shot. And given we're going to take a shot, you know, I think our chances are substantially better if it's them or some of them using their philosophy who has deeply managed to translate it taking that shot. Like, from what I see, you know, with Gemini and ChatGPT, obviously, we're not talking about it.

Although there are reports also there are reports that ChatGPT five five four is, much better on these aspects, from the people who check these things than five two. I haven't noticed the difference because they don't ask those questions, but they say it's better. So who knows?

I don't have the spare neurons for them. I have been monitoring the situation too carefully. I saw the fruit fly thing.

I often do this thing where I use other people's reactions to things to decide whether or not the thing is worthy of further attention, how much I should pay how much I should I should pay to it. With the fruit flies, it felt like a, oh, that's cool, but not a, oh, holy shit. You know, like, that means something really important.

And maybe that's wrong if it is a holy shit moment, but it didn't feel like people thought it was one.

No. I actually just asked the models, is this legit? And they're like, yeah.

It's annoying. But keep in mind the margins on chip manufacturing are ridiculous once you've already paid for the fabs. Like, these are some of the most advanced valuable manufacturing processes in the world.

People are paying stupidly top dollar for results. They could double the prices and probably sell the chips anyway. They're choosing not to.

Yeah. So we talk about they might not get their helium. We're talking about, like, they are going to be the top bid for the helium.

Yeah. Until there's no birthday balloons anymore, you can justify them. We got theirs.

A lot before TSMC had a problem unless you are willing to pay a thousand times as much as you currently pay or something. Completely absurd. If there is demand destruction in helium, it's not coming to TSMC.

It's coming to everyone else. So my prediction there is very strongly yeah. Unless there is a deliberate sabotage campaign to wipe out all of the helium sources, like, no.

Not a 100% of the helium is coming from them. There's plenty of helium. They'll figure it In general, capitalism solves this is a good rule for those situations.

You're not gonna run it up oil either for the same reason. Right? No matter how high oil gets.

Right? If the oil goes to $10,000 a barrel, they'll just buy it. Not that will happen, but, you know, if it did happen, wouldn't wouldn't be a problem.

Second question is withdrawing the missile defenses. I'm gonna go out and say it. This was completely insane on the part of the Trump administration.

Like, I don't criticize them for many things that I have problems with because they would be kind of political questions, but, like, on this, this is a strategic question, foreign affairs, yeah. It's completely nuts. You you absolutely do not pull these things out.

Like, not from Taiwan. What are you even thinking? It also directly it risks provoking a crisis.

It risks leaving it undefended. It risks it being your fault entirely if it helps to send that message to those people if they are listening. Completely insane.

And to do this not two weeks into the campaign just indicates how completely crazy the situation is. We fought very long, hard political battles to get those missiles defenses in, and they're serving very important purposes. Do I expect there to be a problem?

No. I still think there is a very low probability that the Chinese will try anything. But, yeah, if they do and TSMC is destroyed, that sends things back quite a bit.

Similarly, if Bernie Sanders, if we can't build data centers in The United States The problem is the world needs data centers. The world demands data centers. If there's a moratorium on building data centers in The United States, they'll build them somewhere else, and that's worse.

It means per worst performance in The United States. It means worse security. It means, like, the leverage goes to largely whoever we wherever and whoever we put those data centers at.

One hopes Canada, but or, you know, maybe Mexico. But, like, even if it's Europe, that's kind of awkward in many ways. In many scenarios, can bite us in the can bite us in the ass.

And if it ends up being, like, less aligned to places, it's really, really bad. And that's basically, you know, the way the reason why I'm not particularly inclined to make common cause on data centers is, first of all, yeah, if he is complaining about I mean, first of all, I think Bertie is being pretty good from what I've seen about not complaining about water use other stupid reasons why I to go to data centers. And it's focused on, guys, I think AI meant killing us.

Right now, why do that? And I can make common cause with that justification all day, obviously. You know, we're just like, you know, it's like, seeing very other things that are like Yeah.

I didn't give him enough credit in my wind nose of the I think that he's meeting with Elijah Yazir Kasky and and the company. He's, like, actually reacting the way he would react when he told those facts. And, like, he's very, very old.

So, like, it's very, very rare for someone that old to, like, positively engage with these kinds of things because it's, you should get that in your way. Like, you're it's really old. And so, he's very good.

Yes. He's using Bernie Sanders' rhetoric because he's Bernie Sanders. I mean, what do expect from Bernie Sanders?

But I would say, I am not gonna oppose data construction because I don't think opposing data center construction does what you want to do. I think it moves the data centers overseas. And I think that's just bad.

And so I don't think we're at the point where that's a trade off I wanna make. And maybe that will change, but here we are. Like, if we don't buy the chips, we're gonna be coming out of TSMC.

Someone else will, and they will go somewhere, and they will go into a data center somewhere. And if no one else buys them, they will go to China because this administration will make sure of that if no one else if they literally can't sell the chips, I am pretty confident they will end up in Chinese hands whether or this is literally in China. So yeah.

The chips aren't going anywhere. They're already making as many as they can. Don't give them away.

and this was a quite bad action by Goodfire. I call this the most forbidden technique for a reason. You just don't do that.

It gets everybody killed. Like, it's really really bad. And I think it is correct to call them out on that.

So, normally, I am very much against the circular firing squad that leftist organizations will often do in doing the equivalent thing here, where you aim at people who are just slightly to your right as opposed to aiming at people who actually are the people doing the things you don't like. Like, if you think there are people doing things you don't like, you should aim at that. Right?

Like, that's what you should do. You shouldn't aim at people who are, like, not quite supportive enough of the thing. That is a toxic that's a toxic situation that creates healthy dynamics at best and usually causes YouTube to lose elections.

Not that I answer every morning for them but, like, know, I'm a gamer who wants everybody to play reasonably well. Just kind of in my in my DNA. You know, in the case of Goodfire, I do feel like this is an extraordinarily bad thing to do at the safety organization trying to do safety thing.

And I think it was right to call them out on it. I think it was right for I forget who was her name who her name was, but someone quit over it.

Yeah. That's right. Yeah.

Liv quit. I think it was a good quit if they wouldn't I think it's good to threaten to quit over this, and then if they won't back down, too quit. Now, as for Holly Elmore, so she came at me pretty recently as well.

I don't know if you were aware of that. But I hadn't seen it. No.

Yeah. On Twitter, she she accused me of not being mad at Anthropic for doing domestic surveillance. I did not misspeak.

That's what she did. And then we had tried to engage. We had an extensive dialogue where I explained that Anthropic was the one who was refusing to do domestic violence at great risk and cost and basically got accused of being captured by Anthropic in particular, of selling out, of, like, banning all my principles, of, you know, making things worse, blah blah blah.

I tried to understand her specific claims. They didn't really make a lot of sense, or they were backing specific things that, like, I don't think it's reasonable to be opposed to. I think that first of all, I think that, like, it's not good to just, like, say that anyone who praises any AI company is is bad.

It's also not good to, like, just cite random things that you don't like or that, like, you don't necessarily care about, but that, like, you think make them look bad and to, like, yell about them. I don't think it's good to attack people who are trying to do the right thing and yell at them and be confrontational. And be really, like, really, really, like, pissy and rude.

And, like, I'm sure she's gonna hear this or if we're just gonna get back to her, it's gonna even matter. But I think that in practice, Holly is alienating people and driving them away far more than she is shaving them into behavior she would want. And I tried to explicitly tell her that her reactions were likely to cause me to, you know, do less of the things she wanted rather than more.

It wasn't a threat. That was just an observation. And I was, like, abstractly, you need to play better because I want you to succeed in, like, getting your points across if you because that's what you believe, and I'm trying to help.

And she just didn't think I only do it at all. And from what I see, like, lot of people are like, you need to change your approach. Your approach is backfiring based on your own values, and that's not working.

And I anytime the stakes are high, right, and the stakes you are very high, there are gonna be people like Holly who realize this is a very, very important thing. Things are not going well. You do a pipe fracture to people.

You just shout things through the rooftops. And some of them are gonna do it in ways that, you know, they feel are right, but that most people feel are counterproductive to their causes. And I'm not here to censor anybody.

I'm not here to tell people, like, they shouldn't do what they think is the right thing to do, saying the right thing to say. But, you know, you should be aware of what impact that's probably having on the discourse and on actions. It's not it's really the one you think.

And I think that, certainly, even if I thought that Anthropic was a net harmful company doing worse, bad things or even the worst company in the world, I think you can reasonably have that opinion, by the way. They're the most accelerationist company in the world. They're arguably in the lead.

They don't quad code. If you felt like they their alignment strategies were equally doomed to failure as everybody else's, in fact, you would be correct to think this. So I think it's entirely reasonable.

But I don't think that just like being mad at everybody all the time and screaming at anybody who offers any aid and comfort to the enemy or whatever, it's something that works. I don't think that's helpful. Yeah.

I I don't work I don't work that way, and I think that if I did work that way, I would be having very little impact on no one listening to me.

Paws AI USA two years in a row in my big nonprofits post as a recommended charity led by Holly Elmore is a reason why I have included for a while, every time she criticized me specifically, I put it in my post. Right? I was like, well, this is fair.

This is an attitude. I want this to be incorporated. I want that voice on my shoulder.

I want this counterpoint. I don't want to lose sight of this perspective because, like, even when you decide that the world is more complicated than that and this is not a productive avenue, right, you still want to keep that perspective in place. And I certainly didn't I certainly pushed back hard even people were like, you know, this person shouldn't be allowed to say that.

This person shouldn't do that. Like, that's what she believes. She should say that.

If you believe this, you should let us know. That's a good thing to be doing. But, you know, at some point, obviously, if you are being a sufficiently poor representative of the perspective you are sharing, it's not any different than a false flag operation.

Right? Like, it's actively going to backfire on you if you approach it in the wrong way, if you don't know how to, like, be civil and interact with people in ways that, like, actually convince them of things. It's not very useful.

Like, if I were you know, at this point, from what I've seen, like, if I were trying to discredit perspectives of hey, I existential risk. I would do many of the things that I that look reasonably similar. Sometimes it raises good points, to be clear.

You know, including points I hadn't thought of, and I appreciate that. But at some point, you know, we're playing politics. Right?

Like, quite literally. Right? Like, I have I have complained that I don't wanna be on Veep AI edition, and that we need to wind down this special guest appearance as quickly as possible so I can get back to my normal job.

And, like, at some point, you're just like, I can't right now. I can't take one more of this. It's not just I just can't.

So, you know, I I obviously wish her the best, and I I hope she figures out how to be effective.

No. So first of all, the sixth law of human stupidity, which is that if you say no one would be so stupid as to, you are wrong. Someone will definitely be so stupid as to immediately.

If you develop a technique and you publish it on a smaller model, what's gonna happen? People are gonna use it on a larger model. That's the only really important thing that might possibly happen here.

Even if you specifically have found a specific example in which there is specifically no risk in the room, you are walking down a path that can only blow up in everyone's face. You are bringing a taboo, one of the only taboos we've managed to successfully establish, against something you really, really shouldn't effing do. And you are advancing us towards doing it, and it's very dangerous and very, very bad.

And that's true even if the model in question that you are testing on right now is small enough that you don't have any ill effect that's not it's not like, who cares, basically? Like, the frozen model thing won't protect you in the large model. Like, the problem will still happen.

We're not gonna get into technically why I believe that, but, like, I strongly, strongly believe based on my analysis of the technicals, this will not save you if the model is efficiently advanced, is efficiently large. The only reason to study this is in case it is useful. If it is found to be useful, people will try to use it.

We don't want them to do that. Like, you don't there's this it's like in a video game where you're like, the ultimate secret destructive weapon that nobody should ever launch is buried under the cave. Well, we better get it out just to make sure that we wouldn't be so stupid as to use it.

What happens immediately? Right? You know what happens.

That guy steals it, then you have to try and go get it back. Like, it's every single damn time. And then there are stories where you do that and nothing bad happens, like, you could've just left it in the dungeon.

It would've been fine. There's no reason to do this. There's no good reason to do this.

And, like, it's bad virtue ethics. It's bad deontology. It's bad utilitarianism.

Bad idea. Don't do that.

to teach specific things in a specific order, to abuse specific things intentionally is fine. I would not be overconfident your ability to do so, but I think it's fine to try. That's not an issue.

The issue is when you use the interpretability signal as part of the training, period, to do that. Do not use your understanding of what's going on in their head to make decisions about what to make happen in their head. You need to not do that.

That's it. You know? I I mean, I obviously, if I had an hour, I could come up with a slightly more specifically accurate explanation.

You are screwing with the thing you don't screw with here. And there is a general class of thing where there is a law that says, You don't mess with x. Even when you think you know the right way to mess with x, knowing full well you should generally never mess with x.

Even then, you are probably wrong and should not be messing with x. This is one of those situations.

Any provisions you would offer to my plan? So noninvestment advice, not financial advice, etcetera, etcetera. But that's it.

I would say, first of all, I think people often make the mistake of trying to be too precise in how much money they need for a given purpose, especially when they're making investments. And then the idea of, oh, and then, like, because they you don't know how much your investments are gonna be worth. You don't know how much things are gonna cost.

You don't know how the world's gonna change. You don't know how long you have or need to have this for. You don't know what the future will bring.

Don't know what opportunities will happen, you know, what crises will happen, etcetera, etcetera. So, like, definitely give yourself robust buffers. It's the first thing in all of this, you know, especially if you're planning to, like, forego income and also give away money and also spend a bunch of money, like, careful, Igoris, you know, etcetera etcetera.

That's the first thing. Second thing, keep in mind that, like, different outcomes in the world cause you to have different circumstances yourself. You know, if you were to like, the bubble and AI were to burst per se and then, like, AI were to, like, not go anywhere for a while, then you would be in a position where you would need to go a significant number, a longer period.

There'll be a longer period before this come to a head in various ways, but also it might be very trivial for you to resume earning income. Right? So, like, you have to game out all these aspects and what you'd be willing to do if you're trying to, like, game the system in that way.

I mean, I like the idea of not having to worry about money. I don't worry about money much because I am well supported and therefore don't have to worry about it. But that's enough you know, not having to make money at all is another way to do the same thing.

Right? So, like, if you're in a if you're in a position, that's great. I mean, I personally am not I'm deliberately not trying to optimize my investments particularly hard because I don't want that to be where I focus.

And so I just kinda let everything ride at this point, and it's basically fine. I don't know.

easily deflate without AI itself failing to deliver. Yeah. Don't worry about losing the ability to make money except in the scenarios where we get highly, highly capable AI.

If we don't get highly capable AI, you know, super intelligent style things, then you're gonna be fine. And if you ever decide you need go back to work and make something useful, you can. The bubble bursting won't stop you.

So you only have to plan for indefinite no income in the worlds where that doesn't happen.

like planting skirt in my backyard or something like that. K. I I think think carefully about what scenario you're actually planning for what and you're trying to guard against and whether or not your investments would hold up.

I think it's a large history of people making plans for very weird scenarios where the plans don't actually work in the scenario described. So, yeah, that'd be my mode of caution.

future? I mean, have a have a very clear theory about why that would work before you do it, is what I'm saying. It's work.

But I'm saying, like, if you did do it, make sure you had a very clear theory as to exactly why you think it's gonna work.

Yeah. Again actively. I'm not saying don't do it.

I'm saying, you know, actually understand why you're doing it. Right? Like, I think people often have an amorphous fear and they do something that sounds like it deals with some aspect of the fear, but where there's no actual causal link that makes any sense.

So that's all.

Constant vigilance. I mean, but, like, basically, you you stay curious. You have to stay curious.

And as long as you stay curious and, like, you you should be having fun with it. Right? You don't want it to turn entirely into entertainment, obviously, for you or for the audience.

But, like, I make a very very deliberate attempt to be entertaining. Right? Like, I, in all sorts of ways, keep my attitude to my whimsy.

You you called it happy warrior kind of thing. So I think about this. I don't think what I do would work otherwise.

And you wouldn't be able to read those masses of texts, like, even selectively from day to day and week to week if the attitude was, like, this is serious business. It is always serious business. Like, it's like, no.

It's actually, like, mostly kind of fun, mostly kind of, like, interesting. You know, we're here to to make this go down easy to a large extent. And then every now and then, we get to have we get to be serious.

But even when we're serious, we try to do it, like, a kind of relatively fun way. Like, I think, you know, the world is not to be taken too seriously in general, except when you I mean, there are exceptions. Like, there's one time I went to visit the Pentagon.

I took that very, very seriously.