The First Thermodynamic AI Computing Chip - Thomas Ahle

Yeah. So why not try and build a chip that's just inherently random? Meet Thomas Arlie.

how do you know it's actually right? So my background is in theoretical computer science. I used to do algorithms for high dimensional data, locality sensitive hashing.

Then I moved to normal computing to develop thermal computing, also to speed up Bayesian intelligence. Sometimes I think about it as the lovable for chip design. So we take it all the way from your intent, through the design, through optimizing your design, to formalizing and verifying your design, all the way to tape out.

Now I didn't fully appreciate this before. These days, a chip doesn't necessarily start in a factory.

how do you know it's actually correct? The percentage of tests that it got right or, like, even like the fable, it was like, oh, yeah, it got like 80% tests correct. And it's like, yeah, but I talked with the people who made the benchmark and like, yeah, but it did did it get any of them actually right, you know, like Now in chip design, noise is the enemy.

And then you infuse all of this noise, and it'll start to behave according to to to these stochastic differential equations.

So we're it's things like I mean, it's compilers for RTL. It's like the RTL is like the programming languages you use. So if you're building a new chip, you typically don't draw all the different gates and so on.

You write it in programming language like Verilog, which has sort of constructs that are quite it's like a super parallel programming language, which a lot of constructs that map well to hardware. But then there are actually no good open source compilers for this system. So the whole hardware industry, it doesn't have the same open source feeling as like software where all the good stuff is free and there's like this big vibrant market of tools here.

It's much more locked down to like big providers. But people then, yeah, they compile this too and they get like a netlist, they can stop building their schematics. They can also look at it.

You get like these giant things that you can eventually send to the fab. But yeah, so this, but they also need to do like do simulations, for example, for testing, you can do like formal verification.

Yeah, exactly. It's super expensive too. It could cost you what they said, like there was this famous Intel division bug in the nineties that probably cost them between 500,000,000 and $2,000,000,000, and I mean, that's a lot of money in the nineties.

Yeah. And there's I think there are other other examples later on that nearly bankrupted some of these companies like Tinybox that managed to make it through to the fab and it's a really different world again from software where we just, if people fix stuff in prod, you just move fast and break things. That's definitely like yeah.

per seat or something? Yeah. Per for one CPU kernel.

Right. Yeah. Like, say you wanna scale this up in a data center with a million agents running, like, you're gonna like I mean, the computer's already expensive, but not that expensive.

What is that $10,000,000,000 or something, right? Just for those licenses. And I think actually that's one of the reasons also why AI still is not as popular in the hardware space because they haven't been able to train like the models are not as trained to this kind of workloads because they just It's not feasible.

Like you don't have all of the open source code out there to start the training, but you also don't have the tools that they need to learn to use. You don't have the Yeah. All of the Like the task, like you you can't just all of the reinforcement learning on top of it.

I mean, I mean, they clearly are doing some of that. Like, we can also see from model generation to model generation that they are getting better, but it's just it's kind of day and night between like Python or JavaScript. We've been sort of developing these EDA tools in house using AI, having I think I probably have the world records of the longest running agents having like some 20 GPT agents running for around six months now, still making progress.

Yeah. I think for a lot of this stuff, we're relying on the hope of some kind of escape velocity from code complexity that the models are gonna keep improving faster than our code gets messed up. I had those three days of, you know, Fable access and it was really good.

Like it it definitely cleaned up a few things that GBT 5.5 had messed up. But did it though?

Do you think it could be deceptive?

Yeah. No. It's true.

Like, in principle, you could all be be smoke and mirrors. I mean but I think that's why it's good to have also the the hard test. Right?

So I could also see that it was actually suddenly, was making more progress on the more objective tests than I hadn't seen for a while with the other models. So I guess that gives me some trust.

as much as you can and have time for. I mean, I think for compilers, there is some very important architectural design decisions. And then there's a lot of sort of just implement like these like a 100 different functions.

Like they all just, especially like modern compilers, it's like layer by layer by layer. You have to lower these things from like the front end level to the middle interpretation, to the back end layer. And a lot of that, I do definitely don't wanna have to write like It also paralyzes well, I think.

I mean, I'm sure this could be one of the reasons why Anthropic also picked it. We talked about this program bench thing. I think Facebook released where it's a benchmark where the task that I think they took 150, 170 programs.

Some of them are really complicated like FFmpeg is one of them. The AI just has to re implement them without internet access. Yeah.

Basically when it came out, they had like all of the LLMs got zero percent because none of them like were able to pass all of the tests. But I've seen like people like I very rarely see that when people post benchmarks for this thing, they always post this like, oh, the percentage of tests that it's got right or like even like the fable, it was like, yeah, got like 80% tests correct. And it's like, yeah, but I talked with the people who made the benchmark and like, yeah, but did it get any of them actually right?

You know? Like, did it because it's if if the program only passes 70% of the tests, it's probably not right. And in a in a way, this is the thread that we were talking about before.

think humans do it all the time actually. I think there's the whole field of kind of reverse engineering where people Yeah, actually, I listened to something with the FFmpeg people where they're talking about all these codexes that they put in there. And they Like a lot of time, they have no idea what they do.

Like it's just like this obscure blob of code. Like, may they probably don't even have access to the code. They just have a couple of, like, example video examples that were encoded with this, and maybe some people had shared some screenshots, like, with clips from the the movie and then they had to go and and try and read this like encoder bluff and being like, what is it doing?

Like, how could we like write a program that decodes this thing? And somehow people are able to do it. You know, we must have some sort of prior, right?

Like, what what are the typical things people put in a video codecs? Like, oh, there's probably some, you know you know, furrier transforms of some stuff where they're probably like you chunk things up or I don't know. I haven't done that much video encoding.

But yeah, just like completely black box, just try and and keep improving your program and and visualize the output until the end of movie comes out and and you have an idea of, oh, this probably looks how the movie is supposed to look.

Right.

I think they possibly could. This is good point because I tried to get an answer, I think, from Norman Brown at some point on Twitter about this because you see how the better pre trained model you start with, and then they start reinforcement learning on it.

I do think, like, that's why, for example, Methus is so good at doing stuff in our code basis is because it has somehow seen a lot of code and it just seems to have this sort of intuition for bugs and problems and so on. That must be kind of like that. It's seen the patterns before it has some kind of abstract, but it's true.

Like when you actually are doing sort of running them more live, they don't do it very well. Like, they don't and and I think a lot of people these days are thinking about this, how can we do better continual learning so it can just build all of these abstractions and these things really well on the fly? I mean, I know some companies are actively against trying to do that also.

Why? Like I think Anthropic, right? I think Dario said like he considers it like a big safety issue too because you're gonna lose You could easily lose all of the work that has gone into alignment if the agent is learning too much on the fly, then it gets further and further away from the sort of safe checkpoint.

But I think there are enough other people working on it that it'll probably happen. But it's yeah. It's definitely like a big unknown, right?

And I think it'll also have to change a lot of things in how we serve models, for example, because suddenly you need to be able Like if we mean by continual learning that we actually update the weights live, it's gonna it causes a lot of problem problem for the current paradigm because you then suddenly you can't use the same weights for all customers too. I mean, I guess, Thinking Machines has this thing where they have one shared model and then like some loras on top. So maybe something like that can work, but it's still like you have to somehow, yeah, keep doing this all the time.

So we are also working in normal computing on like alternative forms of computation. Like these thermodynamic chips are just sort of unconventional computing. It actually becomes sometimes hard to keep the memories in these analog resistors unless you keep learning at the same time as you're doing inference.

Like you have to use these fancy substrate like memory stickers or something if they have to be permanent, but if you're just using like sort of basic capacitors like you have in DRAM or something that require constant refreshing, So either you just have to spend a lot of energy on that, or you just want to keep the learning going forever. That way they kind of automatically refresh. And that's probably more similar to also how brains work, right?

Like they don't they never they never like just don't what Chumsky or somebody would say about if they were just freezes. I think like the synapses and stuff, they're always adapting.

as a system.

right now is this idea of trying to build hardware that best fits like the models we have right now and just like really make a super efficient inference. You could say like the, in the past, least the issue has been you stop yourself from innovating on the software side if you lock down your hardware too much, right? Like the NVIDIA chips has been pretty good for innovation.

Like they're pretty flexible. Of course, they also have have guided the way we do AI in a lot of ways, like towards matrix multiplications and so on, but still people have been able to innovate. But on the other hand, I think also hardware is getting easier and easier to make with these kind of AI for EDA tools.

So it's sort of shortening the like, maybe now people are thinking about, like, making their CUDA kernels. Right? It's like it's not so unreasonable to think that soon we'll just be doing our AI for making like, instead of CUDA kernels, we'll just, make some custom circuits for every single thing we want.

So if you come up with a if you want your new algorithm to run really fast, you just design, like, a specialized circuit for that, And the AI helps you optimize it and check that it's correct. And of course, you still have the FAP, but people are We're getting better at like batching things for the FAP.

there is the auto formalization which we define as taking human specifications and writing up and turning them into formal specifications in lean, for example. Then, but then of course you also need the proof step where you provide the Yeah, prove that your coach, whatever you have actually satisfies that specification. And I think some companies like Acxiom, for example, they're very focused on this part, right?

Like it's And also in some sense, alpha proof, that's also the main thing it did was it started with a formalization and then the hard part was training the model to provide a proof or disprove. I think it's actually a really nice trick in AlphaProof was that when they did the formalization of the proof, it didn't really matter if they got it right or wrong because if they just asked the model to provide a proof or disprove. So if they got it wrong and it was no longer true, then it would just prove that it was not true.

Like, you could still use it. Of course, like when they actually did the IMO challenges, they wanted the form autoformalization to be correct. So then they did it by hand, but they didn't need it for the training, which I think helped scale it up.

I think yeah. So we can do a similar thing with hardware, by the way. It's pretty easy to just take some chip design and then you can basically just come up with some properties that may and may not be true, and then you can train the model to try and prove or disprove that this thing holds.

So then, but that's all about creating the proof. Then the autoformatization, in some sense it's harder because it's harder to create the training data for it, right? Like I think that's kind of also a story about AI in the last two years since reinforcement learning, like anything you can create good RL environment for, you can probably learn, but anything else is like out of reach right now.

And so some of these chips have thousands of pages of specifications. And if you want to turn that, yeah, you're turning that into a formal model. It doesn't really like if you got just a couple of words wrong somewhere or a couple of numbers, then it doesn't work.

Like, then what you prove is not not important or is not relevant. And I think, I mean, it's always been an issue for in the chip industry like that. And they've kind of tried to solve it by having orthogonal teams.

So they have one team that's designing the chip and one team that's designing the tests and another team that's designing kind of tests of the tests, like where they kind of like coverage. They call it functional coverage where they measure what the tests test and then check everything off. And so hopefully like if all of the three teams have read something the same way and understood it the same way, hopefully they have the right idea of it.

I mean, can try and do something similar with AI. You can argue whether it's really orthogonal if it's the same models that are doing each of the three jobs. I do think it is quite a thought.

Like I do think there is enough entropy in these models when you do long runs and you can definitely find a lot of box if by just doing things many times and see if if there's agreement between them. But it's something it's all but it's definitely like a trust exercise too and a and a human exercise to figure out how do we get the hardware engineers to trust and like, how do we make it easy for them to verify that our formal model fits with what they what they thought, like, and you can come up with Yeah. There's all kind of different tricks you can come up with both to make it more visual, like what your your understanding is and like whether it matches and like ask questions and see if you agree with the questions or show examples or you can also try and do some kind of back and forth.

I don't know. Lots of tricks you can try, but it's an interesting problem. I feel like no one like the Alpha Proof and all of these other people, they never really solved this because their math statement were just one paragraph.

They were not like thousands of pages long. Oh, yes. Yeah.

Interesting. Yeah.

Yeah. I think I think where we're going now is also trying to encompass more of the of the spec creation, right? Like, obviously, when we start out, people already have written down all of these specs and we wanna help them out with those.

But when you do that, you also lack you miss out on the whole intent that got into and the whole process of creating the spec. You don't know if there's just some numbers somewhere in there. You don't know why they chose those numbers and not some other numbers.

And I think you sort of by internalizing more of the process, at some point, you can at least be as safe as any as any human could have been about this chip. Right? And and like there might also just be some ambiguity that no one cares about.

The ambiguity thing is interesting that there's a wonderful talk by Eric Curio called Math Does Not Represent.

seems like this is almost an The training data in a way, if they autopharmize the same problem in different ways, they actually get different problems.

With with smaller chips, like, say you have like just like floating point or a crypto chip or something. I mean, the specification is actually pretty simple for those. It's more when you come up to like the big like system level stuff.

I think software is in many ways ahead of hardware. I mean, in terms of formalization they're behind, but in terms of thinking about these like architecture and architecture together with AI, right? Think a lot of people now they'll run their architecture documents by Claude or somebody and being like, Hey, what do think of this?

Should we move things around here? And I think this whole discussion like it sort of helps the models understand like what your intents are and what you care about and what you don't care about. And just to give us an example to bring it to life, was that DRAM article you published.

Explain that. What we talked about before with the formalizations is very It's like at the RTL level at the very sort of cycle level where you care about Yeah. You wanna prove that the exact thing happens at every single clock cycle.

But of course, there is another kind of formal and that's very important for things like adders or like arithmetic circuits or crypto. But a lot of the hard problems people have are more these, you can say, like kind of system problems, more like protocol level stuff, like the things actually, maybe also more popular in software has been like proving things like there's the system can never deadlock or livelock or sort of like protocol level properties. And like all of the sort of timing requirements between different things makes sense and there's no internal inconsistencies.

And there so there currently, exists different formal languages for these things, like for the cycle level stuff, people use these system very log assertions, SVA type things. For the higher level protocol things, they're more classic things like the TLA plus I think by Valiant or no, Laporte or something. And yeah, so we try and formalize things in both ways because it can have different it's useful for different things.

It's possible at some point they can all merge if you supply. It's kind of like you're doing very high level math and then you you can you can reduce it all the way to the actions if you want to. But I think it's also interesting.

Yeah. I mean, it's a very new field so we're trying to explore different ways and and these time patronets is one way to represent like these super parallel systems that you have, for example, in these memories where all these different banks, they can like, you can do activate like like for example, yeah, if you activate one of these rows, you have to wait for the data to run down to the bottom before you can read it, for example. Then after a while you have to like refresh it because again, it's like these capacitor DRAM cells, they have to be refreshed all the time, but you can also be working in a different memory bank that might be like lots of them on this chip at the same time, so there's certain like But then there might be some circuitry that's shared between the banks, and so certain things that are like interbank dependencies and intrabank dependencies and also bank groups and people build really crazy stuff actually in there.

I mean, that sounds crazy. Like, how does that work? Yeah.

I mean, this was one of the things that really that got me to normal in the first place. Before normal, I was at the at at the Facebook as we call them in the like the research group that does probabilistic computing. Like we were doing like these spatial neural networks where you assume like probability distributions or all your weights and you try and infer like the posterior from like the prior data that you look at.

But a lot of these techniques were kind of slow because you had to do you had to like either try and do lots and lots of repetitions with different random seeds or you were trying to do it analytically. I mean, that kind of maybe takes a little bit of the point out of the probability. But then you could also just but then but then it's funny because you have these chips and like the chip manufacturers, they spend so much time like getting out every single little piece of noise out of their systems and like having these extremely sharp margins for everything, like so much, you know, precision is probably like the most precise business in the world.

And then what do we do with them? We just like add randomness everywhere. And yeah.

So why not try and build a chip that's just inherently random? Like, I mean, yeah, I think the brain probably has bunch of randomness. But here, the first chip we made was then this, you basically, you have this array of of, yeah, capacitors and you have certain resistances between them that you can program and then you infuse all of this noise and it'll start to behave according these stochastic differential equations.

And then you think, okay, what can we do with that? It's sort of It's a new computational paradigm that we've found very interesting to explore where you can say one of the things, for example, you could do with it is it actually turns out that the matrix that you put onto the the chip in the weights differential or the stochastic differential equation, it actually behaves sort of according to the inverse of that matrix.

Yeah. And I think this was kind of one of the issues we had like, because I think Bayesian machine learning was really strong for a certain amount of time when like at a certain point in time before the generative AI because you had like the one output and then it made sense to have this sort of distribution as the output. But now you have these sequences.

Yeah. As you're saying, like you keep putting these tokens out. And and I think no one really cares about what is the uncertainty about like one particular token and having a better distribution on that.

You really want to know after the model have thought about 10 different options and you know it's backtracked and all this stuff, and it comes out with a final final answer. What is, like, how much can I trust this answer? You either have to go like really deep on mechanistic interpretability to try and really carry all of the uncertainty all the way through that, or you have to try and use some more maybe anthropomorphic methods, I think, that are based on like how would humans estimate their uncertainty and try and apply that at really high level.

Yeah. Yeah. We actually did some experiments back also when Transformers first became popular where we tried to do it was before we knew exactly what the company was gonna do, that it was gonna be AI for hardware.

So we wanted to do like predictions and we just started building base law into the model. So for for some particular question, it would try and find like lots of pieces of evidence and then we'll say, is the probability that this if if the answer is yes and if the answer is like different things, like what is the probability that I would see this evidence? And it would do that lots of times, and then at the end, you know, you could use Bayes law to say, okay, then what is the probability that the original statement was true or false?

It actually worked really well, like we did this I did some like internal like prediction game and it it it beat everyone. I'd We had two multiple versions. One of the versions we sort of It was more a neuro symbolic, you could say where you would just like have the model come up with all of these probabilities and then you would manually calculate it.

And then I also tried another version where it's you do I just told the model now they use Bayes law and do it or something. And somehow it actually did a little bit better, so I don't know how it how it did that or maybe it just was able to Yeah. Maybe it's because it was able to look back and think, actually, a couple of these values, probably bullshitted and I'll just ignore that.

But yeah, it's interesting. I know there are some people training Like there are some benchmarks now for where people are trying to predict polymarket and this kind of stuff with LLMs and they have benchmarks for that. But I'm not quite sure if they use these sort of techniques or it's all just end to end reinforcement learning and and they hope it just picks up a good mythology by itself.

Yeah. I I was thinking about there is this older tweet by Elon Musk. I don't know if people were laughing at him.

I mean, I don't know. They he'd make he tweets a lot, but they were he was saying like, why don't LLMs just write the binaries directly or the assembly? Oh, yeah.

I saw that. You know? Yeah.

I I don't see why we wouldn't be able to do it. I just it's a question of if we would want to do it or like like do we think because like, I think there's some fundamental computational problems always and and like everywhere in the world, don't always like recent years, we don't focus so much on them because we're so excited by AI and we wanna we're just interested in all the stuff that AI can do really well. But I mean, obviously, things like cryptography or, like, some of these, like, basic algorithms, like, you never wanna want gonna want the LLM to just do it even if they can do very large number multiplications.

It's just super inefficient. Or, like, you might as well optimize like a circuit or piece of code for that, and compilation has some of the same problems where you want to and also and chip synthesis where you're trying to, yeah, explore all these different designs and so on, so you don't necessarily want to have the LLM do it because it's so slow versus some super optimized loop. Yeah.

Not to jump too much in things, but it's interesting to think about chess again, like, of course you have like the AlphaGo thing where it was like very everything ran through a neural network, right? But today the state of the art is in like Stockfish, of course, is that they took, they did more like hybrid. So they took the neural networks and they made these like like a certain type of neural network that can update really fast when you change the the state.

And then they they combine it with like just super fast search and it actually outperforms the best like open source, like alpha go type chess engines. Oh, really? Using like adaptive fine tuning?

And structured inference? I don't know if you call it adaptive. It's more like they took the classic test search engine and they replaced the evaluation function with a neural net, but like kind of a very shallow wide neural net that they can that is really, really fast to evaluate.

And yeah, I think it could be that for something like synthesis and compilation, there's a similar thing where it's like, you could try and do it all with LLMs, but at some point that's the speed is a bottleneck and like you can throw you you get a benefit from having more knowledge and more intuition on all of this stuff, but at some point, there's also just a hard commutation problem where you just wanna brute force some stuff. Yeah. And at that point, you want to be able to switch to like more classical algorithm.

practice, where can we see a huge uplift? I think it's always interesting when you build hardware because there's always this co design problem with the algorithms, right? The algorithms are so based around the the GPUs and the hardware we have now.

And so you can try and target some of those bottlenecks like you can make hardware like we talked about with the with the thermo DRAM that's super efficient for for the memory. Like that might speed up inference, but it can only speed it up. Like, say we have there's some like 10% GPU utilization now for inference, like you could then you can hope to have like the 10x there, but you don't know if you suddenly have access to that much like that different architecture, what like new algorithms could you build?

Like if you really went all in that. And there's some similar thing with the diffusion stuff, right? It's like now the architecture is not that efficient in terms of like randomness, like the GPUs.

It's kind of people don't really do wanna sample like tons of Gaussian random variables everywhere. And and so they maybe also build architectures where they sort of try to remove that bottleneck by not focusing too much on these things, and then they instead so we can build new hardware that that makes them that more efficient and and that has some performance gain, but to really make the most of it, you then also have to come up with the new models that that sort of go all in on that. It's interesting, this Notion and apps.

Have you thought about this, how all of the apps now they want to be like your central AI? Like Notion has this, they wanna integrate with the other apps and your linear have an AI and they wanna integrate with the other apps and like everyone is trying to like capture like being your central AI assistant. And then all the other apps will just be tool calls.

think Codex, I think they are actually opening up more that you can use your accounts. I saw just yesterday, I think maybe also Anthropic opened up some that you could use some more, just like the SDK you can use with your accounts and but it's just like the price difference is so big that if you have to use API pricing, like nothing is competitive.

chess AI is a bit more is a bit different. I actually worked on test engines for fifteen years. But I I I actually I remember him saying that too and I was like a bit surprised because you'd think that if like sure.

Like if your science is biology or you wanna understand how humans do language, sure, then maybe the LLMs is kind of not so relevant how they learn language. But if your science is more the abstract of what is language, I thought you would be really interested in seeing different systems developing language and you can comparing and see what's common, what's the same, what's different and so on. That would give you like a wider understanding of what is the concept of this thing.

I mean, of course it has to, you have to have some kind of respect for AI language to even like include it if you just think it's like completely just some, I don't know, stochastic parrot thing. Maybe you're like, don't really care about it, I wanna include it in my model of language. But if you do think it's actually doing language, then I think it doesn't really matter if it does the same way as humans or not.

Maybe it's more interesting if it's in a different way. I know. I know.

You mentioned chain of thought, by the way.

Yeah, it's like you want to give the model somewhere to think, right? Like, when you do reinforcement learning, I mean, the chain of thought before the reinforcement learning and after reinforcement learning, think is very different because before it was kind of, you try and prompt it and some tricks, they kind of work and kind of doesn't work, but once you do the reinforcement learning, you need like It's like a Turing machine, right? It needs to have infinite memory and being able to have something to operate on.

Whereas when you just had the transformer and just like single shot output, it's like, I mean, probably in the Chomsky hierarchy, it would be a completely different type of system. Like, I don't know, an automata or something, right? There's like only finite amount of computation it can do, but now it can do as much computation as it wants, and it just has to learn to figure out how to do it.

Like you could imagine building a really simple LLM type system and with access to Chain of Thought, it would be a universal Turing machine. So now at that point, it's just No, because it can have this tape and it can keep reading and putting. I think that would be pretty easy to make a structure like that.

a lot of things are breaking now because of that. Mean, a lot of the open source stuff is like everyone just also like writes new code from scratch instead of trying to to get come together and hone these like shared libraries. But I mean, it is just for a lot of things more efficient than also going through and you can get everything just the way you want it.

But yeah, and I actually think it's very interesting what you're talking about with the spreadsheets and like this channeling, right? Because there's a lot of kind of domain knowledge inside of these tools. And that's actually also what I think like people might be worried about who build these tools, like how you see it like it is to kind of extract that knowledge and like people building clones and so on of their tools.

Like if you have like everyone these days are like locking down all of the data, right, because they're realizing data is becoming so valuable, they wanna keep it themselves or build something themselves rather than have other people use it. And and I think it could be a similar thing with with a lot of, like, specialized tools because the tools basically are data like they're developed over so much time, like, founding all the right patents, everything is built in there.

breaks this social contract, right? Like in the past, I wrote something and asked you to read it, you could at least have assumed that I would have spent 10 times more times writing it than you reading it. But now you're really skeptical about anything because like, why would I wanna spend time reading some stuff that you didn't even read yourself maybe?

Yeah. Like, I mean, but a lot of these are like social problems, right? Like how do we protect also the open source ecosystem from AI generated PRs where like no one has ever read them and then these like volunteer maintainers have to like read through all the slop.

And yeah, it's not really fun. Yeah. Do you Have you talked with anyone about that?

Like people talking about like we need like a GitHub, like social credit system, like a commerce system so people can like down rate you. And like, if you don't have a high enough common, maybe they'll just not read your PRs or We do need that. Yeah.

put gates on people uploading things there as well. Right. Yeah.

And they had this ban. It's like a one year ban if you have a hallucinated citation. Yeah.

Yeah. And creates a feeling of understanding that you might not actually have, right? And it's like, I don't know, doing an exam or copying somebody else's work.

Like you might feel like you wrote it, but you don't, like your brain didn't go through the motions. And yeah, I think it's dangerous also for like if you're worried about sort of AI taking over, you know, because it's not like it's not just that it's getting smarter, it's also that humans are getting dumber, like we no longer like no one like we get lazy in terms of understanding stuff. We don't read the papers.

You just put them in AI and be like, oh, explain this paper to me or something. But yeah, it's strange, right? Because on the other hand, I mean, can be like this big level, like speed, like it can speed you up so much.

So it's like, it's hard to not use it, but like finding out when you should stop and when you should start using it again.

seem to build deeper foundations and discover new things. Yeah. And I guess that's like a tension with like in capitalism or something that the companies, they they don't they're not trying to necessarily develop their employees.

I mean, maybe if they can see a pro the profit motive in it, but at the end of the day, they just wanna get the job done. Yeah. And like, of course, if you spend a lot of energy developing your employees and then they leave and stuff, it's always a bit tough there.

I don't know if there's a way to force it. But I think it's also a little bit self inflicted because I mean, learning stuff is just hard and like it can be like frustrating and it doesn't always get you in the same kind of flow as just prompting or something, right? And it's like, I think, yeah, I've been trying to use this.

I think it was Kapathy who suggested this, like when he was learning some new stuff from LLMs, he didn't like copy paste it or he would have like side by side windows, but he would always like write all of the code by hand. I made a internal app to try and because we hire a lot of AI engineers. Like it's hard to find people who are both like really good hardware engineers, really good AI engineers and also software engineers.

Of course, you have to be that. So we we have to scale up people. We have to scale up hardware people on AI.

We have to scale up AI people on hardware. So I made, like, internal, like, training tools and so on where I try to have the same like blocks in there. If you if I'm block copy and paste in the least, like give people a warning and saying like, you should maybe try and type it yourself.

Like, hey, somehow it like, you you wouldn't think so. Like it doesn't feel like anything. Like if I'm just looking at it and then typing myself, like why would I learn it better?

But it's just empirical that somehow like our neurons and stuff like it's important like to go through the movements. You have to figure out what projects you need to understand in-depth and which ones you're just making like where it doesn't matter as much. Think we also we get tempted with the AI to just start way more projects in parallel.

Right? Because like, oh, I could just start another one while it's working that I'll start another one and another one. And then suddenly you're like, well, there's obviously no way I could actually understand all of these in-depth because now I started so many projects.

So then then you have to tempt, like, to stop yourself and being like, maybe instead of starting the fifth project, I'll go back and try and understand what's actually going on in in the in the first couple of projects. I think maybe there'll be some evolution, like we're still learning all the social stuff around using these tools and and what is effective and what's not effective hopefully will. And maybe we can even build some stuff into the tool system.

Is that what you're saying also with the scoping like to to try and make them better. And it's hard. I mean, another really hard social problem I think right now is how do you do teamwork?

Like that's something I also think about as a manager, like if everyone on my team is like staring at their 10 agents or something working on some stuff, when are they gonna go and talk to their colleagues about this stuff? Or like if they don't understand the code themselves, how are they gonna explain to the other people in the company what the code does and sort of and build together. And I don't think anyone has really solved it well at like at scale.

I heard some podcasts also with the Anthropic co working team. And at least the way I remember it was that the podcast says, hey, how did you make this so good? And they were like, yeah, just everyone on the team coded their own version and then we picked the best one.

And I like, damn. If that's the best we can do in teamwork, you know, it's kind of, I mean, in a way it's cool because it's like, oh, it's like the ensemble method just applied to people and and, you know, it's it's cool you can explore so much more, but it it's also like it does remove all of the teamwork. At that point, it's just everyone working in isolation.

You're just duplicating. So I'm curious how we're gonna solve that. It's performance versus competence.

Thomas, it's been amazing having you on the show. Yeah. It's been so much fun.

Thank you, Tim.