Intelligence is collective, not artificial — Prof. Michael I. Jordan (UC Berkeley / Inria)

It exists in the real world. This is a abstraction, but it's a it's a real thing. It's like f equals m a.

It's a set of, it'll make predictions. So if I write down a game, just like I wrote down f equals m a in some coordinate system, I can now predict what'll happen. I don't think we need to see that.

I think this anthropomorphizing of intelligence and understanding all that is not necessary, not appropriate, and is is a distraction for many, many problems. Why say it understands? I think it's science fiction, and I think science fiction is important for society, but it's also at the level it's being promoted and and and those kind of voices, it's really hurting 25 and 20 year olds.

You know, these these young folks of whom there are huge numbers are excited about technology, and they wanna build things that help their family and help their country, actually more of their family than their country, honestly. And they they they see real opportunities in doing that, and they're kinda being told by the leaders, well, had our fun. We developed a bunch of algorithms.

We we did it, and we were just interested in the pure int you know, understand intelligence even though they didn't understand intelligence. They built, you know, gradient descent algorithms. And now you guys, you can't do this because it's dangerous.

It's gonna it's gonna wipe out humanity with a with a high probability, or it's superintelligible arrive soon, so there's nothing left to do. That's in your lifetime. That is so demoralizing.

So demoralizing. And that that thing, I think that bothers me the most. I mean, the second part that bothers me is there's no economic thinking going on there.

So the current generation is just way too you know, there's not much thought going on, not much intellectual stuff. It's just yeah. It's possible to build it.

It's possible to steal the data from wherever you want to because that's what the Internet allowed to happen and not return any value to the person who originated the data. It's possible to run greedy descent on that, but you need huge amounts of money, but it's now possible to get it from people who aren't thinking very deeply. I I don't think it's bad to build systems you don't understand, but I think this level of detach ment from reality is unusual for human history.

AGI to me is just a bit of it's a PR term, and it it's some people think it's it's fun because you have to have these great aspirations. I think it's just distortionary. I think it confuses young people.

And as I will talk about today a little bit, I think that one of the things I find most alarming about the so called thought leaders that one will see often on podcasts and other venues is the alarmist tone or the exuberant tone. And I think 20 and 25 year olds are watching that and saying, am I gonna be exuberant or am gonna be alarmist? Those are the two choices.

And I hope that this conversation we're about to have is one that makes it clear to young people that there's other ways to approach life and technology. I've never actually thought of myself as an AI researcher. I didn't read an AI book.

The term was coined in the fifties and John McCarthy and others had particular goals in mind for coining it. And they had particular methods in mind, like logical inference and so on that didn't really quite pan out. In the meantime, in the sixties and seventies, know, eighties, something arose called machine learning.

The actual methods like decision trees and nearest neighbor and logistic regression and hidden Markov models were developed in other literatures, mostly statistics, operations research, and so on. And that led to industrial success stories. So supply chains and commerce and transportation systems all used, and still to this day, vast amounts of machine learning.

They used gradient based methods and the cloud was developed to handle machine learning workloads at Amazon, in fact. And so that's the tradition I came up in. I was trying to think about systems building at scale that would also serve multiple people.

The AI buzzword returned, I think, maybe five or so years ago because the data that started to be used was language data. And so the box now is not just making predictions about supply chains or commerce or prices or whatever, it spits out human fluent language. And people said, oh my God, we've solved the old AI problem.

In fact, in some ways, if you define the AI problem narrowly like the Turing test, yeah. But there was this ongoing tradition of machine learning and by that time had incorporated people from all different kinds of fields. And it was really having an impact on industry, still is.

But the AI buzzword returned because of LLMs. And now to my view, it's been a distortionary effect on the path of research, on how we think about where research should go, but also on the path of how do we think about business models and how do we think about where technology is going? And AI wasn't enough, they had to create this big hyped up buzzword, AGI, which, well, we will talk a lot about economics as a source of intelligence, a social intelligence.

And when it's put together with machine learning style intelligence, you can now talk about at scale, not just numbers of computers and amount of data, but numbers of humans.

a little while back. It's funny because I was trained as a statistician and a cognitive scientist, but I'll take it.

Give us the elevator pitch. I was never an AI person. So in some ways it's easy for me to come in and look at people who are self professed AI researchers and sort of say, what are you doing?

What's your point? What's your goal? I think sadly they often don't have a very clear goal.

It's that humans are intelligent, humans are a computer, the brain is a computer. And if we mimic that and take aspects of it and paralyze it and make it more powerful, it'll just do great things. And and it kinda stops there.

It's not that there's a goal in, you know, society that we're gonna we're gonna try to do this or that. It'll just solve problems for us, and then we're we'll be, happy. And it it you know, I I got away from Silicon Valley partly because that's just the way that people talk and I got tired of it.

And there's not a lot of intellectual, you know, let's call it deeper long term thought going on. I now became a rat race and a money race and all that. So yeah, my my perspective, I mean, it comes from a long tradition of other people having sort of social science perspectives on intelligence.

We are social animals, and a lot of our intelligence comes by the fact that we aggregate. We aggregate opinions and thoughts and, you know, we have cultures and so on that retain them. And, moreover, the the society provides a context for our intelligence.

A smart action in one context is not in another context, and it's all very fleeting and contextual in in the moment. And so social science ideas are needed to appreciate what that means. When I say social science, I include economics, so game theoretic.

The context is somebody else out there is trying to take advantage of me or maybe to collaborate with me, and I don't really know. And so I've gotta put off feelers and do signals and create mechanisms where we can interact effectively, and economics studies that in a mathematical way. That attracts me because I am a mathematically inclined person.

I'm not a critiquer of AI. I wanna make it right, and I wanna make it better understand what it means to be intelligent in in this world and and safe and interesting and think about long term issues. And so to me, you have to do that, you know, formally or mathematically at some level.

It's not enough just to build things and put them out there. So when I say a collectivist, I just mean that most of this technology is based on inputs from billions of people. So there's already a collective putting input in, and it's meant to serve billions.

So there's a collective serving. So there's really a big network that's kind of latent there. And then economics critical, so I don't want to just sort of say words.

the model there is the old AI model. Let's just build something intelligent and it's only got upgraded a little bit. It's gonna be a better search engine.

That's fine. I do think the search engine was major progress for humanity, but now it became more of the search engine. It's like a secretary sitting on your shoulder helping you, whispering things to you.

And it's just a dumb business model. I don't think many people really will want that. They'll turn the damn thing off.

They wanna think for themselves. They want, you know, maybe at the end of the day, a summary or something, but they don't want this all the time, you know, they're interacting with this entity thing. It's not a very good business model.

And in the meantime, we have huge healthcare systems and transportation systems and finance systems that are all based on data flows among many billions of agents and, you know, are ripe. They already have a lot of machine learning in them and they're ripe for thinking in a more economic way. What are the agents and what are they trying to get out of it and what kind of cooperation and competition is latent there that you could make improve.

Markets arose thousands of years ago and we learned about some of the principles, but we can improve them. And thinking arose billions of years ago or whatever, but we're not perfect. And not just in terms of thinking, but we're also not perfect in terms of narrowly following our own agenda and hurting other people, even though we don't want to.

Humans are wonderful. We wanna prize human life and creativity and, you know, emotion and love and so on and so forth. It's fundamental.

But, humans are also bad or do bad things. And, that's where technology should be able to aid you. And so you need to think about the system.

The set out these systems. They're not really systems. They're, you know, big statistical boxes, that do inputs and outputs.

That's not a system's way of thinking. There's a lower level system, of course, the computer system, but I wanna be above that. I wanna say what ecosystem does this belong to?

Who's it interacting with? At what rate and what kind of quality and what kind of values are being created? And And when I say value, mean often mean money.

I want jobs out of this thing. I don't want just it to answer and do things for us.

agent systems and we get all of the economic stuff that you're talking about for free. Like, what would you say to those people? I mean, it's just not a good way to think about engineering.

I mean, if you were a chemical engineer back in the forties and fifties, saying, we're just gonna throw a lot of stuff together and make it work. Well, you could do it, but you'd get a lot of explosions and a lot of economically nonviable things, you'd hurt a lot of people. And I think a lot of these people are not thinking about all the people that are being hurt already of Facebook and so on.

It's damaged a lot of young people, a lot of teenagers are having mental health problems. And this is just not something that isn't talked about by computer scientists at all. And now we're talking about yet another level of displacement of, know, jobs may go away, but you know, that's tough.

It'll create new ones, of course, like always. You know, I just don't like to talk that way. You know, so you got to say, well, step back a moment.

What is your point? Are you trying to create a new kind of market where people could come in and have their talents valued and appreciated and or bids could be put out for things that people might need and collaborations can emerge and there could be producer consumer relationships being explored and understood and developed, and this could all be a mix of computation and humans. I think eventually we'll all kind of merge, but along the way, just doing something so disruptive with all of these metaphors that's not good social science or not good mathematics, It's just metaphors.

And yes, you can build it because the previous generation of people created these amazing things that collect data, and we can do gradient descent on it and ad hoc architectures. And yes, that works, it's amazing, but let's not give so much credit to the people that did that. It's the people twenty, thirty years ago who did that.

So the current generation is just way too know, there's not much thought going on, not much intellectual stuff. It's just, yeah, it's possible to build it. It's possible to steal the data from wherever you want to because that's what the internet allowed to happen and not return any value to the person who originated the data.

It's possible to run greedy descent on that, but you need huge amounts of money, but it's now possible to get it from people who aren't thinking very deeply. And so, you know, made me seem more dark than I want to. I mean, there's some lot of good builders, but we also, every previous era of engineering development, electrical engineering, chemical, mechanical, all, had some builders, but they had a lot of concepts and they had a lot of thinkers.

In fact, all of those engineering disciplines had something like Maxwell's equations or Newton's equations to help them kinda here, no. It's just people that are very smart and who can code and then have lots of intuitions, And and it seems to and I don't ever see anything that's it feels deeply intellectual to me. It feels like science fiction.

the best of both worlds.

But then you've got to kinda put things around it. And the things that are beeping around are like buzzwords, like AI safety. It's a buzzword.

Okay? What you really need, I mean, a human, you can't explain to me why you picked this Airbnb over another one or whatever. All the choices you've made today are inexplicable to me that come out of your brain.

And I don't need to know all the whys and wherefores of your choices. And what I need to know is that you're somewhat predictable and that if I make certain options available to you, you're likely to take this one versus this one and therefore I can make my own plans and we can start to interact and so on. So that's part of economics is the economic style of thinking says, I don't understand all these other entities out there, but there's certain rules of thumb that I can use or quantitative predictions I can put in place that allow me to interact and not get hurt and even get value out of it.

So no, I don't think it's necessary to understand all the details. Now the input output behavior, you often have to understand better than we can now. For example, if I'm denied a loan at a bank and the bank uses this big AI program based on past data, I wanna know why.

And why doesn't mean that you look in the internals and show me some circuit. No one's gonna want that. They're gonna want, well, there was like 50 here's like 50 people that are pretty much like you according to the embedding we're using in this big network.

And of those 50 people that are like you, some of them got the loan, some of them didn't. And here, let me just show you what those people are like. And you start to say, oh, I see, they differ from me in this way.

That's actionable to me. I could now change things. So you have to build systems around this predictive system.

That's a nearest neighbor system, for example, and that system will supply what people might consider more like an explanation. And so it's not just trying to go in the internals or something. Again, chemical engineering is, you know, there's certainly thermodynamics and lots of things are understood, but lots of phenomena were not understood for a long, long time.

You mix up a bunch of stuff and, you know, certain waves are created and certain things happen and you exploit that and go and move on. But you understand something about input, about behavior and and constraints and and so on. I think the current generation of neural nets will continue to, you know, they they have very nice scaling behavior.

They'll continue to be there, but they really have to be thought of as a product part of a bigger ecosystem. And then you kind of ask, well, what can the neural net do in this context? And what's it missing?

And what if I have multiple of them?

The issue that we found empirically was that when you ask certain kinds of questions, you know, in particular, we did one where we were looking whether quantum fluctuations in a protein were associated with phosphorylation, meaning the phone the protein was active or not in the cell. And you might think that these fluctuations would lead to strands hanging off or kind of like, you know, bad proteins. Evolution wouldn't use them.

But it turned out that a lot of them seem to be phosphorylated, meaning they're reactive in the cell. That suggests a hypothesis test. Is there an association between yes, no phosphorylated and yes, no quantum fluctuation?

So that's a little two by two table, and you do a statistical test on that. And the problem is that if you just use known protein data that's been crystal you know, there's a crystal structure known, you don't have enough data to test that hypothesis with high power, and so you can't reject the null hypothesis. There's no association even though there looks like there is.

If on the other hand, you use 200,000,000, you know, proteins out of alpha fold, you can test hypothesis with high power, and you reject the null hypothesis. But what we found is that the, confidence interval on that statistic of that two by two table was extremely narrow and away far from the truth, the true value of the the gold standard value. And we found this in domain after domain.

You know? So why is that? Well, what's happening there is that there's probably not many examples in the training set of proteins with quantum fluctuation because it's not been that studied in the past, and it's hard to crystallize.

And so not many examples means that it's quite possible AlphaFold won't give out a great answer, but it won't tell you that. It doesn't give you out error bars, and it doesn't but specifically on the question you're asking. That's where I want the error bars.

And it didn't know about that question when it was built and designed. Okay? All right.

So now I have a good statistical question. What if I add a little bit of ground truth data to the 200,000,000? Can I shift the error bar so it stays somewhat narrow, so I have high power, but it covers the truth?

And the answer is, yeah, there's a methodology. We developed something called prediction powered inference that does exactly that. And so it'll cover the truth just like in a classical statistical setting, but it's using this rather highly biased architecture.

And it's now, it's not biased overall. In fact, its accuracy is high overall. But for the question I'm asking, it might be very biased, and that's gonna happen a lot in science because scientists are rarely interested in just studying the past over again.

They're interested in brand new things on the edge of knowledge, and that's where specifically these foundation models will be most poor and most highly biased. So there needs to be around any foundation model the ability to maybe collect a bit of ground truth data to merge it in with some procedure like this and then to give out a more trustable answer. That's all not fine science fiction.

That's what can be done and what really needs to be done. And I'm sure the AlphaFold people are on board with that, that they would not find that weird or surprising. But a lot of other people out there talk about bias and all that, they either don't worry about it.

Say it'll go away, we have enough data, or they just critique the architectures and critique the outputs, but they have no scientific method in mind that'll help us go forward. So that's kind of the state we're in.

we interpret that as an understanding process or is I don't think we need to. See, I think this anthropomorphizing of intelligence and understanding and all that is not necessary, not appropriate, and is is a distraction for many, many problems. Why say it understands?

You know, some of my heritage comes from seeing in in real life in in industrial settings machine learning algorithms being rolled out twenty, thirty years ago. So I when I first went to the West Coast, I visited Amazon and around 2000, they were using huge amounts of data to do supply chain modeling using the neural networks of the day, it was random forests. And it was really working.

They could make really fantastic predictions of whether certain ships would be delayed in the Indian Ocean or whatever, and so certain parts wouldn't arrive in time. And the overall supply chain takes billions of products and sends it to 100 millions of people per day. And so you cannot, there's no way that any human can understand what's happening in that big box, but it's not necessary, and in fact, you can ask, does that overall system understand transport and logistics?

And the answer is who cares? It's it's it does a very important optimization and and prediction process that allows an engineering system to be built around it. It brings down uncertainty.

It makes you possible to do kind of stockpiling and planning, and that's what you ask for. You don't care whether it's has to have a word like understand or intelligence applied to it. That's for the media.

The media that's that's my problem with a lot of these people rolling out AGI and AI terminology. The media laps it up, and they know that, even though we don't have a clue what understanding or intelligence means. And we and our research realize we don't care or need it.

We want to build good systems. Yes.

but we would try to turn it into some kind of an equilibrium or optimization problem, and here's the information that's available, and here's the data, and here's the power and the error rates. And we try to put a little bit of structure around it of that form. And there's always this creative moment.

Like I remember when in high jumping, I used to be high jumping, interested in high jumping when I was a kid. And you would go up to the bar and you jump over it in various ways, and there were the the barrel roll rolling across, you know, was the that was the technique the Olympians were using. And then there's this guy came Dick Fosbury came along, and he says, no.

If I go backwards, I can do better. And no one had thought about doing that. As soon as he did it, everybody did it, and it went up like by a half a meter or something, I don't know.

And so what process led to that? Was it an understanding process? It was just a little bit of let's try something different mixed in with the ability to try it out and to do tests.

So a huge amount of industrial planning is try it out and see what works. Those are called AB tests. And those are done all over the time and I've got nothing against that.

It's not based on understanding, but it's led to optimized systems that can do things that, you know, people hadn't thought about before. So a blend of that with understanding, but just understanding, you know, I was a cognitive scientist and I, you know, I'm interested in neuroscience, should be interested in those things, they're fascinating, but they aren't the leading edge of thinking how to build systems that work in the world. And they're not the leading edge of trying to believe in the next generation systems.

If you put a lot of people keep saying, well, we got to put logic back in or symbols cause that our came previous kind of view of what humans are doing. Probably humans are capable of doing some logical reasoning and probably have some symbols, whether they're kind of built in some complicated network or they're reified somehow, I don't know. My intuition is as good as yours, but really the goal is to, I tend to be an engineer at heart, a mathematically inclined engineer.

I want to say, what are you trying to achieve? Are you trying to displace teachers? Are you trying to make doctors better?

What are you trying to do? And what would be the abstractions and the points of entry into that problem? And then how can you kind of pull back from that and do it in some general way that's elegant and will inspire others?

decompose it into this new frame of thinking? It's not been done really enough for me to have tons and tons of great examples. But we've been looking at kind of modestly scaled examples where there's a so for example, we looked at a little bit of drug discovery and kind of the the regulation, you know, the so I'm a pharmaceutical company.

I test out all kinds of proteins and I throw them in animals and maybe a few humans to sort of see what's working, and I have some understanding, quote unquote, in other words. I know somebody of the evolutionary biology behind it and so on, and that guides me. But at some point, someone's got to really test this out in the real world and decide regulatory agents has got to come in and say, yeah, that goes to market or it doesn't.

Okay, so now you got a kind of tangled web of scientists and pharmaceutical companies, not just one, but many, many of them, and proteins. And now you've got to think about how that system is behaving. So hopefully the regulatory agency is trying to overall, over the entire system, have the number of false positives below and false negatives below.

That's what the goal is of the problem. So it's a statistical problem. Oh, but wait, a stat classical statistical problem, you would just go gather IID data, independent, identically distributed data from some source.

Here, no, the data is coming from the self interested pharmaceutical companies. What's their motivation? Money and whatever.

Maybe the hell they wanna help people and money. And all of that's kinda hidden from you as a regulatory agency. K?

So now the economic mindset kinda comes into play. He says, well, it's hidden from me, but there it's not arbitrary. You know?

I can kinda probe in various ways. And so that becomes very economic. So, you know, economic economists think about how you set prices.

You know? So if I got a lot of people coming on my airline, there's a thousand people that just arrived who want to go from here to London, every one of them has a different price point and that price point will shift in the moment. How eager are they to get?

It's not just because they have lot of money. It's because they have needs. I don't know what those are.

So what I do is I set up various services and various prices that kind of bracket the possibilities so that overall it's likely I'll make enough money and everybody will be kind of happy and the service will go forward in life. That's what so it's a blend of knowing a few things and admitting that you don't know other things, but putting it in a system that actually can work with that kind of mix of asymmetries and and incentives. So, you know, the incentives there are that, you know, there's a certain service and price.

If you pick that one, you're likely to be able to get on the airplane, you're likely to get, you know, have the goodies you need or whatever. And that then doesn't make you do something, it incentivizes you. And in the pharmaceutical world, if I could get them to be incentivized to mostly send in drugs they've done some testing on, or they have some belief, it's a pretty good one and not just throw arbitrary ones at me, then maybe the overall system will actually have the error rate you want it to.

Okay, because if you don't do that, then if it's a drug that'll make a ton of, you know, a billion people will use, then you're gonna make money whether it really works or not. Okay? So you need it to get to market.

How do you get it to market? Well, you just throw it at the regulatory agency, and there may be as a false positive. They just, you know, got a false positive and they put it on the market.

You make a ton of money. And if there's enough of that, the incentives are all wrong and the overall system will not control type one and type two errors. Those are examples we've actually worked on, but I just hope you can appreciate when all this stuff starts to really roll out in society, it's not gonna be there's a few big LLMs and everyone consults on like a search engine.

That's just not the model. It's gonna be there's local data, like I told you about with prediction powered inference. Everyone's gotta what's coming at them.

There's gonna also be local data because I collected it with some expense and I wanna just give it away. Okay, thank God finally Anthropic is paying people for money. That has got to be the future.

So I'm going to have some competitive value in my data and not just give it out, all right? And so now if you start interacting with lots and lots of people that want to get some value out of the interactions, you have to talk about the incentives. What's the incentive for them to send the data, but not only send the data, send correct data, send truthful data, don't just add or not be adversarial.

consumers and they might have their data and you've got Google and then Google was using the data, the consumers are getting a service, and then Google might sell the data over here. That's kind of like a traditional model. Let's start with that.

Okay. So those are really kind of like bore Adam kind of things. We're being scientists there.

We're trying to say, what's a minimal model that exhibits some of the behavior that we want to study here? So let's think about a data market because data is not just now something you analyze to build a big LLM, it's also something you would sell and buy and has value. And also there's privacy concerns about data.

Okay, so let's put a little minimal model together where we could study that, all right? And so one we've done is called, we call it a three layer data market and it exists in the real world. This is a abstraction, but it's a real thing.

You've got a user or multiple users coming into some platforms. The platforms provide a service like, imagine payment service. And as I use that, they get data from me.

They learn about what kind of purchase I've made and so on. And they use that data to make their service better. That's a good little nice loop there, Right?

The problem is that rarely do they make enough money off of that service. They, you know, take a small cut, that the merchants don't like to give them. So they have to do other things to to to stay in business.

So typically now for a long time now, probably twenty years, they've been selling their data to third party data buyers. And these are not evil people, just trying to, you know, ruin people's privacy. They're they're trying to do market research, learning what what would work and what what are people really doing.

So behavioral studies. And so that data is valued to them. They pay for it.

Alright? So, you know, Google doesn't need this because they created this artificial advertising market, which we could talk more about that kind of super powered all this nonsense. But, other companies like Mastercard, what do have to have to sell their data.

So so now it's a three layer thing. And and as soon as that third layer was, introduced, the the equilibrium has to shift because the, the user who's sending their data in just lost something. They lost a little bit of privacy.

Some third party that I don't know anything about is getting data about me. And I can't just accept that, you know, but I can't walk away. And also there's a stress on the system now.

So in an effective economic system, what would happen is that the, you wouldn't just wait for the regulator to come in, the government say, know, no, this can't be done. What you would do is that the platforms would say, well, we'll offer you a tunable level of differential privacy for some cost. Or we'll just say that our company, I'm Google, I'll offer you level 0.

3 and some other company says, well, I'll offer you level 0.7. Okay, so the user looks at that and says, ah, zero seven, that's better.

I I really care about my privacy, so I'll go there. That company then will get start start to get more data, and their service will get even better. And, whoop, you got a little nice little feedback loop there.

But now the data buyers will look at the data from that person. Point seven means more noise has been added to the data. It's less valuable to the data buyer.

Data buyer will say, I'll I'll spend less. I'll give you less money for that. I'll get more money to Google.

And so now you can see there's conflicting tendencies here. The incentives are aligned, but they're not optimal for everybody. And so now the mathematics is not just an optimization problem.

Mathematics is an equilibrium problem. Yeah. But it's an equilibrium problem that involves statistical, assertions, data, and how much you can predict with this data and so on that.

So you quantify that with error bars and and statistical predictions. So you put that all together in a big mathematical system, and you can find the equilibrium as a function of various system parameters. So for example, is there a minimal level of privacy that regulators could require or not?

Or, you know, is there some heterogeneous privacy budget, You know, etcetera, etcetera. You can put in various those sorts sectors. And now you you do a plot of how the equilibria moved.

And the equilibria have overall utilities for all the three players summed up. That's the social welfare. You can ask how high is the social welfare or that equilibrium versus this one versus this one?

And then another regular could look at that and say, well, I prefer this one because it's overall higher social welfare, and laws could be made at that level. Okay. So even though this is a toy little you know, a little toy model, it has the ingredients that I'm very interested in, predictive models, data markets, but money, incentives, and a real system that really is already kind of working, but people aren't thinking about it very well, just like in the drug discovery domain.

If But you take an economics point of view, you can make the system better.

and then we get these modes of- No, we don't have to simulate it. In that case, you can actually write equations and calculate equilibria. It's it's a stackable bird game, and you can actually find the equilibria.

But, you know, in other cases, you would simulate. But the point is a lot of my machine learning colleagues don't know much about fixed point algorithms and, finding equilibria and how they shift as you shift various parameters and all that. That's economics stuff.

Machine learning people are really good at optimization, but this is not an optimization problem. It's a and there's all these algorithms and and other branches of mathematics that find Pareto frontiers and and do it statistically and do it as a function of size of various markets and size of populations and all that. And it's kind of amazing in this era that the two have almost never met.

The economists never had a lot of data to inform their design of their market, so they just wrote down a bunch of equations and made rational assumptions and all that and then found equilibria mathematically or otherwise. And the machine learning people never thought about the equilibria. They just had a lot of data and they used it to do the obvious thing, predict the next word in a string of words.

But the the, you know, the future has got to be that those came those branches come together. The economics equilibrium perspective is critical, but the, oh, it's gotta be adaptive perspective is critical. And also you alluded to earlier some of the machine learning Silicon Valley types just saying, well, we've got all this data, therefore all the behavioral stuff's already built in.

That's too naive, obviously, but it is a useful point of view in a certain sense. The economists do make rational assumptions they shouldn't have to make. And if you put in data instead of that assumption, you'll probably do better.

You'll have some of the behavioral economics already built in. If you do it outside of any economic thinking whatsoever, you'll just make a mess of things. And of course, that's what Silicon Valley seems to be pretty good at.

What is the difference between like data and the kind of knowledge that you're talking about? Social knowledge is very ephemeral and it's very in the moment. I can walk down the streets of Copenhagen here and there's all kinds of little markets out there and what's available and at what price and what I might like and all that, it's all super ephemeral.

And that's kind of, I think, better way to think about all this. You can't just gather enough data to know that that person walking down the street there, they're going to come buy this product. It just in all of our decisions that are our choices, you know, cannot be that you have enough data to cover all of that.

You know everything about what's going happen even in the next, you know, ten seconds. So you have to be a little more humble about that. I have a lot of ignorance, but that doesn't mean I can't build a safe system, like a market of some kind that people could come in and they could not get cheated and they could get value and it could evolve over time.

It could shift in ways. And I don't have to be, I'm not the God figure at the top, you know, designing the human value function or whatever to put it in so that it responds particularly well to what humans really want in my God vision view of the world. No.

It's gotta be a system that permits bottom up preferences to be expressed in the way the human wants to in the moment. And that's not built in, and the system respects those things and wants to maybe learn more about them and then use them in the moment, and then maybe keep some of it, maybe it's all ephemeral and goes away. But there seems to be a huge naive day about what data, even if you have, you know, whatever exabytes or whatever of data, you're going to miss all the details that are probably the main thing that matters for a particular kinds of class of decisions.

Even AlphaFold, based on huge amounts of data, doesn't do well on certain queries. And yes, they'll patch those and it'll do better and better, but always the new questions that people will ask will always be on the edge of knowledge or often be on the edge of knowledge. And people aren't thinking about that.

They're thinking about, well, I just gotta replace the teacher because the teacher is working not on the edge of knowledge, they're working back in the, you know, all the stuff that's already known. That's fine.

for make markets work, but it's not the only one.

Individual humans create abstractions too that work for them. And when those abstractions are kind of useful enough and they kind of get promoted into the culture and that flows up and down all the time. And indeed, that's something that systems could perhaps help with.

I'm not gonna just trust systems to take on that burden, but it could be helpful. And so indeed, it's not just the individual cognitive entity that creates abstractions, and we should just redefine that. Yes, it comes up, but cultures create abstractions.

And you can study the micro economy of that or whatever or not. You can just sort of say those abstractions are part of the culture and they're useful. They've stayed around and they're useful.

And going forward, it's not that we're gonna just keep the old ones, but we're gonna build systems that allow new ones to emerge. And that is not the God figure figuring it all out and putting them in there. So Silicon Valley says, well, you just got it.

We got so much data and we'll have so much that we can do all of it top down. And they're forgetting somehow that, first of the data came bottom up. The data was all contextual and the data was supplied by people and so on.

But they're also forgetting that it all has got to continue to be at a micro level that is kind of going to be beyond their ability to sense, and we're not going to want them so much in our lives. After the search engine, which I thought was a fantastic piece of technology allowed access and all that sort of, then a lot of it was very prime. We're going to put glasses on you.

We're going to put things around cameras in your home and all that, and we're going to know all the details of your life and we're going to make your life better somehow. That just, that equation did not calculate for me.

but that's something that good intellectuals do. That's what, you know, economics, like there's a whole field of behavioral organization or, you know, how do organizations effectively emerge? And those are that's really interesting.

It's not everything, but, you know, there's a lot known there. And some of it's mathematical, some of it's not, some of it's best practices, but those are the kind of ways that these AI ecosystems should be talked about, not just in terms of neuroscience and metaphors of the neurons and physics metaphors and all the stuff that it was part of my heritage too, but it just felt so lacking when we actually see these things, the rubber hitting the road, as you say. And so yes, behavioral organization, how people are organized into things that promote not only good revenue for companies, but also promote democracy and so on.

So there are people that talk about all these things, and I just don't think they seem to have much presence in Silicon Valley, and maybe that's for good. You know, Silicon Valley let them go. You know, they'll just burn a lot of money and, you know, cause some headaches, and and they'll also create things like search engines.

And and I think a lot of the companies are also really focused on creating value. I do think Amazon is different from Meta. Amazon has got a business model, bring packages to doors.

And behind that, they create some technology to support that, and it's mostly to the good in my view. You have to worry about labor markets and so on so forth, but those are all good things to worry about. But just creating computational artifacts that make predictions and that if you were to wear goggles, you would be able to, you know, live in their world.

It's not a business model. It's a science fiction dream that it may or not be may not be helpful for humanity.

to generate the songs with with AI. Right? Yeah.

They are. And I I'm a you know, I have a a project. I'm a a scientific adviser to something called United Masters, which is an alternative, which has, musicians keep their their their their work and and United Masters connects them to brands and to other kind of opportunities so that they are kind of more like a real artist, not just like that they their their song got streamed and they got a little bit of money.

Because Spotify indeed is not it's close to perhaps a monopoly, but, it's not incentivized to pay pay there's not a pricing. There's monopoly prices, if you will. And so one would hope that somehow the market will fix that, that enough young artists will say, I'm getting screwed here.

I'm not making any money. And another service will emerge. But we are in an era where some of these services do become monopolies pretty quick.

And that's, I leave to my economist friends to think that through and to look back at historical examples and think about, you know, is regulation needed or is or is there other market making mechanisms that'll make this more healthy for human beings? I'm not against Spotify, it's a but you know, it should be part of an ecosystem that actually rewards the artist more. Right now an artist is getting paid very, very little on, and I think I don't believe the prices are being set under competitive mechanisms.

But I think there's this broader macroeconomic view of what are these systems doing and what are they what's their role in society going to be? And with the search engine, I was many of us were puzzled about how they would make money. It just didn't seem like, you know, there was a money making.

And then the whole advertising thing was a bit of a surprise, at least to me, that it would become so huge. Of course, the underlying thing is that people expect things for free. All right?

And so Google couldn't kind of make payments, but I think they made a mistake at some point. I think like with YouTube, when they acquired YouTube, YouTube is more than just pointing people to a website. You know, YouTube is incentivizing creators to create things that people will watch.

At that point, I think a socially responsible Google to critique them a little bit would have said, oh, we've created a market here. We've created a producer consumer relationship. We've got to make that market a little bit more valid, and we could actually have that when someone's watching things, they can have some sort of an economic connection to the person who made it, and then it can be incentives flowing that this person is now incentivized to make more because here's my audience.

Connect it directly. Instead, it was all going through Google, and then Google was putting advertises next to make a ton of money for themselves. And then there's a modest incentive to give back a little bit of money.

That to me was a huge mistake, and then Facebook made it even worse.

I think it's science fiction and I think science fiction's important for society, but it's also at the level it's being promoted and those kinds of voices, it's really hurting twenty five and twenty year olds. These young folks of whom there are huge numbers are excited about technology and they wanna build things that help their family and help their country, actually more of their family than their country, honestly. And they see real opportunities in doing that.

And they're kind of being told by the leaders, well, we had our fun. We developed a bunch of algorithms. We did it, and we were just interested in the pure, understand intelligence, even though they didn't understand intelligence.

They built, you know, gradient descent algorithms. And now you guys, you can't do this because it's dangerous. It's gonna it's gonna wipe out humanity with a with a high probability, or it's superintelligible to arrive soon, so there's nothing left to do.

That's in your lifetime. That is so demoralizing. So demoralizing.

And that thing, I think that bothers me the most. I mean, the second part that bothers me is there's no economic thinking going on there. It's zero.

It's really about cognitive science mentality or neuroscience. So we figured out how the brain works. It's gradient descent with a lot of distributed neurons.

And the fact that these LMs are working so well shows that we figured it out. They wouldn't work so well otherwise. Now, I think that's dubious.

We don't, the brain is way beyond, mean, you ask a neuroscience if this has anything to do with the brain, basically they'll say no. It's a nice metaphor, it's cartoon. Does gradient descent work at massive scale?

Yeah, more than we would've ever imagined. But is it showing its weaknesses? Yeah.

Can it be fixed? Certain areas? Yeah.

You build certain verticals, they'll do good things and it'll make mathematicians go faster, but it won't put them out of business and so on. It's having a big effect on society. I worry more about labor and capital relationships than I worry about it deciding to take over.

So the rest of it, that to me is more on the ground sort of, you know, how does the next generation take technology and work with it? And I don't think that voices like that are actually helping that generation to actually perceive what they should work on and why. Super intelligence versus extinction, those are your two options.

And then goddamn it, those aren't the only two options. There's a huge number of very positive things that can be done at human scale, And let's hope that enough of the young mentalities kind of get behind that. But they don't have enough examples of people out there who made money by making Sam Walton make a life better, you know, not clear.

And and and, I think in previous generations, was a little bit more, here's people that are out there making things that, vaccines or whatever, oh, I want to be like that.

but why is it? Why do they believe that? I mean, they're clever in a way, in a recognizable way at some level.

They're taking all this human cleverness and packaging it in a new way. And again, I think it'll kind of always be missing a little bit of the point because it's not in the moment, it's not the ephemeral stuff, but that doesn't mean it can't be even more clever. And I kind of think that that's okay.

I think that for me, the goal here is not to build a superintelligence and have it dictate or tell or anything like that. It never was. And I'm kinda shocked that some people seem to think that was always the goal.

To me, it just never was. Rather, again, I think I said this in the very beginning, humans are wonderful. You know, I'd hate to have robots taking over from us, and I don't think that's gonna happen.

There's just too much good about human nature and about what humans are, you know, able to produce that are shockingly beautiful and creative and inspiring. We need to support all that. Now the issue though is that the flip side is that we are not we're far from perfect.

Know, our people really hurt a lot of people and they are being empowered to do yet more of it. And we are very narrow minded. We also don't understand, like, you know, people hurt other people often because they don't understand their motivations.

They got a misunderstanding. Well, how many wars are created because someone didn't understand the intentions of the other side? And they said, well, let's just, you know, proactively, let's just bomb them.

That that's just all the time. That's how humans act and think. What's missing there is an appreciation of uncertainty and information signaling and sort of eventually game theory arose to help people think it through a little bit, but it's still extremely rough.

And if you look at our political system, you know, an an aged, you know, charlatan leading a country, You know, this is our optimized human system for making decisions at, you know, of the highest kind. There's so much room for improvement of the human being. And democracy's got to be the way, but democracies right now are a few aged people sitting in various rotundas in various capitals, you know, not knowing what they're talking about mostly.

We have a very broken human system in many, many domains. We have a few that are, I think the universities are pretty good and I think a lot of companies are pretty good and a lot of human associations of various kinds, various skills are pretty good, but we have so many broken ones. And so to me, that's what AI is about.

AI is about helping the things that were too hard for humans and aiding the information flow so that humans could actually make the good decision in the moment that most of them really wanted to make and not making the bad decision that they were afraid they had to make because they didn't know enough. So there's so much to me opportunity if you think about it at that level, that's what AI is about to me. AI is not about this replace the human with the computer, the recursive self improvement stuff.

I mean, it just feels like a metaphor. We work with recursive algorithms, we work with improving algorithms. I don't see that getting out of control, like a virus that somehow, we're gonna work with these systems and we're going to, like I say, hopefully mostly focus on getting right some of the things that evolution didn't quite get right for the human being, especially at scale of 7,000,000,000, evolution perhaps didn't prepare for that.

And focusing on that to me is what AI can be about. So I'm positive. I'm bullish about AI in that sense.

And I'm appalled by the dialogue has become between the people that have all the money and wanna just build something for build its sake and the people that are just intellectually saying it's terrible, it's gonna destroy all of humanity. That's the dialogue in the public eye right now. And that's just, I find that so harmful, and it does bother me that people that worked on it for all these years think that we've reached the end, that somehow the gradient descent is like the brain, and therefore you could take multiple brains and you could fuse them together, and oh my God, it's gonna just do incalculable things.

That's such science fiction. Whether it's even true or not, it's worth not even thinking about it. What's the path?

How do we engage younger people to do things that are actually positive? And what mechanisms are you gonna talk about? What kind of education are you gonna talk about?

What what goals are you gonna set? And the thought leaders are not talking any of that kind of language. And, you know, I I think it's unusual for human history that thought leaders are heading off in these two directions.

airplane crashes at massive scale these days. There used to be a lot when I was a kid. And it's because of the autopilots.

And mostly now planes are flown by autopilots and the human can come in as need be, but it's because of that. So there's this blend of automation with human is actually the most effective way to go. It's again, it's improving.

Humans didn't evolve to be flying this big thing up in the air. And so you can improve upon human ability there. You put the two together, you can do something that's helpful for everybody.

change in weather patterns, you've got some person who did something stupid. It's easier because yeah, up in the air, there's a lot of room. In three d, there's a lot more room than in two d, But in two d, you got all these cars floating, flying around and you've got tens of thousands of people dying each year in each country.

It's a mess at some level, even though it's very important and effective for many of us, so we do it. But a hybrid system that had a lot of autonomy with some human and and so on. But the other thing about at the system level, just putting a super intelligence behind the wheel of a car, dumb dumb way to think about technology.

Is there any hope? I mean, I don't know what you think would be the thing that would make these folks update. By way, I think that Ilya and others have done some great things.

I mean, they built some systems that all of us are not only using, but kind of it's changing our thinking and all. And I think that's kind of what I get out of what they're saying is that I'm a builder. I'm not a You think I'm a guru and a thinker and maybe I think I am too, but maybe I'm really better as a builder and I can build things with the resources that are now available to you.

And again, it's not just the money, it's the whole Internet and the whole, you know, all the things that previous generations of people did that one thing that bothers me a lot about these people, not the Elon Musks or the Sam Aldmans, they're just coming in and taking the cream off the top, you know, from all this effort that people put in. And a lot of these people are, you know, rightly not not just that they wanted the credit, they just are annoyed that this is the direction that these people are not taking it without the appreciation of why were these people building these things? Not for you, but I had other goals in mind.

So I think that, yes, these are some builders and there's some very impressive builders, don't, and I think there's this system that we call it Silicon Valley, whatever that these people live in, and where thinking the more outrageous, the more far flung, the more physics, biology inflected, neuroscience inflected that your language is, the more you sound like a guru and people enjoy that posture and that activity. And it creates great amount of money. They don't care about the wealth perhaps, but it allows them to yet be more prominent because they can now have another company that tries some other crazy thing and then survive it.

As a work, that's a sign of you had a great idea. It's a I wouldn't wanna be in that world. And I am trying to becoming a bit of a historian.

I mentioned chemical engineering, electrical engineering, but you look back at the history, there were some glimmers of some of these kinds of things, but I think this level of detachment from reality is unusual for human history. This level of my crazy science fiction 25 year old dreams are all that's what I'm gonna pursue for the rest of my life, whatever with, you know, come hell or high water. And then at some point, I'll flip because I realize, oops, I didn't really have a great goal in mind at all.

And what have I got here? Oh, I've just spent a lot of money, and I got this thing, and I don't really know what to do with it, and I'm worried about it.

the difference? Oh, well, game theory is a mathematical discipline. It started with von Neumann in the 20s, and it's got many, many branches to it.

And it's a mathematical way of thinking really. And one way I like to think about it is that it's like F equals MA. It's a set of it'll make predictions, okay?

So if I write down a game, just like I wrote down F equals MA in some coordinate system, I can now predict what'll happen. And in the case of F equals MA, I integrate a differential equation. In the case of game theory, write down the game and I calculate the Nash equilibrium or the correlated equilibrium or some other equilibrium concept.

And I say, here's what'll happen in nature because my little mathematical model captures the appropriate ingredients. And for F equals MA, yeah, the the thing follows a parabolic, you know, curve. It means the theory is right.

And then Einstein said it's not quite right, and he makes a better one. And in game theory, same thing. You look at, okay, do those those equilibria actually characterize how systems and organizations and people behave?

Sometimes yes, sometimes no. But those aren't, that's not the end all. So there's all kinds of other equilibria, Stackleburg equilibria and sequential equilibria and various kinds of figures of merit, various social welfare constructs, various regret constructs, and all sorts of things.

It's a whole huge field of its own. And let's think about it eventually kind of being as big as physics because it's all about strategic interactions and so on, not molecular interactions, but now you can also ask the inverse question. In physics, the inverse question would be, I wanna build a bridge.

So my goal is not just to if something follows a parabolic path or something. I want that bridge to stand up. So I invert f equals m a.

Okay? I go from the goal back to the design that would ensure that that thing stood up. Alright?

And so most engineering fields are inverse problems. They go from the goal back to the design. Whereas the the forward direction is science.

You say, here's the here's the setup. Here's the prediction. And is the prediction realized or not?

So, okay, yes, it is. That means the model must be good. So what's the inverse of game theory?

Okay. Well, it's outside of economics, not talked about perhaps that much. Game theory sounds like it's sort of everything.

Well, the inverse of game theory is what's called mechanism design. And mechanism design says, oh, I want a certain outcome in the world that that person gets paid, that the wealth is divided equally, that there's some fairness or some market that's created. What game do I design so that that outcome is realized?

So I'm the designer of the game. I'm not just taking the game as given and then looking at what it predicts. Mechanism design has got many pieces too.

I work in contract theory. That's a part of mechanism design. It says, what if I have two entities interacting, they're not symmetric, that one knows more than the other and they have to interact with each other.

That's that's contract theory. Auction theory is another part of mechanism design where I've got a bunch of people coming in and I think of them as symmetric. I don't know who's got more money than who wants to bid more than others, but I have this mechanism called an auction that reveals their value.

And the outcome is that the person who wanted the painting the most got it. That's one desired outcome. Anyway, long story short, game theory is a super rich, not so old discipline, hundred years now, that's continuing to evolve and continue to supply all kinds of algorithmic ideas for those of us who are in the business.

So I've been mostly a statistician in my career, kind of worried about uncertainty and probabilities and decision making and uncertainty. But when I go to equilibria and games and or or economic ideas, the naive theory is kinda is part part and parcel of the thinking.

Marvel Television's Wonder Man, an eight episode series

now streaming on Disney plus A superhero remake. Not exactly what we'd expect from an Oscar winning director. Action.

I don't know what he thinks of himself as, but I think of him as a statistician with game theory background too. He's in the school of like the Phil Davids of the world and the David Blackwells who spilled out of statistics to do all these other things. And so, classically p values were just kind of a one shot quantity that statisticians would talk about that it was like Fisher that said, I've got a model of what's gonna happen in the world.

It gives a probability distribution on the outcomes. Some outcome arrives, it looks very improbable under that model. The model must be wrong.

That's kind of the p value. And so the p value is the tail probability. The problem is if you do that repeatedly and you look at maybe the smallest p value along the way, that's called p hacking and that gives you wrong answers mathematically and then in practice.

So e values are different. It's an expectation of some non negative random variable or a non negative supermartingale in more generality. So you're watching this evidence kind of accruing and you make sure the expectation of that evidence is less than or equal to one at each step.

And then you can think about a multiplicative kind of evidence gathering, that if it's always an expectation less or equal to one, then it'll kind of stay below one. And if it's non negative, it'll just kind of decay decay away. So under the null hypothesis, I've got this stochastic process, which is kind of decaying away.

Well, I can look at that at any time and sort of assert that it's decaying away, and I can look at it repeatedly and keep asserting that, and I can have control. There's something called Will's inequality that Vladimir and others have exploited that says that can be controlled over the entire path of this thing. So now we can do statistics in a new way.

It's called any time inference. We can peak, we can change, we can gather new data, we can do this in an updated way or day. Very liberating.

And Vladimir's, yeah, one of the leaders of that. And an e value is one of those Martingales stopped at a particular time. By the optional stopping theorem, you could stop it whenever you want.

So that has opened up a lot of connections. In fact, our statistical contract theory, what is a contract? Remember, was like services and prices.

Well, the services are like evidence gathering and the price also is part of the, it's a random variable. And it turns out that we can have an incentive compatibility in contract land if and only if e value in statistics land. So there's a nice tight connection between game theoretic probability and the theory of incentives.

So to me, uncertainty quantification is rarely just here's an error bar, that's kind of classical statistics. And it's more what the context is. Here the context might be a contract or it might be some other evidence gathering mechanism.

there's economics and there's computer science and there's statistics. Well, are thinking styles. I don't even even call them by those disciplines.

So there was a paper by Jeanette Wing, a few decades ago talking about computational thinking. So it says, oh, computer science has developed these thinking styles that are more abstract than just computers. It's modularity and abstractions and APIs and all that.

And why don't we teach everybody in all the sciences and all the disciplines to do computational thinking? And I think that's totally right on, that's great. But lots of algorithms don't come about from those kind of computer science principles.

They come about from thinking about inferential uncertainty and how do I gather data to make predictions about things that don't yet exist and think about incentives. How do I make sure that, you know, incentives are in place? And I called those two kinds of thinking, one of them inferential thinking.

So not just statistics, a lot of fields have inference in them. And then economic thinking, it's not just economics, it's also scientists of all kinds and legal scholars and so on. When you put those three together, you get a pretty good platform for training of the next generation and a pretty good platform for problem solving of the kinds that we've been talking about this entire time.

Just one of the fields, just computational algorithms and optimization, that kind of gives us LMs. Fine, great, but it doesn't give us any of the context around the LM. The incentives kind of gives you the whole thing that we've been talking about.

And then statistics to me is critical. It thinks about what kind of errors I'm going to make, how to make sure the data's controlled so I don't make the errors. And we put the three together, yeah, they also bring kind some partners.

The economists talk to the behavioral psychologists, the computer scientists talk to the physics people or whatever, the statisticians talk to the legal people, whatever. There's a whole sub communities that come together. So to me, if you put on this triangle there and you think is around it, it starts to become a new way to think about academia.

This is the liberal arts of the era, This is the core. Now my colleagues in the humanities might disagree.

And like, what's the difference? Why does the language model not really have any idea about its confidence? You should ask the language model builders because all they're doing is predicting the next word and there's not any thinking about uncertainty quantification in doing that.

And you can graft in ideas, but they're dubious. They're often putting up dubious prior And so you go to the statistician and that's what people have done and they've said, okay, I could just treat it as a black box and I can put conformal prediction around it. It's a nice method, doesn't require a lot of assumptions.

So yes, that's true, but it makes a lot of there's an exchangeability assumption. The data, if you scramble it, you get it's the same. So while I think all of that's really crucial and important, I tend to think more about the broader context.

So I gave an example in that article that you mentioned of a duck who goes to a lake and this is a statistician duck. It's kind of calculated that over the last year, there tends to be twice as much grain on that side of the lake than on this side, two to one ratio. All right, so now the next day I need to decide on the duck which side of the lake I go to.

And the Bayesian duck who has those probabilities would then do the maximal expected value and they'd go to the left side of the lake with probability one, because they're all right. But the actual ducks don't do that. They go to probably two thirds to that side of the lake and one third of the other side, they're hedging.

But it's not just a hedging thing, hedging would just do occasionally go into the other side of the lake. They're actually getting the right ratio. And so the explanation is that you weren't thinking about the context right of this uncertainty, okay?

It's not just you, the individual duck, probably you evolved in a world where there are many ducks. And if all the ducks went to the same side of the lake, obviously you've missed out on a resource. And so is there an algorithm that allows many ducks to cooperate here?

Well, if they all have that same uncertainty, then they can sample with probably two thirds and go to this side versus one third. That's actually a Nash equilibrium of the bigger system. All right, so the right way to think about uncertainty there is that in the context of the population, what should be, how should I use my uncertainty?

Another kind of uncertainty, that's kind of the economic side. Another uncertainty in economics is the one I've alluded to, information asymmetry. You know things I don't know, and you have expertise I don't know about, but we're gonna work together and I'll maybe give you a contract, a menu of options.

But even if I interact with you for a while, still might not know. There's things you're gonna know that you're not gonna give away to me. And maybe you'll hedge, you you'll lie a little bit, so I I don't know about that.

That'll never that's not just sampling. That's a different kind of uncertainty. And then finally, there's what I like to call providence, that's more like a database kind of uncertainty.

If I wanna do a medical operation and you're a doctor and you look at the data for people like me, here's if you do the operation this way, the probability of survival versus this. And I look at that and say, great, but now you tell me, oh, that data was gathered ten years ago. And I'm gonna say, okay, my confidence interval should go up.

Well, classical statistics could talk about that. In fact, I'd be more of a Bayesian to think about that, but it doesn't. It just sort of the data is the data.

And it should be in a bigger system that as data's flowing around, there should always be tagged with metadata about how old it is and that should be quantitatively brought into the uncertainty quantification. We're not doing anything like that right now. And so the poor LLMs, which are basically doing none of the above, have to strike out a little bit in all these directions if they're gonna start to do like what humans do.

We are pretty good at getting these, with a little bit of providence, oh, it's old data, I discount that. We get a little bit of context, oh, there's a social environment here, I should just do the same thing, should randomize. Oh, there's some sampling uncertainty, you know, and so on.

We put all that together almost seamlessly. And then we do this in a social context where if I don't know how to get from here to the other side of town, I will ask someone who looks Danish. I know something about how to gather more data and so on.

So the poor LLM has none of the above. And so what should it say when you ask how sure are you? And all it's doing to the best of my knowledge is that it's just, well, in the past, someone asked a human on the internet, how sure are you of that equation you just wrote down?

And someone said, oh, I'm very sure because of this or that. And I think it just mimics those kinds of assertions, but that's not reasoning under uncertainty.

That's, these are classical areas of statistics. Optimal experiment design. You know, for that subpopulation, I don't have enough data.

I'm making a bad inference. And data collecting in the context of inference, in context of making assertions and doing that repeatedly, that's what statistics has long focused on. So I think give them credit for handling a kind of an active form of uncertainty reduction.

But again, for me, uncertainty reduction in the large comes about from much broader sets of components like a market. Like if I, and I use example in the paper where I wanna have a restaurant like this or pizza and I need tomatoes. And so if I had to forage for tomatoes every day, that would be pretty uncertain whether I would have pizza that evening.

But because there exists a market where someone else did the foraging, there's a stable amount of tomatoes every day. I can build my restaurant assuming that that's true, that my uncertainty for finding tomatoes went down, therefore I can build on top of that and do other things. Markets mitigate uncertainty and they don't do it because someone designed an optimal experiment design or ran or did some multi arm bandit, not directly, but because the market did try various things out, there's incentives for people to explore and exploit.

You can't reason with the sun. Trust us. We've tried.

This summer, it's time to put that angry ball of fire on mute. Columbia's Omnishade technology is engineered to protect you from the sun's harsh rays that can burn and damage your skin. The sun is relentless, but so is our gear.

Level up your summer at columbia.com to spend more time outside and less time slathering on aloe lotion. You're welcome.

Columbia, engineered for whatever.