Deep Learning is Not So Mysterious or Different - Prof. Andrew Gordon Wilson (NYU)

So when I say deep learning is not so mysterious or different, I'm not saying that deep learning is not mysterious or not different. I think it's actually both. It's rather that the ways in which people often think it's mysterious can be relatively well understood, both intuitively through a notion of soft inductive biases, but also formally in terms of rigorous generalization frameworks that have actually existed for many decades.

And a lot of these phenomena can also be reproduced using other model classes. I think deep learning really is distinguished in its relative universality, how broadly applicable it is relative to other model classes. That doesn't doesn't mean that it's anywhere close to being completely universal, but it's sort of a movement in the direction of universality, a significant movement.

It also does representation learning incredibly effectively. It has properties of its optimization objective, its loss landscape, which are relatively different and surprising, like mode connectivity. And so I think deep learning is certainly different and mysterious, but often not in the ways that that people might might believe.

So I'm very excited to be talking to you both, even though it can be challenging knowing that there are so many people watching. But I think there are so many fundamental misconceptions in the way that people understand generalization and model construction and artificial intelligence, it's really important to hear a different perspective around, for example, how it's completely fine to build a huge model that will also have a stronger bias for simple solutions, more of an Occam's razor like behavior than even smaller models. And these sorts of perspectives actually help us understand phenomena that are often seen as very mysterious, like double descent and benign overfitting and overparameterization, and give us a principled approach for thinking about how we're gonna build our own models for whatever properties we're interested in.

There's this fundamental trade off between bias and variance.

So I think the bias variance trade off is an incredible misnomer. There doesn't actually have to be a trade off.

So, yes. I mean, this is what happens when you challenge conventional wisdom. But in some sense, I think that we should always be trying to do that because otherwise, we're just preaching to the choir, and then what's the point?

If no one if you're not changing anyone's beliefs about anything, then maybe it doesn't make a difference. And so I think it's important to really try to understand what do a lot of people believe that might be wrong and then just unpack that. And it's also very exciting and fun.

But it's challenging because, of course, the initial instinct will be to resist whatever you're saying. But then over time, and if you try hard enough and if you talk to amazing communicators like you two, then you can start to have an influence. And I think that's really important because so much progress has been stalled, I think, by just getting stuck on misconceptions.

Like, once a certain number of people believe something, it's very, very hard to change their minds no matter what you say. And I think as a consequence, we've been in all sorts of local minima in machine learning and AI research because we haven't been able to get unstuck from these erroneous beliefs. And there's a whole roster of things like this, like the role of implicit biases of stochastic optimization and generalization, I think, is significant, but also significantly overstated.

How we can have really large models that also generalize well, even when there's a small number of data points is something that is not very well recognized. And in fact, think is one of the primary drivers of scale being important for achieving good generalization. So not just flexibility, the simplicity of bias that comes about through scale.

I think another misconception is this idea that we should change our model depending on how many data points we happen to have available. And this might even be the most controversial one. The reason I don't think we should is because we should always honestly represent our beliefs.

And our beliefs about the process that generated our data typically shouldn't change depending on how many data points we happen to have access to. And you can actually demonstrate that these principles work in practice. So you can have models that will be very good when you have a small number of data points and also very good when you have a very large number of data points.

And so this relates to not necessarily needing to have hard constraints, but instead combining expressiveness with Occam Tracer. Yeah.

And I took a poll at the end and it did shift. And so that was promising. Yeah.

I I sort of joke sometimes that if I hadn't met the airline passenger dataset, I don't know really what my life would be like now because it's driven so much of my research. And it's just amazing how people are biased towards choosing the linear function or the cubic polynomial, even if in practice they're not making that choice. Like on CIFAR, for example, it's not uncommon to use a neural net with tens of millions of parameters to fit a training set with tens of thousands of data points.

And even before deep learning was popular, we were doing nonparametric statistics where we were working with models like Gaussian processes that were inspired by taking infinite limits of neural nets that are more flexible than any neural net you can fit in memory. And other popular covariance functions as well, like the RBF kernel, really are like saying, I wanna use an infinite order polynomial. And so even in these kind of classical statistical models, we're implicitly saying, well, if we're unhappy about the third choice, it's actually because it doesn't have enough parameters.

We want infinitely many parameters, not just 10,000 parameters. And I think another way to say this is that parameter counting is a very bad proxy for model complexity. Really what we care about is the properties of this sort of induced distribution over functions, rather than just how many parameters the model happens to have.

And so you can have a distribution over functions, which is very flexible. It can represent many different solutions to a given problem, but it can also have very strong preferences for certain types of solutions over others. And strong preferences doesn't mean saying that certain things are impossible necessarily.

They can just have Epsilon probability. And I think that that is really meaningfully different than saying, okay, we're we're gonna have a hard constraint, and we're not gonna represent those solutions. A, because it's not an honest representative of our representation of our beliefs to to have those hard constraints.

And b, because we see in practice that when we do have these expressive models with simplicity biases, they're much more adaptive. They're much more automatic. So when you have a small data set, it sort of does the right thing.

When you have a large data set, it also does the right thing. And so you don't need as much human intervention. And arguably, that's the definition of what really machine learning is trying to achieve.

It's trying to build an intelligent system that doesn't require manual intervention.

It is. So there is some hand wavy intuition around loss landscapes, and I think this is born out empirically. So we can sort of understand, for example, that the solutions that we're finding when we're building larger models are more compressible and are flatter, etcetera.

But this is really ongoing research. And I think this is one of the most important questions to understand right now is like, why does scale rigorously speaking produce a simplicity bias? And can we get that bias in a more elegant way than just building bigger models?

It's really hard to choose. In some sense, both. I'm mostly driven by trying to understand things.

And so this can be done in a variety of ways. A lot of our papers empirically try to understand model behavior. And so I feel like this is a scientific approach to machine learning.

And one thing that really motivates me about this type of approach is whatever you learn will never go obsolete. So quite often newcomers to the field and even very experienced researchers feel distressed at the rapid pace in the discipline where you see methods getting published at a conference and then becoming obsolete within a month. Or when you go to the conference, everything you're seeing is sort of, you know, been replaced by some other algorithm.

And wonder, okay, well, there any point to me sort of investing myself significantly in building a model if I know that it's not gonna be used by anyone for any long period of time. And I think one way to address that is really to try to combine what you're doing with an understanding of why things are working. So if you're building a model that gets better performance on some problem, there's a reason for that.

And if you can understand the reason for that, that understanding will outlive that specific model and how widely it might be used. And I'm really hoping to do research that will be relevant in hundreds of years from now. And so I think these questions around model selection, for example, and Occam's razor, people will never stop asking.

Hopefully they'll be able to go back and read not just my work, but work that's been done in this space and think, okay, this is useful to me in thinking about how to approach some of these questions. And so in this respect, I would say I'm a scientist and I try to combine classical theory with empiricism towards understanding model behavior. And I think if you really understand something, hopefully it's something that you can demonstrate in practice.

And so I also try to combine some sort of practical demonstration with a lot of the work that I do. And this process also involves engineering. And sometimes understanding those low level engineering details becomes really fascinating and leads to unexpected intuitions about the principles behind model construction.

I think this is something that's perhaps underappreciated. Quite often, you can have a great idea and whether it works or not depends very significantly on all the low level details like numerical stability and other things like that. And when you get really deep into those details, sometimes you can discover things at a higher level that are also very significant and how we should think about model construction and algorithm design.

Very cool.

So I'm Andrew Wilson. I'm a professor at the Crown Institute of Mathematical Sciences and Center for Data Science at New York University. My work focuses on having a prescription for how to build intelligent systems.

What are the key principles involved in model construction? I think although the field has made an extraordinary amount of empirical progress towards building more performant machine learning systems, we're still at early stages of understanding what principles should we broadly embrace when we're approaching our own problems. And so this involves work on understanding inductive biases, so what assumptions we should be making.

And so this relates to symmetries like equivariances, maybe we're modeling molecules or rotation images could be translation variant. How do we represent those invariances? How do we learn them automatically?

How do we discover interpretable scientific structure in our data that tells us something maybe surprising that we didn't know before that will go beyond a particular application? How do we represent uncertainty towards decision making? Arguably a prediction that's just a point estimate without any kind of error bars associated with it isn't really actionable in the real world.

If you have a autonomous car and it says there's a stop sign five feet ahead plus or minus 10,000 feet, you can't really do anything with that information. But if it's plus or minus one foot, then you can really act on that information. Observing that almost makes you paranoid.

Okay. Now I really need to represent uncertainty because if I don't have that uncertainty, then, you know, machine learning can't meaningfully engage with the real world. And Bayesian methods, I think, are a really great way of reasoning about uncertainty.

And so that also forms a big part of my research program.

And I agree that it's very appealing to say, well, if we know that some constraint applies to our problem, then we should encode it in our model. And I don't think that's particularly controversial. There are some perhaps surprising results that in some instances, even when you know what the constraint is, you can do as well or better when you don't represent that constraint.

And this can do this can be due to the dynamics of how you train your model, etcetera. But generally, the way I would respond to this is, really just say two things. The first is there aren't that many instances in practice where we know exactly what constraints we want to have, even when it comes to things like physical conservation laws.

So rarely are we modeling, for instance, closed systems. You can have a dynamical system and maybe you have like a pendulum with like wind in the air or whatever, and now you have some sort of violation of conservation of energy. And so you want to perhaps instead build models that are just biased towards these constraints without being exactly constrained.

Secondly, when you do this, when you represent so called approximate constraints or you have soft constraints, so you have a model which is very flexible, but it says, well, if we can fit the data in a particular way, then we want to do that. Quite often, it will just collapse down onto those constraints if it provides a consistent explanation of what we observe. And so if you're paying any kind of penalty for deviating from those constraints and you can perfectly explain your data with those constraints, you'll just collapse down onto that.

And so I think as a prescription for model construction, it's often going ful to try to embrace expressiveness, but at the same time have a simplicity bias, can be formalized in terms of compression. And this is really an honest representation of our beliefs in many cases. Like another way to describe my philosophy of remodel construction is just honestly represent your beliefs.

And we believe the real world's a complicated place. And if we combine that belief with the idea that simple solutions that are consistent with our observations are more likely to be true, then we can often see desirable behavior in quite a variety of different settings. And so in terms of like having cake and eating it too, I think one of the most surprising findings that that we and and others have had is that quite often you can increase model expressiveness while simultaneously increasing its biases.

So larger models are often more inclined towards simple solutions. And there are sort of demonstrations of this that have been just hiding in plain sight, like double descent. This idea that as you increase model flexibility, your generalization error first gets lower.

So it improves as you capture more structure in the data, it gets worse as you start to overfit, and then it gets better again. And in that second descent, typically all of the different models that you're considering are fitting the training data perfectly. The only possible way that larger models could be generalizing better is because they have some other sort of bias, like a simplicity bias, rather than being more expressive.

I think time and again, researchers express surprise at the fact that they can have these massive models, billion parameter plus models trained on relatively small data sets that aren't overfitting. But in fact, actually, they've made the models even bigger, they would be less likely to overfit. And so quite often expressiveness and soft constraints can be aligned.

So I think the bias variance trade off is an incredible misnomer. There doesn't actually have to be a trade off. So the idea, I guess, behind the classical bias variance trade off is that your generalization error can be compartmentalized in these two terms.

So bias, sort of how well you are fitting the data essentially, and variance, like how your fits vary depending on, like, you sample different points from this distribution that you're trying to model. And it's true that sometimes if you naively build like a really large polynomial, for example, you can have low bias and high variance. Whereas if you build a small polynomial, maybe you have low variance and high bias.

However, approaches like ensembling are actually a good way of getting low bias and low variance. And it turns out actually building large neural nets are another way of getting both low bias and low variance. You actually have flexibility combined with a simplicity bias.

Right. So overfitting is real. Absolutely.

But the conventional wisdom about how we should approach it, I think, is fundamentally misguided. So and this is rooted in things like the bias variance trade off. Like, let's constrain our hypothesis space so that we can't have a bad fit to the data that will make bad predictions and so on.

Whereas instead, I think if we just embrace the honest belief that there are many possible solutions, even if they're not probable for any given problem, combined with this sort of simplicity bias, we won't tend to overfit. And interestingly, the prescription is almost the opposite of what people think it it perhaps should be in principle. Like, build a smaller model is usually the the prince the the sort of the the prescription for avoiding overfitting.

Or to enforce of simplicity. Exactly. Yeah.

the models are typically fitting the training data perfectly. And so the loss is not really the decisive factor anymore between which models we're selecting. It's something else.

And so those other biases start to dominate. And so this is actually really important, I think, when people talk about phenomena like flatness. So flatness is this idea that if you perturb your parameters, then you can still get a relatively low value of the loss.

And there are all sorts of debates about like the role of flatness and how relevant it ought to be and understanding generalization, etcetera. But I think what a lot of these discussions miss is it's just one of many properties that control generalization. So if I had to choose between a model that has very high loss, but it's very flat versus like a very flat solution versus a model that that finds a solution that's low loss, that's that's relatively sharp, I would almost certainly choose the low loss solution.

And so when you're in that kind of second descent regime, you now are controlling for the value of the loss, and it's just the flatness of the solutions that's increasing. And so that's sort of one way of knowing maybe like what side of the curve that you're on and thinking about how big should you make your model. But I would also just say, make your model always as big as possible.

Just try to combine what you're doing with some sort of simplicity or a compression bias. And there's a question of like how you do that. But I think there are good ways that we have of thinking about how to do that.

Trick question.

I agree with the idea that representation matters. Representation meaning sort of how you're solving the problem even if you're getting the same performance in a particular application. But the reason it matters is because different representations that are achieving the same performance might give you different performance than on different problems.

And so if we're trying to build more general agents, we want to understand what sorts of representations are going to provide a better general description of the real world. And so that means we want to avoid things like shortcut learning and so on. If it's just gonna lead to good predictions and some contrived problem and not really in the real world.

And so there's this question of like, can we understand what sorts of distribution shifts we might typically encounter? And can we build methods that have broadly more robustness to a variety of different types of realistic distributions? I think this connects to things like no free lunch sort of thinking.

So the no free lunch theorems say that every model is equally good in expectation overall problem sampled uniformly from a distribution over all problems. And there are other no free lunch theorems that say no single learner can be good on all problems. The issue I think with these theorems is not their mathematical validity.

What they're saying is correct under the assumptions they're making, but rather that the assumptions they're making are not a good description of the real world. So the real world is a small corner of all possible data sets. It's not drawn uniformly from a distribution of all possible problems.

If we were to do that, we would mostly just get noise. And so the question then is to what extent is the structure across real world problems shared? And at what level of abstraction can we represent that shared structure?

And so my contention is that the distribution over real world data is biased towards low comographic complexity. And so are some of the models that we started to develop.

There had been this sort of approach where you would take some approximate Bayesian inference procedure and pitted against deep ensembles as the non Bayesian alternative. Yeah. And some normally, I I actually don't care that much about what's being called Bayesian or not.

Like, it's same with, like, intelligence, what is intel like, like, whether something's a good representation, all these things. So let's just connect this to whatever problem we're trying to solve. But this was actually quite problematic because the takeaway seemed to be that if deep ensembles were working better than some Laplace approximation or some MCMC procedure, then the answer is to be non Bayesian, to be less Bayesian than we have been historically.

And that's turned out to be exactly the wrong sort of directionality for how we should think about model construction, because for a given computational budget, those deep ensemble procedures were actually doing a much better job of approximating the posterior Bayesian predictive distribution. So doing marginalization. And so, in fact, the prescription should have been, well, we actually need to be more Bayesian.

And so, like, this is sort of a frustrating thing. But it was also very difficult to approach because there had been so many papers where people had just written basically that these deep ensembles are the non Bayesian alternative. And so kind of coming out and saying, well, actually they're doing a better approximation of the Bayesian ideal than all these methods that are being called Bayesian is sort of confronting, just sort of hundreds of papers in some sense at once.

But it felt like a very important thing to do. And we had actually done it subtly in a lot of papers where there'd be some subsection of some paper that mentioned something like this, but that was never really internalized because it was never front and center. So I thought, okay, a blog post is the right way to do this and let's make it all about this.

And I think in the end, actually it was the blog post that changed people's minds. I never saw a paper after that where people were making this separation. But also I think it did sort of strike a nerve a little bit.

Yeah. But how should we approach model construction?

And so when I say I want to embrace expressiveness, there is some some subtlety associated with that idea. And that basically means that it in some cases, maybe we're wanting to represent lots of solutions, we're assigning them almost zero probability, but not zero probability. So they're possible but not plausible solutions in our view.

And then if the data is telling us something that like, well, actually, we really should be paying attention to certain type of structure that maybe would surprise us, the model can actually respond to that. And if that structure isn't actually there, then your model isn't going to perform a lot worse than the model that is exactly constrained in those ways. In terms of how we should approach model construction in general, if you have a soft bias, a gentle encouragement towards certain types of constraints over others, quite often you can do as well as the perfectly constrained models.

The reason is you're paying some sort of penalty, even if it's small, for deviating from that constraint. So if you can fit the data perfectly with the constraint, you'll just collapse down onto that model. And we noticed this in a work we had called residual pathway priors for soft equivariance constraints.

And so this was a Bayesian mechanism essentially to have a distribution over solutions which would be concentrated in some way around certain types of equivariance constraints. Just for the the sake of the audience, equivariance is a generalization of invariance. It basically means you have some transformation t, f of t x equals t of f of x rather than f of t x equals f of x.

commutes in that case with the translation. Exactly right. So if you translate the image in some way, pattern of activations will translate in the same way across the different layers, rather than them just staying exactly the same, which would be invariance.

So in this paper on residual pathway priors, we were interested in the strength of this soft bias. So we basically had a distribution over neural net parameters that had a covariance matrix that lived in some equivariance subspace of our choice plus some orthogonal complement. And so there'd be these weighting terms basically that would represent how strong do we want this bias to be for equivariance.

And surprisingly to us at the time, it didn't matter very much. Like, if as long as you had a very soft bias for the the constraint, it would often be as good as even a perfectly constrained model. Basically, for this reason that, you know, you're still paying some penalty for deviating from the constraint.

And so if it is a good description of the data you would fit it perfectly, then you often collapse down onto that constraint. Although I wouldn't want to dismiss the importance of trying to calibrate these biases. It it can matter in certain instances.

I would also add that we can't get away from making assumptions. So even though I'm in favor of embracing expressiveness and having relatively soft biases for certain types of solutions as opposed to hard constraints, machine learning means learning by example. And we can't do that without making assumptions.

The question is just what assumptions should we be making and at what level of abstraction? And perhaps it's enough in a surprisingly large array of different problems to embrace expressiveness in combination with some sort of simplicity, some Occam's razor bias that can be formalized in terms of compression.

simple models work better. Ironically, given the surprise people often express at this idea of wanting to always embrace flexibility is that, like, before deep learning, the community had started to come on board with this notion that we want arbitrarily flexible models. In fact, the class of models that I was working on in my PhD, Gaussian processes in machine learning, were kind of inspired by this idea that we want really really large neural nets.

And Radford Neil, a statistician at Toronto at the time working in Geoff Hinton's group, was saying, Okay, we want to build models the size of a house and I'm a Bayesian and I'm going to really embrace expressiveness, so I'm going to take an infinite limit of a neural net with an infinite number of hidden units. And this is going to converge actually to a Gaussian process using a central limit theorem argument with a particular type of covariance function. People thought, oh, well, that's amazing.

Let's just use Gaussian processes because they're so much more principled in a lot of other ways. Like they're less sensitive to a bunch of design decisions, etc. They can be written in, you know, a very small number of lines of code and everyone anywhere in the world is gonna get basically the same answer, etcetera.

And so people sort of moved in this direction of just embracing expressiveness. But then once we started working in these kernel formulations, I think people perhaps started to forget that there was this dual space correspondence. And we actually were working with models that were more flexible than any neural net you can fit in memory and finding we were achieving very good generalization, especially on problems with a relatively small number of data points.

So now to to get back to your question, Radford Neil also had an interesting quote, I believe, in his PhD thesis that whenever you have a simple model that performs well, you can always build a more complicated model around it that will perform even better. And he gives this example of handwritten character recognition where you might have irregular writing styles, weird inkblots on the stage on on the page, just some sort of structure that you probably haven't already accommodated in your model that you can try to accommodate, and you'll achieve better and better performance. So what I would do in this situation you described is really try to understand what's the inductive bias there that's leading to good performance, and how can I soften it in some way?

How can I generalize this in a way that still honestly represents my beliefs?

It's a great question. I think we have to be careful about what we mean when we say a complex model. So I think most people would not consider Gaussian processes with an RBF covariance function, just the standard covariance function kernel that's often used to be a complex model, but it's highly expressive.

So it's more expressive than any neural network can fit in memory. It just has very, very strong preferences for certain types of solutions over others. And this enables it to be extraordinarily data efficient.

So one of the main use cases these days for Gaussian processes is in something called Bayesian optimization, where you're trying to maximize some sort of black box objective. So it's not something you have a closed form expression for, like it could be generalization performance of a neural net as a function of some of its hyper parameters, for instance, or some really costly physical simulation as a function of some parameters. And you basically want to query this objective as few times as possible in order to achieve a good result.

Gaussian processes are an amazing surrogate model for this objective, and you use the uncertainty to do the exploration efficiently. So I think that you can have expressive models that aren't necessarily complex. They still have very strong simplicity biases, very strong preferences for certain types of solutions over others, But at the same time, they're representing a wide array of possible solutions to the problem.

And I think that's how you can kind of reconcile what Radford Neal was saying with what we see in practice around things like overfitting. So I think there's a common misconception that the expressiveness of a model and its inductive biases are at odds with each other. The more expressive the model, the weaker its assumptions in some sense.

Like, the the fewer inductive biases it has, the less data efficient it will be, etcetera. And what we found, which has been quite exciting, is that the larger you make, say, big transformers, actually, the stronger its inductive biases. Like, the models get both more expressive and they have a stronger simplicity bias.

And so I think you can expand these two things together in some sense, and this is how you can avoid, say, overfitting and other sorts of issues with not achieving very good generalization. And I think one of the the clearest demonstrations of this is in a phenomenon called double descent. So double descent is this phenomenon where typically on the horizontal axis, have the expressiveness of the model, the number of units, for example, in each layer of a residual neural network.

And on the vertical axis, you have generalization error. And so initially, generalization error decreases as you increase the expressiveness of the model and it's able to just fit the data better and capture more structure. And then it starts to decrease.

It starts to go up. And that corresponds to some sort of overfitting. And then it decreases again, and that's why it's called second double descent, because of that second descent.

And in that second descent, all the models typically have about zero training loss. So the training loss just keeps going down as you increase the expressiveness of the model until roughly the number of parameters equals the number of data points. And what that means is that the larger models in that second descent cannot be generalizing better because they're more flexible.

They're all fitting the training data perfectly. It has to be that the larger models have some sort of bias, is enabling better generalization. And it turns out that this is a simplicity bias, a compression bias that we can measure.

When it's time to scale your business, it's time for Shopify. Get everything you need to grow the way you want, like, all the way. Stack more sales with the best converting checkout on the planet.

Track your cha chings from every channel right in one spot, and turn real time reporting into big time opportunities. Take your business to a whole new level. Switch to Shopify.

Start your free trial today.

a really important observation that neural nets tend to learn structure before they learn noise and things like this. There's a question around to what extent being able to fit noise is hurting their generalization capabilities. So there's this other phenomenon called benign overfitting, where the model fits typically a mixture of signal and noise, but the noise being fitting the noise doesn't significantly degrade its generalization performance.

And this is often seen as something that's specific to deep learning and in the face of everything that we know about generalization. And that's partly because classical frameworks for trying to understand generalization like VC dimension and Rademacher complexity are essentially measuring a model's ability to fit noise. However, there are other generalization frameworks like PAC Bayes and countable hypothesis bounds that we explore, which don't penalize an expressive hypothesis space and instead try to understand what sorts of soft preferences the model has for certain solutions over others.

And we've been able to achieve fairly tight bounds on the generalization performance of these large models using something called a Solomonoff prior. And so a Solomonoff prior says that we actually have a maximally overparameterized model. We can represent every possible program on a computer, but we have exponentially stronger preferences for solutions that have what are called low Kolmogorov complexity.

And so they're very compressible. The Kolmogorov complexity is the shortest possible program that can generate our hypothesis. And the fact that we're able to get these tight generalization bounds for these very large models, and in fact, the models get sort of better, sorry, the generalization bounds get better as we make the models larger, suggests that this is not a bad description of how these models are actually behaving.

Doing induction with a Solmanov prior is called Solmanov induction. And so it seems that when we make these transformers, for instance, very large, we're combining this expressiveness with this strong preference for low Kolmogorov complexity solutions. Another observation that's been made is that models are becoming increasingly general purpose.

And so twenty or thirty years ago, the typical prescription was to encode as much expert knowledge as possible into the model you're constructing and tailor it very specifically to the problem that you're considering because of results like the no free lunch theorems that say that every model is equally good in expectation over all problems sampled uniformly from this distribution over all problems. There are several no free lunch theorems. Another one says that a single learner is not going to be good on all problems.

Now these results are mathematically correct, but they don't really correspond to the real world data generating distribution. The real world is a small corner of all possible data sets. It's not drawn uniformly from that distribution.

If you were to draw data sets from that distribution, you would mostly get noise. And so I guess the question is, well, what is the real world data generating distribution really like? And it seems like there is a bias towards generating data with low Kolmogorov complexity, and our models share that bias.

And this is why we've seen increasingly general systems. So we've moved from feature engineering, basically hard coding structure into our models, to more modality specific models and architectures. So convolutional neural nets for vision, recurrent neural nets, for sequences, language, etcetera.

MLPs for tabular data and regression to transformers for almost everything. And so this isn't to say that transformers have achieved general intelligence. Absolutely not.

But they're relatively speaking more general than the predecessors. And we have seen this kind of movement towards increasingly general models. And our contention is that this has been made possible by aligning with the real world data generating distribution, which seems to have a bias for Lohr Kolmogorov complexity.

And we had this paper on no free lunch theorems inductive biases in Kolmogorov complexity. And I think one of the most surprising findings in that paper was that convolutional neural nets, which were clearly designed for image recognition, so they have locality and translation equivariance and so on, provably have inductive biases for tabular data shaped And as an the only possible reason that could be the case is because they both sort of share this bias for low Kolmogorov complexity.

This is largely an open question, although there are some intuitions, and this is something I'm really excited about pursuing further in my research. Where does the simplicity bias, especially from scale originate? There are some intuitions, so geometric intuitions around the loss landscapes for objectives that we use to train these models.

And so when we're sort of minimizing training loss, we can try to geometrically understand the properties of this landscape that we're minimizing. So it's been observed, for instance, that flat solutions, meaning solutions where you can perturb the parameters by some amount but retain low training loss, tend to generalize better than sharp solutions that have the same value of the loss. And you could make a compressibility argument for why that's the case.

Flat solutions don't need to be represented with as much precision, and so they're more compressible. As you grow the size of these neural nets, the relative volume of these flat solutions starts to exponentially dominate the volume of the sharp solutions.

Yes. So we know why to some extent, but it's our understanding is is somewhat heuristic. And so you could imagine, for instance, having a region of the loss surface with radius r a that's flat.

So you perturb your parameters within that radius, and you have a low value of the loss. And another region with r b that that's that's, you know, where r b is much less than r a. Mhmm.

Now as you grow the number of parameters in your model d, r a to the d starts to really dominate relative to r b to the d. And we seem to observe this in practice. So there was a result not from from my group, from Tom Goldstein's group at the University of Maryland, where they were trying to understand the role of the implicit biases of SGD and stochastic optimization in generalization.

And so quite often, as you perhaps have alluded to, SGD is often thought to be a really integral component of achieving generalization in deep learning. There's this idea that we have these very complicated objectives that we're minimizing, that are very non convex, etcetera. And SGD somehow saves us that the implicit biases of SGD navigate our procedure through some region of the loss landscape that represents low loss solutions that do generalize rather than low loss solutions that don't generalize well.

Well, it turns out you can actually do full batch gradient descent and achieve pretty comparable generalization to what you would get if you were using SGD, even if you don't try to make that implicit regularization explicit in the loss. And there was another paper that showed that if you even do guess and check, so you just randomly sample your solution vector and then stop when your loss is below a certain threshold, the generalization will also be fairly comparable to what you get if you use SGD or ATOM. And so that corresponds to this geometric intuition.

If you're just sort of throwing darts at the loss landscape and then you stop as soon as you have loss below a certain threshold, you're much more likely to be in this region of low loss and good generalization than low loss and bad generalization as you increase the number of parameters in the model. Now, is not an airtight argument. So there are lots of ways that you can increase parameters and models and not really influence the geometric properties of the loss landscape.

However, it does seem to correspond to what we observe empirically and not just in terms of results like guess and check. If we look at something like double descent, in that second descent, we can measure something called the effect of dimensionality of these models. So that's the number of relatively large eigenvalues of the Hessian, which is sort of the number of sharp directions essentially in the loss landscape.

And so that that decreases as as we as we make the model bigger. So interesting.

It's something I've been thinking about. It's very hard to operationalize. So you can evaluate these bounds by computing an upper bound on Kolmogorov complexity, measuring the compressed file size of your model after training.

Mhmm. But in order to use that prior as some sort of regularizer Mhmm. You would need to be considering sort of compression of some whole set of different hypotheses, not just a single hypothesis that's found by the model.

And so there's an open question of how you could try to operationalize that. Like Solomonov induction is sort of an represents an idealized learning system. It's not something that we can really do exactly in practice.

It seems that neural nets are sort of approximating it. And that's evidenced by these bounds and how they're able to tightly characterize generalization behavior of these models. But it's hard to sort of turn into into some kind of regularizer.

I'm also interested in how we can go beyond things like Kolmogorov complexity. So Kolmogorov complexity doesn't distinguish between incompressibility due to randomness. So like noise, for example, in our data versus structural complexity.

And Scott Aronson actually had a really interesting blog post related to this about ten or so years ago, where he was imagining a system. He had this kind of physics analogy where you have coffee and cream, and initially there are separated liquids and you start to stir them together. And as you do this, the entropy of this system is increasing.

I've seen that one. That was shared on our Discord recently. Yeah.

Very nice. Okay.

Actually have a yeah, I got to get ahead of myself. We have an idea, of course, for how we can do this. But so the blog post sort of compares this to and so the entropy of the system is increasing, the Kolmogorov complexity is increasing.

But the intuitive sophistication of that system is kind of non monotonic. Initially, it has low entropy, low sophistication, then sort of intermediate sophistication and entropy, and then again sort of like oh, sorry. And then high entropy and kind of low sophistication.

And so you can think of this in machine in the machine learning context in terms of reasoning about the value of data. So if I sample data from, like, a uniform random uniform distribution, that's gonna be very incompressible. I'm gonna need to memorize it.

It's sort of uncorrelated. But this could be useless for learning a representation, for training my model. I could alternatively imagine some sort of sophisticated cellular automata problem, some sort of game of life problem with very sophisticated generalization or generation rules.

That data actually could have an extraordinary amount of value for learning a representation. There was a paper that looked at something briefly like this called Intelligence at the Edge of Chaos, and I think there are other results like that that are coming out that like, you actually might want to train your models on this data with a lot of structural complexity, even if you really want, you know, in the end, the model has some kind of outcomes, razor bias, etcetera. And so we've been thinking about kind of measures of information that might compartmentalize structural complexity and random complexity.

And this will help us reason better about the value of data and developing priors sort of that are like Solomonoff priors, but might actually be more directly addressing the type of incompressibility that we're interested in.

debate right now. I think it is very closely associated with intelligence in a lot of ways. If we can compress our data really effectively, then in order to do that, we're discovering regularities that are typically gonna help enable generalization.

And in some sense, physical laws, for example, are a great compressed representation of reality. And so I think the compression is really intricately connected with what we mean when we talk about building intelligent systems. There are instances in which this can go very wrong, like in shortcut learning, where, for instance, you might have some spurious correlation, like maybe every time there's a blue pixel in an image, the label is a bird or something like this.

And so the model forgets about the foreground and it just looks for some feature in the background. In terms of generalizing on that distribution, that's actually not a bad idea. That actually is the right thing to do.

So Occam's razor is really a good principle. But if we go out of distribution, so we see like a bird in, you know, with a volcano or something behind it or in a room, then, you know, it's not gonna arrive at the right label. So there's this question of like, are there other principles of induction that will lead to greater robustness under distribution shifts?

I think in general, Occam's razor still is the right thing to do. Like there are many cases where a compression will actually not give you what you want more broadly if you move beyond that distribution. But in absence of additional information, it seems like you can't really do better as a guess as to what's the right strategy.

And so I still think Occam's razor is a very robust principle of induction. And perhaps one answer is just more data. I think this will work well in some cases and not in others.

So we have some work in progress on trying to build transformers for matrix operations. So this is actually kind of analogous to the work that people have done on transformers for multiplication, addition, things like this. So I guess, you know, these systems, LLM, seem very impressively effective in some instances in being able to write and code and solve a variety of different problems we didn't really expect them to be able to solve.

In other instances, they're just shockingly bad. So, you know, how many r's are in strawberry, like reversing strings, counting and just basic addition and multiplication, etcetera. And so there's this debate, I think, about whether we should be giving them tools to do those things.

Like, why not just give them a calculator? I mean, whatever they learn to do in terms of adding and multiplying numbers is still not going to be as efficient as giving them access to to a calculator. So why don't we just do that?

Or like, you know, is there some sort of greater value in having to sort of learn a representation that can do some of these things? Because even though we'll always want to use a calculator, if we're multiplying or adding numbers, being able to do something like that reasonably well might transfer into other settings that we don't really anticipate. And I'm probably more in this category.

Like, I just think intellectually, we should try to figure out how to do this without tools. And so this work that we're doing on transformers for matrix operations, I think, is quite analogous. So you could argue that addition and multiplication are just fundamental primitives for trying to build intelligent systems.

We're not as good as a calculator is, but it might be important for us to have some ability at being able to do these things. The same could be said of matrix operations. This is really the backbone of all sorts of different learning algorithms like Gaussian processes, which we discussed a little, involve solving linear systems with a covariance matrix, computing log determinants, etcetera.

Degenerative models like normalizing flows involve log determinants, dimensionality reduction like PCA, etcetera, involve other matrix operations. And so they're just really ubiquitous as kind of a primitive for trying to build learning algorithms and intelligent systems. And so I think if transformers are ultimately going to become some sort of general intelligence, then they ought to be able to be competent at these types of operations.

And so we were kind of representing then matrices as sequences of numbers and then having the outputs be things like the maximum eigenvalue or, you know, solution to a linear system or whatever other operation we were considering, the spectrum of eigenvalues. And we found interestingly that when you train this approach on Gaussian random matrices, you just sample every entry of the matrix from a standard normal distribution. It will do fairly well at in distribution matrix operations.

So other matrices sample from that distribution that it hasn't seen before, but extraordinarily poorly, even if you go slightly outside of that distribution. So given an identity matrix, all ones in the diagonal, zero everywhere else, that'll have pretty low density, has support under this Gaussian distribution where matrices, but pretty low density under it. Hasn't seen something very much like that.

It will just completely fail. It won't do anything reasonable. And so we considered a number of interventions like looping sort of adaptive test time computation, enriching the training distribution very significantly.

So having sort of this like EINSUM space of structured matrices that we were sampling from. So all sorts of different matrix structures, Toeplitz, Kronecker, etcetera, block diagonal, low rank, and a variety of other interventions. And interestingly, we found once we had done that, this approach actually was able to generalize even to matrices that were out of distribution for this fairly rich sort of training set that we've created.

And so it seemed to move more towards learning an algorithm rather than just doing statistical interpolation on the training data. And so this is, I'm an optimist by nature. So I am very happy about the fact that we can use data to move towards things like algorithm discovery.

But now I remember why I started talking about this. So I think there are instances where it's going to be hard. So autonomous driving is is an example where, like, you have outliers, but they're different each time.

So just training on the outliers isn't going be useful because they're going to be new outliers that look very different from those outliers. And I just have the intuition that more data alone is not really the answer to building robust autonomous driving systems. Yeah.

I just wanna say, as far as why not just give them tools, I mean, my answer folks is because those have to be programmed and built by people.

And I think being able to do certain things well, we're finding might surprisingly relate to doing other things very well. And so I think we have seen this to a large extent with LLM. So we had a paper where we just took a text pretrained LLM off the shelf and applied it to time series forecasting.

So we did everything naively. We weren't even really intending for this to be like a proper project or a paper. We were just curious, you know, if you just give GPT like a sequence of numbers naively encoded to strings and have it extrapolate the next sequence of string tokens, how would it compare to purpose built time series forecasting procedures?

And it just worked way, way better than we thought it could possibly work. It didn't even really make sense. And so we did sort of turn this into a proper project.

We did a little bit of work on trying to improve the tokenization and think about uncertainty representation, etcetera. But most of that paper, it's called large language models or zero shot time series forecasters, was focused around trying to understand how this is even possible. And in the end, it it did start to feel a bit more like maybe it's not just that you can do this, maybe you should do it in some instances.

They did quite well on a variety of benchmarks. And to me, this suggests with like a proper dedicated research effort on LLMs for time series, we could see these systems actually working a lot better than the purpose built models. And so what that shows is being able to predict the next words in sentences can actually transfer to being able to do other things like time series prediction really well.

And we had another paper on LMs for materials generation, was kind of similar. So we took a text pretrained LM off the shelf, a LAMA-two model at the time. And we fine tuned it in this case on atomistic data represented as text, so locations of atoms and energies and things like this.

And the resulting system was able to generate inorganic crystals with favorable properties better than these purpose built approaches and even foundation model approaches that had been trained, on that domain specific data. And so one of the takeaways from that project was like the text based pre training was an indispensable component in being able to achieve good results on materials generation. And we tried to understand also in that paper why that wasn't the case.

We were collaborating with some some chemists at fair in California, and they were just very curious about all Emsen Foundation models. So they're willing to sort of humor us a bit and help us sort of see what we could do. But they were just like skeptical throughout the project until we we saw the results.

And it's like, you can't deny the results are great. You know, why is this happening? And part of it was that in being able to predict the next tokens in strings, you are learning principles of induction like Occam's razor.

And so how do those sort of manifest themselves in context learning? So what it means is if you are and in fine tuning. So what it means is these models are gonna be predisposed to discovering compressible representations.

And that means, for example, salient symmetry. So this was a problem where there was a rotation invariance. And these models actually were very quick to learn these kinds of invariances because of the text based pre training.

It sort of instilled this principle. And so I think this is also an example of how compression can be a broadly applicable principle for induction. It's at sort of the right level of abstraction that you can start to see more relatively universal behavior.

So for instance, there are some theories on the success of foundation models that suggest that different problems are just different projections of some underlying realities. Like, platonic representation hypothesis is an example of this. And so you can represent an image with pixels or with words and train the respective models on those different modalities and they learn similar representations.

I think that can be true in some instances, but I also think different problems are often truly quite different from each other in terms of their low level structure. So like if we have molecules, there's rotation and variance, maybe some other problems, some image recognition problem like character recognition, we might have translation invariance. It doesn't matter if the two is on the left of the screen or the right of the screen, the label is still a two.

These are very different low level feature representations. So like the architectures that you would typically use, for each of those modalities and to respect those different types of invariances would look very different. But what they have in common with each other is they're both ways to compress the respective problems that they're being applied to.

So if you have a Did you know Sam's Club isn't a store? It's actually a club with cool finds and, like, a whole community. It's a club.

Of course, Jason. It's in the name, Sam's Club.

44 to learn more. Model that has this compression bias, then it can discover those salient symmetries. And we had this really surprising finding that vision transformers actually can be more translation equivariant than convolutional neural nets after training, which sounds impossible because conv nets by design are are translation equivariant, but they're not exactly translation equivariant because of aliasing artifacts and edge effects and things like this.

And so this other model, this transformer with no explicit constraint whatsoever, just a soft bias that manifests itself increasingly at scale, is able to discover a solution that has lower equivariance error than the convolutional neural net, which is just absolutely remarkable. So to come kind of full circle to your question, I think that we're discovering more and more that being able to solve certain problems really well will translate in perhaps unexpected ways to being able to solve other problems well. And so I think especially when it comes to things like addition multiplication and matrix operations, like these are just even, like, obviously, like fundamental primitives for building intelligent systems.

Like, even if we can use tools for those things, we want our representations to be somewhat competent at them because it's going to be useful for all sorts of other things we probably haven't anticipated.

I'm so glad you asked. So there's almost nothing I like more than talking about Bayesian inference. And I'm so glad you said marginalization because I feel like that's often overlooked.

Like, when someone says Bayesian, probably the word that comes into your most people's mind is prior. And is the prior good? And how do we know what would a good prior be, etcetera?

And they get worried about that. But really, the prior is not the defining feature of what it means to be Bayesian. It instead, being Bayesian means that you want to represent the honest belief that you have uncertainty over what solution is correct given a finite data sample.

And so if I'm trying to estimate, you know, the bias of a coin that I'm flipping, if I flip it once or twice, regardless of whether it comes up, you know, heads twice or something like this, I ought to have some uncertainty over the bias still. And that uncertainty is manifested through this procedure called marginalization. So I think we can think of regression to get some intuition of this.

So we could imagine like a bunch of different points on a sort of YX plot. And maybe the points look roughly like they're on a straight line, but not quite. We could also imagine that there are many different curves that will perfectly run through all of those points.

They all look different from each other. Some of them will be very wiggly. Some of them will be slowly moving, etcetera.

Some of them will basically just be like a straight line. And we wouldn't be able to say, well, we know with 100% certainty it's this curve that's the right description of our problem. But that's exactly what we're doing, you know, 99, 100 minus epsilon percent of the time when we're training models in deep learning.

That is not an honest representation of our beliefs, and it's going to become a bigger and bigger problem the more expressive our model actually is. Because that means there are going to be many more different settings of parameters that are consistent with what we observe, and we're just betting everything on one of them. And probability theory says, no, that's that's just wrong.

That's not what you should be doing. The summon product rules of probability say you should be doing marginalization. And so that's all to say that Bayesian marginalization, basically looking at all possible solutions that can be expressed by your model class, weighted by their posterior probabilities, is going to be most important when we have a model that's very expressive, has a lot of parameters, so deep learning, for example, especially relative to the number of data points we're considering.

And I think often people think of it in the opposite way that like, oh, maybe Bayesian methods are most relevant in the realm of classical statistics. Like if you're doing some logistic regression and maybe you want to represent some uncertainty, etcetera, but it's not suited for deep learning. It's really the opposite.

The challenge then is how do we do this in a way that's computationally tractable? And I think like with most things in life, the answer of course is nuance. So it's not that we can either be fully Bayesian or not Bayesian at all.

Let's just try to do what makes sense given the resources available to us. And so if we're just doing standard training, you can actually view that as a very crude form of marginalization where you're saying, okay, I'm representing the posterior as a point mass around the most likely setting of parameters. And then we can say, okay, well, maybe the posterior, which is really like the loss functions that we're minimizing are basically negative log posteriors.

Maybe the posterior looks nothing like a point mass and it looks nothing like a Gaussian. It's very multimodal, it's very messy, but we can still do a better job of representing it with a Gaussian than with a point mass. And so let's use a Gaussian approximation.

And indeed, when we do that, we often see better generalization because we're representing all these other complementary and compelling explanations to our problem. And then we can keep taking it from there. Okay, maybe we can develop some MCMC procedure that will explore the loss landscape and capture something much richer than just unimodal Gaussian structure, etcetera.

And again, we see improvements in performance in doing that. And so I actually think that Bayesian methods have been an extraordinary success story in deep learning and beyond. But we don't hear as much about them now as we did maybe ten years ago.

And I think there are number of reasons for that. So to some extent, they're a victim of their own success. So I think there was some low hanging fruit in being able to do better approximate marginalization in neural nets around 2015.

And there was really a lot of progress between about 2015 and 2020 in achieving increasingly better results that will also be kind of computationally efficient. So there are procedures like one we developed called SWAG for instance, which was based on some insights we had into the structure of these loss landscapes that will allow you to do Bayesian marginalization without really any significant additional cost on training. It's a bit more expensive a test time and so on.

It's also deep kernel learning, which basically just requires a single forward pass through the model and you get sort of some representation of epistemic certainty. And these procedures were adopted in practice. And so if I go to some more domain specific conference, like some workshop or conference on materials engineering and things like that, I'll see lots and lots of talks using Bayesian optimization, Gaussian processes, neural networks, with, epidemic uncertainty representation, etcetera.

So this is really useful in practice, but it's very hard to kind of go beyond the useful approximations I think we developed without doing a significantly without making sort of a really significant, you know, ten year kind of style moonshot landing kind of investment in those directions, which I think is worth making. Secondly, there's the advent of LLMs and that did sort of change things in practice. So like we went from million parameter models to billion parameter models.

And the role of epistemic uncertainty is also a little bit less clear, think in some instances when we're working with LLMs. And so, this is just a real technical challenge and I think we're just sort of barely starting to scratch the surface of how to think about it. But the motivation is absolutely there.

Think it's really greater with these big models than it is for virtually any other model class. And in this sense, I don't know how anyone could not be a Bayesian. Like, how could anyone say, I don't want to represent epistemic uncertainty?

I don't like using jargon. So there's like, there's aleatoric uncertainty, which means irreducible uncertainty, often associated. Like we have given a dataset, we have some noise on that dataset, there's nothing we can do about it.

That's aleatoric uncertainty. Epistemic uncertainty is uncertainty that's reducible with more information. And we believe that uncertainty is there.

And there's even an argument that that might be the only uncertainty in the world. Right? Like, things that seem intrinsically random.

Like, could roll a dice Right. And it might seem like, okay, there's a one sixth probability that it'll land on any one of the sides. But if I had enough information, like if I could sort of exactly know the strength of the throw, the wind in the air, the friction on the table, and I had the right physical model, I should be able to predict exactly what side it's gonna land on with each throw.

So that's just saying, okay, the more information we gather, the less uncertainty we have, even for this process that somehow seems intrinsically random. And physicists, modern physicists now probably do believe that some aspects of the universe are intrinsically random, radioactive decay and things like this. But many others have not historically, like Einstein was a determinist.

And so he would have believed that the only uncertainty there is, is epistemic uncertainty. I think for practical purposes, that's not an unreasonable belief. And so it's absolutely crucial that we try to model it in some way.

To not do it is just to do something that's mathematically incorrect and could come at a huge cost. There's also a change I think that's happened in this movement to LLMs in the way that people think about data. So it used to be the case that you kind of have a fixed dataset, and then you throw everything you can at it to get the best possible performance.

Maybe you have some scientific problem and you just really want to do well, and you're limited by computation and other things to some extent. But it's not the case that you have this sort of trade off to manage for a given computational budget between the size of the data and the size of the model. And so that's described by these scaling laws like Chinchilla scaling laws.

And that is starting to become the assumption in mainstream machine learning, that you have almost like an arbitrary amount of data, and you have a fixed computational budget, and you want the best possible performance under that computational budget. In that case, you know, rather than trying to be more Bayesian or something like this, it could actually make sense just to use more data.

Absolutely. Yes. So Bayesian marginalization has this automatic Occam's razor bias.

And I actually started working on Bayesian deep learning ironically when I saw a talk on optimization, which might be viewed as almost the opposite of being Bayesian. So Jorge Nosidal gave a talk at Cornell where I was initially faculty and he was presenting on small batch biases basically of stochastic optimizers. And he had this figure where he was contrasting flat optima with sharp optima and he had this horizontal translation of the loss landscape.

And under that translation, the flat minimum still had reasonably good test loss, but the sharp minimum had very bad test loss. Right. And so he was saying, okay, this is why we want to do stochastic optimization with small batches, because it's gonna be more likely to find these flat solutions.

And in some sense, like the other optimizers, like his LBFGS and so on, were like too good. They were like converging to these sharp minima. If like you just care about minimizing loss, then maybe that's not bad behavior.

But generalization depends more than on just getting a low value of loss, at least with the loss functions that we're using. And so I saw that figure and I thought, well, this is actually really great motivation to be Bayesian because if you're being Bayesian, you're not just betting everything on one solution, you're integrating under a sort of a flipped version of that curve. And so automatically most of the volume will be in these flat regions.

And that's so much more elegant than approaching this from the optimization perspective where you have to say then, okay, I need to rigorously define what it means to be flat. Like, maybe it's gonna be like the largest eigenvalue of the Hessian. Maybe it's gonna be flat Intenser in or something like that.

Exactly. Yeah. Maybe flatness in random directions versus sharpest directions.

Maybe it needs to be parameterization invariance. Like that's that's been often a criticism of of discussions around flatness, etcetera, that it's most ways of measuring flatness aren't parameterization invariant. I have thoughts about that separately.

But anyway, you might then look at Fisher matrices or something. Okay. So then once you've decided on what it means to be flat and no one will completely agree with you, most people will will strongly disagree whatever you choose, then you have to decide like how much am I gonna penalize sharpness.

No one will really know what to do about that either. And like the idea is you wanna build a loss function that is a better proxy for generalization by accounting for things like flatness. But it's just this really messy rabbit hole.

Whereas if you're just doing marginalization, this is all happening under the hood. You don't need to worry about it. And so that's really elegant.

And then there's Occam's razor in terms of model selection. And I'd recommend that everyone check out chapter 28 of David Mackay's information theory and inference learning algorithms book. It's titled Occam's razor.

And I don't agree with some of the stuff in that chapter. In fact, we wrote a paper about this on Bayesian model selection and marginal likelihoods, but it's a very beautiful description of automatic Occam's Razor. And he has these just extraordinary visualizations to try to demonstrate how that's possible.

So he has like these also, you know, everyday examples as well, like this idea that maybe you have like a block behind a tree, but if you had X-ray vision, it would appear to be like two blocks of equal height and color or 10 blocks or something like this. But given that you don't have X-ray vision, and you don't think this is a trick question, you'd be pretty confident it's gotta be one block. And this seems like some manifestation of Occam's razor.

And you might try to rationalize this by saying, well, it would be just a remarkable coincidence to have two blocks standing next to each other of equal height and color. But he argues that this is actually a quantifiable consequence of doing Bayesian marginalization. And so he basically has this conceptualization where you have all possible data sets on a horizontal axis and the probability of generating a particular dataset under your model on the vertical axis.

And that's the marginal likelihood, the probability that you would generate your training data under your model prior. And so if you have the one block model, you're not gonna be able to generate very many different data sets. So most of the data sets on the horizontal axis have no support, no PD given M.

But the ones that it can generate, it's gonna have to give a lot of probability for it, because this is a proper normalizable probability density. Similarly, if you have the 10 block model, you can generate all sorts of different types of observations, but you're gonna have to sort of spread that mass more thinly because this is like a proper normalizable probability density. And so for a particular dataset that's consistent with both of these models, like the block behind the tree example, the one block model is actually gonna have significantly more probability.

And this is even not factoring into account the idea that maybe we have some prior preference for simplicity and that there's like, we think that one block is more likely than 10 block or something like that. He's just, let's just forget about this ratio of prior odds and only consider this ratio of marginal likelihoods. And so it's just a beautiful demonstration of how Bayesian inference automatically encapsulates a notion of Occam's razor.

And this is just a fundamental question in science. Like, if you have arbitrarily many hypotheses that are consistent with any number of observations you can ever record, how do you choose between them? What's the principal way of approaching that problem?

And the answer to some extent, I think, is given by Bayesian marginal likelihoods. There are, very subtle issues, think, with some of the ways that this can be done. And so we wrote a paper all about that.

But largely speaking, it's, you know, something that I think people should acquaint themselves with because it it really is sort of getting at something very fundamental and it's has extraordinary practical value. What was that paper? Was that The the one that we were sort of looking at.

So exam so we have a paper called Bayesian model selection, the marginal likelihood, and generalization. And so that paper is really trying to sell two sides of a story. On the one hand, it's trying to convince the readers that the marginal likelihood is something quite extraordinary and that there's a reason we should be interested in it.

Because if it isn't, then, you know, if you haven't heard of it, like, why even read criticisms of it if it's something that doesn't matter? The other part of the paper is questioning whether it's answering exactly the question we want to be asking when we're doing model selection towards trying to achieve the best possible generalization. And so the question that the marginal likelihood answers is what is the probability that my prior generated the training data?

That's different than what is the probability that my posterior after I've observed the data is going to lead to reasonable predictions? Mhmm. And so we can construct examples that really illustrate this difference.

Like you could have a uniform prior over solutions that you expect to be easily identifiable from the data. And so the posterior will contract very significantly around something that will actually be quite reasonable and make reasonable predictions, but the marginal likelihood will be really bad. And so we construct all sorts of examples where there's actually a misalignment between the marginalize marginal likelihood and generalization for these reasons.

You could also overfit, even though you have sort of some robustness to overfitting. Like, if you're considering arbitrarily many models, you could just get super unlucky and have a model which is like a point mass prior around something that can only generate that training data set, but isn't gonna do anything reasonable. So the marginal likelihood might prefer that, but it's not gonna lead you to a model that's gonna make good predictions.

It is very useful as sort of a heuristic for model selection in many instances. It's got incredible practical value in learning things like hyper parameters that control complexity and Gaussian processes. And I think it often is the right tool for scientific hypothesis testing, which is subtly different.

And so you could this is actually a real historical example. So there was a dispute between statisticians. I think Columbia University thought that general relativity was not the explanation for Mercury's irregular orbit.

It's called Mercury's perihelion. There must be some hidden planet or some orbital debris or something like that. And so Bayesian statisticians actually went ahead and computed the marginal likelihood associated with general relativity explaining Mercury's orbit versus these alternative hypotheses like, some modifications in Newtonian gravity, etcetera.

And because general relativity was so falsifiable, like its predictions were so sharp and consistent with what we observed, it had orders of magnitude greater marginal likelihood than something like a modification to Newtonian physics, where you have to have some distribution over the modification, might sort of enable you to explain what we see, but it also is gonna generate other data sets. And so its mass is gonna be spread more thinly. And so I think that's actually a beautiful demonstration of how the marginal likelihood can be used for for scientific hypothesis testing.

frequential coding and ideas and information information theory. So this is something we've been thinking about a lot actually in trying to go beyond Kolmogorov complexity and very much related to your earlier question connected to benign overfitting. This observation that models first tend to fit structure and then they start to fit noise.

And so if we can think of some sort of like compute limited Kolmogorov complexity, maybe we can start to get some idea of how to do model selection. And the marginal likelihood actually can be written as the so the marginal likelihood is the probability of the data under your model condition on your model. And so you can use the chain rule probability to write that as a product of p of d I given d less than I essentially.

And so you take a log of that and it looks sort of like the log under your sort of training curve as you're observing more and more data. And so these these things are all connected together. And I think this is a very reasonable way of trying to understand how to do model selection properly.

Sort of thinking about compressibility but also thinking about the dynamics of training. How a model's representation evolves with increases in computation.

But would you would you lump potentially grokking in with that as well? That's a great question. So in the intro, at the end of the intro to that paper, I say that I'm not talking about Grokking or scaling laws much in this paper because these phenomena are often not treated as particularly mysterious or distinct to to neural nets.

But I would say that they fit into this sort of suite of generalization phenomena that we can try to understand using some of the same tools. And so the generalization bounds that I present in in that paper now are being used by us and others to try to understand the root of scaling laws. I mean, they seem like remarkable laws of nature almost.

Like if you increase computation by a certain amount, you can predictably improve generalization on a set of tasks. But it's sort of this empirical law. And so we want to understand why.

And this relates to why larger models might have stronger simplicity biases and things like that. And so definitely the generalization frameworks that we present in that paper, that I present in that paper, can, be used to shed light on scaling law behavior. Grokking is not something I've thought about too specifically, but it does seem to be the case that by training for longer, the model is doing some reorganization that enables a more compressible solution.

And so it would be very interesting. I'm sure people have done this, like, like, on the flatness of the solution and things like that as you proceed through grokking. And I think this is actually an older, like many things, older than people might realize.

So like double descent is thought of as like a modern deep learning phenomenon, but it was actually first presented in the 1980s. And so this has been sort of around for a while. And I remember like Ilya Satsgeva and others talking about behavior that was very similar to grokking, where, like, the training loss isn't really changing, but training for longer actually leads to better generalization.

And this relates a little bit, to a procedure we had called stochastic weight averaging. And so the idea there is you want to ramp up the learning rate, to a relatively high constant learning rate, and then maintain a running average of the parameters as you're sort of traversing this loss landscape with SGD or ADAM or whatever else. And what happens when you do that is you're sort of spinning around the periphery of flat solutions.

And by taking an equal average, you get to move inside that region and get a much shorter, like flatter solution. And that reliably led to better generalization. And it's quite convenient because you can just load up a pre trained model and then increase the learning rate, do this for a certain number of epochs and get better generalization as a consequence.

And so think grokking might be related to that sort of behavior as well.

and thought, well, I wanna do Bayesian deep learning. I had done a lot of Bayesian ML and I had done some deep learning, but I hadn't thought about Bayesian deep learning until I saw that figure in his talk. I thought, well, before we get into Bayesian marginalization, let's try to understand the geometric properties of these objectives that we're minimizing first so that we can come up with like good posterior approximations.

And so in some sense, this is a bit Bayesian agnostic. It should be useful even if you're just doing classical optimization. And then we had this, you know, the first thing that we encountered was this discovery we called mode connectivity, which shows that if you retrain your neural net, for example, with different initializations and find seemingly different modes, in fact, you can actually walk from one mode to the other in sub subspace without increasing the training loss at all along the way.

And so before that, it had been believed that the different solutions that we would find, for example, by retraining our model, were isolated from one another. So you walk in any direction and you increase the loss a lot along the way. And there were some results from Goodfellow and others to suggest that intuition.

We showed that you can introduce very simple sort of parametric curves, like a polygonal chain with just a single turn, or a quadratic Bezier curve. You can anchor the endpoints with whatever solutions you find in this procedure that finds your two parameter vectors, w one and w two at. And then as you vary the the parameter of this curve, t from zero to one, you walk from one to the other.

And then the idea is, well, how do you sort of learn this curve? And you can discover these by minimizing your loss uniformly in expectation over the curve. So if you're doing classification, this looks sort of like a line integral of cross entropy loss normalized by arc length.

And it's actually a pretty simple objective to try to minimize because you could just sort of sample uniformly along the curve t and then take gradient steps with respect to the parameters of the curve theta that you're trying to learn. And you can always do this. And the larger you make the model, the less of an arc length they're kind of or sorry, less of a sort of bent turning that you need to do, the more it looks almost like a straight line path between the two solutions.

And so what this showed is that there were these regions within the loss landscape that were extraordinarily flat. So they all had sort of zero loss. And what was especially interesting about them was that the different parameters in these mode connecting curves actually led to models which made very different predictions on the test set.

So of course they're making the same predictions on the training set to have the same loss, but on the test set, they were different representations. And so this meant that you could ensemble them and get much better performance, for example, just uniformly sampling on the curves. And then this led to this stochastic weight averaging procedure, where we were sort of thinking, well, how can we sort of spin around these, like, contiguous regions of flat solutions and find something that's centered within them?

That ended up being fairly practical. Yeah.

Yeah. It's fascinating. Grocking is not something I have thought about too specifically, but it seems to me you probably are entering some region of the loss landscape that doesn't have different loss, but is providing a representation of very different properties.

And so this does relate to procedures like stochastic weight averaging, where again, in the end, the solution doesn't really have a different loss than what you would have gotten training in a standard way.

downhill, if you will, to a simpler solution? You might sort of get to start to I I I mean, is really speculation and I haven't thought a lot specifically about grokking, but it could be that you're just sort of edging your way inside because of gradient noise and so on towards a flatter solution as you continue to train. Yeah.

Yeah. Just some type of yeah. It's really fascinating.

This sort of made me wonder actually after the mode connectivity discovery, like, what what do these lost landscapes really look like? Like, the the path that we considered initially were just one dimensional. Eventually, we started thinking about multidimensional lost volumes, kind of.

And so we had a paper which was creating a simplex. We were basically adding vertices to the simplex and sampling uniformly within the resulting simplex and trying to sort of add vertices such that when we did that, we would sort of have low loss and also maximize the volume of the simplex. And so that sort of enabled us to find these sort of like multidimensional loss surfaces.

And then we had this that all had sort of low loss. And we had this picture, I guess, in the front of that that paper where we started with this sort of original understanding that all the local optima are kind of isolated to wormhole like tunnels between the different optima to this idea that maybe everything is just sort of connected together in some manifold that's embedded in this really high dimensional space. And what seems like a sharp or a flat optima might actually just be like how converged are you with inside that manifold.

Because if you're on the edge, it'll look it'll look sharp because you move in most directions and you increase the loss quite a bit. But if you move in specific directions, it's very flat. And so I think we still don't have a full understanding of what that looks like.

And I think it has really fascinating implications for the generalization behavior. Really hard to think in a billion dimensions.

Help us understand. Mhmm. So my philosophy is that we should honestly represent our beliefs in the way that we do model construction.

And our honest beliefs are usually that the real world is a nuanced place and we're going to want to have model expressiveness in order to represent that nuance. At the same time it can't just be flexibility, it has to be flexibility combined with some sort of simplicity bias. It should have some kind of Occam's razor bias.

It turns out perhaps surprisingly that making transformers and other types of neural net models larger often actually enhances rather than reduces a simplicity bias. And this is in many instances, mostly what's been responsible for the better generalization behavior of larger models. So we had this example with double descent, where in the second descent, all the models have basically zero training loss, but the larger models are generalizing better.

Can't be because they're more flexible, has to be because they have some other bias, some sort of simplicity bias. So how do we operationalize this? I think one way is, well, expressiveness.

So choose a model class that is gonna be able to represent lots of different solutions. In terms of the simplicity bias, well, we're finding empirically that increasing model size can help with that.

Yes. So there are a lot of things you can do. I think and we'll talk about them.

But the question of how can you more elegantly encode this compression bias beyond just making the model bigger is really a fascinating open research question. So my contention, and I might be wrong, is that in many cases, a 7,000,000,000 parameter model is not doing better than a 1,000,000,000 parameter model primarily because it's more expressive. It's actually because of the simplicity bias.

And perhaps things like knowledge distillation can help make this argument. If it's possible to really distill a large model into a much smaller model, then it means that the small model has some setting of parameters that can provide a good approximation to the large model. It's just not able to find those parameters when trained directly on the data.

It needs the help from the teacher model. And so the question is, can we build the 1,000,000,000 parameter model with some sort of like explicit bias that would have otherwise come about through scale in the 7,000,000,000 parameter model and find those solutions itself? I don't think we're anywhere close to being able to do that.

Think it's it's sort of a an open research programme. However, there are little things we can do that will help. So like there are ways of intervening in the training procedures.

So like the stochastic weight averaging we discussed, that will help find a flatter, more compressible solution. There are of course, all sorts of regularizers that can be useful in certain instances. Bayesian marginalization can be very helpful in sort of encoding an automatic simplicity bias in what we do.

And so there are all sorts of interventions that will help us with this. But I think the dream is that maybe we can embrace flexibility in fifteen, twenty years from now by building these non parametric models that like really do have an infinite number of parameters and are more expressive than any model we're using right now.

So I think the bitter lesson is widely misunderstood and incomplete. The bitter lesson describes how over short time scales, it's been appealing to try to encode our knowledge into our procedures in order to achieve good results. And that this is also a cognitive bias that people have.

They like encoding expertise and things like this and being thoughtful and elegant about problem solving. But over longer timescales than a typical research project, computation becomes cheaper. And so quite often more brute force seeming approaches based on search and learning end up working a lot better than elegantly encoding our priors and our constraints, etcetera.

And there are a number of examples in the essay like Deep Blue, for playing chess in the late nineties, speech recognition, where some researchers were trying to model things like physiology of the voice box, etcetera, to try to get any possible advantage. But then they were beat out by statistical procedures like hidden Markov models, and of course, AlphaGo and and other procedures like this too. And so the takeaway seems to have been because of how computation is becoming cheaper over longer time scales.

Quite often, it's going to be more practical to try to build procedures that are based on search plus learning as opposed to feature engineering. I think to a large extent, this is true, but what it doesn't say is that in order to learn, you need to make assumptions. And so machine learning, as we discussed, is learning by example.

And we can't do that without making assumptions. So if we go to like maybe this coin toss example, where we're trying to estimate the bias of the coin, you have to make some assumption. Like before I start doing that experiment, are we assuming a uniform bias?

Are we thinking it's looking sort of more like, you know, centered around point five or something like that, like being unbiased, but it we have support for other things. These assumptions are gonna influence how we do induction. And so, like, we just can't get away from making assumptions.

And so the question is just what assumptions should we make and to what extent can they be universal? And when it comes to things like scaling laws that describe how we can reliably improve performance with increases in computation, if we're able to make better assumptions, we can actually change the scaling exponents, act which mean that we'll get exponential improvements in performance with increases in computation, which is just remarkable motivation for trying to do this. And it's not just a contention that this might be possible.

We're actually starting to see some evidence of this. And so in our own work, we've been very interested in how we can produce structured representations for linear layers and neural nets. And so this might sound very abstract, but there's a very well known example of doing this.

So you can actually start with a fully connected layer, having every possible connection between the nodes and two layers, remove a bunch of the connections and enforce parameter sharing, and you have a convolutional layer. And so mathematically, what you've done is you've replaced a dense matrix multiply with a matrix that has sparsity and locality, and that's a Toplitz matrix. And so there's this question of like, could you actually systematize this process of creating structured layers towards better compute optimal efficiency?

And if you were to do that, what sorts of principles should we be embracing? Should we embrace things like parameter sharing? Because convolution, for example, has parameter sharing.

Should we embrace sparsity? Should we embrace other sort of features? And so we introduced this sort of EINSOM formulation over structured matrices that contains all sorts of different structures as special cases, as well as all sorts of different novel structures.

And this taxonomy of interpretable hyper parameters that kind of control the properties of these structures. So how fast is it to do a matrix multiply? What's the rank of the matrix?

How much parameter sharing there is? And we'd have sort of continuous values for these parameters that would control these things. And what we found is that actually in general, towards compute optimal efficiency, assuming we have as much data as we ever need, parameter sharing was actually not a great principle.

And that was a bit surprising. You also want to have full rank structures and that move from surprising to sort of disappointing. It's like, well, can we go beyond what we're doing with dense matrices then?

Because that's what we're using now. And the answer is yes, if you can get faster multiplies. So we proposed a structure called a block tensor train, which is related to another structure called a tensor train and another structure called a monarch matrix.

It's basically a sum of monarch matrices. And this sort of is full rank, doesn't have parameter sharing, but you can do multiply faster than you can with a dense matrix. And this allows you to build wider layers for a given computational budget.

And this did have sort of a meaningful effect on the scaling exponents. And so this is sort of at a proof of concept level and a lot of work would need to go into parallelized implementations and so on of these structures. But it showed that it's possible.

And we're not the only one. So there are other groups that have been looking at neural tangent kernel inspired ideas to understand different regimes of learning, like easy versus hard feature learning and so on, and trying to think about, well, what principles are actually gonna modify these scaling exponents? And other things like, I think Albert Gu and others have been looking at, it's like chunking and transformers and the inductive biases that come with that.

And it's like whether we can change these inductive biases towards better scaling exponents. And so I guess in short, learning requires assumptions. And so I don't think we can neatly compartmentalize elegant ideas from having successful learning combined with computation.

We really do need to these things not only are not at odds with it with one another, they really strongly go together.

Upfront payment of $45 for three months, $90 for six months, $180 for twelve month plan required, $15 per month equivalent, taxes and fees extra. Initial plan term only greater than 50 gigabytes. Me slow when network is busy.

See terms.

What's stopping us? It's a great question. So I'll say that I I think convolutions are a great idea.

And if you're in this setting that I described earlier where you have a fixed dataset and you want to do something reasonable in order to achieve the best performance, this will often work really well. And even if we're moving towards soft inductive biases from hard constraints, would normally advocate for having some sort of convolutional inductive bias, even if it's not a hard constraint anymore. And that's what we were doing with residual pathway priors.

And there's been some interesting work on convolutional VITs and things like this to try to have these sorts of soft biases for more efficient learning. The reason that we found in this instance that parameter sharing didn't seem to be a good principle towards better compute optimal scaling is because we were in this setting where we could have as much data as we ever needed, and there wasn't much of a generalization gap. And so we basically just need to fit the data as efficiently as possible.

And so this basically led to the principle of having as many possible parameters per FLOP. And you might sort of wonder then, well, can you go beyond one parameter per flop? And I think things like sparse mistress of experts actually allow you to do this, because you have some sort of gating function which might be acting over different MLPs, and you only have some subset of them that are active.

And so you kind of divide your computation by k over e, where k is the number of active efforts experts relative to a model with the same number of parameters. And so we also tried to push that principle a bit further in this work where rather than having the mixture of expert gating function operate over whole MLPs, it was actually operating over individual linear layers within the MLPs and the attention projection matrices. And so you would represent these layers as like sums of structured matrices, and then you would have this gating function acting over the rank index of this sum.

And this would allow like much finer grain rooting decisions across the experts and did lead to sort of better efficiency for a given amount of computation. But the short reason I think is we found parameter sharing to not be helpful because you basically just want to reduce your loss as efficiently as possible. And you have sort of as much data as you could ever need.

And there isn't much of a generalization gap.

So one of the things I'm most excited about is developing AI systems that can discover new scientific theories at the level of general relativity or quantum mechanics. And we haven't even really scratched the surface in being able to do this. It's not even clear how data driven that process would be, how much it would look like symbolic logic if you were to think about what Einstein did when he proposed relativity and how we might want to write that down as an algorithm and automate it on a computer.

And so I would love to see progress in this direction. I think that some of the ideas that we've discussed around compressibility could play an important role in how we think about selecting for scientific hypotheses. Also, ideas around universality, like what sorts of assumptions might be more universal than others and at what level of abstraction.

But it's something that we haven't really made progress on at all, despite a number of very exciting research projects in AI for science, where neural nets, for example, are being used as black box function approximators in some sort of pipeline targeted at a very specific application. And I think this is extraordinary work and it's really a way in which machine learning is clearly doing a lot of good in the world. But I think it's time to try to go beyond that paradigm towards really giving us new scientific insights into the data that we didn't have before.

And in fact, that's something that I'm really more excited about than anything else in terms of how technology might develop in the future. Like if I were to go a thousand years in the future, one of the first questions I would have is, well, do we understand something about physics that we didn't before? Do we understand how the brain works?

And this is sort of a conventional sort of like approach to science where the theory is really the quantity of primary interest and the applications of course are important, but they're not primarily why like no individual application is primarily why we care about the theory. Like GPS would break within minutes if we weren't accounting for gravitational time dilation in general relativity. But Einstein wasn't thinking about GPS when he proposed relativity.

And if you have the theory, then you can sort of suggest all sorts of applications that otherwise wouldn't have been on the horizon. And like we probably could train a neural network to correct for gravitational time dilation without understanding what's going on. But that wouldn't be nearly as exciting or useful as as having the theory of relativity.

Mhmm. Yeah.