Captaining IMO Gold, Deep Think, On-Policy RL, Feeling the AGI in Singapore — Yi Tay

The thing that I find the most useful about, like, these models in general is, like, when I have these big spreadsheets of a lot of results and I just need plots of it. I think models can quite go with a screenshot and make a plot of this. I hate making this method like stuff about it's so annoying.

There were so many moments this year where AI suddenly crossed that, like, the emergent thing. Why the AI coding is one of that we just discussed? I think, like, Nano Banana also got to the point where I usually, like, if you make these images, it's just, like, a little for fun.

You just troll your friend or something like that. But, like, Nenoma actually really got so good.

It was like a vice thing that we put AGI in. Yeah. I think that, like, one of the reason why we work on this model is that we wanna get to AGI.

And I just it was a vice thing that we added to AGI to the job posting. Yeah. But there's no, like, formal name of the team yet, but it's basically the Gemini team Singapore.

Yeah.

Amazon as an AGI team and you guys have an AGI team.

You try official hot takes on this. No. You have officially AGI in your job title?

No. It's not a thing, ma'am. It's this is not a thing.

Yeah. Yeah. It's just, you know, we we just want to signal the north star of we're beating this model to get to AGI.

Yeah. Yeah. No.

I I wasn't really fishing for hot takes. Okay.

Oh, yeah. So I guess coming back is very interesting because it felt and returned to Google, like, everything including your LDAP, your username is all the same. It's like you play Pokemon, you you leave it aside and then you go back and you click continue.

Save game. You save game and then continue game. It's like that.

Obviously, the last one point five years, while I was away, many things have changed. Brain is now part of GDM and stuff. So I think that, obviously, a lot of things have changed, but I think overall, the coming back has been pretty seamless.

Of us, obviously, I I love Google infrastructure and I think debuts are great and stuff like that. Yeah. And I'm very glad to be back to Yeah.

To Google Infra. Yeah. And was the intention always that you were going to work on DeepThink?

No. Not really. I think I miss research a lot, like doing like research.

Not like super fundamental research, but like close to model research. Right? But I really miss being at the frontier and trying to go beyond that.

Right? So I really miss that a lot. And I think when I came back, big thing wasn't a thing, and I don't think there was any plans actually.

It was just like, I'm just going to work on research and see what happens. Yeah.

Yeah. I think reasoning these days reasoning and RL is like probably quite is RL reasoning comes I spent a lot of my past life, I call it the past art, working on like architectures and pre training. But I think now I more I will like condition more into RL research.

I'm not like old school RL, but again RL and old school RL. And and to be honest, I had almost no RL background coming back. But I think, like, like, RL is the main means of modeling these days.

And yeah. So I think it was pretty easy to jump back in. And I think a lot of fundamental skills in research is for general purpose and universal, and and it's it's quite easy to innovate even in a toolset that you're not super used to.

And yeah. So I think RL is basically the main modeling toolset that we play around with Yeah. These days.

But I know I understand it's very super super is objective and they have some, like, overlap. Right? Yeah.

Yeah. So I think like the biggest analogy of the own policy and off policy is basically that off policy is basically like when you SFT something, it's off policy. Basically, you take some other model, larger model stuff, and then it's basically like this off somebody else's generated outputs, trajectories, and whatever.

I think odd policy is mainly like the core idea of, like, modern LMRL where you, like generate and then you reward the model based on its own generations. And then the model trains on its own generations. Yeah.

So it's more it's a bit like self distillation to some extent. You the model generates its own output and then you reward it and then trains on its own output. So I think on policy analysis is basically this idea of, like, model training on its own outputs and letting the model, like, generate its own trajectories and then let letting some reward verify it and then the model train its own outputs.

I think this is more generalizable in general. I think there's still a lot of, like, science out and still to be done about the gap between SFT and and RL itself. But I think basically on policy and on policy.

Right? And I think bringing this analogy back to real life I mean, this on policy in us is more like humans, we are more on policy because we go around the world, we make mistakes, and then we are, okay. This is what like, imitation learning is supposed to be somebody else Not first principle.

There's nothing else you want to do and then you just copy. So I think, yeah, this philosophy, bringing back this philosophy to life is quite, like, powerful. Like, when I like, now I have a kid and everything, I want my kid to try stuff and then you tell them, like, okay, this is, where this went wrong, where this went right and stuff rather than, okay, you just copy everything somebody else does.

And then you are out there. And technically, SFT is still imitation. But I think for humans, it's a bit of this.

Right? Because if you basically, like, like sports. Right?

You play sports. You start off by imitating, like, hardcore imitating. But then you cannot imitate forever because you need to, like imitation I I don't know if this is good energy, but watching a lot of tutorials and stuff is more like imitating.

We learn try to learn certain movements and stuff like that. But then, like, on pause, know, it's like going to the game itself and trying to get a reward sooner from that. Right?

But so I think that humans do need some form of imitation learning, but, like, I think everybody starts off by imitating. But then again, the human and model kind of is not it's just fun to have analogies, but we shouldn't, like, take things, like, super literally and stuff like that.

What is this stubbornness?

Okay. I think it's a good it's a good strategy. I think also this brings a little bit to, like, when new paradigms happen, like, how fast people are to adopt it or, like, to invalidate their understanding of things.

I think as scientists, we definitely a lot of times, we do have to keep as the field progressed, we do have to keep, like, invalidating our own world model. It could be, like, a certain way is the way to do, like, something all along and suddenly something comes along and, you know, it is. Yeah.

Yeah. You can be very proud of your priors until this, like, becomes your prison. Yeah.

I know. Yeah.

Oh, yeah. Yeah. Mean, I like, when we train our models, they they sample multiple times.

So, yeah, to some extent, there's some form of Yeah. Self consistency. Is that directly let's talk with this theory.

Yeah. Self consistency is a little bit more Budgy. The more nuanced version of if we talk to Danny, that is not majority voting for sure, but it's it's more I agree.

I agree. Yeah. It's more nuanced version of that.

But I think parallel thinking definitely is related to self consistency.

in modern, like, LIM

Oh, you gotta be at a like, you gotta be at a photo taking to to get the credit.

No. No. Like like, to get the the credit for being the IMO.

But it's we hit it in the order. So it's it's just a joke. It's not an yeah.

By the way, okay. Could you tell the story of studying this IMO thing? Apparently, was done in one week.

So let me, like, be a bit more clarify a lot a lot of things. Right? So the IMO effort has been, like, very long standing.

So Tang and basically, Kuo has been busy working on this. Even last year. Right?

Last year, they got it's like, was not bad at Google at that time. Like, they had the alpha geometry stuff, and then they were, like, alpha proof and stuff. So it's it's a very long standing effort.

But I think this year was the we wanted to try to, like, use actually use Gemini as an end to end model.

I don't know. I personally was believe always believe in if we are not like, in retrospect, it's easy to say this, but it's a bit like, if the model can't get to IMO gold, then can we get to AGI? Basically so it's basically at some point, we have to use these models to to try this Olympic competitions.

And I think that one of the goals this year was like, okay, we're to do an end to end, like, type of index on model. That's where, like, my involvement came in. So basically, I was not, like, involved in the IMO effort only until the model training part.

So I have to say that that Tang did did most of the IMO thing. I just trained the model with a bunch of other Okay. What what does that work involve?

What are some things? So so basically, we we we just prepared the model checkpoint for the actual IMO itself. Right.

So that that's also something that's easily overlooked about the IMO thing was that many times you you want to chase benchmarks. So stuff like that is always like a thing that you can kinda keep running and running and keep climbing until you get there and then you but like the IMO was a life completion. Like, some members of the team were in Australia for the thing, and there was like, this happening thing was happening live was unfolding live.

Oh, it's a very alpha goal. You receive the thing, you, like, punch it into your system, and then you, like Yeah. Yeah.

Yeah. So so, like, some of the professors from Tang's team were, like, in when they went to the IMO itself and stuff like that. The conference I don't even know where the IMO is a conference, but it feels like there were people there, like in Australia.

And then so it was a life thing, and there were people who actually the job was to run inference on on on this IMO p one to p six that came out, and they also came out on, like, different days. So it's, like, different sets, like, one day, one day, two, something like that. So the fun part is that I knew nothing about IMO, like, at all.

I'm not, like, a kid that took part in IMO. Like, I was too down for that. A piano player.

And, yeah, I'll ever play the piano. But what I only knew was that, okay, we delivered the checkpoint, and that checkpoint was used to do the IMO goal. But then, like, there was somehow a week in London where everybody gathered there.

So everybody was flying to London, and then this photo was taken there. And then you get to see how all the different parts, like, come together. And, like, also being in the other rooms in the rooms with the other, like, co captains, and then it felt a little bit like a hackathon thing.

I think So okay. So that's what I think what you mean is that, like, some in some way, this thing with is encoded in the purpose of the model somehow. Yes.

Yeah. I mean, it's just whether at the of the day, you you just believe in like this, like, connection is one one model, lots of parameters. But mean, there's also two yous.

Right? There's also two yous which, you know, and stuff like that. But I think to some extent, the model we, like, I think we should be able to get to a point where to like, the past when the LMS first started, the model couldn't even be a calculator.

Now it can somewhat be a calculator. So technically, like a tool, like a calculator is somewhat encoded in the parameters of the model. So I think we we we will eventually get a point where whether there's things that cannot be expressed in the parameters of the model is like an open question.

I mean, we don't know where is the limit, but I think we will keep pushing and pushing this limit. So whether like something like a lean system or like some other things to solve other like, you know, physics engine or something, still we still continue to push that that boundary. Yeah.

But I actually don't know, like, whether there were a lot of debates about symbolic system versus endo yes. That's the word I was trained in. I actually don't really know whether there was like, to me, I was just like, oh, let's train the model, then then some someone told me to train the model, and then I train the model.

Basically, there was, like, overarching, like, people at like, the IMO effort that that decided this. And I also think that because the basically, specialized systems are very, like, one off systems that are, like, we could create, like, a chemistry engine, and create a math engine, and we could create the thing. Right?

But at the end of the day, you want one model for everything. So Yep. So I think this kind of fits that direction a little bit more where you have one model for and then this model was also, like, launched as Gemini deep thing, as a general purpose Gemini deep thing.

So it's basically unchanged, but with maybe some config toned down a bit. Yeah. So the the the inference time config was like the one serve to most people is different.

But and the full IMO conference config was present like shipped to some mathematicians just because of the inference cost. Right? But that was good enough to be a general purpose model.

I think my take is that this intuition was what led to the trying to go towards one model instead of because these specialized systems, there's no end. Right? You can create many specialized systems.

Yes. The the most I can see in the future is there'll be a model then that that is there's something that really cannot be subsumed by a model, then you just use a tool or something like that. Right?

But my prediction is that, like, the most things can be can be subsumed by the model. I think yeah. And maybe the AI researchers are quite good at hill climbing.

Right?

So I think there's good there's a lot of aside from proof, there's a lot of domains that are, like, non verifiable and I think not not easy to verify. So it's like when people mean nonverifiable, it's, like, nontrivial to verify or, like, just not as not as easy as, like, the solution of, like, a math problem because it pulls a long form and it's also that's why it's nontrivial to to verify unless convert it to lean and then you do all the kinds of things. Right?

but I was related

And, yeah, basically, how does the effort look to come together? So I think there were four captains for the IMO two from London. Jonathan was from Mountain View.

I was from Singapore. So I think four of us basically trained this model together. And I think one I was also trying to see what Tang was saying.

But I think one one interesting thing was that, like, we're all in different time zones and we're off. And there's something also very interesting about, like, passing on the job. There's no, like, really, like, fixed work for how to work together between captains.

So it's more like, oh, I'm going aboard the plane now. I'll be at AFK for twelve hours. So if someone takes So you're JB sitting in run?

Sometimes there are bugs and stuff. The job comes down sometimes. So basically, it's very ad hoc and it's very liberally it's really between the captains how we decide to, like, work together.

And yeah. But I think it was a kind of interesting time also because we were all flying. Like, I think the London folks were not having to fly, but I had to fly and Jonathan had to fly.

And then, like, when you visit another country, you have another like, if you visit an office, have many meetings. So there was in and out meetings, and it was pretty, like, interesting. And I also think that we nobody really knew whether we would get a goal at that time because IMO actually has had hasn't happened.

Yeah. It it was interesting and exciting. And then I think, like, the whole process of this getting verified by the IMO committee and, you know, like, you know, there was, like, okay.

But we're not going there. But I had to learn a lot about how the IMO works. Right?

Apparently, the gold score is not even it's not even a fixed number. It's like a bell curve. Very on the Right?

So it was like a time where you just look at the score, you're like like, was even, like, looking at the watching the human participants and then seeing, like, what the scores were because whether Gemini will get gold depends on, like, how the humans do. So you're, like, looking at all if a certain percentage, you're, like, what's the do we get gold? To some extent, you don't have any control over that.

So But you're just curious. Right? Because Yeah.

That's why I would say that it's definitely more, exciting Like, there's more adrenaline than, like, just running on a benchmark and getting a number. It's was, like, a process that that that took some time. Yeah.

Yeah.

I I wouldn't say, like, surprised, but, like, it was more like a pat on the back kind of surprise. We actually made a lot of progress in we, as in collectively, the the all the engineers and researchers working on Gemini, there's a lot of progress being made. They just look at how much we went in one year.

Yeah. And I also think that it's just five years ago. Right?

Not two years. Like, five years, you just you just imagine the outcome. Like, you just look at the state of AI now, like, just generally the the IMO and the ICPC goal and, like, also even, like, things like Nano Banana.

If you just look at the AI progress now and five years ago, I think people would think that we already reached like AGI. Some form of AGI. There's some form of AGI.

Yeah. We're just moving If you just travel like, you take these checkpoints and you travel back into five years ago, someone would make a drama about this, but I think it's really quite impressive, like, the field has moved so quickly. Yeah.

Yeah. Yeah. The hard parts you would say were scaling inference.

In what aspect? Like, hard in terms are you in as massive?

Okay. That's not really like, it was in the so I think I tweeted about it at some point. Right?

Yeah. Bringing up the tweet. So the I'm quite cat was basically like okay.

Let let me think of where to start with this. What do people need to know?

I think I probably should be listening this. There's actually nothing specifically done for Pokemon. Of course.

Yeah. Of course. There is nothing specifically done for Pokemon.

And I think that I think Logan had this tweet recently about the recent Jeremiah three on Pokemon Crystal. And Pokemon Crystal is like was so much more efficient. I think Pokemon is like so I used to play a lot of Pokemon, and I'm a big Pokemon fan in general.

And I think it's a it's a great like you say, it's a great long horizon benchmark and stuff like that. And I think it's good to check-in once in a while on these benchmarks that, like, almost never get contaminated or let people actually, like, don't spend time to like, do climb, match max. It's, like, kind of silly to, like, build, like, okay, like, what are you working on?

If somebody will be, oh, I'm working on Amy. I'm working on HLV. I'm working on Pokemon Maxine or something like that.

That's kind of, like, funny. We did interview the Cloud based Pokemon AI. I think his name is David.

like the harness thing or like also whether the model has access to like game state information or is it computer visual?

but I don't know whether I'm jumping off tangent or something like that. I solving Pokemon is gonna be more of, like, how fast you solve it. And then, like, the thing that I have not really seen so far is like whether the model can complete the Pokedex.

Why is that? I think it need more challenging. No.

Completing the Pokedex is so hard. You need to plan. You need to like you need to search up like information.

Like, there's some things if you don't like go online, basically you need to have a little bit of deep research in Yeah. Yeah. In this.

Yeah. The the model will just never know, like, that it needs to trade. K?

If it's able to go online, post on forums, then find someone to like, hey, Claude, can I trade with you this Pokemon to evolve? Some Pokemon needs to be traded to be involved. Yes.

Or mated. Yeah. Yeah.

But anyway, I I don't know. I have not seen the model if you're able to complete a book of death. But completing a book of is really hard actually for model.

So I was I think that's actually an interesting like like an interesting one. Yeah.

would be once let's if let's say we we have a model that is capable of doing that, what can we make it do that we cannot do today? Well, there's a lot of planning involved. Right?

There's real deep research. Like planning there's also a lot of planning involved and it's more I think Pokemon is the Pokemon game, the is very linear. Right?

The thing that Pokedex is involve a lot of, like Backtracking research. Yeah. A lot of research and a lot of yeah.

So it's probably a different nature yourself.

Don't have high confidence that it will do it. I I don't know if someone's actually tried. Probably not.

Right? No. I I think the hard part is actually, like, trying to use the like, synthesize the web knowledge and then apply it in in in the game itself with all that visual state going on and stuff like that.

It probably will just solve in one or Yeah. Yeah. It's not It is challenging.

It's not challenging, but it's not like super interesting.

which is what didn't. Oh, yeah. Yeah.

But then when it comes to this study, it's mostly just exhaustive search thing. The model to try and try, like, humans guide for that. Yeah.

Yeah. Okay. So that's actually less interesting to you.

Interesting. Okay. Actually, to be okay.

When you think about this, like, not super super interesting, but it's okay. It's just that I have not seen the model to try to do this. So it's Yeah.

The story. Yeah.

But Novo Knowledge is actually an interesting thing that that I think is gonna be quite a big thing. Being able to generate Yeah. Novo Knowledge.

Google has done stuff there, which I don't you're probably not that close to those teams that has done AI scientist work. There's some things that have been it's like, for example, if you freeze the model weights at, like, twenty fifteen, the freeze time at twenty fifteen Mhmm. And then you even with the current model, let's say you have okay.

Let's assume there's no leaking of information somehow. Can if you ask the model what's the best ML, okay, not twenty fifteen, like twenty twelve or something. It will just tell you the SVMs are, like, the best.

Right? This is the way that machine learning works in general. Right?

Then then the question is, can you invent the transformer? Right? It might not be able to like, even today models, they might not even be able to invent the transformer.

Like, if you freeze the time at a certain time and even you bring the tech I mean, the the model is a transformer, I just say there's no assuming there's no leakage. That's you're like, like, Probably possible. Yeah.

Right? So so I think there's still a lot of open questions about whether the model can really, you know, innovate and generate, like, really novel Yeah. Knowledge.

I think these days, like, reasoning itself is very vague and it's very open. So it's like most mostly different people have different definition of what reasoning is. Right?

So I agree that, like, this chain of thought is, like, basically, when people think of reasoning, they associate it with a chain of thought and also this is what happens in in in the thinking and then, okay, reasoning. Right? But I think these days, it's more like I said in earlier part, it's like reasoning RL is almost like this, like, basically, it's anything that is, like, post training to elastic capabilities, basically.

It's like RL and post training to elastic capability. So I think the actual, like, technical definition of reasoning is making models better with thinking and post training. K?

Yeah. So basically, like, RL ing the model to think better. Right?

And thinking is more like thinking traces and thought trajectories and stuff like that. Right? There's also this line of work for late like latent thinking and and stuff like that.

Like, when the lender thinking and discrete token thinking is, like, gonna be the same thing or, like, something like that is, like, open question. Meaning adding extra tokens to your vocab that represent all Yes. All these I forgot what's the name.

Like, these academy papers that do these loopy things or Track token. Or, like they basically instead of decoding discrete tokens, you actually simulate this by doing this in latent space. Right?

Uh-huh. So when you do chain of thought thinking and, like, reasoning, you basically decode extra tokens hiding in the thinking pack, and then you decode stuff. But, like, latent thing is basically you just don't decode Yeah.

Tokens.

Do you actually see much reasoning traits on the Internet?

And it will just go up over time because, like, people are publishing it. Yeah. But I think if the sources are, like, quite clear, you can actually, like, filter away those because it's usually people put it on GitHub or put it on Do you want to filter it?

Maybe you don't. There's there's a choice for the for the Yeah. Quite real quite literally.

Oh, yeah. So I actually don't really use much AI coding in the past, but I think we reached a point where AI coding has started to become really useful. Like, okay.

So before AI coding, the most the the thing that I find the most useful about, like, these models in general is, like, when I have this big spreadsheets of a lot of results and then there's gonna be plots of it. I think models can quite go with a screenshot and make a plot of this. I hate making this method like stuff about it's so annoying.

Okay. But that's basically, like, one thing that I can remember about, like, how I use AI in the past. But I think AI coding has started to become the point where I run a job, I get a bug.

I almost don't look at the bug. I place it into, like, anti gravity and, like, I thought that would fix the bug for me. And then I relaunched the job at Tedlight.

Beyond, like, Vibe coding, it's more like Vibe training Vibe ML or something like that now. I would say it does pretty well most of the time. And it's actually, there's there there are classes of problems that it's just generally I know this is actually really good for and in fact, maybe probably better than like, would have to spend twenty minutes to find like, figure out the issue and then fix the thing and then redoing.

So yeah. It is very interesting because I do say like, level one vibe coding is you actually know what to do, You're just too lazy. Yeah.

It's just, ah, just do it for me. Like, I've done this a thousand times. Like, just go fix it.

Like, I know exactly what to do. Here, you're saying it's kinda like the next level where you actually don't even know. It's it's investigating it for you.

As long as, like, the answer looks right, you'll just strip it. At the start, I was a bit I I did check it and look at everything. And then at some point, I'm like, okay.

Maybe the model looks better than me. So I'm just going to let it do its stuff, then I will relaunch the job based on the fix that the model gave me. And I think the models will just keep keep getting better and better.

Yeah. So, yeah, it's something that that Yeah. But I also think that I recently, there's this there's anti gravity.

I think I think also because these tools were not like that. In Google infrastructure, it's not that easy to you don't I'm not that familiar with what is available outside. And when I was at the start, I didn't really I think the models were not, like, so good, like, one and a half years ago.

So it's also like a forcing function that lies also people are like, oh, try anti gravity is a game changer and stuff. Like, okay. So I just started using and yeah.

Yeah. You spent some time with Varun recently.

as a user. There were so many moments this year where AI suddenly crossed that, like, the emergent thing. So I think AI coding is one of them that we just discussed.

I think, like, Nano Banana also got to the point where I usually, like, if you make these images, it's just, like, a little for fun. It just throw your friend or something like that. But, like, NanoMetal actually really got so good that You can use it for the best.

Yeah. That that you can use it for, like, basically yeah. So it's getting really good.

And I think, yeah, this year, the stuff like and even things like the past, many of these LMs were, hallucinate things a lot. But now, I just trust it automatically.

Gemini internal for only your code base and your work. Oh, actually, I'm not sure though. You see, these things are just like Yeah.

Really?

So I think these are these are all interesting questions. I want you to say one quick thing first. Right?

So I think that when it comes to, like, whether a model can be a, like, a junior suite or or, something like that. Right? So I think if you think of it this way of if a job from a one one x SWE, one time SWE can be replaced by a model itself.

But let's say you are a manager. Right? The objective that the metric you track is like your time.

And then if you can have a model that saves you like the same amount of time as the work that your reports do, but you don't actually, like, replace one person per se, but you yeah. A little bit from everybody. Yeah.

You're right. Then you can I I definitely agree that, okay, like, when you count the net time save, there are times where the model can fix bugs that, like, what have cost me, like, one day? Right?

One day is huge. Right. And these things are definitely, like well, if you I don't know whether anybody has done any, like, real, like, metric evaluation or these type of things, but if you use time as a real metric and then not as a number of okay.

Maybe three hours, like, kind of metric. Right? But these things are not like like, gonna replace one person one as as it is, but more like a passive aura that buffs everybody.

The in game terms. Right? So yeah.

I often think of myself as a bard cause I tell stories and I I plus everybody around me, you know, I think I couldn't that's that's a that's an ideal situation for me in a D and D group. Oh, okay. Yeah.

Oh, there are times when models get try to get lazy and try to fix something in a like, they are still we will have they get lazy and then they try to, like, guess, like, me into thinking that, like, the bug is fixed. So there there are still classes of there are classes of problems that are, like, very easy for the model and very hard for humans. There are some things that are very easy for humans, very hard for model, reverse paradox, and stuff.

So it's still very hard to characterize these these things into these proper quadrants and stuff like that. So I will say that the capabilities on models these days are good enough to be, like, really helpful, but, like, it's still you it's a bit like it still has some. But yeah.

But I think this will like, this I don't think there's anything that to to be done to specifically, like, focus fire. These things is more like general capability improvements. The model just get better over time, and then these things will just, like, go away.

You say that okay.

isn't hill climbing just a sequence of doing that until you reach AGI? Okay. So I I get your point.

I think that it's true that sometimes a lot of progress on the whole is just a series of small incremental changes that, yeah, that push. I think that's accurate. That's true.

There's also it also feels that there's also a lot of, like, small mind like, seemingly minor, for the lack of better word, like, that push AI to the state where it is today. So I definitely agree. So nothing, against people who are, like, focused fire.

But it's just that when I mean that, like, it might be not easy to focus fire on things that are, like, not very easy to characterize. So it's just that, like, when you have there's something targeted. Right?

Yeah. Like, okay. I wanna improve this capability, add some data.

So I think, like, defining, like, the evals, defining the problems and stuff, it's, like, characterizing it. And if it can be characterized, then then okay. Then fine.

But I think it's, what I was trying to say with the coding is that these things are not even some of the class of problems are, like I don't work on coding, but, like, people who maybe work on coding, they know, like, they have, like, terminology for different types of failures. But so maybe somewhere somebody is focusing fire on this, then make the model better. That's great for everybody.

So attention and transformer have been is like a core idea in the recent times. Like, pre training and scale is the thing that made attention and transformers actually shine. Right?

Because without I think the first transformer paper was this like a machine translation thing. And then basically, GPT and BERT were the ones that like actually showed the full, like, big potential of this idea. So in terms of is industrial, like, really, really all we need?

Like, probably no. But I think it's like one of the from the architectural point of view, also may maybe no.

It will be a transformer, I think. Really? Like like, you people it depends on what you call it, but I think unless the paradigm shifts completely, which is mean, as a scientist, you cannot like like completely say no to like like that this will never happen.

But my feeling is that it's been like what? Like almost ten years since the transformer. In 1917.

'9. 2017. Nine years, right, since since the transformer.

I think we have not replaced self attention. Like, it's it's some form of it. Like, you could rename it.

You could name it something else. You could sometimes You can do local global yeah. It's still a transformer and then and I think, like, that's not going, like, anywhere.

Unless the whole thing with, like, back prop, like, everything, like, goes like, the whole thing just changes completely. Like, then there's a different story. There's a different conversation to have.

But if it's still within the same scope and bounds so I spent a lot of time thinking of about architectures and, like, whether there's alternate architectures and stuff like that. Yeah. Okay.

At the sequence processing level, like, there is the ultimate There is sequence to sequence transformer. It's probably the self attention is there was this whole big era, which I was also involved in this era where people try to, like, undermine the attention as much as possible. Like, they try to remove it, simplify it, make it efficient, like this whole, like like, efficient attention era.

At the end of the day, the outcome was always like, oh, we remove all the attention, but we have one layer of self attention there and it still works. Like, that's at the end, like, of the always a story.

still attention but modifying it quite lot. I I would consider local and global attention to be, like, still still attention. Just like how much you're skipping.

Yeah. Yeah. The the only question is that if the formulation changes too much, your QKV becomes like ABCD, MGD or something like that or some Okay.

Maybe I'll give you some motivating constraints Yeah. In order to do this.

But it's just not really about sequence to sequence, it's more about like the whole gradient design and backpropting. Right? Mhmm.

It's not architecture itself. That's that's a problem that we that is more of, like, the learning paradigm itself rather than the architecture itself. I think the architecture is just basically, the interface between the learning algorithm and the tokens.

I think it's more about the learning algorithm itself. And this like, continue learning this, like, there's many ways to think about processing many, like, insanely large, like, context. Right?

Like, 200,000,000, 1,000,000,000 tokens or something like that. Right? Like, whether it's gonna be like, you have a new learning algorithm that every time you run inference, you you learn on it.

Right? Then you can technically have some kind of memory, like human being as learning as I'm talking to you. Right?

So that that's also, like, one way. The other way is, like, whether, okay, maybe somebody will say that, okay, the attention is, like, just too expensive for 200,000,001 billion context, so we need new architecture. Or some people will say that, oh, we just improved the chips, like accelerators.

So I think many ways to interpret it, but I think it's like if it's about there's a lot of fundamental things that if it's about continual learning and stuff, there's a lot of fundamental things about the learning algorithm and stuff there as well. Yeah. That will have to change?

I think, like, the learning paradigm and architecture and stuff like goes, like, hand in hand. And I think as the field progress and ideas just stack on top one another. Right?

So there's also this thing about ideal that it was proposed has to be compatible with all the work that have been done before to to shine. Right? It's a bit of variant of this hardware lottery, like, by by Sarah.

It's not the hardware lottery that I wrote about the t GBUs failing, but it's it's the original Haraway lottery. But it's more like a bit of lottery of, like, the things proposed have to play well with the things that were proposed before. So it's a bit like going down this local optima to some extent.

So now we are, like, in this local minima of, like, transformers everything everything. Right? Maybe it's not easy to, like, get totally out of this because also a lot of people's investmentization and optimizers have been have been done.

So the things that play well need to play well with the ideas before. And the the way I see it now is it's very difficult to, like Yeah. Come out of it.

Okay.

There's this, like, misunderstanding a little bit about, like, oh, the last five years has just been, like, scaling things, scales. Okay. Please tell me more.

Yes. I I You you made that joke about scale. Now we scale research researcher salaries.

Okay. Let's not go there. But but I think that ideas, like, matter.

And I think that there have been a lot of good ideas in the last five years. It's just that maybe it's just not so it's not been, like, blindly like, if you took a MLP, right, just, like, without self protection, and you just okay. I'm going to throw, like, $100,000,000,000,000 on this and scale up that thing and the thing.

It's never gonna work. It's never gonna work. Yeah.

Yeah. So there's no, like there's part of it that's also, like I think the bitter lesson gets used too much in like, too conveniently used around. But actually, there's there's also a little bit of not a bitter.

There's also a sweet lesson where it's like ideas matter. And I think even to to today, right, like, people downplay ideas and stuff like that. Yeah.

Do you think the rate of new without being specific about what ideas, because obviously you can't share, but like, do think your rate of ideas has increased or decreased because there's like kind of a law of diminishing returns? Are these smaller I think the number of ideas is always proportionate to the number of researchers working on a certain problem. So by definition by definition, it should increase.

But I think the number of ideas that actually work is not decreasing compared to the last like, it's we're not in the era of diminishing returns yet. Wow. So I think the ideas are still very important, and there's still very good ideas.

There are game changers that are being invented. Yeah. Yeah.

of like In terms of what? Like serving?

honestly, I don't think about this this memory bound that much. So maybe it doesn't yeah. So I would disagree with it, but I don't have high confidence in anything.

And because you're mostly on the research side, less on the inference side. Yeah. There's yeah.

Maybe the inference guys will be like Yeah. Yeah. Yeah.

I don't think about I don't recall when I think about serving. So, yeah, maybe I don't think about the inference that much. Yeah.

And I don't know what that means. Data efficiency and basically, but you're asking the question in a way of asking, like, how much repeats is tolerable?

like learning more like extracting more from varied data points is definitely valuable, but I think that's all related to the fact that we're like running off tokens in the world. So I don't work on data for pre training. I think things that I say were General state of industry, not nothing The general state of industry.

Right. So I think that the don't even know whether data has, like, diverged. Like, the way that these things are done, it has diverged too much across this labs and no open There's a lot of cross cross pollination for sure.

Cross pollination? Okay. Okay.

Yeah. But I don't think about data that much, like, pre training data that much this Yeah.

classically, you take 20. The data efficiency of humans is definitely way higher than than than models. Yeah.

The only question is that where does this thing come from? Is it actually like putting more flock on every token?

to see. Nobody said pre trained was dead. I know.

Yeah. But what is the world model itself itself?

No. I don't think about world models that that often. Think because world models are just not really well defined in the first place.

or, like, AGI capability that is not easily unlocked right now on our current path of scaling. Yeah. Yeah.

So I I think I think when it comes to, like, like, data efficiency, I think it's more like a I'm believer of finding ways to spend more flops per token. Right? Because you actually, basically, if you are data bound, you want higher data efficiency because you can learn more from every data point.

You squeeze you squeeze out more points. Right? So things like that can extract more can use more flops on every token is definitely like a form of data efficiency.

Then there's the learning algorithm. Right? Because I think there's this there's a different scaling law for, humans is this, machines is this, like, dogs are this, cats are this, there's this different Nice.

I just wanna That's the earlier chart. Right. Yeah.

Yeah. That there's this famous famous famous chart. And point one and point two are just like not entirely like the different things because it could be that better architecture is actually just spending more flops per token.

So if you are you come to a point where you are very data bound but not compute bound or you just find algorithms that spend a lot of compute on every token on every token. So I think the overarching point is just that, okay, that that that that it's a learning algorithm thing for data efficiency. And then if whether the correct way is actually just apply more faults per token, just to squeeze out more from every Hory data, every data point.

Also, because humans actually don't like like, are exposed less when you say less or more data, it's very ambiguous because they are technically, like, on twenty twenty four, 37, and then you caffeine is mostly visual. Yeah. They have a lot of, like, different dials inputs.

Right? And whether they actually spend more flops on everything they listen is also a question. Because maybe they are just beta efficient just because I mean, think somebody needs to count, like, how much flops the brain used to process, like, how much maybe they're just spending more compute than every token.

And also maybe the learning algorithm is different. So but I agree that data efficiency is very important given that that I think we're gonna like, there's limited amount of data in the world. One more thing before we go into DSI.

Aura environments that are like that have a lot of domain expertise are probably very valuable. And actually, I don't know specifically about what Aura environments people are actually especially buying. But what was the thing that you're not satisfied by?

LM's are just used everywhere now, but whether it's actually like a Like a big LM. Like, whether it's like a generative retrieval type of Yes. Models, like, it's like another question.

Correct. Is it? We don't know.

All we know is that they have said that they have swapped out their current REXS for an LM based REXS. That's all they found. Okay.

Okay.

But you have actually some background in IR. You care about IR. No.

I don't care about IR, man. But I think DSI okay. Like, DSI or or generative retrieval was like, I think one of my favorite works in the old of like I have some IR background in like when I was doing a PhD, I did some REXIS work.

I did some retrieval work with REXIS and stuff like that. So I have some IR and REXIS background. So I think generative retrieval and generative Rexis is very very conflicted.

DSI started as a retrieval thing. So we did, like, natural questions, like ranking of, like You learned the index. Yeah.

Like, doc documents that are rare everything. The it started off as I think that's actually, we did an interview with Yanik, like, and Don, we did an interview with Yanik when when when the paper came out, like, long time ago. So at that time, we wanted to, like, reimagine retrieval and search.

So we wanted at that time, LM, we were still using t five models. At that time, it was, like, not we're not in the LM era yet. It was pre LM era.

It was like, okay, pre training works kind thing. And then there were like some pre training models around. So we wanted to reimagine retrieval.

Right? But retrieval rexes, they're all the same formulation ranking retrieval problem. Right?

And then that's where we started to imagine retrieval as one giant, right, and that that encodes everything in the memory. But we try so many different like, so many ideas actually, my collaborator, Vin, was the one that came up with either so many ideas that basically and the start of this whole genetic retrieval was actually basically literally just trying to give a document like identifier and just predicting like raw brute force predicting like this. Actually The ID.

Yeah. Like, it actually works because the models can memorize some something. If you look at the literature from all the way to things like doctor Vag, it's very off like model the words have no meaning than this ID in in a vocab, but it's an artist's number.

Right? And technically, the model should have enough capacity to predict. But I think semantic IDs was an idea that that basically you have some, like, semantic association and then you actually try to break down the search space hierarchy.

Alright? So how this will evolve into REXIST was at a time after DSI came out. Right?

So Ed Chi's group and Mahesh, the guy who who they did some exploration of applying DSI to to Raxi's. And that's how that generative generative Raxi's recommended system paper came out. Yeah.

I didn't even know he was involved. That's crazy. There was like basically us, like, transferring this, like, basically, okay.

DSI works. We we try to try it on on Raxi's. And then I think the recommended system, people have a slightly different way of doing somatic IDs, but it's basically that's because the the domain is slightly different.

But after that, I think we were done with the invention, anyway, this one The rest is details. The the rest are are details. So over time, I also left Google and stuff like so over time, these things evolved a little bit here.

I think also saw, like, something like Spotify is also using something like YouTube, Spotify, like, they they use this style of, like, somatic IDs, this style of DSI, models. I think from the research community point of view, like, the DSI work was the first one that, like, decode semantic tokens. But then when we went to I don't know.

Academic community is, like, strange in a way that, like, they will do things like, oh, this is gender retrieval. It's not genetic rexes. It's like they will do this kind of like random things that is like a a bit strange.

But yeah. I really think this was like the whole history of this gender retrieval. Apparently, there's also a lot like of people working on.

I don't follow actually, I don't follow Addis at all now. It's just not even in my mind. But there was once I went to even in Singapore office, there are SWIS actually, like, working on generative retrieval.

They don't work on I don't know whether they're still working on it, but I met a person that tried to explain generative retrieval to me. It was quite funny that, you know, I kind of like co invented generative ritual. But, yeah, I think this is this whole IRR thing has been it's just an interesting phase.

And I I definitely think DSI is one of my more creative works that I've done. That is not like really LM, but it's like under the general principle of apply ML to everything. If the Googler is working on general retrieval, would that be like AI overviews?

Is that something similar?

It is not that's not just about paraphrasing. It is about understanding query intent. I b m 25 is a really strong baseline, actually.

B m 25 is a really strong baseline. Yeah.

depending obviously on the query set. I didn't really think about it this way before, but because I've done modeling in many different domains, including like search and even in the search community IR community, there's also set a benchmark and stuff like that for like, that people will hill climb on and some like there's Amy of or like a of I don't know why they call these days anymore. But generally, the modelling dynamics of IR task is very different from like, a Raxi's task is very different from standard language tasks or like vision tasks and something like where we kill clientele and where you train models and like, the way that modelling things interact with this environment is very different.

So I think that I honestly I hated working on, like, Rexis and RuxiVo stuff. Okay. I'm just looking about old days.

When you work on, like, T5, you work on you change change architecture, you try to improve complexity, you use superglue. Like, this is the olden days. You are on like, even now when you train LMs, you just do zero short, two short, stuff like that.

Your things the because as a researcher or engineer, you just interact with the environment a lot by this you're just like, okay, RL by this environment. But REXIST and IR has a very strange feeling to it. Strange feeling in the sense that it feels like you're the like, whatever works is like you are in this in a world where the the gravity is different or like you are in a world where the modelling things that feel intuitive are not intuitive.

So it feels like a very strange space to so I wrote some papers back in my days on, like, phraxes and stuff like that. Every time I ran some modeling experiments for for phraxes and stuff like that, I I didn't enjoy the it it feels like the environment was rude. It feels like the the vibes are just like like What makes it rude?

It just feels transactional. No. No.

Not not not not transactional. Like, I I don't know how to describe it. For example, like, if you play, like, sports like the tennis perimeter, where you hit the ball, you have a very nice feeling like hitting the sweet spot.

When you do modeling in traditional dome, when you get the feedback back, you feel like everything sounds right, everything feels right, everything right. But Rexis and IR is like a problem where it's like you hit the shuttlecock and you hear a glass shatter like randomly. It's like just feel this weird, sexy world that that Cause and effect are too far apart.

Like, it just feels strange. And then sometimes maybe the metric, like I think the Red Sea is they use, like, all the NZCG effects. And then the BM 35 is strong, and then you just and then you get, like, worse than the, like, the the BM 35.

Back in the day, you stack two LSTM to three LSTM, you're like, woah. I see life. It's just the game.

An unrewarding area to work in. It's it's just weird. Also, the IR community and the retrieval community is also, like, always behind the mainstream.

And then now it's just probably gotten even more worse because of LM and stuff. So okay. I'm getting into hortic territory, but is this like, some of conferences are just, like, behind NewRichs and ICML and stuff.

Yeah. Some conferences are just, like, they're just like applying things that that this can They're downstream of They're downstream. Yeah.

They're downstream. Okay. So it always feels very uninspiring to to to to work on on this.

Look, there's a reason that you left and But it was like a side quest. Like, when I woke up on the air, it a side quest thing. Yeah.

Yeah. Okay. I understand.

field. I kinda understand why. Because the academic benchmarks for those tasks are just so far detached from Yeah.

They're so far detached from what industry is. I didn't work on any of these, like, in in like, like, the the thing. But that's from my academic point of view.

Oh, then all you need are online Evalsa. Right? Yeah.

Yeah. I test and I go.

impetus for starting GTM in Singapore? So we will talk about the event first. So the event was mostly so Kwok and I are gonna start the team, and then I think before I came back, we discussed this for some time.

Jeff was very supportive of this. He was in the region many times in Vietnam and Singapore last like, about around the time when I was gonna come back. And I think that so this event itself was Kuo and Jeff was visiting, and we just inspire the community here.

I think that it's also a bit more like a soft like setting the tone right for the start of the Gemini team in Singapore. And I think it's a very rare instance where you get somebody like Jeff and Kuo Hua, the true pioneers of AI in the world, to be in one room. And then, are you there as well?

And I think like many people told me Oh, true pioneer of AI, but I was there to a live tweet. Yeah. Yeah.

I think having them all in one room and then giving these talks, many people came out to me and said they were very inspired by their presence in the region. So starting a team and starting something is also very like, there's also no one moment that it's not okay. Press the button, it starts.

Right? It's like a process. Right?

So we hire people and then people join one by one and stuff stuff stuff something like that. Right? So I think this event was more, I would say, like, to set the vibe.

I think it's possible for Singapore to be close to the frontier. And I think that we're having the true pioneers of AI here. We want to give this basically, more like an inspiring thing.

And also get left for Kok and Jeff to meet the people here. And and so Jeff was here last year, but Kok hasn't been here for some time and he's gonna have a team here. So it's like also nice to bring him around and meet the people here.

Yeah. So it was a really, like, amazing event. We met along as well and Who a lot of people don't know has a CS degree.

He's like one of the few PMs with a CS degree. Yeah. Yeah.

I would say that the context of the meeting was more like partially, like, also, he wanted to learn more about the IMO stuff. Oh, really? And then also about Jeff.

He wanted Oh, because they they invited you without knowing that these guys were coming or something like that. But I was a bit is like some yeah. Jeff and Kwok and me, well, we went to visit a chat with at the East Anga.

And we discussed a little bit on dipping, discussed a bit on IMO. And then I think the rest of it was more like Jeff and Lee Sinon was talking more about, like it became less about AI and tech, more about very macro economical, political thing, which was I was very out elemental with, so I was just I I just You were in a suit. Yeah.

I I was just talking about the dictating and the IMO and stuff like that. Yeah. But he seemed genuinely quite surprised that AI has reached this point.

This is a very good, like, question. Also, it's related to the thing about identity. Right?

Because I think you also moved from SF and Singapore quite a bit. Right? I was in my mother view this, like, one, two weeks ago.

And I'm here, but, like, almost all my my my if you just look at my sibling, aside from my family, like, everybody I talk to is, like, somehow in the Bay Area or, like, just because of work and everything. I think the geography matters. Okay.

Firstly, the the most like thing that is like logistical is like probably the time zone. So you literally want the twenty four hour coverage around the world. No.

There there are understand. There are advantage no. What I want to say is that the difference is possibly but like, it's also more like like people define the location more than location define Okay.

The people somehow. Okay. Time zone, we get to time zone a little bit.

Like, there are pros and cons. I think that's pros and cons. Right?

So you're bullish on on Asia, Singapore people, the talent pool. I think we managed to find, like, amazing amazing people. But I will also have to say that this type of things is more like talent attract talent.

I think most of the time, like, people are very excited. Like, the vibe I get is that people are very excited because it's, like, course team and my team and we're working on library core things related to to to AGI. So I feel like the talent we can get from the region is really good.

But it's only it's only because it's us as well we can unlock this talent. Otherwise, might join some other place. Mhmm.

But Yes. And move to to The US. Yeah.

Yeah. Yeah. About the identity wise, I will say that I definitely agree with you that, like, why it doesn't matter.

Like, that I think that the advantages of Singapore itself or, like, just anywhere, like, it's also that you are like, okay, the world is very good, so technically, you can interact as much as you want. You can also go there. But I think Singapore has this advantage where you can go close and you can go far.

I think the Bay Area is like so much about I have friends in London and New York, they would just never move to the Bay Area. So I'm not against Bay Area. I think Bay Area is a great place.

Right? But it's just yeah yeah yeah yeah everywhere. Right?

I think sometimes if you have some, like, mental space and energy to, like, to have some other culture and then, like, you know, like London, Singapore, New York, they have their own culture. Right? But the Bay Area culture is just like AI.

Right? Like, you just go anywhere, you just hear AI everywhere. Right?

Even the billboards and stuff like that. You think it'll be much. Yeah.

Although I did see some billboards down here. For AI also? Yeah.

No. I was like, what is this? Is is culture infecting Singapore?

I do think that to some extent, if you want to do research in you need a little bit of peace and quiet somewhere. Right? So this island may be good for that, but then you can you still we're still, like, able to, like, be connected.

Right? Okay. So I think that's mainly talent wise, I think people are strong here.

What are you hiring for? You're still hiring. Right?

We're hiring like like my team will work on like RL and reasoning for Gemini and Gemini Deep Thing. I think we care more about like talent density now. So we're not like also like growing that big, this small team first just because compute per capital is probably like important and yeah.

So I think that's something that that we're hiring for now. Basically, I personally just there's a lot at the end, but I think, like, generally, there's a lot of people, like, who are very capable. But I think what I'm looking for mainly is either you have, like, a track record of RL research or, some even not necessarily the RL, but or like some exceptional achievement in like coding competitions or like some exceptional achievement somewhere, then there's like the kind of people that that we we want.

Yeah.

Right? So you don't I forgot if I said that or not. But to far, I stand just to to to stand I stand here.

I think we'll definitely be very happy with people that are like very high stats and just like even without much statistical knowledge. No. No.

Stats. Yeah. Yeah.

The stat points. Child wiz int. Like, just high-tech stat points people.

that I actually went to recruit them, like, personally. Right? So it's I you see see the work and then you send them DMs.

Right? So I get a lot of Yeah. That's paper.

No. No. Like like for hiring, generally.

So yeah. They just need guidance. They just need, hey, work on these like five things.

It showed me an interesting result in any of them. Okay. So if somebody comes up with something and then does something that you feel that is very tasteful and aligns with what, like, researchers in the labs, like like, want, and they come up with that independently, you know that the function that protoceremia itself is good.

Right? Like, if you just go and tell somebody to do this, like, you you can you just get the signal that this guy can execute. Right?

Yeah. I would say that this is is more there's so much competition just because of like the like everybody wants to get into AI.

because that's also a big thing for you. I think being healthy is important to be to do good research. Right?

And I think I think I've been probably in one of I'm probably in a peak physical health now. Yeah. You look great.

Yeah. Thanks. And I think it's also impacted my work in a good way.

I didn't go to extreme, but I was quite also quite data driven when I came I will, like, have my own email, then I will track this. I I was I was still supposed to make a blog post about this, but I I feel like I'm not, like, really at the end game yet. So, like, when I get there but, yeah, just to just for people who don't know, I think I lost 23 kilos.

This year? Actually, across one year. Yeah.

One and a half years. Yeah. So '23.

Yeah. Basically, literally from the last forecast to now. Yes.

Yeah. Yeah. There's an ablation study now.

20 kilos. Yeah. And I think, like, my HRV heart rate variability has went up by two two times, and my rising heart rate has dropped by 30 beats per minute.

30 beats per minute? That it was like eighty, ninety, and now it's one sixty. Oh, yeah.

Eighty, ninety is super high. Yeah. Yeah.

I was unhealthy. Yeah. Yeah.

Yeah. Okay.

a thing. So when you're hungry, you're just thinking of food. So I think to me, it's like these disjoint things.

But when you but it's hard to do work when you're hungry. Yeah. Okay.

Thank you so much. Yeah. Thanks.

It's really great. Yeah. Have a great time.