🔬 The Coolest Diffusion Research Isn't in LLMs — Evan Feinberg & Sergey Edunov, Genesis Molecular AI

I remember very clearly in, like, 2017, 2018 talking about GANs and how generative adversarial networks and how they're clearly the future of image generation, obviously. They didn't work very well for for proteins or protein ligand systems. And we sort of had to wait for the right primitive to get created, and that turned out to be diffusion, which turned out to be a much more useful primitive for the space.

What's kinda cool is right now for people that are are interested in really core fundamental AI research, actually, some of the most innovative diffusion research is happening in our field. It's happening in three d structure prediction right now. No one would have predicted that then, but now, like, that's a pillar of diffusion, I'd say.

Hey. I'm Evan. I'm the founder and CEO of Genesys Molecular AI.

And like Sergei, also also a physics major. I was a bit different from everyone in my family growing up. Almost everyone was a medical professional of some kind, and my sister became an accomplished TV writer, playwright, novelist, and my love was physics and computer science.

But my mental model of an adult was, well, you should help people and help patients ideally. So I was always searching for the right the right way to do that. And after, you know, arriving at Stanford doing my PhD in Vijay Pandey's lab, we were excited in the mid twenty tens of everything amazing going on in machine learning to use a data term for for images and and for language.

And around the same time that that Sergei was at fair working on a lot of big graphs, I was at Stanford working on many small graphs. Because molecules are really networks of atoms and bonds and spatial interactions. And if you're at the right place and the right time to bring to bear our backgrounds in in physics to improve AI algorithms for looking at molecules, We published a few papers in the area of graph machine learning.

And sort of like Sergei, I thought, well, I won't need this physics again because, you know, there's machine learning. But turns out the massive amounts of simulations on GPUs of proteins that we ran also came in quite quite handy as Genesys evolved. And we've been really excited in the past few years to figure out how to build foundation models for a totally new domain and make them useful for patients.

Yeah.

I totally agree, Brandon. It's and the amazing thing is or one of the one of the really remarkable things is that when we were founding the company initially as a a spin out of the research we were doing in AI at Stanford, we were afraid that we could be too late. I mean, do you remember that when you and I first met around that time, right, just seven plus years ago, as you rightly point out, almost a different generation in the area.

And at that time when we were a little seed company, there were already incumbents in the space who had raised orders of magnitude more more money than us. There were already some really exciting academic papers that were out, and we had to constantly answer the question sort of like, is there room for another company doing AI in drug discovery? Which is a crazy thing to say looking with with a backward facing light.

And, you know, hindsight's twenty twenty, I suppose. And the claim that we made then is the same that we made we're making now, which is that there's 20,000 protein coding genes, and each of them can can cause a disease. And in the same way that in other domains of AI, it's not like they were zero to one, and then suddenly that problem was solved.

The reality has been there's been no single iPhone moment. Even the iPhone required iterative development over time to become what it is today. And people love to talk retrospectively about the ChatGPT moment, but the first ChatGPT models that became widely used were vastly less useful than than they are now.

And I think we should look at drug discovery from an AI context in the same way that as we go, we should expect to solve more and more problems. And there will be large epiphany leaps. There will be iterative improvements, but they'll compound over time.

And that's why I think in the same way that ten years ago was very early, but clearly value was starting to be created for drug development then from a from an AI perspective. Fast forward ten years, our technologies, which we'll talk about in this discussion, are vastly more useful than they were then. In the next ten years, we'll see another exponential improvement as well.

In the same way that your car, in terms of autonomy, can do things today. We're we're recording this in San Francisco. You look outside and and see see Waymo's.

We'd be blown away, you know, a few years ago by by those technologies.

It's funny. It's just, I mean, breaking the fourth wall a little bit. I I came in not not really knowing any of these questions, but Brandon really is knowing how to speak my love language because because, like like, the other people in this room, you know, though my my background is much more in the the physics and CS side, I've been very, very focused in a deeply passionate way about the area of how can we make medicines with AI for over a decade.

So I've seen so so many tectonic shifts as a view. To give you one concrete example, and and I'll give a little background for those who don't have as much biological background. Drug discovery is akin to finding a key for a lock, where the lock is usually a protein, in some cases, a nucleic acid.

Right? And where the drug is the key, and it's usually some some small molecule or a peptide or or or an antibody or some other modality that wants to bind that lock, that protein, that receptor, and change something about it. If it's an enzyme, you wanna usually stop it from functioning.

Sometimes you wanna activate that endogenous protein, but it's to is to introduce an external molecule, an exogenous molecule to change something about the biological pathway. A necessary but not sufficient part of that process is finding a molecule that binds well to that receptor or that protein. Of course, that's not sufficient.

We also need to make sure it does not bind to certain anti targets, certain proteins you wanna avoid, because that often mediates toxicity. You need ADME and toxicity, which is typically, like, 30s properties to make sure your molecule is safe in addition to being effective and gets to the right tissue. These are all critical, and all are necessary and none are sufficient.

But one concrete example to answer your question about protein ligand interactions is that we had a hypothesis for a long time that if we can predict the three d structure, the three d coordinates imagine for the CS people, like a point cloud, the three d positions of your drug and the three d positions of the protein. And if you could model that with high accuracy, that will lead naturally to measuring binding affinity or or predicting potency more accurately. That was a hypothesis that fundamentally cannot be tested because models for predicting those three d poses, the three d structures of complexes were so bad.

Or if you can predict this, it requires so much computational, like, resources that you might as well just solve the problem. Right. Like, you might as well solve the three d crystal structure or cryo EM structure is what you're saying, which which can cost tens of thousands of dollars.

It can be months or years. Right. Entire PhD theses and postdocs can sometimes work literally twenty four seven trying to solve one three d structure.

And the hypothesis was that if we make this orders of magnitude faster, sometimes more accurate, which we can talk about, with artificial intelligence or before that with with molecular dynamics that you would thereby not only accelerate drug discovery, but enable the discovery of medicines for targets that were previously thought to be undruggable. And that was just a hypothesis, and it and it took until the last few years to show that if we systematically improve those the accuracy of those those predictions, if we can prove the accuracy of a protein ligand three d complex, we can make potency prediction more accurate as well. And that's something that I am personally extremely gratified to feel like we've been showing because that was just a hypothesis when we started a company a few years ago.

any, like, pure ML practitioners in the audience just because machine learning is initially constructed for pattern recognition. And it's the same with when you fire up Claude or CatchyBT or what have you. It's going to do best when it's closest to the training data, and it's going to pattern match off.

Yeah. So I think that has been the big sense of urgency in AI meeting the physical world is how to extrapolate, how to make generalizable models. And that's a we've been really hard to work on.

Maybe maybe Sergei wants to give some more more color on that. Maybe to step back a little bit and discuss what Perl is.

that is impossible to do without good models. Mhmm. It's like finding a needle in a haystack.

Yeah. Where everything except your needle is very, very dangerous or or just doesn't kind. Or they are needles at the Right.

Right. Or finding hay in a needle stack Yeah. Might be a more Oh, yeah.

Yeah. Yeah.

a lot more credible than I am talking about the Go for LLM translatability to aerospace. Sergei is being humble, but Sergei led the LAMA two research team at at at META when when he when he's still there. So he's trained one of the most widely used language models in the planet.

But I guess what I can add on the on on more of the the physical interpretability side is I like to look at at AI models in terms of the inputs, the model themselves, and the output. And we find that it really improves outcomes as well as interpretability, to your question, when you could include as many physical priors as possible without, of course, you know, biasing the model to what we puny humans believe. But I've always had the view that AI fundamentally is representation learning.

And to me, that's actually not that different than what's happened in the language and the vision domains where we're enforcing a real human prior on images when we started using convolutional neural nets. Is we're saying, well, pictures are grids of pixels. There's something inherent about that.

That's that's a human construct. And we constructed convnets on top of that. And language as sequences of tokens in one dimension, We first built RNNs.

We've then built transformers, but it's still that's baking in a pretty pretty serious prior on how to to look at that data. And we don't view it very differently here. In our case, from the input perspective, the disadvantage that we have in our field over the more traditional demands of AI is that we don't have the Internet to work with.

Like, we can't just download Reddit posts and, you know, buy some subscriptions to the Wall Street Journal and, like, train a model and voila. The pretrained model works fairly well. The closest equivalent that Sergei mentioned is the RCSB protein data bank, which has more like a couple 100,000 crystal structures, cryo structures, others.

Although there's a lot more there's a lot of tokens per structure, so there's a lot more latent information that that figure would describe. So we have to be clever on the input side, which is generating more pretraining data as as RJ's mentioning, using physics as much as possible. In terms of the model architecture itself, the idea is to not have the model have to relearn as little physics as possible, so it is less likely to overfit.

And then at the output stage, enforcing enforcing physicality, I think that also goes to one of our main focuses as a company, which is we've been focused from the beginning on small and medium sized molecule discovery. So what does that mean? So small molecules tend to be drugs that you can take as a pill.

So it's orally bioavailable. It's how most people think of medicine. And medium sized molecules.

So think about macrocycles, peptides, modalities that break the traditional rule of five, but are still growing growing modalities in medicine. That said, of course, small molecules are still 65% of FDA approved drugs. So we're talking about the biggest part of the pie.

We're building this with the judicious focus from day one on what do drug hunters, what do medicinal chemists, what do CAD scientists, what do they need to to discover drugs faster and better than they could before? And so we wanted to to build that sort of human, not interpretability, but usability from the beginning and also physical usability. We want outputs from these models to be used by physical methods that require three d coordinates to actually make sense.

Use the term force field. We want the outputs of our model to be useful as inputs to a force field. There is this a few papers that came out.

One was in Cell, I think, last year, which showed that for all of the claims about alpha fold solving drug discovery, people try to take alpha fold produced protein structures, use them for traditional docking, and found no value in it. Those pockets just weren't high resolution enough. They couldn't be used by a physical screening method.

And so we wanted to be able to build our systems that'd be useful by humans, useful by all the many adjacent and powerful tools from the physics and computational chemistry communities from day one for to have that interoperability.

As you rightly point out, if we go from from the very beginning to the very end, first, we need to identify what target is causing the disease, what signaling cascade, what's going wrong, what's causing the phenotype of the patient. And that's not a trivial exercise, but there are many, certainly not all, but many diseases where the cause is actually very clear. The most clear are the the monogenetic diseases, but even in others, it's from various panels.

It's clear that this overexpressed or overactive or deleted gene or protein is causing the disease. So we know many, but not all of the universe of causes of diseases. Then once a target is identified, we have the process of finding a drug for that target that's ideally selective, very potent.

It gets the right tissue. Some diseases are multisystem. Some are very specific to to a certain organ or tissue.

We wanna make sure that drug can get that tissue. And then there is the process of GLP tox and IND enablement, IND investigational new drug. You have GLP in this case is not good.

Not not Yes. We're we're not. Yep.

We're practices. Yeah. Thank you.

We're not talking about the metabolic space right now. There's there's clinical trials. They're usually ascending size.

Your phase one starts typically more more with safety. Than phase two and phase three, and and as you point out, ideally, approval at whatever regulatory body, the FDA or NDA or or what have you. Our contention is that the highest leverage application of artificial intelligence is the drug discovery and drug design process.

And the reasons are that even though they're they're all valuable, and I cheer on all of the peers, many of whom I know, who are working at all the different parts of the stack. But I think the first thing to make clear is just like a a vision model is going to be very different than a coding model. They might share some similarities under the hood, but requires real focus to do any one area correctly.

There's a similar, if not greater difference from the sort of models you need for target identification, I. E. Biology to drug discovery to preparing for regulatory filings to helping segment patients for clinical trials, these are all very important, complementary, but ultimately distinct problems.

I think about all of the times where a patient goes to a doctor, gets told that they have a very clear diagnosis. And in addition, the physician will tell them there's been breakthroughs where we understand why this disease happens. We've sequenced your genome even.

We we know that there are sequenced your tumor. We know it's causing your condition, but we do not have a selective therapy. There is no precision medicine for your condition, and it has the hope that still at least they know what's wrong with with me, but they don't really know how to treat it.

And our aim is to have as many moments as possible where patients are told they have a very specific condition and there's a specific treatment for them. That's going to require bending the cost curve of develop discovering developing new medicine, but also importantly, solving those cases where there's certain, quote, undruggable targets, undruggable proteins that we need to figure out how to drug, where those targets have have proven resistant to traditional methods or or even intractable in some cases.

then then the answer, it doesn't matter. Is that kinda what you're saying? Yeah.

RJ, if you because people love to debate about success rates in in clinical trials. But the reality is if one focuses on those drug candidates that are aimed at proteins that have a close genetic linkage to a disease and or at least where the biology is well understood, where the animal models translate well to disease, where the molecule is predicted to have good pharmacokinetics, as in the levels in the blood in patients are or in serum are gonna be high enough, those molecules have fairly high FDA approval rates. The success rate from phase one to end of phase three is substantially higher than average.

People love to cite the ten percent success rate, but that's really a lowball because we often know what genes, what proteins, what targets are causing disease. They're just really hard to drug or the molecules we put in the in the patients are inferior to the the the target product profile that that they deserve. In the cases where if you just look at the preclinical data, you focus on good targets with good biology that are predicted to be distributed well and have good safety profiles, those molecules are very likely to get approved and create tremendous value for patients.

And so our view is I think about the physicians that run trials that are clamoring for those kinds of molecules. I think about the patients that are looking for more selective therapies.

Known biology is orthogonal to the ease with which one can drug that target.

many of the, quote, easier targets Mhmm. For a variety of reasons have been sort of many of them have been drugged, not all. Mhmm.

But even in those cases, people love to to sort of have this this false dichotomy of easy versus hard targets and, well, this one's drugged versus it's picked over, therefore, it's not. But we at at Genesys do most of our work through large pharma. Mhmm.

Like, most of what we do is providing AI services basically to to to to major pharma companies like like Gilead, and we recently announced our our expansion of our of our collaboration with Insights, which we're very excited about. And I can't give details of specific targets that they're working on, but what I can say is there there's a real range from first in class chemical matter. What that means is we believe this target causes a disease.

There's no known molecules that bind to that target. We need to find the first binders ever, the true zero to one cases. But there's a a wide variety of targets we work on that are more, I'd say, one to ten cases where sometimes there's a there is an approved agent or sometimes there there's molecules that are only preclinical, but they're not optimal, where if you can improve upon those preclinical molecules and get them to development candidates, they're ready to get into patients, or if you can improve upon the existing clinical or approved agents, you can create a lot of value for patients.

I'll I'll give an ex a public example. We're we're not working on this target at all, but you just look at the the progression of ALK inhibitors, ALK inhibitors, and you look at, to make it very concrete, charts of patient survival. You might have said, well, when the first ALK inhibitor came out, why do we need another?

We've drugged ALK. We're done. But it is a qualitative, not just a quantitative difference when you when you compare patient survival curves from first generation ALK inhibitors to later generation ALK inhibitors of how much longer those patients live, how many more get benefit from it.

And so I think there's not only value in first in class, there's enormous value for patients in best in class too. And and I think both are areas that that we focus on, if that makes sense. Okay.

Yeah.

We're limited in what we can say about pharma Of course. Companies are are famously secretive and which makes sense. Yes.

What I can say, which was recently publicly disclosed as an example Yeah. Is we just expanded our partnership with Insights, and that started approximately, you know, a little over a year ago, the initial work together. And that spanned the two sort of bookends of the drug discovery process in some ways.

So Yeah. One of the the the programs we worked on together was a case where a very challenging target where chemical matter existed, but there's a key binary event which is getting to a DC or a development candidate. And that is the molecule or ideally a set of molecules, all which are possible to be the agent for a phase one clinical trial.

And we had to to do some work together where we we take our foundational models, our base models, we fine tune them on InCites data, and use them in in a close collaboration to to sort of work together to to get to a DC faster. And we're getting substantially closer in in that case, which is one of the reasons we're excited to to expand our work together. And on the the flip side of that is one of the other areas that worked together was on a protein with also very nice linkage to a very severe disease where there was no known chemical matters, like no patents, no papers.

In this case, no co crystal structure of a ligand, another synonym for a small molecule, that bound to that protein. So we had to find the first ever known chemical matter to bind to it, and then we progress those what are called hits. Hits are like the first molecules that are that bind you to your protein, progress those into inhibitors that are active and called biochemical assays, which are more enzyme based and cellular assays, which means that in a actual cell living cell model, your molecule is active as well.

So it was really based on on on a concrete set of accomplishments together that we want to expand the collaboration. And, unfortunately, outside of that, there's really limited that any of versions of what we can say. Yeah.

But the objective of Genesis is to create medicines that patients wish they had. And the way we'll be able to do that is by working with as many pharma and biotech partners as possible for whom their comparative advantage is discovery, clinical development, commercialization. And our comparative advantage, of course, is in AI, where where we can put those those two expertises together in in a synergistic, not just an additive way, and and make medicines together that otherwise would would not be possible.

Yeah. So that's the name of the game that we're in this for is those those clinical outcomes that you're pointing out, and I am very excited to be able to share those as as they arise in the coming years.

then the downstream predictions are also gonna be wrong. I just wanted to say for your question, I really appreciate your mid two thousands TV reference about Oh. You know, enhancing forensic images.

I just wanna say I appreciate that before we get back into get back into details here. We talked about poses.

A note for sort of the the wider AI audience that probably resonate both both with practitioners as well as users is, you know, obviously, something that's blossomed in the in the past six months is agents. And we love agents. Who doesn't?

It it's there's a lot of I'm gonna say a few necessary but not sufficient conditions, I'd say, in in my my sort of response to your really deep question there. We all remember what agents were like, let's say, mid last year. And let's just say there is positive value and there's negative value, and both can be amplified by agents.

And why is that? Well, agents are only as useful as the underlying models that they're orchestrating. And let's think about coding.

If your coding model even makes subtle but but real bugs, your agents are just going to amplify those issues. And you're gonna end up with not only slop, but something that that may be anti useful that might give the user incorrect information. We all remember what that what that age was like mid last year that made a lot of people lose patience, I would say, with claims about LLMs for for agentic engineering.

Something changed. Like, clearly, a threshold was was was was met. And even these models are still not perfect, the utility of LLMs for software engineering are so obvious now.

I mean, obviously, it's been a huge tailwind and very useful for us for for replacing a lot of the drudgery coding and getting it focused a little bit more on some of the the the more strategic issues that matter. You could draw a direct analogy from that to what we're talking about here in in that we obviously know it will be no surprise, but we're working on an agentic platform for twenty four seven drug discovery. You can imagine just fleets of hundreds of med chemists and CAD scientists working nights and weekends all the time for for your drug targets.

The the the code name for that gen is Sapphire. The prerequisite for that was we needed the underlying models for pose, three d three d complex version, potency, ADME to all be good enough for an agent using these models twenty four seven to create molecules that medicinal chemists would actually want to make and not laugh at. And if your model is sitting at 1.

8, 1.9 angstrom RMSD, that's slop most likely. And let me be really direct about it.

What you're hearing so you're hearing bimodal distribution from people of the utility of a three d pose and how, quote, real it is. The reality is that for a highly potent ligand, almost certainly, there is a large portion of that molecule with a very well defined three d position down to even half angstrom. And if you don't believe me, you can open up the electron density diagram in the PDB.

It's all available online, and you can you can literally see, in some cases, aromatic rings with missing density in the middle. So it's not just a construct on a blackboard that your organic chemistry professor told you. It is you can literally see a doughnut, a torus of electron density for an aromatic ring.

However, there's going to be solvent exposed areas oftentimes that will be less important for binding affinity, but maybe are important for solubility or or other properties of your molecule. Those sometimes are less defined because sometimes in reality, to your point, they're more dynamic. They're flopping around solvent.

But the critical piece as a as a upstream indicator that's valuable for predicting the free energy of binding is to get the core of your molecule that specifically interacts with the protein to be correct to subuncertain resolution. And why does this matter? Just to use it intuitively.

So a hydrogen bond. Everyone in the audience has probably heard about a hydrogen bond. It's the most critical forms of noncovalent interactions.

It's how nucleic acids are held together. It's how most ligands and proteins interact. It's how proteins form secondary structures.

Hydrogen bonds have a very specific angle and distance. And the distance is from the donor, the acceptor heavy atom, it's 2.7 angstroms to 3.

3 angstroms. And if I do my math right, that's a 0.6 angstrom gap.

And outside of that, it's not a hydrogen bond. If it's less than that, it's a clash. If it's more than that, the interaction is is much, much weaker per pretty quickly.

And angstrom, for those who don't know, is one tenth of a nanometer. So drug discovery really is a science of resolution. And if your accuracy is not sufficient, it will therefore not be useful for the downstream things that you care about, which is both potency prediction, but also prospective design.

What molecule do I make next? So that that's why our view is that the history of innovation in startups is that the ones that do best are ones that focus on one well defined but very important problem. And we think our ability to get higher resolution predictions stems from our judicious focus on small and medium sized molecule design rather than boiling the ocean.

Yeah. So that's the what and the why.

Have you ever heard of SWE bench? So Gemini does pretty well in SWE bench. Sometimes Gemini publishes models that win on some of those software benchmarks.

Raise your hand if you're using Gemini to write code right now instead of, you know, the obvious other name competitors? Yeah. Not me.

No one. Like, why would you do that? It's it's obviously worse in in practice.

And I I'd say actually that in this case, if you look at the provenance here of how that happened, RMSD less than two came originally from again, Brent and I have been in this space forever, from docking studies. Long before AI models for post prediction, when physics based docking studies from academic institutions, because usually large you know, the the proprietary software makers didn't wanna benchmark their methods against other methods, so academics had to get licenses and try it. And then they'd introduce our MSC lesson too, which is that's surprising.

They're academics. They're not using these things to make drugs. They're they're using it to to to write papers.

And so that got sort of the providences that that kind of got repurposed by the AI community. But the actual trend is much more in the direction that we're saying. So the first big innovation there was posebusters, we'd say, put out by a lab at University of Oxford, which which pointed out that RMSD itself is insufficient, and we need to look at physical validity as well.

So that's postbuster is what I'm talking about, not the benchmark, but the metric. So so Oxford improved that. And in the latest release from OpenBind, we just published our benchmarks at Genesys on OpenBind set.

We'll we'll talk about that in a bit. Okay. We fully plan on that.

But but if you look in in their, you know, that's their original publication, their default metrics are using RMSD, Posebuster's validity, and LDDT. So there's clearly an acknowledgment that's already been made that RMST lesson two is insufficient, and the field is now rapidly evolving to acknowledge that.

model modeling strategies? I would say there was an eval crisis in our field that is now in transition right now as our field, which was previously, let's say, lot more quiet. But if you go to NeurIPS and ICML every year, year after year, their workshops get a lot more crowded.

I will say that the last line of PyTorch I've written is much further back in history than the most recent line of PyTorch that Sergei has committed, so I I wanna make sure that he gives his his opinion as well. I wanna make sure that I'm gonna directly answer your question. I wanna make sure that I address something you said, like, a little bit of time ago that I think kinda got lost, which is you asked about how we do other things, including the ADMET prediction.

Yeah. So I just wanna make a comment about that. Just I I can spend all day, and I'm happy to diving in details about about Pearl and the evolution, and and we will.

I just kinda wanna give a shout out to the fact that it it is one important pillar, but not the only pillar that matters in the trajectory of a drug discovery campaign. And you mentioned that PotentialNet paper, which we're happy to to see have been become influential since you and I were working in the space when it was very much in the future, let's say. But now we now it is the future, so it's great.

Another paper we published around the same time was on neural networks for ADME prediction. And we published two, actually. Right.

ADME, ADMAT, can you Oh, sorry. Thank you. Yeah.

Absorption, distribution, metabolism, elimination, and toxicity are the five prongs that comprise AdMet.

over 30 or so assays, each of which you could imagine if if you're a neural net person, like a a multitask neural network or multi head, it's gotta predict over 30, you know, three properties, each of which, if it's in the wrong range, means your molecule is not a drug. It's just a tool. And so these are things like solubility, which Sergei mentioned, important for formulation as in can it be made into a pill that you can take orally, Your oral bioavailability, whether or not you're inhibiting certain enzymes called the cytochrome p four fifties and its different variants.

HERG, a critical channel, which if you inhibit it too much can cause cardiotoxicity. So these are it's an alphabet soup of things that most people here haven't heard of.

It's sparse problem. It's sparse in the public domain. I'd say there's actually a range from really directly predictable properties to ones, let's say, that, as you point out, are actually amalgams of other signaling events.

three d work on ADME prediction before anyone else was. And there was a bunch of work that ended up launching launching the company. You know, back in the day when AI could still be in peer reviewed journals, not just sort of like random white papers on on archive, We published molecule net, which has been cited a few thousands of times at some point.

And we we also published this paper showing that multitask graph neural networks were the best at a time for doing ad net prediction on large pharma datasets. And that that paper is one of the most cited papers on AI for ADME ever now. A lot of papers, as you know, it's like you write them and you kinda move on, but that work both those works have been quite influential as well.

So our history from the beginning has been to work on not just one problem, but to focus on drug discovery. And in so doing, being able to have the bandwidth to focus on building all of the ML models that are needed for drug discovery and not get distracted by the biological discovery processes that are needed, the target ID side, or or the clinical trial side, but really focus on all those tasks that are required for drug discovery. And also molecular generation.

Right? These are all important. And I just wanted to make that point at at the beginning that as you rightly point out, Brandon, PEARL is a three d structure prediction model, and many of these ADMET properties can be posed as those.

So it is therefore useful and and not just on target potency prediction as well. All of them matter, and we've we've had to to to tackle all of them. Do you use Perl for all these tests, or they're focused on some of them?

We'll sharing some things publicly in the coming period, but I think we've had we've had enough news of results lately, I think I think so far. Mhmm. If if that makes sense.

Every every time you publish something, it's actually it's it's it's a lot of work for the team to to put together. So we'll we'll do that in the coming period.

where in retrospect, it's everything seems obvious, and you can draw really linear, you know, processes of how we got from point a to to point z. But the reality is always more interesting and much much messier and much more nonlinear, I think. And for Sergei and I who've been training or taming neural nets for for over a decade, we remember a lot of different eras in the space.

And we would have been so excited if we could tell ourselves the capabilities of our technologies ten years ago. And if we told ourselves how we got there, it might have seemed somewhat obvious, but but still still there are some things that would have been really difficult to foresee. So one example is, you know, the concept of using generative AI for for the molecular space is not a new concept, but the reduction of practice would have been very difficult a decade ago.

And so as one concrete example of that, I used to co co run the Stanford AI Salon. We had fun organizing, you know, Stanford and the Gates Computer Science Building, and we'd get some interesting speakers and, like, 25 people having wine and cheese. And they're all now, like, some very, like, just straight up famous people.

Right? But back then, hey. I was a much smaller field.

And I remember very clearly in, like, 2017, 2018 talking about GANs and how generative adversarial networks and how they're clearly the future of image generation, obviously. And so there was a lot of work then also in well, can we just apply GANs to produce confirmations of proteins? Or or can we try to use them to produce protein ligand poses?

And for all the same reasons that those models were really tricky to train for images, mode collapse was the most famous sort of problem. They didn't work very well for for proteins or protein ligand systems. And we sort of had to wait for the right primitive to get created, and that turned out to be diffusion, which turns to be a much more useful primitive for the space.

And interestingly is actually a lot of image and video models, some some are using diffusion, but some of those have actually gone autoregressive. And so what's kinda cool is right now for for people that are are interested in really core fundamental AI research, actually, some of the most innovative diffusion research is happening in our field. It's happening in three d structure prediction right now.

We no one would have predicted that then. But now, like, that's that's kind of a pillar of diffusion, I'd say. That was unpredictable ten years ago, everything I just said.

And and in parallel to that, long before the advent of three d diffusion models for generative tasks in chemistry, we were building a variety of tools for the problems of drug discovery at hand, some of which were using physics based methods for predicting potency or or even for helping predict certain ME properties. And same with molecular generation. That that is, like, you know, using different techniques to generate new molecular ideas.

As Sergei pointed out, there's 10 to the 60 drug like molecules. Searching that space efficiently is hard. So we've been working on that problem.

And those things happen to be available when we wanted to to take what was then the very nascent area of cofolding, which was clearly exciting, but not at all ready for primetime, not all ready for what a medicinal chemist would want to use in their day to day work, and take it to the realm of useful and, in some cases, irreplaceable, like, clearly superior to to a non COFLD method. And it kind of just happened that we had been building those other primitives. And so they were available to us so we could put them together to build out, for example, the synthetic data pipeline or to use some of the inference time techniques.

Like, we were we were ready to do those rapidly because we had been working on orthogonal techniques in ways of approaching problems. So there was some thing here that were that was contemplative, but, like, almost any discovery, like the discovery of penicillin I'm not saying what we're doing is on same order of magnitude or benefit to humanity as as as penicillin, but there's always this element of if you're laser focused on a problem for long enough and spent enough time in the lab or, in our case, on a computer banging your head against the wall of these problems, you can create the luck, as it were, that will enable some of the developments.

That's amazing. Yeah. And diffusion's slow, so it's about the same speed as that sometimes.

Give it a week. It is true. It's like I I'm a big believer that companies are typically really good at one or two things and do best when they can can focus on it.

And in the same way, we've been just really doggedly focused on developing the best AI models for drug discovery. Like, Insight has that level of maniacal focus on optimizing drug discovery and development. And it's another sort of trend people like to be talk talking about is sort of, of course, the rise of of China in in biotech.

It's the elephant in the room that there's there's no point avoiding it. It's it's so in the zeitgeist right now. And where so so many Western companies have, you know, had become to rely more on CROs to do a lot of their wet lab work, you know, Insight became, like, so state of the art in terms of the experimental capabilities they had in house.

Their productivity is is is extremely high. And as Sergei said, it it's a it's a it's a match board in heaven for us because what we thrive on is continuous learning of the models. So we wanted to have design, make, test, analyze cycles that are as rapid as possible and continuously fine tune, or in some cases, depending retrain the models based on what we see in the lab.

And so that partnership is one of, if not the first ever, that enables that sort of true joint foundation model training on historical and and also perspective data. And as ML practitioners, we all know here that data is is just such a critical input in the whole critical ingredient. So it's it's just extremely exciting for us, and and that's a range, you know, from reflecting our our conversation so far here, you know, from structure, potency, and a variety of ad net properties as well.

So so I think immediately, we'll we'll improve the strength of our models and and be really powerful for for for generally accelerating drug discovery.

of those realized compounds in, what, a day, a week, a month, a hour? It depends. Like, some compounds are really easy to knock out.

They're really known, like, a a very well characterized reaction where the usual conditions just work. And sometimes it's, oh, this coupling actually didn't work like the literature said it would, and we have to try different conditions. So so it depends.

as Sergei mentioned, we're we're really excited about reinforcement learning because it circumvents the problems of having to do very fast design make test, analyze cycles from the lab to feedback in the model. Ideally, we can throw more and more GPUs at the problem and have the model self trained in the same way that we once did for board games in the late twenty tens or or most recently for coding. Like, that that's a key north star for us.

However, before anyone tells you that there will be a country of geniuses in a data room just solving drug discovery, there are some problems in our space that don't lend themselves to RL as as naturally and that will require experimental intervention. And so to drive in the specifics of of why, to give some examples. And I so I you know, coding was always my north star, what I wanted to do with my life, but I I spent a fair bit of time myself in the wet lab as a very mediocre experimental scientist before I found my true joy back in, you know, when when I got to graduate school, and I can just focus focus on on coding all day.

The reality is quite messy. So in order to make a new molecule, you have to synthesize it. And what that means is different chemical reagents and catalysts and temperatures and solvents coming together in the right way in the right protocol try to make your new molecule.

And once you do that, you need to purify it. If you don't do that, your positive or negative assay read could be an artifact. You need to characterize it, so you need to use NMR and other techniques like mass spec to indicate that what you made in your vial is what you actually sought to make in the first place, which is not as trivial as it sounds either.

That's not true only true for us, by the way. For anyone doing material science for AI, it's the same strive challenge where did I make what I actually made is actually a not trivial question at all. It's easier in small molecule drug discovery than it is in materials, but it's still nontrivial and takes it takes time.

We've seen so many cases where screenings of large molecules sounds great in paper because you can test millions, billions of compounds in one go, so you have more shots on goal. And and a bonus, you have data for your model. Right?

You just got millions of data points. Wow. Okay.

The reality is that the translation of high throughput screens, whether it's Dell, DNA encoder libraries, or or more traditional screens, the r squared of those predictions to, like, the actual business of resynthesizing a molecule, de novo, and doing a low throughput, high fidelity experiment is a shockingly low r squared. That false positive rate is enormous for a variety of reasons. And so that's that's one of the many reasons why people are so excited about AI making those predictions because in theory, they can be even cleaner than a lot of the wet lab work, especially the high throughput work, and yield to higher enrichment and higher true positive rates.

drug discovery frequently involves finding the outliers. Yeah. One of the many aspects that's so wild about our space is that the properties that one is seeking in a molecule very often anti correlate.

Binding tends to improve the more hydrophobic, the more greasy your compound is because protein binding pockets are greasy. Guess what? That makes your solubility worse.

Yeah. And then you wanna improve your solubility often by adding polarity to your compound. And suddenly, for those who remember high school biology, cells are surrounded by lipid bilayers.

Your molecule might not get through the cell membrane because you made your molecule two polar. So the properties you want often anti correlate, and that's where you get where multi parameter optimization of molecules ends up feeling like playing whack a mole. So you're often searching for real outliers, and that often requires making molecules that are fundamentally novel.

And the kinds of chemistries that today can be automated are fairly constrained. And so your ability to search chemical space broadly for those really top, top Pareto optimal compounds is is actually very limited.

Yes. We would love for that to happen.

I think if if one is willing to focus on one problem and say no to distractions Mhmm. It's amazing what what can happen. And I think one is that their talent density is very, very high, And they're willing to to what is not universally done now in large or medium sized pharma, which is build so many experimental capabilities in house.

Mhmm. And I actually think that it's possible more companies to try to do that given the global dynamic of pharmaceutical development that's changed so dramatically.

the cutting edge experiments goes down. It can. It's been a debate for hundreds of years about the benefits of vertical integration.

It comes to all the areas of of risk and challenges and upfront cost, but has all the benefits of controlling the process end to end. Mhmm. When it works, you can do amazing things when you do things end to end.

Yeah.

When we started that company, the origin of the company was fundamental deep learning research methods that we had developed at Stanford. Right. And the objective we were trying to solve for was how can we have the greatest impact for patients?

And at the time, you know, the the founding of the company was purely an AI research. There was no real precedent at that time for a pure model company really making it in the realm of just creating value for partners in pharma or really most legacy industries. And the other, I think, component was the company being founded on AI research.

I think we really wanted to show people that we were serious about the biotech domain. So, you know, I think as part of the origin of, let's call it, the Genesis Therapeutics Yeah. Will seem a lot more serious to to to the to the, you know, all the the tried and true folks in pharma, the diet in the wool, med chemists, drug developers who we who we really wanted to work with.

So I think there is an element of an attempt at nominative determinism there a bit, but but also an acknowledgment that I don't think anyone had really shown at that time that you could be a pure AI model company Yeah. And really make it in in the space. And that was obviously, this is 2019.

So we're we had the benefit of being real forebears pioneers in the space. We had the downside of being pretty early. Yeah.

Thought to be late, but turns out we're still early innings. As the company grew and and evolved, we really restructured kind of like a double helix in that we had extremely strong focused AI research, and we were fortunate to be able to recruit some of the most experienced and accomplished drug hunters, like people on our management team and board who have discovered, co invented, developed many FDA approved drugs. You talk to most med chemists.

Most med chemists feel very fortunate if they're able to work on one molecule that gets FDA approval in their whole career. And we're really fortunate to convince these sort of accomplished drug hunters to come join our management team and and board at the beginning and and steer the show. So the outcome has always been the same.

Everything we're talking about, it only matters if we end up helping patients and their families at the end of the day. And we have felt from the beginning that the the way to do that at scale is to primarily partner with large, medium sized pharma companies and biotech companies so they can do what they're best at, which is novel biology, clinical development. We can do what we're best at, which is AI, and bring those two together.

In addition to that, as Sergei has said, there's numerous advantages to having in house wet lab. Data generation, clearly. Dog feeding the models.

Getting really direct human feedback from real med chemists, what we need to build in an extremely candid way. And, of course, the third thing is that the single most valuable thing in the universe, what I like to joke, is the the mirror of error said tests. If you ask people what is what is the one thing you most want?

And you really thought about it, most people would say a medicine for themselves or a family member. And so there is just immense intrinsic value to having one's own pipeline of programs. And when when we interact with pharmaceutical partners, on average, they love that we're working on molecules ourselves because it means that we're not just a bunch of keyboard jockeys, but we live the dream that we are pitching.

And we need to use these models and practice for ourselves too.

because of that shift? Like, what's going on? To finish that that that narrative arc there, the the name change was just to reflect what we were really doing in practice, which is we are an AI company as we have been from day negative one Yeah.

When we were, you know, coding in PyTorch point 17, building, you know, the first you know, when Sergei was scaling transformers on PyTorch, the first one to do that, and we were scaling graph neural nets in PyTorch to one of the first ones to do that. And so it's reflecting what what our identity has been from day negative one, but also alluding to the fact that we are very serious molecule makers ourselves, and and we are full stack in in that way. We are also to be clear, our mission is to create as many medicines as possible as would otherwise not have been created.

And I think the way to get there is to put our technology directly into the hands of as many drug developers as possible, not just our own, but in big pharma, mid cap pharma, and biotech. And that's partially why we've been building our our agents so that our models, which are have become so much more powerful, can be easily used and orchestrated in the hands of med chemists, CAD scientists who might have never ever in a line of code in their life.

human being or really be any human because, like, the long the long arc of history starting from when we decided that hunting and gathering, I don't know about that. I think I'm gonna settle down and and and do this whole agriculture thing. I think that started thousands of years of drudgery.

Sometimes you wonder why did we actually do agriculture because everything I've seen seems like agriculture was probably a net loss on the short term. It was the only benefit I see in the long run is that if you were a hunter gatherer and you got sick or born with a genetic illness or got injured, that was like, you're in trouble. There's not much more we could do for you.

The medicine just didn't really exist. But ultimately, I think the crowning achievement in my mind to civilization is where the sick people among us in society are not seen as burdens, but people who we wanna help. And that isn't true in the history of the animal kingdom, including humans.

It's a sad reality, but it's true. Whereas now, because of medicine, we we see people who are sick as people who we want to help, who we can help. And we can help them, they could become even more productive members of society if that's the thing that you care about.

If to fast forward that to now, I think that so much of creative work, day to day work, if you look at as a pie chart, time is our most precious resource. So much of it was being taken up by repetitive tasks that actually didn't require much thought or creativity. And now it's like we've created, like, this additional headspace for deep thinking.

And for us, you're solving some of the most technically challenging but but important problems, we can use these tools to become wildly more productive and creative. And that goes to the med chemists, drug hunters, cat scientists who are our direct users. They can be grand strategists for a drug discovery campaign.

And when you have hundreds of drug discovery scientists working twenty four seven on large data centers, just the output you get in terms of discovery is just gonna be greater, especially if it's used in the hands of expert scientists who know what they're doing.

There is induced fit throughout the training set. Yeah. Yeah.

This is probably inherited from the the the physics based simulations that you're using. It certainly helps. I think the other component that's that Sergey alluded to was we haven't just been benchmark maxing on the same we obviously publish in runs and poses.

We publish the state of the art numbers and runs and poses last year When we published the Perl technical report with NVIDIA at GTC at the end of last year, we used runs and poses. We used posebusters. Perl does the best on those benchmarks.

We did that. However, as Sari is leading to, the gap from Pearl to next best model is even larger on the partner drug targets that we work on. And what that's in part reflection of is the objective of our company is not maxing benchmarks.

The objective of the company is creating value for biotechs and pharma companies that are usually working on hard drug targets that are different from what's in the PDB. And so we've had to build the models in a way that are able to extrapolate. And the open buying challenge was just the latest example of that.

Before we wrap up, we always ask two questions.

because of the shortage. NVIDIA has been a great supporter of us. They've invested twice in Genesys, and we collaborate closely, like, you know, not just in terms of the capital, but we optimize kernels together.

You know, we co Pearl was coauthored by Genesys and NVIDIA. They publicized the OpenBind results with us. We've been very grateful for that.

They're a very, very, very big company, and, you know, we're we're not the scale of a of customer like a meta or something. So we're grateful that they've given us time and attention nonetheless. But I think part of that is a self serving aspect here, which is that, obviously, like, no one can time time the market.

But the amount of hype investment in the traditional LLM space for chatbots, coding sort of thing, at some point, the amount of alpha will not be what's desired compared to the amount of investment. And I think regardless of the vicissitudes of the economy, medicines will always be high in demand.

Yep. We think those are all great tailwinds for us. And then what what about you?

What what's the bottleneck besides GPUs? Oh, I had the same answer as Sergei. For for sure.

If I could wave a magic wand, I'd love to have just a enormous h 100, h 200 cluster.

sort of unique both in their subdomain and more broadly. We saw as a recent guest, you had one of the the the progenitors of ESM fold. Yeah.

Alex Reeves. Yeah. Yeah.

They cite our model architecture paper that we published in open sourced a few months ago. So the work that you do here at Genesys as an AI researcher, there's lots of opportunities for publication, open source, and influencing the field beyond the obvious ways to influence the field by working with top customers that are actually making medicines that change people's lives. So it's both incredibly intellectually engaging and a lot of opportunities to influence the field and and get your name out there.

But if you want, you could also just stay being a cog in the machine of an LLM company and No shade. No shade. That's it.

Yeah.

Yeah. We we love the show, so we really appreciate taking time to to to chat. Yeah.

Thank you for having us.