This episode recaps Vercel's ShipAI event, focusing on their new workflow development kit, the evolution of their AI SDK, and various internal and external agent initiatives. Malte Ubl, Vercel's CTO, discusses the company's 'dogfooding' philosophy, the strategic decision to maintain low-level abstractions in AI tools, and their increasing investment in Python support. The conversation also delves into the practical applications of AI agents for tasks like SRE, lead qualification, and abuse analysis, alongside Vercel's 'Agent on Every Desk' program and future security models for AI-powered development.
Alright. We are here in the remote studio. Thanks again to f dot ing for Nanyas' space with Malte Ubou, who's CTO of Vercel.
Welcome. Hey. How's it going?
Glad to be here. Did I get it right, Ubel? I've actually never pronounced it out loud until, like, just now.
Yeah. That was completely perfect. It rhymes with Google.
Ah, okay.
which, like, I I think still people don't know enough about Wiz. It is, like, no longer a secret, but yeah. You can't use it.
So, like, unless you work at Google, in which case, you probably know what it is.
there's no reason to really know. Anyway, suffice to say that you are responsible for a lot of the web as it is today. So thank you for spending some time with us.
You're also obviously now, building the next web, as as we say, with, Vercel. And we we we can cover framework defined infrastructure. I I think you you probably saw I have a I have a lot of interest in self provisioning runtimes.
We can cover v zero.
which, we're trying to sort of recap, right, for the general, Lanespace audience who may not be watching Vercel as closely as I do or or you do. So, like so bay basically, just generally, what I guess is your message to the broader AI engineer audience on what Vercel is doing with AI? Yeah.
I think the the the super high level view is that what we're really trying to do is, like, we we're, like, the biggest fan of the AI engineering movement, and we are also fans of you know, we're not not just, like, going super hard on on hype and the big ideas and and talking about things, but, like, being very concrete about, like, you know, agents are very exciting, and you can actually build them. Right? And so, like, I think our entire conference was about both making that easier, right, and and discovering the right abstractions as we're kind of figuring out what the what people actually wanna do, right, which is emerging as we speak, then the way Vercel always does these things is by building things ourselves.
Right? And so that is both in terms of products. So agents that are products that you can purchase from Vercel and stuff that we do basically in our back office to make our own operations more efficient.
And so this kind of building of apps lets us ground kind of what we do in in that reality, and then and then, you know, like, kind of extract the abstractions that we feel are really helpful to to then put that on the road. And I think the the probably most talked about thing that we shipped at the at the conference was our new workflow development kit, which really, really is just a way to make writing, like, workflows, like, very idiomatic. It's something that just becomes kind of first class.
It's something you do every day. You think about it. You don't write 15 design docs just because you wanted one of them.
It's just something you do literally every day. I think, like, since you your audience also more generally, like, in the think probably, like, listening to what people talk about, I think that's when I talk about our work development kit, but also, like, more generally, like, what is what are workflows? What are agents?
How are they related? Do you use one or the other? I would actually love to talk about that as well.
and so forth. Yeah. I mean, as as listeners might know, I have a long history of workflows at at Temporal.
And I think what's weird is a lot of people are discovering this for the first time. You don't really learn about this in CS classes. You don't really learn about this in, like, boot camps or anything like that because it's not really a unit of compute and storage that is taught.
you know, Uber and, like, Stripe and and everyone else. I don't know if there's a version of this at Google. I mean, there is.
There's a version of this at every single company that has been doing anything in computer since 1950. Yeah. Like but what's not necessarily the case is that it has been abstracted in any way.
Right? But, like, when I when I run a, you know, bank transaction system in 1975, then I invent this. Right?
Maybe I'm in pure batch processing world, and I kind of avoided it. But but the reality is that I built this. Right?
And so either I use something very productized, which which obviously temporal innovated on that being a thing, or I use something that's ad hoc. Right? So I go and say, well, I need I need some kind of queue that tracks the work.
And I mean, I need a database to store the state at any given point. And I I don't know. Maybe I write a con job that makes sure that the stuff on the queue doesn't get stuck.
Right? And so I like, I think what's extremely common in essentially every transaction processing system that has ever created is that there either is an explicit abstraction for workflows in it or someone made one ad hoc because otherwise, the thing just doesn't work. Yeah.
Yeah. Totally. So the headline thing for people who maybe haven't dived into workflows enough is that you can sort of wait and and resume code.
long running. Like, literally, you can run an infinite loop inside of your serverless code, and that breaks a lot of people mental model if they don't really understand that the code pauses and resumes. You can wait multiple days, and it doesn't matter.
It doesn't cost anything. Actually, don't know if it doesn't cost anything. Maybe I don't know if you charge No.
It it literally does not cost anything. So, yeah, you can you can run compute for infinite amount of time. You can then also whenever, like, one of these steps fails, automatically retry them, stuff like that that makes things more reliable.
Yeah. And so, like, it it has a lot of parallels to long running orchestration problems for agents. If you want to do human in the loop as well, it's a simple task of waiting for what's this API that you guys have?
It's not I wanna say signals, but you you have something like resolve webhook or something. Yeah.
to a single, but, like, the idea is that you basically make a webhook, like an ephemeral one, which is just a URL that you can ping. And so the the realistic flow would be, you reach that step where you want human approval. You know, let's say that you you write you get the webhook URL, and you write it to some database.
And let's say the user now, they log into their computer two hours later, they have a queue of things they need to approve. That's from the database. They click on one.
They say approve. Now what just happens is that that system now calls that webhook.
you were now able to await that webhook, and now it resolves, and you could just proceed with the program. Yeah. It's it's very elegant.
I would say that it eliminates some complexity that we introduced at Temporal, and then that's probably for the better.
And the other thing I I think just obviously as someone who is, like, in this space a lot, and I I've seen all the solutions. You you made you made it open source, which is another above and beyond thing. You could have made it proprietary, but you didn't.
Yeah. I think the the way we think about Vercel, and I think we I mean, I don't wanna go too deep into that tension, but I think about open source as having essentially three business model. K.
The first one is Red Hat, where you just sell support. You think it's open source. The second one is open core, where you are the only one that gets to monetize it, but everyone else gets to run it if they want.
Right? And and Vercel maybe has not invented this, but certainly kind of is the most successful edit model where you say, okay. I have this software library, and it's truly open source.
Everyone can run it. It comes with, like, adapters for every place on the planet, and that makes it really popular. And then we get a piece of the pie.
And so our strategy is to grow the pie while what we actually see is that the our our pie piece is relatively constant size in proportion to the pie. Right? And so we can drive the open source project.
And so that's why, like, you know, I don't wanna say we're, like, you know, we're we're in it for the for the business model, but, like, I think it's a business model that has more winners than the the alternatives.
and I ran into this in temporal. Like, these are gonna be extremely valuable workloads that you're gonna put on workflows. And so you wanna you wanna you wanna set you want some ownership, you want some auditability.
In practice, like, who's gonna actually run it themselves? Probably not. But you want the option.
You want to check you wanna check the check 100%. Yeah. And I I actually do think people will run it themselves, and that's great.
Awesome. So there that's workflows. By the way, I think the most disgusting is the use of directives.
You know? Use cache. Use no memo.
Use Nemo. Use whatever. So fun.
So fun. What's your I don't know if you have a take on directives in general. I don't I I to be honest, I don't feel super strongly.
I do find in particular, the inside of the workflow dev kit, I find the use pretty elegant. The I could imagine Yeah. I could imagine other other ways of doing it.
I think we we did post a blog post about, like, all the alternatives we considered. Because there are some that are, like that you think about after five minutes, and after two hours of thinking about it, you realize, yeah, maybe this isn't such a good way to do it. But there could be other ways of doing it.
Like, I think we're working with season 39 to bring decorators into more places, which would kind of make this literally the same. The same thing would happen with Above the function instead of below the function. So it's not a big difference, but it would become you know, for example, you you could make it part like, TypeScript be aware of it without a TypeScript plugin, which we already provide.
Right? And so I literally was vibe vibe coding this thing on Sunday, and I said, okay, Claude, you have no idea what news workflow is because it came out on Thursday, but here's the docs. And by the way, you know, put it on my side.
And So I did it. Installed the TypeScript plugin for me, so I had like the perfect EX, and it just worked from scratch.
It was it was great. Okay. Awesome.
So we can come back to for anytime you want. But I just wanted to keep moving on on, like, all the stuff you had announced. We should probably also just touch on AI SDK.
I know you're not, like, as closely involved to that team, but obviously, one of the most successful open source projects.
except that you guys have the perfect package name. So, you know, I think that that helps a lot. I actually don't I'm not sure how well how much that helps, but it's great.
Like, there's a there's a fun background from what people thought AI was for ten years ago. But, yeah, I think, you know, we we announced version six beta, and I think the the big I mean, it's not really news because, you know, these things are open source and you can follow them very closely. Right?
But, what it does introduce as a stable feature because it's already as kind of experimental in in a s t k five is a direct agent abstraction, which so far wasn't there. Right? People would build agents with AI SDK, but they would have to do it in a bit more Beobonds fashion.
What I do wanna mention is actually because you mentioned it's very successful, which is true. And I think the the reason why it's successful is because we could strain ourselves to be humble about what we know our users might wanna do. The example I would like I like to give, like, when you build, like, a new web framework in 2025, you know exactly what people are gonna do.
Like, the the the it's such a well, explored space. Who as the person doing it probably has done it three, four, five times in their life and failed and and learned from that and and tried the other things. Right?
There's it's so mature. Like, it's the most mature thing. Even ten years ago, that was also true.
Right? That's why Next. Js is so good.
Because when when Guillermo started building it, he knew exactly what to do. He knew exactly what the app would be. Almost nothing has changed.
So the AI app space is the absolute opposite. Like, we know absolutely nothing, and we still know absolutely nothing. Things are emerging, but like but it's but we're so early.
And so if you put a very thick abstraction, then it's probably gonna be the wrong abstraction. You have to so you have to be humble and say, okay. I need to stay low level so that this can be flexibly used as trends emerge.
Right? And so that's why we didn't have to rewrite AI SDK when everyone went from writing chatbots to writing agents because the we stayed at a level where that, you know, almost looked the same. Right?
Composite stuff. Stuff people wouldn't on top. And I think that's why that's why it's successful because it doesn't you know, we didn't say, okay.
We know what the apps are gonna look like, we're gonna do this, like, Hollywood principle. Don't call us. We call you style framework.
We're, like, you you just have to fill in the blanks. It's super structured. You'll be happy.
Right? Like, we didn't do that even though that was, like, so in our DNA. Like, we did have to, like, really restrain ourselves.
But, like, that's why it's successful because it's so low level. Right? And so that's why, you know, on the other hand, know, that's why we don't have an agent abstraction yet.
Like, every other competing library, like, leads with that. Right? Yeah.
Like, OpenAI's SDK day one was like that. Master, etcetera. And, you know, I mean, I'm not saying it's bad, but and obviously, that's that's more accessible.
Right now, you know, you have to understand. Like, an agent's tool for the loop. What do I do?
I use the stream text function and gives it give it tools. Okay. Add it we added all kinds of, like, control already in the SDK version five, where you can prepare the step.
You can select the tools on every loop. Like, you can you can, like, do all these things in a pretty advanced fashion. Half of those other frameworks are built on top of SDK anyway.
Right? Like, so that it's it forms the the basis. And so what we're doing now is we're bringing what is emerging as the patterns that people built over and over again into as abstraction into the into the library as these as the users just kind of are solidifying.
Yeah. One I have interviewed enough agent framework builders and model and BigLab model people that I actually find that I can push back on you. Okay.
Go ahead. It's it's it's really interesting because I think you are saying basically, you're saying we will be at the jQuery where they kind of at the jQuery era. Right?
Like, we are where we don't know what we want yet. We're building all these, like, tools to make our the smallest possible things easier, and then it composes up, and we're just starting to emerge of agents. I would say that the BigLab people are obviously the opposite, but they're not they're coming at it not from, like, a DX point of view.
They are they're very big model pill. They they want everything to go through the model. The reason they want the Hollywood principle of we'll call you is because they want the model to control the tool calls, the reasoning, what have you.
And I feel like there is a there's a mentality. Obviously, you can have frameworks that do both, but there's a mentality in the BigLabs, you work to BigLabs, that you always want to give the wheel to the model. And then for you guys as framework developers and people who are software builders, it's more comfortable to build, like, the the the smallest possible thing instead of, like, the the sort of AGI thing, if that makes sense.
Yeah. I actually don't think about it in in in those dimensions. I think the like, because I 100% agree that, like, people, I think, have to be willing to let go and let the tool oh, sorry.
Let the model kinda take control, right, and to to get emergent behavior. And certainly on coding agents, that works incredibly well. But that that that I'm I'm totally on on on on point with.
Right? And that that's yeah. SDK does this very well today.
Like, all the agents that I've personally built work like this. That's not the same it's but the other thing is, like, how do I now embed this this into an application?
Right? Like, the model apps couldn't care less because they're not really building applications. And so that's that's something that a company like Vercel thinks about a lot.
Like, again, what what do what does the developer actually want to express, and and how do we let them do it? And so, like, I think one of the key things that that people wanted and so want is they want streaming because these models are slow. Mhmm.
And so suddenly, this almost obscure subgenre of programming where people are like, you know, it's two 500. I'm just gonna not ship it. Right?
Suddenly, it's thirty seconds and you and it becomes absolutely important. And so we give people the tools to get to build streaming applications in a way that feels intuitive. Right?
And I think that unlocked a lot of, like, value there because that was, like, genuinely hard, and and we made it easy. And so that those are kind of the things that we're looking for, but that, you know, are not obvious when you're kind of mostly concerned about the AI part. Fair enough.
And and I think, like, the design space is has more dimensions than what I tried to simplify it out to. Yeah. Just oh, just one more thing on AI SDK, and then we can move on to, like, the other agent stuff.
And so see you guys pronounce so much. It's so hard to cover. So Vercel is a, like, a house of frameworks.
Right?
all of them legends in their own right. What's one, I guess, philosophy that you're also applying from your all your years and all your people who work on frameworks, right, that that that is informing you? I have one, and feel free to counter propose, which is what Sebastian Mark Boger, who's obviously the tech leader of React, is used to say, which is have a small API surface area.
I feel like that has maybe been, like, not as important or, like, there are other overwhelming priorities. But I just wanna get a sense of, like, you know, what governing principles really resonate with you. Yeah.
I I mean, Seb and I are talking about this a lot.
side where it's like, no, but I actually wanna just control this. Give me one API for this. Just one more, bro.
I I I wanna be, like, you know, in control, and I wanna be able to configure it, and I wanna, like, define the the control the defaults the way I see it. So I think I mean, the but it's good to have, like, tension around these things. I think the the thing that that, like, coming down from Guillermo is just the absolute founding principle of ourself is that we never give you an abstraction that we haven't used ourselves.
Like, dog footing is ultimately the thing. Right? Like, that like, AI SDK was extracted from v zero.
And then we build it, and we kind of diverged a little bit, and then we took on the the the substantial work to bring back v zero actually fully hosted on AI SDK. And then we learned back, and we made sure that migration isn't too hard, which the users appreciate as well, and so and so forth. Right?
And so there's this this this constant feedback loop where if you don't have that which is like this sounds like so obvious. Right? But the the reality is that framework builders are usually not application builders.
Yep. And so they build ivory towers that when they're hyper geniuses or they get lucky, they're happen to be good. But, like but but if you want to do this in a reproducible fashion with a high hit rate, then the only thing you can do is you you have to you have to try that stuff out yourself.
And, like, that's what we do every day. I I I really like that principle. Obviously, a good idea.
spend some time working on framework level issues. Happy to move on to Vercel agent and maybe the agent in every desk program, which, you know, I think you're you're kind of also championing. So, like, yeah, let's talk about the use cases.
Like, you that you guys use internal agents within Vercel, and what emerged? Yeah. Let's let's structure this two ways, because I do think there's a difference between kind of the agents that we're building internally versus the stuff that that's, you know External.
A Vercel product. Right? Yeah.
And which you can we usually can use today. We we've been I thought they were the same thing.
No. They're not the same thing. That's actually it's I think that's quite quite important.
Like, we also we're distinguishing between, like, agent as a service. Right? And so the the Vercel agent, that's what it is.
Right? It's ultimately an agent as a service product similar to, you know, Codex in the cloud or the the Cursor agent. Like, not not not as in, like, it's the same product.
Right? But as in, like, these are things you where you go somewhere and you say, I would like to use this agent, and then, I don't know, maybe you give them a credit card and it works. Right?
Which is different from the stuff that we run internally. But let's talk about the Vercel agent for a second. Like, we we've been basically, I think our strategy overall is to have an agent that helps you build applications on Vercel.
This is, you know, there is some overlap with coding agents, but like, I think the the thing that's unique about about the Vercel situation is that because we have we have your runtime data, we have you see your error logs, we know where the preview deployments are, that they always will exist. We know how to start the dev server. We already have the secrets, so there can be, like, a quite quite integrated solution for something that otherwise can be quite hard.
Right? If you ever onboarded, for example, I mean, you have onboarded Devon a few times, but you probably have, you know, been in that situation where it's like feels like onboarding a junior employee. Right?
And so some of these things, like, if you're within the resell ecosystem, become much more simple. And so in that world, we we are we've, you know, we've we've been chipping away on different things. Right?
Like, a while ago, shipped a code review agent, which I think is is is really good and well integrated. And the thing that we announced last week is our broadly DevOps agent, which is actually tied to our anomaly detection system. So whenever there's anomaly that we detect on your on your production side, it kicks off the agent, and the agent does an investigation of what's going on.
From a technical point of view, what this agent has is has several tools. It can it can make any observability query against your project. It says a query builder.
It can execute the queries. It has a way to read logs, obviously, with another with with queries as well. So what's what's really magical is that it's it's just very good at this.
Like, like, by the time you click on the anomaly, it will, like, almost all the time, just very precisely tell you what happened. It shows you all the graphs that I looked at. It just, like, I don't know, man.
It's just so much easier than doing it yourself. Like, it it takes away certainly minutes of work. But I think what I'm actually very excited about, and I think this is an overall pattern that that that we see with with agents, is that there in many situations is this what we in search call recall precision problem.
And that also happened with anomaly detection. With anomaly detection, you you have to tune it. Right?
And you either tune it to be very aggressive, and then it fires and you and the pay worst case, it pages you in the middle of the night and nothing was wrong. Right? Just the team in Asia sent a newsletter, and and so the traffic went up.
Right? Or you tune it not, like, aggressive enough, and so you miss events. And with an agent, you can just say, okay.
I'm actually gonna have this tuned very aggressively, and I'm not waking anyone up. I'm I'm telling the agent, and the agent can take two minutes to run. It I'm actually fine with that because no one would have reacted in that amount of time in a very reliable fashion.
And now it can actually look at a time series, can look what happened, it can look at the IP addresses that that are making the request, it can look at the type of error messages. Right? And it can make it, like, a call whether to escalate to on call and wake someone up or to say, okay.
This is completely fine for someone maybe to take a look next day, which is I think the perfect decision for agents to make. And so that's that's something I'm very excited about, that you have this, like, coworker that has no sleeping problems in the loop, and and they get woken up instead of you. Yeah.
has been a long time coming. And I'm actually on the record at the start of this year. I made a podcast saying, like, oh, I don't think anyone's gonna do AISRE.
So I'm very excited. I haven't tried it out personally. I I have seen you tweet about it, and I I think, yes, obviously, that is the goal.
That is the dream. We should not we should have make Brian Johnson happy and have good sleep. But, you know, we're we're we're not we're not exactly there yet.
And I think, like, the the question is really fold, which is time series analysis is not exactly within the distribution for language models. There have been a lot of people doing, like, time series models. There's a lot there there's a deep field of anomaly detection, which is basically what you're doing.
And, you know, there's a question about, like, is this a solved problem or, you know, like, how much can we trust it? And then I think, you know, the other one is, like, aligning the the human preferences. Right?
Like, sometimes, like, I don't know until I've seen a few examples of, like, oh, yeah. This one, you should wake me up. The other one, you should not.
And then pretty much, like, when I solve the problem, you know, it goes away. So, like, the next problem, you're always fighting the last war in in in SRE. Yeah.
So I I give you I give you a bunch of things you can take over. Yeah. I think that the the the like, you have to try the product.
It works it works really well. The the so we don't do the anomaly detection in the LLM. Right?
The anomaly detection is as, you know It's a separate part of the for self product. Separate part is our it's a pipeline that works in our time series database.
Right? And and launched independently of this. And so but once you have this, the so our experience is that, you know, if you give these agent a tool that does queries, it's really good actually at at digging into individual parts of the time series.
And the other thing it has access to is logs. And logs are actually they're just text. Right?
So if you say if you see from the time series, like, what happened? Can I somehow figure out, like, what this is? And then you head over to to logs and get do a deep deeper dive.
Now you're kind of more in the in the world where the where the model is comfortable. One thing that we don't do today, but we'll do in the future, is that we also give the model x that particular agent access to your source code so that it can, first of all, figure out, like, what does the error message mean. Right?
And every time, you can actually make a PR to just fix it. But every so often, that'll be possible. And so it would would then also do that.
I think, like, people like Datadog and Sentry are trying to do that obviously because they're observability platforms,
but they never own the code. And so, like, they're they're always limited in what they can do. You know?
Like, and 100%. But I also, like, I I mean, mean, I what I wanna qualify, where where I'm so happy about how how it works is, like, you know, you know, it's just a small part of the overall problem. Right?
Like, we're actually not here to, like, build the AI SRE that replaces that job function. Like, that's another part of what I feel pretty passionate about, that it's at this moment, they're like, agents are both extraordinarily effective and still very ineffective. And you have to find you have to find the right problems.
And then when you find the right problems, they are super magical. And then if you wander beyond, then they don't work. And so, like, that's the magic.
Right? And what we see is that getting triggered on, like, an increased error rate works well. And certainly, making the decision what to change in the firewall, I would not let the agent do that yet.
Right? Like, it it'll be it's too just too dangerous. Or do DNS migrations.
Exactly. Exactly.
That's how they'll be That'll be AGI. Yeah. It's it's interesting on on all that stuff.
Yeah. I mean, I think that there's this there's this growing consensus of, like, where agents are doing well and where agents are not. I think, you know, for me, like, meeting notes are solved, you know, like, simple, like, UI changes are solved.
I guess, like, what what else, you know, in that list of, like, things that are solved that and, like, are reliable every day, what what do you put in that bucket? Yeah. I think the the what so I had a section in my keynote about this, and, like, it boils down to this question.
Like, basically, where the idea is that you go around your company and you'd ask people, like, what do you hate most about your job?
but they're you know, they would have already been automated if they were automatable without an agent in many cases. They often, like, do require some kind of text, like, mini judgment, etcetera. Right?
So, like, people people do these things, and so that that's I think that that that question that thus yields a sweet spot where, like, these the the problems are probably easy enough for for a current generation agent to handle, and they're also often very high business impact because this is actually pretty substantial part of people's jobs. Again, that's why they hate it because it's like takes so long. And so we we ended up at our conference talking about three agents that we built internally, two of which we open sourced.
Again, so people have a starting point because these are custom agents. Like, they're not they're they're not software as a service things that you just install. So these are custom agents.
But, like, the the first one is one that handles processing of our incoming contact sales requests. Lead qualification. Yep.
And that we there are obviously a lot of startups in that space. Right? Like so I think that's that's very much in the in the in, like, the soft case where, like, you give it a tool for LinkedIn, you give it a more generic tool for Google, give it a bit of an objective and, like, de qualify, like, how what do you what do you care about?
Right? You give it a, like, a way to analyze, oh, this is really a support request. Okay.
Hand it over to the support team. Right? Like, there's there's a few cases like that.
It's not so complicated. So that one is, I think, is is perfect. And we open sourced that so people can make their own.
The other one that we falls into a similar category is abuse analysis. So we get abuser, like, And so in this case, it's really the agent essentially doing the prework. So we still have a human person, like, look at the prework and then make the decision what's to hap what happens in the end.
But what were they gonna do? They were gonna go to the reported website. Right?
They were gonna go look at the account and figure out how what the age of the account is, and they were gonna see if they paid their bills and, you know, whatever. Right? Like, there there's a stuff there's a list of things they will do.
And so what you can do is you can just make it so that when they eventually look at the ticket, it already has all this information. And if the page looked like a Facebook login page, then a current day LM is also able to to do that, like, that adjustment call, and then you just quickly check if it's okay, and and and you move forward. Yeah.
Amazing. And then I think there's one more data analyst agent. Yeah.
Yeah. Exactly. I mean, this is also something we we just wanted for ourselves.
Right? Is to do We have one have one to do internally. Yeah.
Yeah. Like and I think it's it's that one that one's also open source. I think the the the idea is that, yeah, you wanna you wanna ask questions against your data warehouse.
And we were very unsatisfied with the current solutions because they ultimately didn't have access to enough information about the data model. And, like, we are not promising that we have the magical tool that you give you give you your your prompt and it spits out SQL, just access your your schema, but we get we essentially develop just a structured way to document the semantics of your data so that then the agent is good enough. Right?
And so we've been using that internally quite successfully. Amazing. So, yeah, the for for those who don't know, Riselle also has a agent on every desk program where you can just sort of reach out and it's is it like a forward deployed engineering situation where, like, you have, like, a SWAT team that comes in and helps people?
Yes. I think that's but it's also not appropriate for every company. Right?
So I think the like, my take is that it's if I'm a large company, I have a lot of efficiencies again, but it's also quite daunting to to ship my first agent. And so something like a forward deployed engineers, which we are doing indeed, like, does help quite a bit in that scenario. I think, like, as a startup, I don't want a Fort Worth deployed engineer in my office.
I just wanna see the open source project and and feed it to Cloud Code and then give it my own problem and say, build me something like that, but, like, here's what I want different, and and that should also be successful. Right? So I think we we are kind of with this particular program, really going for just unblocking people who feel that they just don't know what to do.
They they hear the hype. Right? They don't know how to pick the right project.
We talked about this. Like, how do you actually find the project that's gonna be both successful and high impact? And and then secondarily, okay, now that I have the project identified, how do I how do I do it?
And I think that their, like, forward deployment is effective. It's something that I you know, as a framework engineer, I've thought about all my life, where, like, you need to have someone kind of guide you the first time you do something, and then the second time, maybe, you build an agent yourself. So we we don't wanna stay there.
Right? We want we basically sign contracts with companies saying, okay, you have to commit to building three agents. And if you do, we're gonna help you like, we wanna build the first one for you.
And then the second one, we are gonna be there essentially by your side, you know, maybe not literally, but like on on a phone rotation. Right?
and you can still reach out obviously. But but if everything went well, now this is a company that's empowered to build its own custom agents. Yeah.
You you know, you you're gonna be helping your biggest customers, and, obviously, that's gonna be leading to a lot of good products ideas. Right?
with your the people in the Vercel ecosystem and not just Vercel alone. 100%. You just discover things that probably the 500 person startup would not have discovered.
I think, you know, one one last thing, you know, just to to leave off the the whole topic, and we can we can add in anything on the on the ShipAI side. Actually, anything else on on ShipAI that you really wanna cover and get us a soapbox about? I don't know if we covered everything.
I think, like, one one point that we we we haven't talked about is that we like, Vercel as a company just has been investing in Python quite a while.
I think maybe the audience of this podcast might be also be excited about this. AI SDK, obviously, currently is is a pure play TypeScript system. We we do find Python really interesting.
What we have done over the last weeks is we have shipped, like, zero config support on ourself or all the popular Python frame framework like Flask and and FastAPI. Zero config means that you kinda get the Vercel experience where you throw your stuff over the fence, and and we're gonna run it for you. No questions asked.
And then just as another thing, like, we have, for example, also shipped a Python SDK for our API just to kind of, again, show that we are engaging with the with that ecosystem. It's very obviously something that that is in a way new for Vercel, but, like, we have been making hires and and infrastructure investments to to to make Python really well supported on the cell. It is also on our fluid compute program, which gets you, for example, active CPU pricing.
So you get to run Python in production, and you only pay when you have compute, and otherwise, it's it's free, which is very nice if your back end takes thirty seconds to respond because it's a AI model.
are very nicely meshed. Like, it's almost like there's this fate that you're driving towards, you know, workflows that you you needed fluid compute in order to do all these fancy things or at at least make it easy to ship and discuss stuff. Right.
Like, I think there's some overlap. Right?
like, with workflows, literally nothing's running between steps. Right? So it's literally free.
That's also why it's free because there's literally just nothing happening versus flu compute, which which does have the same property except it's it's more agile. Right? Like, usually in a workflow, I don't know, you're doing the FFmpeg thing, then you're doing the AI model.
Right? Like, you're not talking millisecond latency. There there's some overhead on each step versus flu compute being like the VM being literally on, which does operate on a different kind of level.
Yeah. Excellent. And then on the Python thing, what happened to always build on JavaScript?
You know, like, isn't that the Britt and Ike line? Basically, I think I guess the the broader the non cheeky question is, are we, like, fifty fifty Python, JavaScript now? Is that the future?
There's no particular language that will win, or, you know, do we do we not have an opinion? I don't think I have an opinion. I saw today that that, like, TypeScript is now, as of of today, is the biggest language on GitHub.
Right? Which last year, you had to still cope that TypeScript and JavaScript is, like, drastically bigger than Python. Honestly, know, obviously, I don't really care.
I think both communities are very relevant, very large, and we are investing in supporting them. Like, the way for SaaS infrastructure works, because we essentially just run VMs, it is not actually hard for us to support Python. We will eventually do PHP and Ruby, but we also, I think, care a lot about the details.
And so the essentially, the reason why we haven't done it yet is because we do invest substantial amount of time to make the DX actually good and feel native to the ecosystem.
And so that's why it's, like, technically easy, but, like, in practice, actually very, very difficult for us to support these these things. So we're we're taking it carefully, but, like, we are you know, there's no technical restriction on our system where we couldn't support all of these different ways of running code. Yeah.
Totally. I mean, that's a very fair response. And the fact is that I think when you're a serious AI cloud, you have to support Python.
So there's there's no way there's no way around it. Okay. I wanted to zoom out.
One other thing that we care about in AI engineering is AI leadership, which is leadership of AI engineers. And, obviously, your role as as as CTO has changed a lot since you joined.
you have apps. What comes to mind when I when I say how has the CTO role changed for you? Yeah.
So I joined Vercel, like, a little bit less than three four years ago, and that was definitely that was before ChatGPT. I I mean, I came directly from Google, and I I think I had some insights there of what was happening and but, like, but Vercel certainly wasn't living in this world. That was kind of the tail end of the last really big crypto wave and and and, you know, and everything else.
Right? And so so then suddenly, the the AI revolution happened. And the thing that I definitely had to walk, like, work through is, how do I transform the company into something quite different?
And I think the the solution that we have come to, like, feels really good, and I think there's a lesson there to be learned for companies that haven't done this transition yet, is that you have to do something that feels native to your company. And so the the two big bets that we made early on, one being v zero and one being AI SDK, I think they felt, like, native to Versal in the sense that v zero was especially originally designed as a as a tool for for making web pages. Like, the full stack stuff came later when with the Sono models.
And, originally, it was very much a web development tool. And the SDK was a framework for building AI apps, and because we're a framework company, it felt really native. And I think you you have to, like, make that you know, you have to be honest with yourself, like, what product, even if I do have to, like, change to building something else, does kind of extend naturally from what I'm doing rather than being very kind of like, being something that is entirely different, and no one believes me that I would be the right place to buy that from.
That that I think that's just generally timeless advice. Have manager to IC ratios gone up? Do know what I mean?
Like That's a really good question. I don't I don't track that very closely. Okay.
Think that's It's a thesis of I'm I'm I yeah. I would be supportive of it going up. It's a it's a big question, like, role of the engineering manager.
I have two reports. Nice. Nice.
Right? So, like, that that's the way we're organized. Our we've mean, it's a privilege of a CTO that you don't you know, VPE is usually the people people managers.
Yeah. Like, but but that's actually not not the point. The point is that it's, like, the basic the really the one thing that Google really got right is that it has very strong ICs.
It has IC levels all the way up. Right? There's To eleven.
Seven day at eleven, and and the and so we're doing the same thing, and so we had to have someone as an IC at at the at the at the top level. Right? And so that's why I think the this is, think, another thing that that you have to be willing to as a company to live in this world where you don't make your strongest engineers have the choice of, like, not making more money or becoming a potentially very bad manager.
Right? Because, like, there's little correlation from being the best engineer to being the right manager. Right?
You could be. Sometimes you have, like, these moments of, oh my god.
it's essentially crapshoot whether that happens. Yeah. That that's the other thing that I find obviously, PETA principle applies, which is the the the principle of promoting people to the level of incompetence.
I think the other interesting thing that people are finding is that PMs and designers are now starting to contribute more to code because they feel they can just vibe code something. And maybe that's good. Maybe that's bad.
I don't know if there's standards around this that have been established inside of Vercel where there used to be clear code owners. And now because we feel that we are coding agents, we we can just we feel we can do a lot more, and maybe that's caused some politics somewhere.
100%. Like, we wanna be at the avant garde of doing this. Right?
Like, as makers of v zero, like, we we highly encourage all our employees to, like, contribute code. I think one thing we haven't talked about, and we might have time to go into, but, like, we are very deeply working on a way to build apps that follows the threat model that assume that the developer doesn't know what they're doing. And also, they're using AI that also doesn't know what they're doing.
And so I wanna be able to build an app that is secure even if the developer is incompetent. Right? But today, that's not the case.
Right? Today, I assume a developer Everyone has trust. Has competent.
Full trust. Right? Like the and and and so and so we have, I think, very strong progress there, and we're working with large lots of large companies that have data, for example, where you have the idea that you say, well, auth cannot be part of the app because they're they're not gonna get that right.
Right? So auth has to be extracted from the app. In fact, which data you can see, they also cannot be under control of the app because, again, you're gonna get it wrong.
Right? And so we're we're building these systems that that try to have a minimum amount of security fully independent of the quality of the app. And I think that is part of the future of how people will build stuff.
Okay. And this this is probably a good idea anyway. This is Vercel level, not so much v zero or Next.
Js level? Like, in front Yeah. In a way, in an integrated fashion.
Right? Because I'm gonna build the app in v zero, and now I wanna deploy it to my fellow coworkers. Only the right people should be able to access it, and when they access it, they only should see the right data.
Yeah. I I think this is very exciting. I think, like, the closest people that come to this is WorkOS, basically.
And, yeah, I think there there should be more, like, native agent native, if we can call it that, infrastructure that, you know, lets people build and vibe code safely, which we all we all wanna enable them. They just cannot be trusted. This is very insightful.
I actually, I'm very excited to see that. Yeah. Pin me when it comes out.
But otherwise, thank you for spending the time to to recap ship recap all Vercel's stuff. It's that I cannot imagine a better person to to talk to. You you're so you're so generous and friendly and engaged.
You know, I definitely don't feel like all CTOs are as engaged with, like, regular developers as you are. I'm trying my best, and I'm spending too much time on x. But, yeah, this is super fun.
Thank you so so much for having me.
Shared via Hopper