Building Faster with AI
Warning: This text was generated using AI and has not yet been reviewed by humans
It's really great to see all of you. What I want to do today, since this is built as startup school, is share with you some lessons I've learned about building startups at AI Fund. AI Fund is a venture studio, and we build an average of about one startup per month. And because we co-found startups, we're in there writing code, talking about customers, designing features, determining pricing. And so we've done a lot of reps of not just watching others build startups, but actually being in the weeds, building startups with entrepreneurs.
What I want to do today is share with you some of the lessons I've learned building startups, especially around this changing AI technology and what it enables. And it'll be focused on the theme of speed. So it turns out that for those of you that want to build a startup, I think a strong predictor for startup's odds of success is execution speed. And I actually have a lot of respect for the entrepreneurs and executives that can just do things really quickly, and new AI technology is enabling startups to go much faster.
So what I hope to do is share with you some of those best practices, which are frankly changing every two to three months still, to let you get that speed that hopefully lets you have a higher odds of success.
Before diving into speed, you know, a lot of people ask me, "Hey Andrew, where are the opportunities for startups?" So this is what I think of as the AI stack, where at the lowest level are the semiconductor companies, then the clouds and hyperscalers built on top of that. A lot of the AI foundation model companies built on top of that. And even though a lot of the PR excitement and hype has been on these technology layers, it turns out that almost by definition, the biggest opportunities have to be at the application layer because we actually need the applications to generate even more revenue so that they can afford to pay the foundation, cloud, and semiconductor technology layers.
So for whatever reason, media and social media tends not to talk about the application layer as much, but for those of you thinking you're building startups, almost by definition the biggest opportunities have to be there, although of course there are opportunities at all layers of the stack.
One of the things that's changed a lot over the last year in terms of AI tech trends — if you ask me what's the most important tech trend in AI, I would say it's the rise of agentic AI. About a year and a half ago, when I started to go around and give talks to try to convince people that AI agents might be a thing, I did not realize that around last summer a bunch of marketers would get a hold of this term and use it as a sticker and slap it on everything in sight, which made it almost lose some of its meaning. But I want to share with you from a technical perspective why I think agentic AI is exciting and important and also opens up a lot more startup opportunities.
So, it turns out that the way a lot of us use LLMs is to prompt it to have it generate output. And the way we have an LLM output something is as if you're going to a human — or in this case an AI — and asking it to please type out an essay for you by writing from the first word to the last word all in one go without ever using backspace. And humans, we don't do our best writing being forced to type in this linear order. And it turns out neither does AI.
But despite the difficulty of being forced to write in this linear way, our LLMs do surprisingly well. With agentic workflows, we can go to an AI system and ask it to please first write an essay outline, then do some web research if it needs to and fetch some web pages to put in their own context, then write a first draft, then read the first draft and critique it and revise it, and so on. And so we end up with this iterative workflow where your model does some thinking and some research, does some revision, goes back to do more thinking, and by going around this loop many times, it is slower but it delivers a much better work product.
So for a lot of projects AI Fund has worked on—everything from pulling out complex compliance documents to medical diagnosis to reasoning about complex legal documents—we found that these agentic workflows are really a huge difference between it working versus not working. But a lot of the work that needs to be done, a lot of the valuable businesses to be built, still will be taking workflows—existing or new workflows—and figuring out how to implement them into these kinds of agentic workflows.
So just to update the picture for the AI stack, what has emerged over the last year is a new agentic orchestration layer that helps application builders orchestrate or coordinate a lot of calls to the technology layers underneath. And the good news is the orchestration layer has made it even easier to build applications. But I think the basic conclusion that the application layer has to be the most valuable layer of the stack still holds true, with a bias or focus on the application layer.
Let me now dive into some of the best practices I've learned for how startups can move faster.
It turns out that at AI Fund, we only focus on working on concrete ideas. So to me, a concrete idea—a concrete product idea—is one that's specified in enough detail that an engineer can go and build it. So for example, if you say, "Let's use AI to optimize healthcare assets," you know that's actually not a concrete idea. It's too vague. If you tell me it's software to use AI to optimize healthcare assets, different engineers would do totally different things, and because it's not concrete, you can't build it quickly and you don't have speed.
In contrast, if you had a concrete idea like, "Let's write software to let hospitals let patients book MRI machine slots online to optimize usage"—I don't know if this is a good or a bad concrete idea. Actually, businesses are already doing this. But it is concrete, and that means engineers can build it quickly. If it's a good idea, you find out; if it's not a good idea, you will find out. But having concrete ideas buys you speed.
Or someone might say, "Let's use AI for email personal productivity." Too many interpretations of that. That's not concrete. But if someone says, "Could you build an app that integrates with Gmail automation that uses the right prompt source, right filter, entire emails"—that is concrete. I could go build that this afternoon.
So concreteness buys you speed, and the deceptive thing for a lot of entrepreneurs is that vague ideas tend to get a lot of kudos. If you go and tell all your friends, "We should use AI to optimize the use of healthcare assets," everyone will say that's a great idea. But it's actually not a great idea, at least in the sense of being something you can build. It turns out when you're vague, you're almost always right. But when you're concrete, you may be right or wrong. Either way is fine. We can discover that much more fast, which is what's important for startups.
In terms of executing on concrete ideas, I find that at AI Fund, I ask my team to focus on concrete ideas because a concrete idea gives clear direction and the team can run really fast to build it and either validate it, prove it out, or falsify and conclude it doesn't work. Either way is fine. So we can do that quickly, and it turns out that finding good concrete ideas usually requires someone—could be you—could be a subject matter expert thinking about a problem for a long time.
So for example, actually before starting Coursera, I spent years thinking about online education, talking to users, holding my own intuitions about what would make a good edtech platform, and then after that long process—I think YC sometimes calls it "wandering the idea maze"—but after thinking about it for a long time, you find that the gut of people that have thought about this for a long time can be very good about rapidly making decisions. As in, after you've thought about this, talked to customers, and so on for a long time, if you ask this expert, "Should I build this feature or that feature?" you know, the gut, which is an instantaneous decision, can be actually a surprisingly good proxy. It can be a surprisingly good mechanism for making decisions.
And I know I work on AI, you might think I'll say, "Oh, we need data." And of course, I love data, but it turns out getting data for a lot of startups is actually a slow mechanism for making decisions. And a subject matter expert with a good gut is often a much better mechanism for making a speedy decision.
And then one other thing: for many successful startups, at any moment in time you're pursuing one very clear hypothesis you're building out and trying to sell. And a startup doesn't have resources to hedge and try 10 things at the same time. So pick one, go for it, and if data tells you to lose faith in that idea, that's actually totally fine. Just pivot on a dime to pursue a totally different concrete idea.
So that's what often feels like at AI Fund. We're pursuing one thing doggedly with determination until the world tells us we were wrong, then change and pursue a totally different thing with equal determination and equal doggedness.
And one other pattern I've seen: if every piece of new data causes you to pivot, it probably means you're starting off from too weak a base of knowledge, right? If every time you talk to a customer, you totally change your mind, probably means you don't know enough about that sector yet to have a really high quality concrete idea, and finding someone that's thought about a subject for longer may get you on to a better path in order to go faster.
The other thing I often think about is the build-feedback loop, which is rapidly changing when it comes to how we build with AI coding assistance. So when you're building a lot of applications, one of the biggest risks is customer acceptance, right? A lot of startups struggle not because we can't build whatever we want to build, but because we build something and it turns out nobody cares.
And so for a lot of the way I build startups—especially applications, less so deep tech, less so technology startups, but definitely application startups—is we often build software (this is an engineering task), and then we will get feedback from users (this is a product management task), and then we'll go back. Then, based on the user feedback, we'll tweak our views on what to build, go back to write more software, and we go around this loop many, many times, iterating toward product-market fit.
And it turns out that with AI coding assistance—which Andre talked about as well—rapid engineering is becoming possible in a way that just was not possible much more feasible. So the speed of engineering is going up rapidly, and the cost of engineering is also going down rapidly. This changes the mechanisms by which we drive startups around this loop.
When I think about the software that I do, I maybe put it into two major buckets. Sometimes I'll build quick and dirty prototypes to test an idea. You say, "Build a new customer service chatbot. Let's build AI to process legal documents, whatever"—build a quick and dirty prototype to see if we think it works. The other type of software I do is write maintainable production software, maintain legacy software, but these massive production-ready code bases.
Depending on which analyst report you trust—it's been hard to find very rigorous data on this—when writing production quality code, maybe we're 30 to 50% faster with AI systems. Hard to find a rigorous number, maybe it's plausible. But in terms of building quick and dirty prototypes, we're not 50% faster—I think we're easily 10 times faster, maybe much more than 10 times faster.
And there are a few reasons for this: when you're building standalone prototypes, there's less integration with legacy software infrastructure, legacy data needed. Also, the requirements—reliability, even scalability, even security—are much lower. And I know I'm not supposed to tell people to write insecure code—right, feels like the wrong thing to say—but I routinely go to my team and say, "Go ahead, write insecure code." Because if this software is only going to run on your laptop and you don't plan to maliciously hack your own laptop, it's fine to have insecure code, right? But of course, after it seems to be working, please do make it secure before you ship it to someone else. And you know, like leaking PII, leaking sensitive data—that is very damaging. So before you ship it, make it secure and scalable, but while you're just testing it, it's fine.
And so I find increasingly startups will systematically pursue innovations by building 20 prototypes to see what works, right? Because I know that there's some angst in AI. A lot of proof of concepts don't make it to production. But I think by driving the cost of a proof of concept low enough, it's actually fine if lots of proof of concepts don't see the light of day.
And I know that the mantra "move fast and break things" got a bad rep because it broke things. And some teams took away from this that you should not move fast, but I think that's a mistake. I tend to tell my teams to move fast and be responsible. And I think there are actually lots of ways to move really quickly while still being responsible.
And in terms of the AI assistance coding landscape, I think it was three, four years ago code autocomplete—popularized by GitHub Copilot—and then there was a cursor, windserve generation of AI-enabled IDEs, which I use quite a lot. And then starting, I don't know, six, seven months ago, there started to be this new generation of highly agentic coding assistants, including that she's using o3 a lot for coding. Cloud Code is fantastic. Since GPT-4 release, it's become—and ask me again in a few months, I may use something different. But the tools are evolving really rapidly, but I think Cloud Code—this is a new generation of highly agentic coding assistance that is making developer productivity keep on growing.
And the interesting thing is if you're even half a generation or one generation behind, it actually makes a big difference compared to if you're on top of the latest tools. And I find my team is taking really different approaches to software engineering now compared to even three or six months ago.
One surprising thing is we're used to thinking of code as this really valuable artifact because it's so hard to create, but because the cost of software engineering is going down, code is much less of a valuable artifact as it used to be. So I'm on teams where we've completely rebuilt a codebase three times the last month, right? Because it's not that hard anymore to just completely rebuild a codebase. Pick a new data schema—it's fine because the cost of doing that has plummeted.
Some of you may have heard of Jeff Bezos's terminology of a two-way door versus a one-way door. A two-way door is a decision that you can make. If you change your mind, come back out, you know, reverse it relatively cheaply. Whereas a one-way door is you make a decision and if you change your mind, it's very costly or very difficult to reverse.
So choosing the software architecture of your tech stack used to be a one-way door. Once you built on top of a certain tech stack, you set a database schema—really hard to change it. So that used to be a one-way door. I don't want to say it's totally a two-way door, but I find that my team will more often build on a certain tech stack, a week later change your mind, let's throw the codebase away and redo it from scratch on a new tech stack.
I don't want to overhype it. We don't do that all the time. There are still costs to redoing that. But I find my team is often rethinking what is a one-way door and what's now a two-way door because the cost of software engineering is so much lower now.
And maybe going a little bit beyond software engineering, I feel like this is actually a good time to empower everyone to build with AI. Over the last year, a bunch of people advised others not to learn to code on the grounds that AI would automate it. I think we'll look back on this as some of the worst career advice ever given because as better tools make software engineering easier, more people should do it, not fewer.
So when many decades ago the world moved from punch cards to keyboard and terminal, that made coding easier. When we moved from assembly to high-level languages like COBOL, there were actually people arguing back then that now we have COBOL, we don't need programmers anymore—like people actually wrote papers to that effect—but of course that was wrong, and programming languages made it easier to code and more people learned to code.
Text editors, IDEs, AI coding assistants—and as coding becomes easier, more people should learn to code. I have a controversial opinion, which is I think actually it's time for everyone of every job role to learn to code. And in fact, on my team, you know, my CFO, my head of talent, my recruiters, my front desk person—all of them know how to code, and I actually see all of them performing better at all of their job functions because they can code.
And I think I'm probably a little bit ahead of the curve—probably most businesses are not there yet—but in the future, I think we'll empower everyone to code. A lot of people can be more productive.
I want to share with you one lesson I learned as well on why we should have people learn to do this, which is when I was teaching generative AI for everyone on Coursera, we needed to generate background art like this using Midjourney. And you know, one of my team members knew art history, and so he could prompt Midjourney with the genre, the palette, the artistic inspiration, had a very good control over the images he generated. So we ended up using all of Tommy's generated images.
Whereas in contrast, I don't know art history. And so when I prompt image generation, I could write, "Please make pretty pictures of robots for me," right? And I could never have the control that my collaborators could. And so I couldn't generate as good images as he could.
And I think with computers, one of the most important skills of the future is the ability to tell a computer exactly what you want so they'll do it for you. And it will be people that have that deeper understanding of computers that will be able to command a computer to get the outcome you want. And learning to code—not that you need to write the code yourself, steer AI to code for you—seems like it will remain the best way to do that for a long time.
With software engineering becoming much faster, the other interesting dynamic I'm seeing is that the product management work—getting user feedback, deciding what features to build—that is increasingly the bottleneck. And so I'm seeing very interesting dynamics in multiple teams over the last year. A lot more of my teams have started to complain that their bottleneck is on product, engineering, and design because the engineers have gotten so much faster.
Some interesting trends I'm seeing: three, four, five years ago, Silicon Valley used to have these slightly suspicious rules of thumb, but nonetheless rules of thumb, like have 1 PM to 4 engineers or 1 PM to 7 engineers—this like PM (product manager) to engineering ratio, right? Which should be taken with a grain of salt, but it was typical of a 1 PM to 6, 7 engineers.
And with engineers becoming much faster, I don't see product management work—designing what to build—becoming faster at the same speed as engineers. I'm seeing this ratio shift. So literally yesterday, one of my teams came to me, and for the first time when we're planning headcount for a project, this team proposed to me not a 1 PM to 4 engineers, but to have 1 PM to 0.5 engineers. So the team actually proposed to me—I still don't know if this is a good idea—for the first time in my life that I saw managers propose to me having twice as many PMs as engineers. That was a very interesting dynamic.
I still don't know if this proposal I heard yesterday is a good idea, but I think it's a sign of where the world is going. And I find that PMs that can code or engineers with some product instincts often end up doing better.
The other thing that I found important for startup leaders is because engineering is going so fast, if you have good tactics for getting rapid feedback to shape your perspective on what to build faster, that helps you get faster as well.
So I'm going to go through a portfolio of tactics for getting product feedback to keep shaping what you will decide to build. And we're going to go through a list of the faster, maybe less accurate, to the slower, more accurate tactics.
So the fastest tactic for getting feedback is look at the product yourself and just go by your gut. And if you're a subject matter expert, this is actually surprisingly good if you know what you're doing.
A little bit slower is go ask three friends or teammates to get feedback, to play with your product and get feedback.
A little bit slower is ask three to 10 strangers for feedback. It turns out for when I built products, one of the most important skills I think I learned was how to sit in the coffee shop, how to sit in a—when I'm traveling, I often sit in the hotel lobby. It turns out, learn to spot places of high foot traffic and very respectfully, you know, grab strangers and ask them for feedback on whatever I'm building. This used to be easier when I was less known. When people recognize you, it's a little bit more awkward.
I found that I've actually sat with teams in hotel lobbies with very high foot traffic and very respectfully ask strangers, "Hey, we're building this thing, do you mind taking a look?" And I actually learned in a coffee shop, there are a lot of people working, a lot of people don't want to be working, so we give them an excuse to be distracted—they're very happy to do that too. But I've actually kind of made tons of product decisions in a hotel lobby or a coffee shop with collaborators just like that.
Send prototypes to 100 testers if you have access to a logic group of users and prototype to more users. And these get to be slower and slower tactics.
And I know Silicon Valley, you know, we like to talk about A/B testing. Of course, I do a ton of A/B testing, but contrary to what many people think, A/B testing is now one of the slowest tactics in my menu because it's just slow to ship. It depends on how many users you have, right?
And then the other thing is, as you use anything but the first tactic, some teams will look at the data, they make a decision, but the missing piece is: when I A/B test something, I don't just use the result of the A/B test to pick product A or product B. My team will often sit down and look carefully at the data to hone our instincts, to speed up, to improve the rate at which we're able to use the first tactic to make high-quality decisions.
Often sit down and think, "Gee, I thought this product name would work better than that product name. Clearly, my mental model of the users is wrong." To really sit down and think, to update our mental model using all of that data to improve the quality of our guts on how to make product decisions faster. That turns out to be really important.
All right, so I talked about concrete ideas, speed up engineering, speed up product feedback. This is one last thing I want to touch on, which is I've seen that understanding AI actually makes you go faster. And here's why. As an AI person, maybe I'm biased to be pro-AI, but I want to share with you why.
So it turns out that when it comes to mature technology like mobile, you know, many people have had smartphones for a long time. We kind of know what a mobile app can do, right? So many people, including non-technical people, have good instincts about what a mobile app can do.
If you look at mature job roles like sales, marketing, HR, legal, they're all really important and all really difficult. But you know, there are enough marketers that have done marketing for long enough, and the marketing tactics haven't changed that much in the last year. So there are a lot of people that are really good at marketing, and it's really important, really hard, but that knowledge is relatively diffused because the knowledge of how to do HR—it hasn't changed dramatically in the last six months.
But AI is emerging technology, and so the knowledge of how to do AI really well is not widespread. And so teams that actually get it, that understand AI, do have an advantage over teams that don't. Whereas if you have an HR problem, you can find someone that knows how to do it well, probably. But if you have an AI problem, knowing how to actually do that could put you ahead of other companies.
So things like: what accuracy can you get for a customer service chatbot? Should you prompt, fine-tune, or use a workflow? How do you get voice output to low latency? There are a lot of these decisions that if you make the right technical decision, you can solve the problem in a couple days. If you make the wrong technical decision, you could chase a blind alley for three months, right?
And one thing I've been surprised by: it turns out if you have two possible architecture decisions, it's one bit of information. It feels like if you don't know the right answer, at most you're twice as slow, right? One bit, you know, try both. It feels like one bit of information can at most buy you a 2x speedup. And I think in some theoretical sense, that is true. But what I see in practice, if you flip the wrong bit, you're not twice as slow. You spend like 10 times longer chasing a blind alley, which is why I think going in to have the right technical judgment really makes startups go so much faster.
The other reason why I find staying on top of AI really helpful for startups is over the last two years, we have just had a ton of wonderful GenAI tools or GenAI building blocks—partial list, but prompting, workflows, evals, guardrails, RAG, voice, async programming, lots of ETL, embeddings, fine-tuning, graph DB, how to integrate models—there's a long and wonderful list of building blocks that can quickly combine to build software that no one on the planet could have built even a year ago.
And this creates a lot of new opportunities for startups to build new things. So when I learned about these building blocks, this is actually a picture that I have in mind. If you own one building block, like you have a basic white building block, yeah, you can build some cool stuff. Maybe you know how to prompt. So you have one building block, you build some amazing stuff.
But if you get a second building block, like you also know how to build chatbots, so you have a white Lego brick and a black Lego brick, you can build something more interesting. If you acquire a blue building brick as well, you can build something even more interesting. Get a few red building bricks, maybe a little yellow one, more interesting. Get more building bricks, get more building bricks, and very rapidly the number of things you can combine them into grows kind of combinatorially or grows exponentially.
And so knowing all these wonderful building blocks lets you combine them in much richer combinations. One of the things that DeepLearning.AI does—I actually take a lot of DeepLearning.AI courses myself because we work with great—we work with I think like pretty much all the leading AI companies in the world and try to hand out building blocks.
But when I look at the DeepLearning.AI course catalog, this is actually what I see. And whenever I take these courses to learn these building blocks, I feel like I'm getting new things that can combine to form kind of combinatorially or exponentially more software applications that were not possible just one or two years ago.
So just to wrap up, this is my last slide. I then want to take questions if y'all have any. I find that there are many things that matter for startups, not just speed. But when I look at the startups that AI Fund is building, I find that the management team's ability to execute at speed is highly correlated with its odds of success.
And some things we've learned to get you speed is: work on concrete ideas. It's got to be good concrete ideas. I find that as an executive, I'm judged on the speed and quality of my decisions. Both do matter, but speed absolutely matters.
Rapid engineering with AI coding assistance makes you go much faster, but that shifts the bottleneck to getting user feedback on the product decisions. And so having a portfolio of tactics to go get rapid feedback—and if you haven't learned to go to coffee shops and talk to strangers, it's not easy, but just be respectful, right? Just be respectful of people—that's actually a very valuable skill for entrepreneurs to have, I think.
And I think also staying on top of the technology buys you speed.
All right, with that, let me thank you very much. [Applause] Happy to take questions.
Q: As AI advances, do you think it's more important for humans to develop the tools or learn how to use the tools better? Like how can we position ourselves to remain essential in a world where intelligence is becoming democratized?
I feel like AGI has been overhyped, and so for a long time there'll be a lot of things that humans can do that AI cannot. And I think in the future, the people that are most powerful are the people that can make computers do exactly what you want it to do. And so I think staying on top of the tools—some of us will build tools sometimes, but there are a lot of other tools others will build that we can just use—but people that know how to use AI to get computers to do what you want it to do will be much more powerful. Not worry about people running out of things to do, but people that can use AI will be much more powerful than people that don't.
Q: Thank you so much. I have huge respect for you and I think you are a true inspiration for a lot of us. My question is about the future of compute. So as we move towards more powerful AI, where do you think that compute is heading? I mean we see people saying let's ship GPUs to space. Some people talking about nuclear power data centers. What do you think about it?
There's something I'm debating what I wanted to say in response to the last question about kind of AGI. Maybe I'll answer this and a little bit the last question. So it turns out there's one framework you can use for deciding what's hype and what's not hype. I think over the last two years there's been a handful of companies that hyped up certain things for promotional PR, fundraising, influence purposes. And because AI was so new, a handful of companies got away with saying almost anything without anyone fact-checking them because the technology was not understood.
So one of my mental filters is there are certain hype narratives that make these businesses look more powerful that have been amplified. And so, for example, this idea that AI is so powerful, we might accidentally lead to human extinction. That's just ridiculous. But it is a hype narrative that made certain businesses look more powerful, and it got ramped up and actually helped certain businesses' fundraising goals.
"AI is so powerful, soon no one will even have a job anymore." Just not true, right? But again, that made these businesses look more powerful, got hyped up. "We are so powerful that by training a new model, we will casually wipe out thousands of startups." That's just not true. Yes, Jasper ran into trouble. A small number of companies got wiped out. But it's not that easy to casually wipe out thousands of startups.
"AI needs so much electricity, only nuclear power is good enough for that. You know, that wind, solar stuff—not." That's just not true. So I think a lot of this—GPUs in space, you know, I don't know. It's like, go for it. I think we have a lot of room to run still for terrestrial GPUs. But I think some of these hype narratives have been amplified that I think are a distortion of what actually will be done.
Q: There's a lot of hype in AI and nobody's really certain about how we're going to be building the future with it. But what are some of the most dangerous biases or overhyped narratives that you've seen people talk about or get poisoned by that they end up running with that we should try to avoid or be more aware of and allow us to have a more realistic view as we are building this future?
So I think the dangerous AI narrative has been overhyped. AI is a fantastic tool, but like any other powerful tool like electricity, lots of ways to use it for beneficial purposes. Also some ways to use it in harmful ways. I find myself not using the term "AI safety" that much. Not because I think we should build dangerous things, but because I think safety is not a function of technology, it's a function of how we apply it.
So like an electric motor—you know, you can't—the maker of an electric motor can't guarantee that no one will ever use it for unsafe downstream tasks. Like, an electric motor can be used to build a drill, a machine, an electric vehicle can be used to build a smart bomb, but the electric motor manufacturer can't control how it's used downstream.
So safety is not a function of the electric motor, it's a function of how you apply it. And I think the same thing for AI. AI is neither safe nor unsafe. It is how you apply it that makes it safe or unsafe. So instead of thinking about AI safety, I often think about responsible AI because it is how we use it responsibly—hopefully—or irresponsibly that determines whether or not what we build with AI technology ends up being harmful or beneficial.
And I feel like sometimes the really weird corner cases that get hyped up in the news—I think just one or two days ago there was a Wall Street Journal article about AI losing control of AI or something. And I feel like that article took corner case experiments run in a lab and sensationalized it in a way that I think was really disproportionate relative to the lab experiment that was being run.
And unfortunately, technology is hard enough to understand that many people don't know better, and so these hype narratives do keep on getting amplified. And I feel like this has been used as a weapon against open source software as well, right? Which is really unfortunate.
Q: Thank you for your work. I think your impact is remarkable. My question is as aspiring founders, how should we be thinking about business in a world where anything can be disrupted in a day? Whatever great model, product or feature you have can be replicated with AI code and competitors in basically hours.
It turns out when you start a business, there are a lot of things to worry about. The number one thing I worry about is: are you building a product that users love? It turns out that when you build a business, there are lots of things to think about: the go-to-market channel, competitors, technology, moat—all that is important. But if I were to have a singular focus on one thing, it is: are you building a product that users really want? Until you solve that, it's very difficult to build a valuable business. After you solve that, the other questions do come to play.
Do you have a channel to get to customers? What is pricing long-term? What is your moat? I find that moats tend to be overhyped. Actually, I find that more businesses tend to start off with a product and then evolve eventually into a moat. But consumer products, brand is somewhat more defensible. And if you have a lot of momentum, it becomes harder to catch you. But enterprise products, sometimes if you have a—maybe moat is more of a consideration if there are channels that are hard to get into enterprises.
So I think—sorry—when AI Fund looks at businesses, we actually wind up doing a fairly complex analysis of these factors and writing a two- to six-page narrative memo to analyze it before we decide whether or not to proceed with it or not. And I think all of these things are important, but I feel like at this moment in time, the number of opportunities—meaning the amount of stuff that is possible that no one's built yet in the world—seems much greater than the number of people with the skill to build them.
So definitely at the application layer, it feels like there's a lot of white space for new things you can build that no one else seems to be working on. And I would say, you know, focus on building a product that people want, that people love. And then figure out the rest of it along the way. Although it's important to figure it along the way.
Q: Hi professor. Thanks for your wonderful speech. I am an AI researcher from Stanford and I think your metaphor in your speech is very interesting. You said the current AI tools are like bricks and can be built upon accumulation. However, so far it is difficult to see the accumulative functional expansion of the integration of AI tools because they often align on the stacking of functions based on intent distribution and are accompanied by dynamic problems of tokens and time overhead. So which is different from static engineering. So what do you think will be the perspective of a possible agent accumulation effect in the future?
Hey, just some quick remarks to that. You mentioned agent, token cost—my most common advice to developers is, to first approximation, just don't worry about how much tokens cost. Only a small number of startups are lucky enough to have users use so much of your product that the cost of tokens becomes a problem. It can become a problem—I've definitely been on a bunch of teams where the cost—you know, users like our product and we started to look at our GenAI bills and it was definitely climbing in a way that really became a problem.
But it's actually really difficult to get to a point where your token usage costs are a problem. And for the teams I'm on where we were lucky enough that users made our token costs a problem, we often had engineering solutions to then bend the curve and bring it back down through prompting, fine-tuning, RAG, or whatever.
And then what I'm seeing is that I'm seeing a lot of agentic workflows that actually integrate a lot of different steps. So for example, if you build a customer service chatbot, we'll often have to use prompting, maybe optimize some of the results with DSPI, build evals, build guardrails, maybe the customer service chatbot needs RAG part of the way to get information to feed back to the user.
So I actually do see these things grow. But one tip for many of you as well is I will often architect my software to make switching between different building block providers relatively easy. So for example, I have a lot of products that build on top of OpenAI, but sometimes you point to a specific product and ask me which OpenAI are we using? I honestly don't know because we built up evals, and when there's a new model that's released, we'll quickly run evals to see if the new model is better than the old one. And then we'll just switch to the new model if the new model does better on evals.
And so the model we use week by week—you know, sometimes our engineers will change it without even bothering to tell me because the evals show the new model works better. So it turns out that switching cost for foundation models is relatively low, and we often architect our software—AI Suite is open-sourcing that my friends and I worked on to make switching easier.
Switching cost for the orchestration platforms is a little bit harder. But I find that preserving that flexibility in your choice of building blocks often lets you go faster even as you're building more and more things on top of each other. So hope that helps.
Q: Thank you so much. In the world of education in AI, there are two paradigms mostly. So one is AI can make teachers more productive—automating grading and automating homeworks. But another school of thought is that there'll be personal tutors for every student. So every student can have one tutor that gets feedback from an AI and gets personal questions from them. So how do you see these two paradigms converge and how would education look like in the next five years?
I think everyone feels like a change is coming in edtech, but I don't think the disruption is here yet. I think a lot of people are experimenting with different things. So you know, Coursera has Coursera Coach, which actually works really well. DeepLearning.AI is more focused on teaching AI, also has some built-in chatbots. A lot of teams experiment with autograding. There's an avatar with me on the DeepLearning.AI website you can talk to if you want—DeepLearning.AI.
And then I think for some things like language learning with Duolingo, that has become clearer—some of the ways AI would transform it. For the broader educational landscape, the exact ways that AI would transform it—I see a lot of experimentation. I think what Khan Academy, which I've been doing some work with, is doing is very promising for K-12 education. But I think what I'm seeing is frankly tons of experimentation, but the final end state is still not clear.
I do think education will be hyper-personalized. But is that workflow an avatar? Is it a text chatbot? What's the workflow? I think I feel like the hype from a couple years ago that with AGI soon, it will all be so easy—that was hype. The reality is work is complex, right? Teachers, students, people do really complex workflows, and for the next decade we'll be looking at the work that needs to be done and figuring out how to map it to agentic workflows.
And education is one of the sectors where this mapping is still underway, but it's not yet mature enough to the point where the end state is clear. So I think we should all just keep working on it.
Q: All right. Thank you so much Andrew. My question is I think AI has a lot of great potential for good but there's also a lot of potential for bad consequences as well such as exacerbating economic inequality and things like that and I think a lot of our startups here while they'll be doing a lot of great things will also be you know just by virtue of their product be contributing to some of those negative consequences. So I was curious how do you think you know us as AI builders should kind of balance our product building with also the potential societal downsides of some AI products and essentially how can we both move fast and be responsible as you mentioned in your talk?
Look in your heart, and if fundamentally what you're building—if you don't think it'll make people at large better off, don't do it, right? I know it sounds simple, but it's actually really hard to do in the moment. But at AI Fund, we've killed multiple projects—not on financial grounds, but on ethical grounds—where there are multiple projects we looked at, the economic case is very solid, but we said, "You know what, we don't want this to exist in the world," and we just killed it on that basis.
So I hope more people will do that. And then I worry about bringing everyone with us. One thing I'm seeing is people in all sorts of job roles that are not engineering are much more productive if they know AI than if they don't. And so for example, on my marketing team, my marketers—they know how to code. Frankly, they were running circles around the ones that don't. So then everyone learned to code, and then they got better.
But I feel like trying to bring everyone with us, to make sure everyone is empowered to build with AI—that'll be an important part of what all of us do, I think.
Q: I'm one of your big fans and thank you for your online courses. Your courses make deep learning much more accessible to the world. And my question is also about education. As AI becomes more powerful and widespread, there seems to be a growing gap between what AI can actually do and what people perceive it can do. So what do you think about—is it important to educate the general public about deep learning stuff and not only educate those technical people and make people understand more what AI really does and how it works?**
I think that knowledge will diffuse. DeepLearning.AI—we want to empower everyone to build with AI. So we're working on it. Many of us work on it. I'll just tell you what I think is the main danger. I think there are maybe two dangers. One is if you don't bring people with us fast enough—I hope we'll solve that.
There's one other danger, which is it turns out that if you look at the mobile ecosystem, mobile phones, it's actually not that interesting. And one of the reasons is there are two gatekeepers: Android and iOS. And unless they let you do certain things, you're not allowed to try certain things on mobile. And I think this hampers innovators.
These dangers of AI have been used by certain businesses. They're trying to shut down open source because a number of businesses would love to be a gatekeeper to large-scale foundation models. So I think hyping up dangers, supposed false dangers of AI in order to get regulators to pass laws like the proposed SB 1047 in California—which thank goodness we shut down—would have put in place really burdensome regulatory requirements that don't make anyone safer, but would make it really difficult for startups to release open source and open-weight software.
So one of the dangers to inequality as well is if these regulatory—you know, awful regulatory approaches—and I've been in the room where some of these businesses said stuff to regulators that was just not true. So I think that some of these arguments—the danger is if these regulatory proposals succeed and end up stifling regulations, leaving us with a