AI算力为何不是矩形
论证要点
- 典型容量合同给客户固定数量GPU和固定期限，画出来是矩形；但真实负载会增长、尖峰和回落，买峰值会闲置，买少则可能缺需求。
- Modal的做法是跨数千客户池化需求，在底层锁定容量，让每个客户按需弹性使用；聚合需求越平滑，池子越有用。
- 作者承认反向金融风险：若按当前价格锁定容量而GPU价格后续下跌，供应商可能持有昂贵供给；他不预期立即崩盘，但认为建业务时需考虑。
- Modal的沙箱产品源于观察到语言模型能生成代码、需要运行环境；早期采用有限曾讨论关停，后因编码代理、后台代理和强化学习带来新需求，现约占收入三分之一。
- 弹性依赖快速启动：Modal通过可用GPU缓冲、自定义文件系统、进程检查点和GPU内存快照，减少启动新模型副本的工作量，影响应用响应和成本。
- 作者从Spotify推荐系统经历出发，认为构建AI应用时意识到需要更好的基础设施，这成为Modal的起源。
对实践的启示
对做AI应用的人，应避免按峰值买固定算力，优先选择能弹性伸缩、快速启动的基础设施；同时关注供应商的容量池和定价风险，沙箱等代理运行环境可能成为重要成本与体验因素。
作者承认的反方与局限
作者承认，如果供应商按当前价格锁定容量而GPU价格后续下跌，可能持有昂贵供给；他不预期立即崩盘，但认为这一可能性在建业务时值得考虑。
原文摘引
A typical capacity contract gives a customer a fixed number of GPUs for a fixed period. Plot capacity against time and you get a rectangle.
Erik describes Modal’s bet as a kind of duration transformation: pool demand across thousands of customers, secure capacity underneath, and let each customer use what they need when they need it.
If a provider locks in capacity at today’s prices and GPU prices later fall, it can be left holding expensive supply.
▸ 展开原文全文（57 段，抓取自 Latent Space）
Warm welcome to Basil Chatha, our newest cohost from the Forward Deployed side of the AI Engineering world! We’ll be ramping up a new show with him and couldn’t pass up an interview with Modal’s CEO Eric on the occasion of their monster Series D!
Two years ago, we spoke with Erik Bernhardsson about his vision for truly serverless AI infrastructure: developers should be able to run code without provisioning clusters, predicting load, or paying for idle resources. CTO Akshat then joined us to talk about Modal’s infra primitives and stack to offer those primitives and agent experience.
A lot has changed in the past 5 months. Since their $5B series C, Modal now reports it crossed $1 billion in annualized run rate, raised a $750 million Series D at a $16B valuation, and has spun up nearly three billion sandboxes. In this conversation, Erik joins Vibhu to explain the infrastructure and economics behind that growth—and why demand for compute rarely looks like the contracts used to buy it.
Watch the full conversation on YouTube →
GPU demand isn’t rectangular
A typical capacity contract gives a customer a fixed number of GPUs for a fixed period. Plot capacity against time and you get a rectangle. Real workloads behave differently: they grow, spike, and fall throughout the day. Buy enough for the peak and you pay for idle hardware; buy less and you risk missing demand.
Erik describes Modal’s bet as a kind of duration transformation: pool demand across thousands of customers, secure capacity underneath, and let each customer use what they need when they need it. The smoother the aggregate demand, the more useful that pool can be. It’s a useful companion to our conversation with SF Compute’s Evan Conrad: Evan focuses on making capacity tradeable; Erik focuses on making it usable in smaller, elastic slices.
There is a financial risk on the other side. If a provider locks in capacity at today’s prices and GPU prices later fall, it can be left holding expensive supply. Erik says he doesn’t expect an imminent collapse, but thinks the possibility matters when building the business. Jump to the GPU price discussion →
The sandbox that almost didn’t make it
Modal launched sandboxes after Erik and cofounder Akshat Bubna noticed that language models could generate code—and that someone would need a place to run it. Early adoption was modest enough that the team discussed shutting the product down. Then coding agents, background agents, and reinforcement learning created new demand. Erik says sandboxes have grown to roughly a third of Modal’s revenue, a result he didn’t predict at launch.
That growth also changes what infrastructure has to do. An agent may create an environment, run code, inspect the result, and try again many times. The latency of starting and restoring those environments becomes part of the experience. In our recent interview with Akshat, we explored this shift from developer experience to agent experience; here Erik adds the product and business story. Jump to the sandbox discussion →
Fast startup is an infrastructure feature
Elasticity only works if capacity can arrive quickly enough to matter. Modal’s serverless GPU write-up describes the machinery: a buffer of available GPUs, a custom filesystem, process checkpointing, and GPU memory snapshots. Those pieces reduce the work needed to bring another model replica online. It is a concrete example of how infrastructure decisions show up in an application’s responsiveness and cost.
We also discuss Erik’s path from Spotify’s recommendation system to Modal, how the company buys compute, alternatives to NVIDIA, forward-deployed engineers, and why code review is optional at Modal. Jump to engineering culture →
Show notes
- Modal — the AI infrastructure platform Erik founded.
- Erik’s writing — his essays on infrastructure, markets, and building companies.
- Modal’s Series D announcement — the company’s latest reported growth and sandbox milestones.
- How Modal achieved truly serverless GPUs — the startup and scaling techniques behind elastic inference.
- Modal’s resource solver — how it selects capacity across changing prices and availability.
- Scaling to one million concurrent sandboxes — an engineering account of sandbox scaling.
- Our first interview with Erik — the original serverless infrastructure thesis.
- Our interview with Akshat Bubna — infrastructure and developer experience for agents.
- The Programmable Data Center — Evan Conrad on GPU trading and physical settlement.
Timestamps
- 0:00 What Modal does
- 1:25 From Spotify to building Modal
- 4:19 Founding before the generative AI boom
- 7:43 First customers and fundraising
- 10:00 Building for inference and autoscaling
- 13:21 Where Modal gets GPU capacity
- 15:39 Renting vs. owning infrastructure
- 19:45 The risk of GPU prices falling
- 24:18 Is AI demand infinite?
- 26:12 Alternatives to NVIDIA
- 28:25 What Modal can learn from neoclouds
- 30:26 GPUs as an asset class
- 31:55 Modal’s products: functions and sandboxes
- 36:03 The case for smaller training workloads
- 41:21 How agent sandboxes are changing
- 46:48 Anthropic and sandbox security
- 49:29 Using agents to test security
- 52:51 Developer experience in the age of agents
- 57:45 What Modal’s forward-deployed engineers do
- 59:55 Modal’s approach to product management
- 1:03:29 Optional code reviews and engineering culture
- 1:05:56 Cloud, on-prem and bring-your-own-cloud
- 1:09:25 Capacity, fundraising and growth
- 1:13:07 Scaling the team
- 1:17:28 Erik’s tweets: ephemeral sandboxes
- 1:18:31 The problem with rectangular GPU contracts
- 1:21:20 Inference providers vs. model builders
- 1:23:25 Where models improve next
Transcript
Automatically transcribed from the revised edit; names and wording may contain errors.
0:00 Thanks for coming on, Eric Fern-Hartson from Modal. Could you explain the business a little bit and maybe how you got here? Yeah. So we build, the easiest way to put it is we build AI infrastructure. So we offer a compute platform and a storage platform that enables a lot of customers like Dacagon, and Suno, and Runway, and Cognition, and many others to essentially build AI applications. So what that means in practice is a lot of inference, a lot of sandboxes, sometimes training and fine-tuning, sometimes batch jobs and other things. So we have basically built a software SDK that enables customers to move really fast and build this. We manage all the capacity underneath, make it easier to deploy and scale and run these things in production. So you were on Light and Space like two years ago, a lot has happened since then. How big were you back then, like whatever numbers you can share and comparatively, how big are you now? I actually don’t remember where we were, but it’s probably like at least 50X, there’s revenue and users and users. Yeah. I use it actually for a side project that I have. Nice. Yeah. It’s like, yeah, turn 2D images into 3D models. Yeah. So yeah, just hosting a couple of models here. What do you think? Any feedback? I mean, I love it. Nice. Good. I mean, yeah, that’s why I use it over any competitors or anything. Good. So before you started Modal, you were at Spotify and also like better.com, I think.
1:30 That’s right. Yeah. Yeah. What were you doing at Spotify? I actually did a lot of different things at Spotify, but my main contribution was I built a music recommendation system. The first version I joined after school in 2008, 2009 maybe, and spent seven years at Spotify. And most of that time was building basically from scratch and music recommendation system and built up a small team and we launched a bunch of features like related artists and Discover Weekly and a bunch of other stuff. So a lot of, a lot of matrix factorization, a lot of compute intensive stuff. And arguably like a lot of what sort of later became the genesis of Modal was just building AI applications and realizing like the