Jev 与 RLCD：为软件而非聊天造模型
论证要点
- Diogo 认为自 ChatGPT 成功后，业界丢掉了自回归聊天微调 LLM 之外的所有模式，API 前沿模型在对齐、拒答和可靠性上都走错了方向。
- Jev 的核心创新是 RLCD（Reinforcement Learning for Calibrated Decisions），优化 System One 任务上具有认知诚实概率的答案，而非人类评分反馈或可程序验证输出。
- RLHF 导致幻觉、谄媚和对人类的永久依赖；RLVR 能解 Navier Stokes 但加剧锯齿状智能，且难以与其他软件集成。
- 作者提出“最苦的教训”：你优化什么就得到什么，ML 中最重要的部分根本不是 ML，选对北极星（如集成进工具调用而非用户偏好点赞）会让一切顺理成章。
- TypeSafe 拒绝公开基准，优化每美元智能；自视为数据实验室而非模型实验室，认为正确的任务和数据可能比算力更重要。
- Jev 目标是消失在软件背景中，像 regex 一样不起眼；开发者应把 AI 工作流分解为小的、可测量的决策，用结构化状态替代巨型提示。
对实践的启示
做 AI 应用的人应重新审视优化目标：不要只追求聊天体验或刷榜，而要把 AI 拆成软件中可测量的小决策，关注校准、可靠性和每美元智能。若认同作者，应探索 Jev 式编程原语和结构化状态，而非再做一个低努力 API 克隆。
作者承认的反方与局限
Theo 提出了合理的反对意见；作者也承认 2023-2024 年因个人和组织低估，未能成功训练出目标模型。
原文摘引
You get what you optimize for and the bitterest lesson in ML is that the most important part of it isn’t ML at all.
a core goal of Jev is to “disappear into the background” - eg as unremarkable as regex
▸ 展开原文全文（87 段，抓取自 Latent Space）
Tickets for AIE NYC and applications for the invite-only AIE CODE now open. Join us!
We have an unusual relationship with today’s guest: for years since coauthoring the InstructGPT paper, Diogo Almeida had been saying that API-available frontier models have been going down the wrong path, everything from the alignment to refusals to reliability perspectives, that we have dropped every mode other than autoregressive chat-tuned LLMs because of the overwhelming success of ChatGPT.
In a launch video now viewed ~40M times (by comparison, GPT4o was 22M, Fable 5 was 15M, Navier Stokes was 74M, and 6 Astra was 137M), Diogo introduced Jev and it immediately took over the AI timeline — we’ll skip full Jev explainers because our favorite AI influencer/educator has probably already done one. We also collected:
- the official patterns and cookbooks you should see first, from Allie
- Jev usecases
- speed based - games and computer use
- the voice + computer use example we discuss at 1h34 mins
- The must not miss Doom demo
- Driving cars in games
- Excalidraw
- virtual try-ons
- “Smart Games”/smart NPCs
- guided responses in text messages
- Jev for coding agents has an official guide
- jev for linting
- compacting tool calls
- reasonable pushback from Theo - Diogo has published a note on the Tyranny of the KV Cache that you should read as a followup after the pod for Jev + coding agents, because of his belief that Cache Rules Everything
- Programming Languages built atop Jev (Diogo’s fave)
- Jev for analytics replay and user journey review
- “dark data”
- “smart software”
- a core goal of Jev is to “disappear into the background” - eg as unremarkable as regex
- Jev as a judge
- Jev memes
Instead we’ll focus on what we can uniquely offer — a broader philosophical and mission-based understanding of how and why Jev was created, and what you should expect next in terms of future models from TypeSafe (ReasoningJev?) and what usecases and ideas you should work on vs the 55th low effort clone of Jev’s API or doing a generic JevBench benchmark - something Diogo has rejected publicly.
Why RLCD: Three kinds of RLHF, and why they are ALL the wrong north star
Diogo knows a good deal about RLHF, given that he was on the team that pioneered post-training at OpenAI — and traces the three branches to Christiano et al 2017 (the robot backflip demo), Stiennon et al 2020 (learning to summarize) and his baby, Ouyang et al 2022 (InstructGPT). From there on, every innovation from Function Calling to Structured Outputs to Reasoning felt like a hack on top of the string based, sequence to sequence prediction paradigm. As he mentions on the pod, from 2023-2024 he struggled unsuccessfully, due to both personal and organization underestimation, to train a model that
Jev’s core innovation is "Reinforcement Learning for Calibrated Decisions”, a novel, unpublished technique that optimizes for “answers with epistemically honest probabilities on System One tasks” rather than human rated feedback (RLHF) — which causes hallucinations, sycophancy, and permanent reliance on humans — or programmatically verifiable outputs with rubrics (RLVR) — which solves Navier Stokes but exacerbates jagged intelligence and doesn’t integrate well with other software.
We’ve talked about the calibration problem before on the pod, but probably the single best place to understand why RLCD became necessary is Diogo’s AIE talk, which discusses why a generation of training helpful AI assistants for humans has impaired them for training models for composable, programmable AI for automation.
At the end he also teases his contrarian opinion on scaling laws - which teases how to build a modern neolab without the billions of dollars the major labs have…
The Bitterest Lesson: Tasks and Data beats Compute
We spend a good amount of time discussing Diogo’s essay on the Bitterest Lesson:
His point is that “You get what you optimize for and the bitterest lesson in ML is that the most important part of it isn’t ML at all.” - and picking the right north star, eg upvoting for user preference vs being integrated into tool calls - makes everything else fall in line.
We’re excited to catch up with a freshly dyed Diogo to discuss:
- Why AI can solve extraordinarily hard problems but still fail to automate basic work
- What System One Models are and why Jev is built for software rather than chat
- RLHF, mode collapse, calibration, and the hidden costs of optimizing for human preferences
- Why refusals become a problem when AI is buried inside software dependencies
- Why TypeSafe rejects public benchmarks and optimizes for intelligence per dollar
- The “bitterest lesson”: why the right task and the right data can matter more than compute
- Why TypeSafe thinks of itself as a data lab rather than a model lab
- RLCD vs. RLHF and RLVR as fundamentally different North Stars for AI
- Why reliability and robustness matter more than simple determinism
- Jev’s programming primitives and how intelligence maps into software control flow
- Why developers should decompose AI workflows into small, measurable decisions
- How structured state replaces giant prompts and system messages
- Why Diogo thinks AI should eventually disappear into the background of software
- The “inverse SaaS-pocalypse” and how AI could supercharge existing software
- System One vs. System Two intelligence and the limits of reasoning models
- Dark data, computer use, real-time intelligence, and Jev’s biggest early use cases
- Why Jev could reshape coding agents built around a single-model architecture
- Why Diogo says he wouldn’t pre-train with $1 billion
- The OpenAI journey that led to TypeSafe and why he thinks many neo-labs are approaching AI incorrectly
- Coding agents beyond the KV cache, shared state, sub-agents, and the multi-agent future
Diogo Almeida
- LinkedIn: https://www.linkedin.com/in/diogomda
- X: https://x.com/CompleteSkeptic
- TypeSafe AI: https://typesafe.ai/
Timestamps
00:00:00 Jev Launch Week and the AI Economic Revolution
00:02:50 What Is Jev? System One Models and Programmable AI
00:05:54 RLHF, Mode Collapse, Calibration, and Yann LeCun
00:10:29 Programmatic AI, Refusals, and Safety Alignment
00:17:21 Why TypeSafe Rejects Public Benchmarks
00:20:43 The Bitterest Lesson: Data, Compute, and the Right Task
00:24:59 RLCD vs. RLHF and RLVR
00:28:42 Why Powerful AI Still Hasn’t Automated the Economy
00:39:55 Reliability, Robustness, and Determinism
00:48:11 Model Versioning, LTS, Speed, and Intelligence per Dollar
00:54:04 Inside Jev’s API and Programming Primitives
00:58:28 How to Build with Jev: Structure, Decomposition, and Small Decisions
01:18:28 The Inverse SaaS-pocalypse and AI Disappearing into Software
01:33:21 Computer Use, Dark Data, and Jev’s Biggest Use Cases
01:38:48 How Jev Could Reshape Coding Agents
01:41:00 AI Safety, Frontier Pacing, and the Limits of RLVR
01:48:03 Why Diogo Wouldn’t Pre-Train with $1 Billion
01:55:19 The OpenAI Story Behind TypeSafe
02:01:41 Why Diogo Thinks Most Neo-Labs Are Getting AI Wrong
02:08:00 Coding Agents Beyond the KV Cache and the Multi-Agent Future
Transcript
Introduction: Jev Launch Week and Developer Momentum
Swyx [00:00:00]: Okay, we’re in the studio. A special occasion because this week, Diogo, my good buddy, launched Jev, and it’s been taking over the complete timeline. How do you feel? What’s it like to be you right now?
Diogo Almeida [00:00:16]: Emotionally?
Swyx [00:00:17]: Yeah.
Diogo Almeida [00:00:17]: Never been worse. Like, I’m a ragged corpse of a person right now because there’s so much going on, and I’m like a technical CEO, so I have, like, a lot of fires to fight.
Swyx [00:00:29]: Yeah.
Diogo Almeida [00:00:29]: But mentally, I feel—I say this all the time, and I’ve been saying this kind of for years in my over-under events. Like, I feel like the entire AI field is like one of those, like, carnival house of mirro