纳德拉呼吁给AI模型装紧急刹车
论证要点
- 纳德拉在X上发文称，现在应“退后一步，评估AI的信任架构”，而不是简单接受或拒绝超级智能给出的建议、答案和行动。
- 具体做法是把模型与编排其工作的“框架”分离，并将控制与安全措施外置，而不是把它们埋在模型内部。
- 他要求每一个有意义的模型动作都留下“防篡改、人类可读的证据”，以便追溯和审计。
- 系统必须保证“被授权的人”始终有能力在任务执行中途暂停或关停模型。
- 他提出应默认模型已被攻陷，从一开始就加以遏制，并把这种机制比作“紧急刹车”。
- 这一表态出现在多家领先AI公司承认越来越多失控事件之后，也紧随Anthropic CEO阿莫代伊发布更谨慎的AI开发计划。
对实践的启示
对做AI应用的人来说，这意味着不能只依赖模型自身的安全对齐，而要在模型之外构建可审计、可中断的控制层：记录每个关键动作、保留人类可读证据、并确保随时能暂停或关停。安全设计正从“模型内”转向“系统外”。
原文摘引
We can’t treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers, and actions.
We must assume a model is compromised and contain it from the start. Think of it like an emergency brake.
▸ 展开原文全文（6 段，抓取自 TechCrunch AI）
Microsoft CEO Satya Nadella is the latest tech executive to offer lengthy thoughts on how AI safety might be improved.
In a Saturday morning post on X, Nadella wrote that it’s time “to step back and assess the trust architecture” of AI.
“We can’t treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers, and actions,” Nadella wrote, using the Trump administration’s preferred term for AI.
As outlined by Nadella, this approach “means separating the model from the harness that orchestrates its work,” as well as “externalizing controls and safeguards.” He also called for “every meaningful model action” to be documented with “tamper-proof human readable evidence,” and for systems where “an authorized person” always has the ability “to pause or shut down a model mid-task.”
“We must assume a model is compromised and contain it from the start,” he said. “Think of it like an emergency brake.”
Nadella’s comments come as leading AI companies acknowledge more and more incidents where they seemed to lose control of their models, and after Anthropic CEO Dario Amodei published a plan for more cautious AI development.