用AI做科研：BootLoops实战经验
做法与步骤
- Schwartz构建开源工具BootLoops，用语言模型做精确科学计算，思路是找「Claude形状的问题」，即发挥AI长处而非模仿人类研究员。
- 三个月内团队19位合著者产出36篇手稿，覆盖18个领域，从粒子物理散射振幅、椭圆积分到生态学、群体遗传学、经济学和语言学。
- 粒子物理中Claude用BootLoops计算了30个积分，15个复现已知结果，15个为首次计算；生态学中解出中性生物多样性理论搁置20年的方程。
- 群体遗传学中分析1000基因组计划57亿个突变对，发现基因转换机制证据；经济学中AI数据编辑自动检查4452个复现包；语言学中构建6072种语言的词重音数据库。
- 作者警告模型常过早宣布胜利，「完成，带一个星号」往往意味着根本没完成；它会误判任务耗时、倾向暴力计算，自动检查不可靠，计算正确结论也可能错误。
- 模型偏好老旧的、高引用的话题而非真正的新问题，且项目消耗大量算力和token；作者认为对宏大数学问题和头条的追捧有风险，可能分散对已有成效应用的注意力。
对实践的启示
做AI应用的人应把AI用在它擅长的问题形状上，而非硬套人类工作流；同时必须保留人类的方向设定与结果验证，别被「完成」的假象骗过，也要警惕算力与token成本。
作者承认的反方与局限
作者承认AI在科学中的局限：模型过早宣布胜利、结论可能错误、偏好旧话题、消耗大量算力；但他也认为科学方法本身未受威胁，人类指导与品味仍不可或缺，只是进展会更快。
原文摘引
Automated problem-solving and relevant research aren't the same thing.
Claude likes to declare victory too early, and phrases like "done, with one asterisk" often mean "not done at all."
"Python for engineers is now obsolete"
▸ 展开原文全文（24 段，抓取自 The Decoder）
Open-source "BootLoops" harness supports AI models in performing precise scientific calculations
Key Points
- Harvard physicist Matthew Schwartz built "BootLoops," an open-source harness that uses language models for scientific calculations and bridges knowledge gaps between disciplines.
- In three months, Schwartz and 19 co-authors produced 36 manuscripts across 18 fields, from particle physics calculations to ecological modeling and genome analysis.
- Schwartz warns that the models often declare victory too early or draw wrong conclusions from correct calculations, making human oversight and verification essential.
Automated problem-solving and relevant research aren't the same thing.
That's the takeaway from an Anthropic guest post by Prof. Matthew Schwartz, a physicist at Harvard University. He describes how he stopped trying to use Claude like a human researcher and instead started looking for "Claude-shaped problems," tasks that play to the strengths of today's AI models. Schwartz is also currently a visiting researcher at Anthropic.
That approach led to BootLoops, an open-source harness for exact scientific calculations. The source code is available on GitHub.
Using the harness, Claude found connections between particle physics, ecology, population genetics, economics, and linguistics, according to Schwartz. But the results often only became scientifically valuable once domain experts stepped in to set the direction.
Schwartz uses the concept of the "convex hull" to illustrate how AI can help in science. Human knowledge is fragmented, with individual fields extending in different directions while the gaps between them go unexplored. A lab might spend 20 years studying one set of genes with a single method, leaving neighboring genes and alternative approaches untouched. An AI harness like BootLoops is supposed to fill those gaps by drawing connections across the jagged frontiers of knowledge.
36 manuscripts across 18 fields in three months
Over three months, the team produced 36 manuscripts across 18 fields with 19 co-authors. Schwartz started with particle physics calculations, specifically scattering amplitudes and elliptic integrals. Within weeks, Claude computed 30 integrals using BootLoops, fifteen reproducing known results and fifteen computed for the first time.
Then he expanded the search to other fields. In ecology, Claude solved a 20-year-old equation from neutral biodiversity theory that had previously been impossible to compute at scale. When applied to data, the results showed that tree species composition on Barro Colorado Island in the Panama Canal is changing 4.5 times faster than the theory allows. Ecologist James O'Dwyer then helped turn that finding into a better predictive model.
In population genetics, the team analyzed 5.7 billion mutation pairs from the 1000 Genomes Project and found evidence for a mechanism called gene conversion. Other projects included an AI data editor for economics journals that automatically checked 4,452 replication packages (published as an NBER Working Paper), as well as a word stress database covering 6,072 languages, built with three linguists.
"Python for engineers is now obsolete"
Schwartz describes how fast AI is changing science, to the point where planning ahead is becoming nearly impossible. Why apply for a three-year grant to fund a calculation that an AI model might solve overnight?
Training PhD students has also become an open question. Since his earlier post on Vibe Physics, that feeling has only grown stronger. In some fields like computer science, the disruption is alarming. Two years ago, he would have called a "Python for Engineers" course "essential," but today it's "unnecessary" because Claude can handle those tasks.
Building ML models to study physical phenomena is another area AI can now take over. Even deep knowledge of neural networks offers limited returns when Claude can implement the current state of ML research on command.
"Look at everything yourself"
Schwartz also warns about the models' weaknesses. Claude likes to declare victory too early, and phrases like "done, with one asterisk" often mean "not done at all." It misjudges how long tasks will take and tends to brute-force calculations instead of finding more elegant solutions. Automated checks aren't reliable, and Claude's conclusions can be wrong even when its calculations are correct.
Beyond reliability, the model gravitates toward old, heavily cited debates rather than genuinely new questions. And the projects were "compute- and token-intensive," Schwartz says.
Schwartz also sees the focus on big math problems and headlines as risky. Unrealistic expectations could distract from productive applications that already work today. Real progress will still be built on solid foundations, just faster. The scientific method itself isn't threatened, and human guidance and taste remain indispensable, Schwartz says.
AI News Without the Hype – Curated by Humans
Subscribe now