AI挖掘四百年档案:发现遗忘的陨石与犀牛
做法与步骤
- 受历史学家Breen用AI发现渡渡鸟新目击记录的启发,作者作为软件工程师,决定用AI代理工作流并行搜索多个历史谜题。
- 先用AI深度研究助手筛选出13个最可能用现有在线数据解决的历史问题,要求每个答案都能追溯到原始文档扫描件。
- 构建研究流水线:读取GLOBALISE转录的435万页荷兰东印度公司档案、荷兰报纸、美国报纸和船舶日志,用嵌入模型将570万段落转为语义指纹。
- 用廉价快速的System One决策模型Jev初筛数万条命中,仅将标记的段落交给Claude Haiku精读、翻译并提取日期地点,再由Claude Code代理核对原始手写页扫描件。
- 在搜索新发现前,先用已知事件(如Breen的渡渡鸟、1783年Laki火山喷发、1815年Tambora)测试流水线,确保能发现已知目标。
- 首个真实发现:1812年巴达维亚报纸转载的孟买公报信件,记载了印度Cokurrgaum村附近从雷云中坠落的铁质陨石,重约四磅,带黑色外壳。
对实践的启示
对AI应用开发者而言,展示了如何用廉价小模型做初筛、大模型精读、代理核对原始文档的流水线,以低成本处理海量非结构化数据;对研究者,AI可加速档案发现,但人的判断和验证仍不可或缺。
作者承认的反方与局限
作者强调这是AI辅助研究而非完全自主调查,自己始终参与引导方向、决定取舍,并指出AI模型可能自信地出错,因此每个结论都必须追溯到可查证的原始文档。
原文摘引
I think there's something honorable about adding to what we know.
Before you trust a search that finds nothing, you have to prove it can find something you already know is there.
This was AI-assisted research, not a completely autonomous investigation.
▸ 展开原文全文(25 段,抓取自 Hacker News)
How I used AI to investigate millions of historical records and surfaced a forgotten meteorite report, three lost rhinos, and unrecorded volcano eruptions.
Last week, on October 1st, 2026, the historian Benjamin Breen published a post called Using Opus 5.5 to discover a new eyewitness account of the dodo. He used AI to search digitized records from the Dutch East India Company. The Company dominated the spice trade and ran a string of ports across Asia from 1602 until it collapsed under its debts and was dissolved in 1799.
A project called GLOBALISE has turned millions of the Company’s handwritten pages into searchable text. Searching those transcriptions with AI, Breen found a 1615 ship’s journal in which sailors on Mauritius “caught many tortoises, dodos”. A new eyewitness record of the extinct dodo, sitting in plain sight for four hundred years.
I read the article and immediately thought, I can do this.
Not because I’m a historian, but because I’m a software engineer who happens to be very adept at piloting AI agents and creating agentic workflows. Breen’s post was the inspiration, and I wanted to extend the approach: I could search for many different historical mysteries in parallel, and I could chain together a few different AI models to filter results and automate a few of the cumbersome, time consuming manual steps.
But the appeal went beyond the technical challenge. I think there’s something honorable about adding to what we know. Adding even one new data point to a couple of niche fields felt like a small but worthwhile contribution to make. I thought it would be really cool if I could do that.
I powered on my small but capable home AI lab and got to work.
The Plan
I started by asking an AI “Deep Research” assistant a question: which open historical questions are most likely to be solvable with data that already exists online? It came back with thirteen candidates, ranked by how complete and accessible the data was, how much AI could help, whether anyone had already done it, and, most importantly, whether an answer could be checked against an original page. That last one matters more than it sounds. AI models can be confidently wrong, so every claim had to end at a real document: a scan of the actual handwritten letter or the actual printed newspaper, with an archive reference that anyone can look up and read for themselves. The list included animals in the Dutch East India Company archives, a giant volcanic eruption from 1808 that nobody has ever located, felt earthquakes in old Dutch newspapers, unrecorded meteorite falls, and ships that completely vanished.
Then I teamed up with Claude Code, Anthropic’s AI tool, and we invented this research pipeline together.
First came the reading material: the GLOBALISE transcriptions of the Dutch East India Company archive (4.35 million pages, from the 1600s to the 1790s), the Dutch national library’s digitized newspapers, two centuries of American newspapers, and a few ship logbooks for good measure.
If I sat down to read just the Dutch East India Company pages myself, at two minutes a page, eight hours a day, five days a week, it would take me about 70 years. And that’s before the newspapers. My homebrew AI lab got through the entire archive in a single twelve-hour overnight run.
The trouble with old documents is that nobody spelled anything the same way twice. Handwriting and old print come out of text recognition full of errors, and 17th-century Dutch spells “rhinoceros” about fifteen different ways. A plain keyword search would miss most of what I was looking for. That overnight run was my graphics card turning every passage into a mathematical fingerprint of its meaning, 5.7 million passages from the Dutch East India Company archive alone. That let me search for what a passage was about, not just which words it happened to use.
Even then, a single search could return tens of thousands of hits, far too many for a person, or even a big AI model, to read affordably. Instead, the first read went to a tiny, fast “System One” decision model called Jev, which answers only narrow questions: Is this a real animal? Is it wild? Where is it? It costs a few cents per million words. Having it read 59,000 mentions of elephants cost me about three dollars.
That’s the part of this project I’m proudest of, and it was my idea, not the AI’s. When I proposed using Jev as a filtering mechanism, Claude Code’s Fable model didn’t yet know what a System One type decision model was; I had to explain the idea before we could build the pipeline around it. In most research like this, the bottleneck is a person reading candidate passages one by one and deciding which are worth a closer look. Putting a cheap, fast judge in that seat automated the initial screening and freed me to focus on the strongest candidates and the direction of the investigation. The pages I examined closely had already survived two rounds of machine reading.
Only the passages Jev flagged moved on. Claude Haiku, a bigger model, read those few dozen closely, translated them and pulled out the dates and places. Then the Claude Code agent opened the scan of each original handwritten page to check the transcription against the original. Before calling anything new, I checked it against the catalogues the specialists themselves use.
A quick note on how AI was used in this project: I stayed actively involved in the process throughout and piloted the AI through the course of the entire project. I steered the investigation toward new questions and sources, decided which leads were worth pursuing, and recognized when I was grasping at straws and needed to move on. The systems could search and read at a scale I couldn’t, but deciding where to go next, or when to stop, still took my judgment. This was AI-assisted research, not a completely autonomous investigation.
There was one more rule, and it mattered most. Before you trust a search that finds nothing, you have to prove it can find something you already know is there. So before I went looking for anything new, the pipeline had to find Breen’s dodo, the Laki eruption of 1783, Tambora in 1815 and a dozen other known events.
It’s like testing a metal detector by burying your own wristwatch in the front yard. You know it’s there. If the detector can’t find it, you know you have a problem with your detector to address before sweeping elsewhere for real treasure. These known historical events were my buried wristwatch: a way to check that the pipeline could find something before taking its failures to find anything seriously.
The controls passed, and I began the real search. Shortly thereafter, my custom-built AI research pipeline started surfacing pages that may not have been read by anyone since the clerk who originally wrote them filed them away, stories that human eyes may not have read in hundreds of years.
The Meteorite History Forgot
The first real find of the project came from somewhere I didn’t expect: a newspaper printed in Batavia (today’s Jakarta) in the year 1812. Batavia sits on Java, the large island in what is now Indonesia that was the centre of Dutch power in Asia for nearly two centuries, and the newspaper was printed there during the few years (1811–1816) when Britain, not the Netherlands, ran the island.
On 19 December 1812 the English-language Java Government Gazette reprinted a letter from the Bombay Gazette of 26 August. An officer with a British force camped near Pandharpur, in what is now Maharashtra, wrote home:
“Captain M— is in possession of a great curiosity viz. a stone precipitated from a Thunder-cloud near the village of Cokurrgaum three days ago (the 6th August). It weighs I should think four pounds at least, is very heavy for its size, being greatly impregnated with iron, and coated with a thin black crust, as if Gunpowder had exploded around it.”
The thunder was heard “like a rustling fire of Musquetry for about half a minute.” The stone had buried itself