The Find/Work Test: Why AI Invents Records for Family Historians
AI invents records because of the request, not the tool. The Find/Work Test is one question family historians ask first: find something, or work on it?
AI invents records for family historians because of the request, not the tool. The Find/Work Test is one question to ask before typing anything: am I asking it to find something, or to work on something I already have? A find request leaves the tool nothing to answer from. A work request gives it a real document, and gives you something to check the answer against.
I learned this the expensive way. In 2023 I emailed a publisher asking about a book that had never been written. AI had given me a title, an author, a publisher and a year, specific enough that I went looking. They wrote back, patiently, to say no such book existed.
I told the whole story in AI Did Not Make Me a Genealogy Genius. This page covers what the test is, what each answer means, and why better tools have not made it unnecessary.
What the Find/Work Test is
The Find/Work Test is a single question asked about a request before it goes to an AI tool. It sorts every request into one of two kinds.
A find request asks the tool to produce something it does not have. Find me a source on this topic. Find the record that proves this.
A work request hands the tool material and asks it to do something with it. Here is this deed, so read it. Here are three documents, so tell me what connects them.
The test takes about two seconds, and it is the cheapest quality control available in AI family history research. It is also the first question of Assess in the AIM Framework, where I call it a finding problem or an understanding problem.
What each answer means
If the answer is find, close the tool and go to the records. Searching an actual index, catalog or archive is the work a find request was standing in for.
If the answer is work, you are about to get something useful. Everything the tool says can be held up against a document sitting in front of you.
The cases that feel in-between are almost always find requests wearing work clothes. Tell me more about this family sounds grounded because a family was mentioned, but nothing was handed over. Here is the 1850 census page with this family on it, so tell me who is in the household based on the census law for 1850 is work.
Why find requests produce invented records
A generative tool is not a search engine, and there is no database of information it is pulling from. It is not connected to the records and it is not looking anything up.
So when a request asks AI to find, the only thing available to it is to generate something shaped like an answer. It is extraordinarily good at that. My invented book had an authoritative title, a real-sounding author and a plausible publisher, because those are exactly the features a real citation has.
A bad answer that looked bad would be harmless. This one looked like the best research day I had had in months, right up until I checked.
I was not being careless, and neither is anyone this happens to. I was following up and asked ChatGPT to find a book on the topic. It wasn't until I emailed the publisher to purchase the book that I realized there was a problem. Many of us are making the same mistake every day.
Why better AI models did not fix it
Three years on, the tools are better, and they cite more and hedge more. Family historians are still being led into invented records anyway.
The tool changed. The request from genealogists did not. A find request asks a generative tool to do a job it does not do, and it will keep doing that job badly however good it gets.
"It's better now" sends a family historian straight back to asking a tool to find an ancestor. The answer will be beautiful, and it will fall apart in 2026 exactly as it did in 2023.
A work request still gets checked
Passing the test does not make an answer right. It makes the answer checkable.
I handed a tool two census pages for the same household. A wife listed in the first was missing from the second, and without being asked, the answer explained that the couple had divorced.
It happened to be true. But the marital status column did not say so. That was the tool's own theory of why she was gone, set down as if the record said it.
Because I had handed over the records, one question caught it: show me where in the record it says that. The record is the check. Holding every answer up against its record is the habit underneath the Research Quality Check, which I run before trusting research enough to write from it.
What the test is not
It is not a skill to develop, and there is nothing to get good at. It is one question, asked before typing.
It also does not finish the research. It tells you whether an answer has something underneath it. Whether the whole body of work on an ancestor is sound enough to stand is a separate question, and the Soundness Standard is where I answer it.
Ask the question before your next request. If the answer is find, the records are waiting. If it is work, you are holding exactly what the tool needs, and giving AI a job is what to do with it. The rest of my notes on AI for genealogy live there.
Questions about the Find/Work Test
Why does AI make up genealogy records?
A generative tool is not connected to any records. Asked to find something, it produces something shaped like an answer, with plausible titles, authors and dates. And of course, it can absolutely create fictional historical records.
Does a work request mean the answer is correct?
No. It means the answer can be checked. Ask where in the record each claim appears.
What should I do when the answer is find?
Go to the records. Search the actual index, catalog or archive.
