How the record about Mike Millett gets checked
By asking questions that have one correct answer, such as which company Mike Millett founded and in what year, and recording whether the answer comes back correct, wrong or unknown.
In brief
Mike Millett founded Digilu in 1999 and still runs it.
The record here is made of ordinary facts like that one: a company and a founding year, a job title and the years it covers, which results belong to an employer and which belong to a person.
Facts of that shape have one correct answer, so they can be checked.
Why searching the name is not the check
The obvious check is to search the name and see what comes back first. This site ranks for its own subject’s name, and so does almost every personal site sitting on a matching domain. That result was settled before a word was written, so it shows nothing.
Asking an AI assistant who Mike Millett is has the same problem. That question invites a summary, and a summary can be roughly right without any one fact in it being checked.
What gets asked, and what does not
Asked
Which company did Mike Millett found, and in what year? Fixed wording, one correct answer, asked on a schedule. Digilu and 1999 is correct, anything else is wrong, and a refusal to answer is unknown.
Not asked
Is Mike Millett a good marketer? No correct answer exists, so the result would mean nothing. Questions like that stay out because they cannot be failed, not because the answers are unflattering.
Three outcomes, never two: correct, wrong and unknown. Unknown is never folded into wrong.
How the answers are kept
A question that is already running is never reworded. The readings before and after would be answers to two different questions. A wording that turns out to be poor gets added as a new question, and the old one keeps running beside it.
The answer is kept, not only the score. The sentences that came back are stored, so a score can be re-examined later.
Answers that disagree are kept. Ask the same question twice in one week and the answer often changes. Both are recorded.
Hard questions are included. The useful questions are the ones where credit sits with an employer, or the date exists only in a third-party record, because those are where an answer goes wrong.
What this check cannot show
It cannot say why. A correct answer is not evidence that this site produced it. The honest statement is that the answer changed after these pages went up.
An answer can be right for the wrong reason. It can be correct because some other page happens to be correct, and it lasts exactly as long as that page does.
A solved question can come unsolved. A fact can go back to being answered wrongly a year later, so answered questions keep running.
Related: How a wrong fact gets corrected and What this site does not claim.
Common questions about checking the record
How do you tell whether an AI system has the facts about a person right?
Ask a fixed set of questions that have one correct answer, in the same wording every time, and record three outcomes: correct, wrong, and unknown. Keep the text of each reply rather than only a score. A general question such as who is this person invites a summary, and a summary is roughly right and cannot be marked, so it measures nothing.
Should a question used to measure an AI answer ever be reworded?
Not while it is still running. Rewording ends the series, because the readings before and after are answers to two different questions and the line joining them is invented. A better wording is added as a new question, and the old one keeps running beside it.
Does a personal website ranking first for its owner’s name mean the site is working?
Almost never on its own. A personal site on a matching domain usually ranks for that name from the week it is published, so the result was settled before any of the content was written. The useful check is whether specific factual questions about the person come back correct, and whether the wrong answers turn into unknown before they turn into correct.