How it was built4 of 11
Why it sounds sure when it is wrong
These systems are built to sound right, and sounding right and being right are separate things it was never taught to tell apart.
How to explain it, by age
Same idea every time. What changes is how much of the machinery a kid can hold, so pick the version that fits and go up a band when it stops being interesting.
Ages 4-6
Keep it to guessing again. "Sometimes it guesses wrong, but it still sounds very sure. It never says oops." Little children find a confident wrong grown-up extremely funny, which is a useful place to start.
Try this
Ask it something about your family that it cannot possibly know, like the name of your cat. Let her hear it make something up. Laugh about it.
Listen for
Her laughing rather than believing it. At this age the goal is that the machine is fallible and slightly silly.
If she already has more access than most. If she asks it questions already, agree a rule: anything important gets checked with a grown-up.
Ages 7-9
Name the mechanism. "It was taught to give answers that sound good. Nobody ever taught it the difference between an answer that sounds good and an answer that is true, because it has no way to check. So when it does not know, it does not stop. It produces something that looks like an answer."
Try this
Ask it for the plot of a book that does not exist. Invent a title. It will usually produce a detailed summary of a book nobody wrote. That single demonstration is worth an hour of explanation.
Listen for
Her wanting to test it herself. Let her, and let her get good at catching it out.
If she already has more access than most. Introduce the habit of asking where an answer came from, and noticing that the source it names may also be invented.
Ages 10-12
Introduce the training incentive. "It was tuned by people rating answers as good or bad. Answers that seemed helpful and confident got rated higher. Nobody could check every fact, so what actually got rewarded was seeming right. That is why it would rather produce something than admit it does not know, and why it will invent a source that looks exactly like a real one."
Try this
Have her ask for three sources on a topic she knows well, then check whether each exists. Fake citations are the clearest possible illustration, because they look completely correct until you look.
Listen for
Her distinguishing "I checked this" from "it sounded right". That habit is the whole point of this concept.
If she already has more access than most. Point out that this gets harder rather than easier as models improve, because the errors get rarer and therefore less expected.
Ages 13+
Go to the structural problem. There is no internal signal that separates generation from recall, so the system cannot flag its own uncertainty in the way a person can. Add scale, and the internet fills with confident synthetic text, some of which becomes training material for the next model. That feedback loop is a real and unresolved problem in the field.
Try this
Have him find a topic where he genuinely knows more than average, and probe until the answers degrade. Expertise is the only reliable detector, and noticing that is itself the lesson: he cannot evaluate what he does not already know.
Listen for
The uncomfortable conclusion that these tools are least reliable exactly where he is least able to notice. That is the correct and unsettling takeaway.
If she already has more access than most. Discuss what it would take to build a system that could say "I do not know" honestly, and why that is much harder than it sounds.
Why this is the most practical concept here
Your child will meet this within a week of using any of these tools, probably for homework. If they do not have this concept, the confident answer wins, because confident answers have always been a reasonable proxy for correct ones.
That proxy has now broken, cheaply and at scale, and no previous generation had to unlearn it.
Make catching it out a game
The tone matters more than the content here. Approached as a warning, this becomes another adult telling them technology is bad, and it slides off.
Approached as a game where the aim is to catch the machine being wrong, it becomes something they do voluntarily and enjoy. Kids are extremely good at this and take real pleasure in it. That pleasure is doing the work, because a child who has personally caught a confident machine inventing three books will never again read fluency as proof.
Misconceptions to address
- That it lies. Lying needs an intention to deceive. This is a machine producing what tends to follow, with no mechanism for knowing whether it is true.
- That it will say when it is unsure. It can produce the words "I am not sure", and that is also just predicted text rather than a report on its own confidence.
- That the errors are rare or obvious. The dangerous ones are small, plausible and buried in otherwise correct answers.
- Adults fall for this harder than children do, because fluent confident prose reads as authority to anyone who grew up before it was cheap to produce.
Running this with a class
Ask the class to have a model produce five facts about your school or town. Verify each one together against a real source. The mixture of correct and invented material in a single confident answer is the lesson, and it lands far better than being told to check sources.
Last reviewed August 2026