Learn by clicking · ~4 minutes · Day 26 of 30

How to spot bad AI output.

Bad AI output means an answer that sounds right but is not: a made-up number, a wrong date, a citation that does not exist. The polished name for this is a hallucination (the model stating something false as if it were fact). Here is the one idea that makes it spottable:

How it sounds

fluent, confident, specific, detailed

≠is not
the same as

Whether it is true

names, numbers, dates, sources: checkable

🗣️ A model predicts likely words, not verified facts 📈 Confidence is a writing style, not evidence 🔍 The fix is a habit: verify the checkables

Think of it like a very smooth talker at a party. They are charming and they sound like they know everything, so you take their word for it. Sometimes they are right. Sometimes they confidently invent a fact. Your job is not to be impressed by the delivery. It is to check the parts that can be checked.

02 / 05 · The myth

"If it sounds confident and detailed, it must be right."

This is the single most expensive belief people hold about AI. Let us name it as a myth, then correct it.

The myth: a fluent, specific, confident answer is a reliable answer. The more detail it gives (exact figures, dates, quotes), the more we trust it.

The reality: detail and confidence are the easiest things for a language model to produce. It generates the most plausible-sounding next words. "Plausible" and "true" overlap a lot, but they are not the same thing, and the gaps are exactly where mistakes hide.

🎭 What confidence is

A property of the writing.

– Fluent grammar and a sure tone
– Specific-looking numbers and dates
– Named sources, real or invented
– Costs the model nothing to fake

✅ What correctness is

A property of the world.

✓ The number matches reality
✓ The date is the real date
✓ The citation actually exists
✓ You can verify it independently

The key: a confident wrong answer and a confident right answer look identical on the surface. So you cannot tell them apart by reading harder. You tell them apart by checking the checkable parts: names, numbers, dates, citations, and anything legal or financial.

03 / 05 · Spot the error

Watch it work. Catch the fabrications.

Below are real-shaped AI answers, each with subtle inventions hidden inside. Click any underlined phrase that looks off. We will tell you if you caught one, and why it is wrong.

🔒 This game runs entirely in your browser. Nothing is typed, sent, or stored anywhere. The "answers" are canned examples, not live AI.

💡 The pattern: the fabrications are the most specific bits (a stat, a date, a named source). That is the tell. Specificity is where a model is most likely to confidently invent.

04 / 05 · Make it a habit

5 checks before you trust an AI answer.

You do not need to fact-check every word. You need a quick reflex on the parts that matter. Run these whenever the answer feeds a real decision.

1. Verify the checkables. Names, numbers, dates, and quotes. The more specific a claim, the more worth confirming.
2. Ask "what is your source?" Then open the source yourself. Invented citations look real until you click them.
3. Cross-check against your own docs. Feeding the AI your real files (this is called RAG, retrieval-augmented generation: the model answers from your documents instead of memory) reduces hallucination. It does not eliminate it, so you still verify.
4. Be extra slow on legal, financial, or medical claims. A wrong number in a contract or invoice is not a typo, it is a liability.
5. Keep a human on anything that matters. AI drafts; a person approves before it ships, sends, or pays.

This is the hands-on companion to the deeper safety lesson. For the full picture of guardrails and review, see Day 24: AI safety →

05 / 05 · Done

You now read AI output more carefully than most people who use it daily.

You know that confidence is a writing style, not proof. You can name the myth and bust it. And you have a five-check reflex for the parts that actually matter: names, numbers, dates, citations, and anything legal or financial.

Spotting bad output by hand works. The stronger move is building AI that has less to fabricate: grounded in your real data, with checks and a human in the loop where it counts. That is what we build, end to end.

We build AI grounded in your real data with checks in place. Have Rabbithole build it →

Next: the deeper AI safety lesson →

Day 26 of 30 free, working AI lessons and kits for small business.

↩ All 30 lessons See it doing real work: RAG kit → Lead-catcher kit → Quote-bot kit →