✦ ✦ ✧
Future-Ready · AI Literacy

When AI Gets It Wrong

Hallucinations & Fact-Checking
The model inside a chatbot doesn't look up facts — it predicts likely words. Search and citations help, but the sentence is still generated. So a confident answer is not the same as a true one.
📌
The six cases that never existed
June 2023: lawyer Steven Schwartz — 30 years of experience — researched six court cases with ChatGPT; his colleague Peter LoDuca signed and filed them with a Manhattan federal judge. Names, dates, docket numbers, quotes: all perfect-looking. All invented. Judge P. Kevin Castel ordered the two lawyers and their firm to pay $5,000 together.
🤖Why Chatbots Make Things Up
🚫MYTH: It looks facts up
"a book of true facts"
Not what happensThere is no encyclopedia inside. It isn't searching, copying, or checking anything.
🚫 Hallucinating isn't lying — lying means knowing the truth and hiding it.
🔮TRUTH: It predicts words
The court ruled ? one likely next word… then another… 🚀 autocomplete with rocket boosters
What really happensIt learned the patterns of language from mountains of text, then builds an answer one likely word at a time.
💡 That's why it sounds smooth — but "sounds likely" ≠ "is true".
🧾How a Perfect-Looking Fake Gets Built
① A court case citation has a very predictable SHAPE: Name v. Name , numbers year ← easy to imitate ② The model fills each slot with a word that SOUNDS likely: Varghese v. China Southern Airlines …and on it goes ③ Out comes a citation that is flawless — and completely made up: Varghese v. China Southern Airlines DOES NOT EXIST ✓ right format ✓ right legal words ✗ real case
A made-up answer delivered as fact is called a hallucination. It's not a rare glitch — it's a side effect of how the tool works.
⚖️Confidence ≠ Correctness
How CONFIDENT it sounds smooth · detailed · totally sure always full How ACCURATE it actually is ? ? ? ? ? ? ? ? ? ? ? ? unknown 🎯 The tone tells you NOTHING. Only checking does.
😬
The fake cases were already filed when Schwartz says his doubts set in — so he asked ChatGPT whether the cases were real. It cheerfully said yes, and claimed they were in trusted legal databases. Wrong again. Asking an AI to check its own work is like asking a student to grade their own exam.
📉It Keeps Happening
2023 Google's Bard wrongly credits the Webb telescope ≈ $100B market value lost June 2023 6 invented cases filed in a real courtroom $5,000 fine 2024 Air Canada's bot states a refund policy that isn't real BC tribunal, on these facts: CAD 812.02 mid-2026 1,600+ court rulings noting fake citations a routine hazard, not a bug
✅Your Defense: The Verify Habit
Run EVERY named fact an AI hands you through these four steps. 1 SPOT IT Circle every name,date, number,quote & title. 🔎 2 CROSS-CHECK Find it in 2+independent,reliable sources. 📚 3 OPEN IT Does the citationexist — and saywhat AI claims? 🔖 4 YOU DECIDE Weigh the evidence.The final truth callbelongs to a human. 🧠 ⚠️ Skipping steps 2 and 3 is what sank Schwartz — the checking is the step that saves you. "Independent" means the sources didn't get it from each other — or from the same chatbot.
🔍Trust Check: Where Do You Cross-Check?
✅ Reliable places to check
  • An encyclopedia
  • A library database
  • A trusted news site
  • A museum or university website
  • The original study, book, or expert
🚫 Not a real check
  • Asking the same AI to confirm itself
  • Another chatbot's answer
  • A random comment under a viral video
  • A meme a friend texted you
  • "It sounded confident and detailed"
🧰AI's Real Job Description
👍 Brilliant at
Drafts · brainstorms · outlines · practice questions · explaining an idea a new way. A thinking partner that helps you start.
👎 Not for
Being your encyclopedia · writing your final essay · doing your quiz · being your final answer. Never let it be the last word.
🔑Key Terms
🫧Hallucination When an AI states made-up information as confidently as if it were a true fact.A student who'd rather invent an answer than admit being unsure — even though "I don't know" was allowed.
✅Verify To make sure a claim is actually true by checking it against trustworthy information.A guard scanning every "fact" at the door of your brain.
🔀Cross-check To compare a fact across two or more independent sources to see if they agree.Ask three friends who weren't together — do their stories match?
🏞️Source The place information originally comes from — a book, website, study, or expert.Follow the river upstream to the spring where it starts.
🔖Citation A note telling readers exactly where information came from, so they can check it themselves.A treasure map pointing to the exact page of the exact book.
🌉Reliable Describes a source with a strong track record of being accurate, careful, and honest.A bridge you've crossed a thousand times that never wobbled.
🤨Skepticism The healthy habit of asking "how do we know that?" before believing a claim.A scientist raising one eyebrow: "Interesting. Show me the data."
🌍Where This Shows Up in Real Life
⚖️Law firms
After a public tracker logged more than 1,600 court decisions worldwide where judges found or implied that someone leaned on AI-invented material — a floor, not a census — many judges now require lawyers to certify that a human verified every case they cite. The verify habit is literally written into the job.
📰Newsrooms
Professional fact-checkers trace every name, number, and quote back to an original source before a story is published — the exact cross-checking habit from this lesson, performed on deadline every single day.
🤯 Wild but true

• Both Cambridge Dictionary and Dictionary.com named "hallucinate" their 2023 Word of the Year, adding a brand-new definition just for AIs that confidently make things up.

• Air Canada argued it shouldn't be responsible for what its own website chatbot said — as if the bot were a separate legal entity answering for itself. British Columbia's Civil Resolution Tribunal called that a remarkable submission, found negligent misrepresentation on that customer's facts, and ordered CAD 812.02. It never examined how the bot was built, and set no worldwide rule.

• Bard's one wrong sentence about the James Webb Space Telescope may be the most expensive sentence an AI has ever written. (The first photo of a planet outside our solar system was actually captured in 2004 by a European Southern Observatory telescope.)

⭐
Remember this:
  • AI predicts likely words instead of looking things up — so it can state pure inventions with total confidence. That's a hallucination.
  • Confidence is not correctness, and an AI cannot reliably check its own work.
  • Use AI for drafts and brainstorming, but cross-check every named fact in an independent, reliable source. The final truth call always belongs to a human.
✏️ ClickClass Anchor Chart · When AI Gets It Wrong: Hallucinations & Fact-Checking
From ClickClass — hundreds of free printables at clickclassedu.com/printables