Pathway 1 · Lesson 0145 min

How Generative AI Works

LLMs, tokens, and the mechanics of intelligence

Learning Objective

By the end of this lesson, you will be able to: Explain how Large Language Models generate text, define tokenisation and hallucination, and describe why AI outputs require human verification.

74%

of leaders cite AI inaccuracy as their top risk

Engage

"If AI is so powerful, why does it sometimes confidently make things up? Understanding the answer changes how you use it forever."

Explore

A clear, visual explanation of how LLMs are trained and how they generate text — from 3Blue1Brown. Watch this before reading on.

Explain

The Core Mechanic: Next-Token Prediction

Every AI response you've ever received — from ChatGPT, Gemini, or Claude — was generated one token at a time. A token is roughly 3–4 characters (about ¾ of a word). The model doesn't 'think' about your question. It predicts: given everything I've seen so far, what token is most likely to come next?

This happens thousands of times per response. Each token is chosen based on statistical probability learned from training on billions of text documents. The result looks like reasoning. It isn't.

📖

Tokenisation

The process of splitting text into smaller units (tokens) that the model processes. 'Artificial intelligence' becomes approximately ['Artif', 'icial', ' intel', 'ligence']. Token limits affect cost, speed, and how much context the model can hold at once.

Explain

Training: Where the 'Knowledge' Comes From

LLMs are trained on enormous datasets — books, websites, code, academic papers — scraped from the internet. During training, the model adjusts billions of internal parameters to get better at predicting the next token.

What this means for you:

The model has a knowledge cutoff — it doesn't know about events after its training ended

It has absorbed biases present in its training data

It has no access to your organisation's internal documents (unless you provide them)

It cannot browse the internet unless given a specific tool to do so

Explain

Temperature: Controlling Randomness

Temperature is a setting that controls how 'creative' or 'random' the model's outputs are:

Low temperature (0–0.3): The model picks the most statistically likely token every time. Outputs are consistent, factual, predictable. Use for: analysis, summaries, data extraction.

High temperature (0.7–1.0): The model samples from a wider range of probable tokens. Outputs are more varied, creative, surprising. Use for: brainstorming, creative writing, ideation.

⚠️

Hallucination: The Most Important Thing to Understand

Hallucination is when an AI generates plausible-sounding but factually incorrect information — with full confidence. It's not a bug. It's an inherent consequence of next-token prediction. The model doesn't 'know' facts; it predicts what text would plausibly follow. This is why you must verify every factual claim, citation, and statistic an AI produces.

Elaborate · Worked Example

Real-World Example: The Confident Wrong Answer

Context

A lawyer asked ChatGPT to find case law citations for a legal brief.

The Situation

ChatGPT produced six detailed case citations — complete with court names, dates, and case numbers. They looked completely authentic.

The Outcome

None of the cases existed. The AI had hallucinated entirely fabricated citations. The lawyer submitted them to court and faced sanctions. The lesson: polished formatting is not evidence of accuracy.

Key Concepts at a Glance

Concept	What It Is	Why It Matters for You
Token	~3-4 characters; the unit AI processes text in	Token limits affect how much context you can provide
Temperature	Controls randomness (0=consistent, 1=creative)	Set low for analysis, higher for brainstorming
Context Window	The AI's working memory — max tokens per conversation	Exceed it and the AI 'forgets' earlier instructions
Knowledge Cutoff	The date training data ends	AI doesn't know recent events — you must provide them
Hallucination	Confident generation of false information	Always verify facts, citations, and statistics

💡

The Autocomplete Analogy

Think of LLMs as 'autocomplete on steroids.' Your phone's autocomplete predicts the next word based on what you've typed before. LLMs do the same thing — but trained on the entire internet, with billions of parameters, predicting thousands of tokens in sequence. The outputs are remarkable. But the mechanism is statistical prediction, not comprehension.

Try It · Reflect & Apply

Reflect & Apply: Think of a time you used AI and got an output that seemed wrong or surprising. Based on what you've just learned about next-token prediction and hallucination, what do you think happened? Write your answer below.

Your notes are saved in your browser for this session.

Evaluate · Knowledge Check

1 / 3

What is the fundamental mechanism by which LLMs generate text?

Fluency Practice Studio

AI Reality Check Studio

Make the decision

Decide whether the task requires a creative draft, a careful factual answer, or a human-only judgment before you open an AI tool.

Practise at work

Choose one real output you use at work. Ask AI for a draft at a low temperature, then identify three claims that need an independent source check before you could share it.

Quality check

Can you explain which statements are generated prediction rather than verified knowledge? If not, treat the output as a draft, not a source.

Spaced revisit

In 48 hours, notice one AI output that sounded convincing. Pause and label what you verified, what you inferred, and what you still do not know.

External video references

Keep Learning on YouTube

Open in a new tab

How Large Language Models WorkEmbedded lesson videoA clear, visual explanation of how LLMs are trained and how they generate text — from 3Blue1Brown. Watch this before reading on.Watch · Watch on YouTube

Ready to move on?

Mark this lesson complete and continue to "The AI Landscape".