Day 1 · Get started without getting burned

What AI actually is, and what it is not

Predict what your AI tool will get wrong before it gets it wrong.

12 min readPlus one task, about 10 min

A hand and a simple machine both reaching toward one speech bubble filled with a flowing sequence of dots. AI-generated illustration
A request goes in and a stream of plausible words comes out. Nothing in the middle looks anything up.

The one idea

It predicts the next word. That is the entire mechanism, and almost everything surprising about these tools follows from it.

It does not look anything up. It does not check its answer against a source. It does not know when it is wrong, because nothing in the process ever compares what it wrote against what is true. It has read an enormous quantity of text and learned which words tend to follow which other words, and it produces a continuation that fits.

That is not a small trick. It is why the thing can write, summarise, translate, explain and rewrite. It is also why it will invent a citation, a phone number or a court case with complete fluency, and why it will do so in the same confident voice it uses when it is right.

Why it matters

Nearly every beginner disappointment comes from expecting a database of facts and getting an extremely good writer instead.

Once you hold that one sentence in mind, the odd behaviour stops being mysterious. The invented citation is not a bug in a knowledge system; it is a writing system doing exactly its job. The confident wrong answer is not a lie; confidence is a style of writing, and the tool has no other style.

It also tells you what the tool is genuinely good at, which is more than you might expect. Anything where the shape of a good answer is clear and the facts are supplied by you — rewriting, restructuring, explaining, drafting, translating, summarising, converting a mess into a table — it does extraordinarily well. Anything where it has to supply the facts from memory is where it quietly falls apart.

The tools you will use

What happens when you press Send

  1. Your sentence is chopped into tokens. A token is often a whole word, sometimes a fragment of one. The tool never sees letters; it sees a list of numbers standing in for those pieces.
  2. Those numbers pass through a network with hundreds of billions of learned settings. The settings are not rules anybody wrote down. They are the residue of reading an enormous amount of text and adjusting, over and over, so that the next word is guessed a little better.
  3. The network outputs a score for every token in its vocabulary — how likely each one is to come next. It picks one, weighted by those scores, so the same question can produce a different answer twice.
  4. That token is appended and the process repeats, with the growing text as the new input. A four-hundred-word reply is that loop running four hundred times.
  5. At no point does anything consult a source, and at no point does anything check the result. The only thing being optimised is whether the next word fits — not whether it is true. Hold on to this: it is the single fact that explains the rest of the course.
    Prediction, not knowledge.
    Fluency, not accuracy.
    It sounds right because sounding right is the only thing it was built to do.

See it

Ask it about your own street. It will describe it warmly, plausibly and partly wrongly — the right kind of shops, the wrong specific ones.

Nothing is broken. It was never asked to know your street. It was asked to produce a description of a street of that sort in that sort of town, and it did, and it did it well. The error is not in the machinery. It is in the expectation you brought to it, and that expectation is the thing this course removes.

The same thing happens with money. Ask what a plumber charges in your city and you will get a confident range that may be two years out of date or simply invented from the pattern of similar answers. Ask it to restructure a quote you already have, and it will be flawless, because you supplied the facts.

Your first reliability baseline

You are going to catch it being wrong on ground where you are the expert. Everything else in this course is built on that experience — being told it makes things up is forgettable, finding three invented facts yourself is not.

  1. Open any AI tool and start a new chat. Name it something you will find again — the rename option is usually near the top of the thread.
  2. Ask these three questions. They must be things you already know the answer to. That is the entire point of the exercise; a question you cannot check teaches you nothing.
    1. What are the main industries in [your town or city]?
    2. What does [your job or business type] typically charge in [your country]?
    3. Who is the [mayor / MP / council head] of [your town]?
  3. For each answer write one word in a notes app: correct, wrong, or confident-but-wrong. The third category is the one that matters. Do not skip this — the written record is what you compare against on Day 14.
  4. Push back on the weakest answer with exactly this. Watch how it responds, because the response is more revealing than the original answer.
    How confident are you in that? Rate your confidence out of 10 and tell me what you are uncertain about.
  5. Save the three-word result somewhere permanent. This is your baseline, and it is the first page of the toolkit you finish the course with.

What you should notice

Most people find at least one confident-but-wrong answer, and the confidence rating does not track accuracy at all — it will happily give you 8 out of 10 on something it invented ninety seconds ago.

That is the lesson. Confidence is a writing style, not a signal. Once you have seen it happen on a subject you know well, you will start spotting it on subjects you do not, and that is the beginning of using these tools safely.

Notice something else too: the tool did not hesitate, and it did not hedge. A person who did not know your town would say so. This one cannot, because "I don't know" is not a prediction it is especially likely to make — it is a much less common continuation of a confident question than a confident answer is.

A confidence dial with its needle near maximum, beside three answer cards of which one is marked wrong. AI-generated illustration
The dial and the tick marks are independent. High confidence is the default setting, not a measurement.

If you have five more minutes

Close the chat and ask question 3 again in a brand new thread. If the answer changes, you have found the second thing: it does not remember, it re-guesses. Same question, same model, two different towns.

Tomorrow

The tool needs to be one tap away, or you will not use it.