← Back to overview
Module 1 · What the tool does with your text

What the tool is, and what it isn't

🕑 5 min read
Goal of this lesson: You understand that you are working with a text predictor rather than a knowledge base, and why that explains everything that goes wrong later.

You are probably already using an AI tool at work. Having a text rewritten, a long document summarised, an email put into English. It often works surprisingly well, and that is exactly why so little thought goes into it.

This course is not about everything you can do with it. It is about the four questions underneath: where do my words end up, is what I get back correct, whose is it, and what do we agree on here. Everything in this course traces back to what kind of thing such a tool actually is. So that is where we start.

It guesses the next word

A language model is trained on enormous quantities of text. What it has learned is which words tend to follow which others. When you ask a question, it does not look up an answer — word by word, it assembles the text most likely to follow your question.

That sounds like a technicality, but it explains almost everything else in this course. An answer that is correct and an answer that is invented are produced by exactly the same process. There is no step in which the model checks whether what it is writing is true.

Think of someone who has read an enormous amount and speaks very fluently, but looks nothing up and never says they don't know. You have probably met people like that. You use them to think out loud with, not as a source.

So: not a knowledge base, not a calculator, not an archive

Three things follow from that one property. It is not a knowledge base: it looks nothing up in a reference work, not even when it cites sources. It is not a calculator: to a language model, numbers are simply text, and a sum that looks right can be plain wrong. And it is not an archive of your organisation: what it says about your company is inferred from what has been written about companies like yours in general.

Some tools are now built more cleverly: they really do search the web, or work out a sum in a separate program. That helps, but it does not change the core. The text you end up reading is still predicted — including the sentence describing what the source supposedly said.

Why it always sounds certain

The hard part is not that an AI tool makes mistakes. The hard part is that you cannot tell from the answer whether it contains one. A colleague hedges: 'I think that's how it works, but do check.' A language model writes the most likely text, and the most likely text is a fluent, confident answer.

So you get no signal telling you when to pay attention. That means you have to build in a moment of checking yourself. Module 2 covers that, and that routine is the single most important habit in this course.

What this means for your work

The practical conclusion is simple: use an AI tool for work where you can judge whether the result is right. Rewriting a text, summarising a document you have read yourself, producing a first draft that you then adjust — there you are the expert and the tool is the instrument.

The moment you ask for something you cannot judge yourself — a date, an amount, a clause in an agreement, the name of a supplier — the roles change, and you have to verify it elsewhere. Not because it is bound to be wrong, but because you cannot tell from the answer whether it is right.

Remember: An AI tool predicts text; it knows nothing. That is why an invented answer sounds exactly as convincing as a correct one — and why the checking has to sit with you, not with the tool.
Try it yourself
  1. Open the AI tool you use at work (or, if you don't use one yet, any free chatbot).
  2. Ask something you know the answer to for certain and that few people will have written down. Pick something public — what your organisation does, which town you are in, how many sites there are — and not something confidential; lesson 1.3 shows exactly where that line runs.
  3. Look not at whether the answer is right, but at the tone. Does it say anywhere that it isn't sure? Does it sound any different from an answer that is correct?
  4. Write down in one sentence what struck you. You have just seen the problem that module 2 offers a solution for.
  5. Keep that sentence. It comes back in lesson 5.3, in your own working agreement.
Stuck?
  • My tool did give a source, so it must be right? Not necessarily. A citation is predicted text too. Some tools genuinely look things up, but even then you have to open the link yourself and check that it says what is claimed. Lesson 2.1 shows you how to do that quickly.
  • I usually get good answers, isn't this overdoing it? That you usually get good answers is true, and it is precisely the risk. You get used to not looking, and that is when the one wrong answer slips through.
  • Does this mean I shouldn't use it? No. It means you use it for work whose outcome you can judge. That covers a very large part of your job — module 4 is about which part exactly.
  • What if I use a newer or more expensive version? It makes fewer mistakes on average, but it still makes them, and just as convincingly. So the checking stays.
Check yourself
  • What does a language model do when you ask a question — and what does it not do?
  • Why can't you tell from an answer whether it contains a mistake?
  • Why is an AI tool not a reliable calculator, even when the answer is full of numbers?
  • For what kind of work are you the one who can judge the result, and when is that not the case?
Create a free account to track your progress. Create a free account