TL;DR
An LLM — large language model — is the software behind tools like ChatGPT and Claude. You give it text, it gives you text back. It got good at this by reading an enormous amount of writing, so it's fluent on almost any topic — but it only knows what it read, and it can sound confident while being wrong. Everything else you'll build in this lab — tools, memory, agents — sits on top of that one text-in, text-out engine.
The idea: text in, text out
An LLM does one thing: you give it some text, and it writes text back. That's the whole interface. Ask a question, it writes an answer. Paste a paragraph, it writes a summary. Every time, it's the same move — read the text it's given, then write text that fits.
How did it learn to do that? By example. It was trained on a huge amount of writing — books, articles, websites, code — and picked up the patterns in how people put words and ideas together. Nobody handed it a database of facts to look up. It learned what tends to follow what. That's why it can write fluently on almost anything, and even answer a question no one wrote out in advance: it's working from patterns, not reciting stored answers.
An LLM has one job: read the text it's given, and write text back.
Analogy
Think of it as autocomplete that has read a huge slice of the internet. Your phone suggests the next word from your texting habits; an LLM continues your text from patterns it picked up across a vast amount of writing. It isn't looking anything up in the moment — it's going with what fits.
Two things this means for you
The model learned from text that was collected up to a certain date, and it learns by fitting patterns rather than storing facts. Two consequences follow — and both shape how you'll build with it:
- It has a knowledge cutoff. It only learned from material up to a point in time, and it has never seen your private documents. Ask about something newer or internal and it simply doesn't know. If you need it to use that information, you have to include the text in what you send it — which is exactly what tools and knowledge bases (later modules) are for.
- It can be confidently wrong. It writes text that fits the patterns it learned; it isn't checking a source. Usually that lands on the right answer — but sometimes it produces something that sounds right and isn't (people call this a hallucination). Treat its output as a fluent first draft, not a verified fact.
Common mistake
Don't assume the model "knows" anything you didn't give it. If your answer depends on current events or private data, you have to put that text into the prompt — that's exactly what tools and knowledge bases (later modules) are for.
Key takeaways
Key takeaways
- An LLM is software that takes text in and writes text back — that's its whole interface.
- It learned by reading an enormous amount of writing, so it's fluent on almost any topic.
- It only knows what it read, up to a cutoff date — for anything newer or private, you supply the text.
- It writes text that fits rather than looking facts up, so it can be confidently wrong.
Optional — how 'write the next bit' becomes a whole answer
The model doesn't write a whole answer in one shot. It writes a little at a time — a few characters, usually a word or part of one. It looks at all the text so far, adds the next bit that fits best, then looks again and adds the next, over and over. Do that a few hundred times and you get a full sentence, paragraph, or essay.
Two things follow from working this way:
- It writes start to finish, left to right — which is why you often see an answer stream in piece by piece rather than appearing all at once.
- There's a little chance in what it picks next, so the same question can come back worded differently each time. You'll get a dial to control how much that varies later in this module.