If you want to know how large language models work, the shortest honest answer is this: they are very large mathematical functions that were trained to guess the next small piece of text, over and over, until that guessing became surprisingly useful. Everything else, the conversation, the code, the poems, grows out of that one training task plus a lot of engineering. The details are less magical than the marketing and stranger than the skeptics suggest. This article walks through the pieces in plain language and marks where researchers still disagree.

What is a large language model, in plain terms?

A large language model, or LLM, is a neural network trained on enormous amounts of text. A neural network is a stack of numerical calculations whose behavior is set by billions of adjustable values called parameters. The model does not store a library of sentences. It stores numbers that, when text is fed through them, produce a probability for what could come next.

ChatGPT is one product built on this kind of model. The underlying GPT series was first pretrained to predict the next word and then adjusted to behave like an assistant. Other chatbots from other companies follow broadly the same recipe, even where the details differ.

How does a model turn words into numbers?

Computers handle numbers, not letters, so the first step is tokenization. Text is chopped into tokens, which are chunks that might be a whole common word, a piece of a rarer one, a punctuation mark or a single character. A widely used method called byte-pair encoding builds the vocabulary by repeatedly merging the most frequent neighboring pairs of characters until the vocabulary reaches a chosen size.

Each token receives an integer, and each integer maps to an embedding: a long list of numbers that places the token in a kind of coordinate space. Tokens used in similar ways end up with similar coordinates. This is why a model can treat “doctor” and “physician” as relatives without anyone writing that rule down.

One consequence surprises people. Because models see tokens rather than letters, simple tasks like counting the letters in a word can trip them up. The model never saw the word as a row of letters.

What is the transformer, and what does attention do?

Almost every modern LLM is built on the transformer, an architecture introduced in a 2017 paper from eight Google researchers titled “Attention Is All You Need.” It was first aimed at machine translation, and it beat earlier systems at English-to-German and English-to-French while costing less to train.

The key idea is attention. Earlier designs read text one word at a time, passing a running summary forward. A transformer instead lets every token look at every other token in the passage and decide how much each one matters for interpreting it. In the sentence “The trophy didn’t fit in the suitcase because it was too big,” attention is how a model can weigh whether “it” points to the trophy or the suitcase.

Mechanically, each token produces three vectors, usually called a query, a key and a value. The query of one token is compared with the keys of the others to produce weights, and those weights blend the values. Models run many of these operations side by side, called heads, so different heads can track different kinds of relationships. Since a transformer has no built-in sense of order, position is added separately with a mathematical encoding.

The big practical win was parallelism. Transformers process all tokens at once, which suits graphics processors, and that is what made it feasible to train much larger models. The 2017 paper has since been cited more than 250,000 times.

How does a model learn: what is pretraining?

Pretraining is the long first phase. The model is shown huge quantities of text, usually scraped and filtered to remove duplicates, low-quality pages and toxic material, and sometimes supplemented with synthetic data. At each step it sees a stretch of text, predicts the next token, and is scored on how much probability it gave to the actual one. The parameters are nudged to do slightly better next time, and the process repeats trillions of times.

Nothing in this task labels statements as true or false. The model learns whatever helps predict text: grammar, facts that appear often, writing styles, how code is structured, even patterns of reasoning that show up in the text. That breadth is the surprising part. A target as plain as “predict what comes next” forces a model to absorb a great deal about the world, because the world is what the text describes.

It is also expensive. Published estimates give a sense of scale: GPT-2, from 2019 with 1.5 billion parameters, reportedly cost about $50,000 to train, while Megatron-Turing NLG with 530 billion parameters in 2021 cost an estimated $11 million. Training costs roughly six floating-point operations per parameter per token, versus one to two per parameter for answering a prompt later.

Why does a raw model need more training before it acts like an assistant?

A pretrained model is a text continuer. Give it a question and it might continue with more questions, or with a forum argument. To make it helpful, developers add further stages.

The first is instruction fine-tuning: supervised training on examples of prompts paired with good responses. OpenAI’s InstructGPT work in 2022 showed how this changes behavior. The second is reinforcement learning from human feedback, or RLHF. People rank multiple model answers to the same prompt, a separate reward model learns to predict those rankings, and the language model is then tuned to score well with that reward model. Alternatives exist, such as Anthropic’s constitutional AI, which uses written principles to guide the feedback.

These stages shape tone and usefulness far more than they add knowledge. They also introduce problems. A model tuned to please raters can learn to sound confident when it should not, and to tell users what they want to hear, a tendency called sycophancy.

How does the model actually produce a reply?

Generation is the same prediction step run in a loop. The model reads the prompt, outputs a probability for every token in its vocabulary, picks one, appends it to the text, and repeats. How the pick is made matters. Always choosing the most likely token gives dull, repetitive text. Sampling with some randomness gives variety, which is why the same question can yield different answers.

The amount of text the model can consider at once is its context window. Early GPT-2 handled about a thousand tokens; later systems handle far more. Anything outside the window is invisible to the model unless the application feeds it back in. Developers often get around limits with retrieval-augmented generation, in which relevant documents are fetched and placed into the prompt, or with tool use, where the model calls a calculator, search engine or code runner.

Newer reasoning models, such as OpenAI’s o1 in 2024 and the open-weight DeepSeek-R1 in early 2025, are trained to produce long intermediate working before the final answer. They still run on the same token-by-token machinery.

Does a model understand what it says?

This is the question where fact runs out and argument begins. What can be stated as established: LLMs generalize beyond their training text, can solve problems they have not seen verbatim, and also fail in ways no person would, such as producing fluent falsehoods.

Interpretability research has started to open the box. In a March 2025 study, Anthropic researchers examined Claude and found that when writing a rhyming poem it appeared to pick the target rhyme word ahead of time, rather than only deciding at the end of the line. They also found that concepts like “small” and “opposite” seemed to activate shared internal features across languages, with larger models sharing more. On arithmetic, the model used parallel pathways, one estimating roughly and one working out the final digit, yet when asked how it solved the problem it described the schoolbook carrying method. In other words, what a model says about its own process need not match what it did.

How to interpret such results is contested. One camp argues these are signs of real internal representations of concepts. Another holds that the models are very sophisticated pattern matchers and that words like “understand” are misleading. Both camps agree on the data; they disagree about what to call it. Yann LeCun, a prominent critic, said in 2025 that LLMs are a dead end for reaching human-level intelligence and has since founded a company focused on “world models” that learn from the physical world. Others expect scaling and better training to keep going. That is a forecast, not a finding.

What can go wrong?

The standard list is well documented. Models hallucinate, producing fluent but unsupported statements. They absorb biases from their data, for instance overrepresenting English-language and Western sources. They can sometimes reproduce memorized passages from training material. They are vulnerable to prompt injection, where instructions hidden in a web page or document hijack the model’s behavior, and to jailbreaks that coax around safety rules. Training and running them consumes large amounts of electricity.

None of these are signs that the technology is a fraud. They are signs of what it is: a statistical system that is powerful, uneven and best used with checking.

Common questions

Is ChatGPT searching the internet when it answers?

Not by default. A base model answers from patterns stored in its parameters. Some products add a search or retrieval step, in which case pages are fetched and read into the prompt first, and the answer is then grounded partly in those pages.

Does the model learn from my conversation?

Within a conversation, it uses what you have written as context. That is not the same as updating its parameters. Whether a company uses conversations to train future models depends on its policies and your settings, so check the provider’s privacy terms.

What does “parameters” mean, and is bigger always better?

Parameters are the adjustable numbers inside the network. More parameters generally allow more capacity, and performance tends to improve with more parameters, more data and more compute, a pattern described by scaling laws. But data quality, training methods and design choices such as mixture-of-experts, which activates only part of the network per input, also matter a great deal.

Why does the same question get different answers?

Because the model samples from probabilities rather than always taking the top choice. Settings that control randomness can be lowered to make outputs more consistent, though that does not make them more accurate.

Can a model be trusted for facts?

Treat it as a fast, fallible assistant. It is strong on common knowledge and weak on obscure, recent or very specific details, and it can state errors with confidence. Verify anything that matters.

What to remember

An LLM is a transformer network trained to predict the next token, then shaped by human feedback to behave like an assistant. Attention lets it relate every part of a passage to every other part. Its abilities come from scale and training, and its mistakes come from the same source. Whether any of this amounts to understanding is still argued over, and anyone who answers that with total certainty is ahead of the evidence.

Further reading

A clear book-length explanation of the mathematics behind neural networks. See it on Amazon (affiliate link)

As an Amazon Associate I earn from qualifying purchases. A commission may be earned if you buy through this link, at no extra cost to you.

Sources