How AI Actually Works, Term by Term

Start here

AI isn't a black box. It's a machine with named parts.

What goes in

The building blocks.

Token

The smallest unit of text a model reads, processes, or generates. Think of a token as a Lego brick that comes in different sizes, where some are whole words and others are just the start or end of a longer one. The model can't read continuous letters, so it breaks your sentences into these chunks first. It matters because every token you send and every token it writes counts toward your cost, your speed, and how much the system can hold in memory at once.

Embedding

A numerical map that places words with similar meanings close together in a high-dimensional space. Picture a giant city map where neighborhoods stand for concepts, and the distance between two points tells you how related the ideas are. That's how a model figures out that bank and river belong in one neighborhood while bank and money belong in another. Embeddings give the model a mathematical grip on nuance and context without ever reading a dictionary.

Context Window

The maximum number of tokens a model can handle in a single turn before it loses track of the beginning. Imagine a whiteboard where a teacher can only fit so many lines before she has to erase the top to make room. The window sets the boundary for what the model can see and reference while you talk to it. Push past it and the system quietly drops earlier details, which is where confused or contradictory answers come from.

Prompt

The exact text you type to tell the model what to do with everything it knows. Think of it as a recipe card that tells a chef which ingredients to use, how to mix them, and what the finished dish should look like. The model reads it, matches it against its internal maps, and starts generating a response that fits. The clearer you frame the task, the context, and the constraints, the better the answer you get back.

Inside the model

How models learn.

Parameter

A trainable number that decides how strongly one piece of information pulls on another inside the network. Picture a massive mixing board with millions of sliders, each one controlling how much a given word or pattern should affect the next step. The model nudges these sliders during learning until the balance is right. Parameters are where the model's knowledge actually lives, and the count of them sets the ceiling on how complex a problem it can handle.

Attention

The mechanism that weighs how relevant each token is compared to all the others in the sequence. Imagine a librarian reading a stack of documents with a highlighter, marking the passages that answer your question and skipping the rest. Attention is what lets a model link a pronoun back to the right noun or catch a contradiction buried in a long paragraph. Without it, every word would look equally important and the structure would fall apart.

Training

The first phase, where the model adjusts its parameters by studying patterns across billions of documents. Think of a student cramming for one enormous exam, reading thousands of books and practice tests until grammar, facts, and reasoning sink in. The model keeps correcting its sliders based on how wrong each guess was until the predictions get sharp. Training builds the foundation for everything else, and the quality of the data decides what the model knows and how it thinks.

Fine-tuning

Training a model that's already learned the basics on a smaller, targeted dataset to specialize it. Picture a general practitioner doing a focused residency to become a surgeon, keeping the broad medical knowledge but drilling specific procedures. Fine-tuning shifts the parameters so the model follows instructions better, adopts a tone, or masters a niche without throwing away its original education. It's how you turn a broad knowledge base into a tool that fits your exact job.

At runtime

How they run in the real world.

Inference

The phase where the trained model takes your prompt and generates a response in real time. Think of the doctor's actual consultation after years of school, applying what they know to your symptoms right now. Inference is where the ongoing cost lives, because it runs every single time anyone uses the system. The speed, accuracy, and price of your experience all come down to how well this step is handled.

Quantization

A technique that lowers the numerical precision of a model's weights to save memory and run faster. Imagine compressing a high-resolution photo into a smaller file so it loads quickly while still looking clear enough to recognize. Quantization shrinks the parameters and the attention math, so a model can run on cheaper hardware or serve more people at once. It's what puts powerful models on a laptop instead of a data center, though push it too far and the accuracy starts to blur.

RAG

Short for retrieval-augmented generation, a method that pulls outside information into the context window before the model answers. Picture a researcher checking a reference book or a database before writing, so they can cite real sources instead of guessing. RAG injects fresh, accurate data into the prompt without retraining the model on it. It keeps answers grounded, cuts down on hallucinations, and lets you wire the model into your own private knowledge.

What's next

Where it's all heading.

Agent

A system that uses a model to plan multi-step tasks, make decisions, and call outside tools to get work done. Think of a project manager who breaks a big goal into smaller jobs, hands them out, checks the results, and adjusts when something breaks. Agents go past single replies by chaining reasoning, tool use, and memory together. That's what turns a chatbot into something that can actually finish a task across several apps.

Scaling Laws

The predictable relationship between model size, training data, and compute, showing how performance climbs as you spend more. Picture a graph of a road trip where extra miles bring diminishing returns past a point, but still get you farther. Scaling laws tell engineers how much compute buys a given jump in capability, and when it's smarter to improve the data than to just make the model bigger. They steer where the whole industry puts its money, and they explain why some models cost millions to train while others cost pennies to run.

Putting it together

Seeing the machine clearly.