Foundations

This section builds the mental model everything else in the library depends on. If you only read one section, read this one. The goal is not to make you able to implement a transformer from scratch, but to give you an accurate intuition for what a language model is doing so that the rest — retrieval, prompting, agents, evaluation — makes sense.

Documents in this section

• What an LLM Actually Is — a next-token predictor, and why that simple framing explains most of a model's behavior.
• Tokenization — how text is chopped into the units a model reads, and why that has practical consequences for cost and correctness.
• Embeddings — turning tokens and text into vectors, and what "meaning as geometry" buys you.
• Attention Explained — the mechanism that lets a model relate every token to every other token.
• The Transformer — how attention, feed-forward layers, and residual connections stack into the architecture behind modern LLMs.
• Sampling, Temperature, and Decoding — how a probability distribution becomes the specific words you see.

Start with What an LLM Actually Is and read in order.