How to Build Your Own Tiny LLM From Scratch

The 5-stage pipeline behind models like GPT and Claude, explained without pretending you can train a frontier model on your laptop. Most people use ChatGPT and Claude every day and have no idea how these systems are actually built. That is not a moral failure; the field is simply wrapped in strange language. Tokens, pretraining, fine-tuning, reward models, RLHF, alignment — it sounds like a wall of math, but the core pipeline is not impossible to understand. You do not need to be a frontier-lab researcher to build a clear mental model. You just need one clean map. Here is the honest version: you are not going to train the next GPT or Claude from scratch on your laptop. Frontier models require massive datasets and millions in compute. The real goal is to understand the pipeline so well that you can build a tiny version yourself and stop treating AI like magic. Once you see the machine, you use it differently. Who This Is For This guide is for AI builders, developers, technical founders, and curious power users who want to understand what’s actually happening under the hood of modern language models. This is for you if: ✅ You want a simple mental model of the LLM pipeline ✅ You want to understand why models hallucinate or why some feel more helpful than others ✅ You want to build a small learning version instead of just reading theory This is not a guide to competing with OpenAI or Anthropic. This is a guide to understanding the shape of the system. What You’ll Learn By the end of this guide, you’ll understand: ✅ How raw text becomes tokens — and why models don’t “read” like humans ✅ Why pretraining is next-token prediction — the absurdly simple objective behind LLMs ✅ Why base models hallucinate — and why fluency ≠ truth ✅ How SFT turns a raw model into an assistant — teaching taste, not just facts ✅ What reward models, DPO, and RLHF actually do — the secret sauce ✅ Why alignment improves behavior — but adds tradeoffs 🏗️ The 5-Stage Pipeline at a Glance Before diving deep, here is the bird’s-eye view of how a raw pile of text becomes a helpful assistant: Data: Preparing and cleaning the raw material. Pretraining: Training a base model to predict text. Supervised Fine-Tuning (SFT): Teaching it to follow instructions. Preference Modeling: Teaching a separate system what a “good” answer looks like. Alignment Optimization: Polishing the model’s behavior using feedback. Now let us walk through it. The five-stage pipeline in one view. 📁 Stage 1: Data & Tokenization Before there is a model, there is text — a lot of it. Beginners usually misunderstand this stage. They think the hard part is collecting more data, when the harder part is actually deciding what data deserves to stay. Raw data is messy (spam, duplicates, toxic content, boilerplate). The model becomes the statistical shape of what it sees. This is why Data Contamination (when test data leaks into training) and Deduplication are critical engineering tasks. 🔤 How AI Actually “Reads” (Tokenization) Models don’t see words; they see Tokens — chunks of text converted into integer IDs. This explains the mystery of why a model can write brilliant code but struggles to count letters in a word. It’s not a spelling machine; it’s a token sequencer. 🔧 What to Build Yourself: Take a small dataset, clean it, and run it through OpenAI’s tiktoken or a Hugging Face tokenizer. Watch how normal sentences split into integers. 🚂 Stage 2: Pretraining (Building the Engine) The objective here is almost absurdly simple: predict the next token. The model sees a sequence, guesses the next token, and its parameters are nudged slightly if it’s wrong. Repeat billions of times. To predict the next token accurately, the model implicitly learns grammar, facts, Python syntax, and cause-and-effect. The output is a raw “Base Model.” It has language ability but no “assistant” behavior. It doesn’t answer questions; it just completes text plausibly. This is exactly why hallucinations happen: the engine is optimized for plausibility, not truth. Never treat fluency as proof of accuracy. 🔧 What to Build Yourself: Train a tiny character-level language model using Andrej Karpathy’s nanoGPT in a Colab notebook. Feel the core loop: Input → Predict → Compare → Adjust → Repeat. 🧑‍🏫 Stage 3: Supervised Fine-Tuning (SFT) Now you have a base model, but you want an assistant. The shift is simple: show the model examples of the behavior you want (Question → Good Answer). The objective is still token prediction, but the data is now highly curated. SFT requires much less data than pretraining, but its quality dictates the model’s “personality.” If your examples are vague, the model becomes vague. Fine-tuning teaches taste, not just information. However, there is a risk: Catastrophic Forgetting. If you fine-tune too aggressively on a narrow task, the model forgets its broader base skills. 🔧 What to Build Yourself: Build a tiny 100-example instruction dataset. Fine-tune a small open model (like Llama 3 8B) using Unsloth or Axolotl via QLoRA on a single GPU. Raw text becomes tokens before the model learns anything. ️⚖️Stage 4: Preference Modeling Two answers can be factually correct, but one is clearer, safer, and admits uncertainty. You can’t write a hardcoded rule for this, so you collect human preference data by ranking multiple model outputs. In classic RLHF, this data trains a Reward Model — an automated judge that scores answers at scale. But if the reward model is flawed, the main model learns loopholes to get high scores while giving terrible answers. This is called Reward Hacking. Modern alternatives like Direct Preference Optimization (DPO) skip the separate reward model and train directly on the rankings. 🔧 What to Build Yourself: Take 50 prompts, generate 4 answers for each, and rank them. Look into DPO tutorials to see how to train on these pairs. ️🛠️Stage 5: Alignment Optimization (The Polish) This is the final feedback loop: Generate → Score → Improve. Using the preference signal, the model learns nuance, safety, refusals, and its distinct tone. However, there is a cost: the Alignment Tax. Making a model safer and more cautious can sometimes reduce its raw capabilities on complex, unstructured tasks. This stage is what makes two models with similar base capabilities feel completely different to the user. 🔧 What to Build Yourself: Follow a basic DPO tutorial using your Stage 4 dataset to see how feedback visibly shifts model behavior. 📊 Summary: The LLM Pipeline Cheat Sheet LLM Pipeline Cheat Sheet showing 5 stages: Data, Pretraining, SFT, Preference, and Alignment 💡 Conclusion: Why This Changes How You Prompt Once you stop seeing the model as a magic brain and start seeing it as a trained system, your relationship with AI changes completely: Prompts make sense because you are changing the statistical context the model is continuing from. Examples work because they act as in-context demonstrations. They shape the pattern the model continues from, but they do not change the model’s weights. RAG (Retrieval) is essential because it feeds the engine facts instead of relying on lossy pretraining memory. 🚀 Resources to Start Building Today Stage 1: Data & Tokenization - OpenAI tiktoken: github.com/openai/tiktoken - Hugging Face Tokenizers: huggingface.co/docs/tokenizers Stage 2: Pretraining - Andrej Karpathy’s nanoGPT: github.com/karpathy/nanoGPT - His YouTube tutorial: www.youtube.com/watch?v=kCc8FmEb1nY Stage 3: Supervised Fine-Tuning - Unsloth (fast fine-tuning): github.com/unslothai/unsloth - Axolotl: github.com/axolotl-ai-cloud/axolotl Stage 4–5: Preference & Alignment - Hugging Face TRL (DPO tutorials): huggingface.co/docs/trl - DPO paper: arxiv.org/abs/2305.18290 Start with Stage 1. Build the smallest version you can. Then move through the pipeline one stage at a time. — - Enjoyed this guide? Follow me for more practical AI engineering content. If you build a tiny LLM using this pipeline, share your results in the comments — I’d love to see what you create. How to Build Your Own Tiny LLM From Scratch was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.