Fast-learner loop (use for every module): 1) Look at the diagram first — get the shape. 2) Read once, fast. 3) Say it out loud in 30 seconds. 4) Take the quiz. Anything you miss, re-read only that part.
LLMs — what you must be able to explain
How an LLM call works
Prompt (system + user) → tokens → model → completion. Context window, temperature, max tokens. Stateless by default — you pass history each time.
Prompt engineering
Clear instruction, role, constraints, examples (few-shot), output format, delimiters. Separate system instructions from user data to reduce prompt injection.
Structured output
Ask for JSON with a schema, validate, retry on parse failure. Never trust raw model text in a pipeline without validation.
RAG in one minute
Chunk documents → embed → store in vector DB → retrieve top-k → pass as context → answer with citations. Use when knowledge changes or is private.
Key concepts table
| Concept | One-line answer | Follow-up they may ask |
|---|---|---|
| Token | Roughly ¾ of a word; billing & context limits are counted in tokens | How do you reduce token cost? Shorter prompts, summarise history, cache, smaller model for easy tasks |
| Temperature | Controls randomness; low for classification/extraction, higher for brainstorming | What temperature for data extraction? Near 0 |
| Hallucination | Model produces plausible but false output | Mitigate: RAG, citations, validation, human review for high-risk steps |
| Embeddings | Vectors representing meaning; similar text = nearby vectors | Used for search, clustering, dedup, classification |
| Function / tool calling | Model returns a structured request to call your code (e.g. get_order) | You validate args, execute, return result to model |
| Fine-tuning vs RAG vs prompting | Prompt first; RAG for knowledge; fine-tune for behaviour/format at scale | Fine-tuning is rarely step one |
A production LLM call (Python)
import json, os
from openai import OpenAI
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
SYSTEM = "You classify support tickets. Reply only with JSON."
def classify(text: str) -> dict:
r = client.chat.completions.create(
model="gpt-4o-mini", temperature=0,
response_format={"type": "json_object"},
messages=[{"role":"system","content":SYSTEM},
{"role":"user","content":text}])
data = json.loads(r.choices[0].message.content)
assert data["category"] in {"billing","bug","sales","other"}
return data # validate before you trust itPrompt template you can reuse
SYSTEM: You are a data-extraction assistant.
TASK: Extract invoice fields from the text below.
RULES: - If a field is missing, use null. Never guess.
- Dates in YYYY-MM-DD. Amounts as numbers.
OUTPUT: Only JSON matching {invoice_no, date, total, currency}
TEXT: <<< {document} >>>Evals & quality (junior-level, high-signal)
- Golden set: 30–50 labelled examples you re-run after every prompt/model change
- Track accuracy, parse-failure rate, latency and cost per 1k requests
- Log prompt, model version, output and user correction for review
- Add guardrails: schema validation, max retries, fallback model / human queue
Say this in the interview: “I don't change a prompt and hope. I keep a small golden dataset, run it after each change, and compare accuracy and cost before shipping.”
Cost & latency levers
| Problem | Levers |
|---|---|
| Too expensive | Smaller model for easy tasks, shorter prompts, caching, batching, summarise history |
| Too slow | Streaming, smaller model, fewer tokens, parallel calls, cache repeat answers |
| Unreliable format | JSON mode / schema, validation + 1 retry with error feedback, fallback |
flowchart LR
U["User / System Prompt"] --> T["Tokeniser"]
T --> M["LLM"]
M --> O["Completion tokens"]
O --> V{"Validate
JSON schema?"} V -->|Valid| S["Store / Next step"] V -->|Invalid| R["Retry once
with error feedback"] R --> M
JSON schema?"} V -->|Valid| S["Store / Next step"] V -->|Invalid| R["Retry once
with error feedback"] R --> M
flowchart TD
D["Documents"] --> C["Chunk
500-1000 tokens"] C --> E["Embed"] E --> V[("Vector DB")] Q["User question"] --> QE["Embed question"] QE --> V V -->|Top-k chunks| P["Prompt = question + context"] P --> L["LLM"] L --> A["Answer + citations"]
500-1000 tokens"] C --> E["Embed"] E --> V[("Vector DB")] Q["User question"] --> QE["Embed question"] QE --> V V -->|Top-k chunks| P["Prompt = question + context"] P --> L["LLM"] L --> A["Answer + citations"]
flowchart LR
P["Change prompt
or model"] --> G["Run golden set
30-50 examples"] G --> M["Compare: accuracy
parse failures / cost / latency"] M -->|Better| S["Ship"] M -->|Worse| P
or model"] --> G["Run golden set
30-50 examples"] G --> M["Compare: accuracy
parse failures / cost / latency"] M -->|Better| S["Ship"] M -->|Worse| P
Test yourself
Module 1 Test — LLMs, Prompting & RAG
Tap an answer — explanation appears instantly. Pass mark: 70%. Retake until you pass.
For invoice extraction, the best temperature is…
Why: Extraction/classification needs repeatable output. Near 0.
Your LLM returns broken JSON 5% of the time. First fix?
Why: Constrain the format, validate, retry once with feedback. Escalate models later.
Private company policies must inform answers. Best first approach?
Why: RAG gives current, citable, private knowledge without training.
What is an embedding?
Why: Embeddings power semantic search, clustering and RAG retrieval.
How do you cut LLM cost fastest?
Why: Route easy work to small models and control token volume.
Prompt injection risk is reduced by…
Why: Treat user content as data, not commands; validate outputs.
0 / 6 correct