Question bank — click to reveal model answers
What is an LLM and how does it generate text? LLM
It predicts the next token from context learned in training. You send a prompt; it returns a completion token-by-token. It's stateless — conversation history is re-sent each call.
How would you reduce hallucinations? LLM
Ground with RAG/citations, low temperature for factual tasks, schema validation, ask for 'unknown' when unsure, human review for high-risk outputs.
Prompt engineering: how do you structure a good prompt? Prompting
Role + task + constraints + format + examples. Delimit user data, specify output schema, test on a golden set.
What is RAG? When do you use it? LLM
Retrieve relevant chunks from your data and pass them as context. Use for private/changing knowledge instead of fine-tuning.
REST API vs webhook — what's the difference? APIs
REST: you call them (pull). Webhook: they call you when an event happens (push). Verify webhook signatures and dedupe retries.
How do you handle a 429 rate limit? APIs
Respect Retry-After, exponential backoff with jitter, queue requests, cache, request a limit increase if justified.
API key vs OAuth vs JWT? Auth
API key: simple server secret. JWT: signed, expiring token with claims. OAuth: delegated scoped access on behalf of a user with refresh tokens.
A webhook keeps delivering duplicates. Why and what do you do? Webhooks
Provider retries when you don't return 200 fast or time out. Return 200 quickly, enqueue work, deduplicate by event id.
How do you test an AI workflow? Testing
Unit tests for code, golden-set evals for model behaviour, integration tests with provider sandboxes, parse-failure and latency tracking.
Workflow is failing intermittently. Walk me through debugging. Debugging
Reproduce, check logs/correlation id, isolate provider vs our code with curl, check auth expiry/rate limits/payload changes, fix + regression test + document.
SQL vs NoSQL — how do you choose? Databases
Relational/integrity-heavy → SQL. Flexible documents/high-write events → NoSQL. Most junior systems start with Postgres.
How do you make a pipeline idempotent? Systems
Idempotency keys, unique constraints, dedupe tables, upserts — safe retries without duplicates.
How do you use Claude Code / Codex day to day? AI Tools
Plan first on larger tasks, small scoped changes, review every diff, run tests, never paste secrets, keep context files updated.
What do you do when the AI coding tool produces wrong code? AI Tools
Reproduce with a test, show it the failure, narrow the scope, review the fix — I own the final code quality.
Design: classify 10k support tickets/day with AI. System Design
Webhook/API ingest → queue → small model classify to JSON → confidence gate → human queue for low confidence → DB → daily report. Evals + cost tracking.
How do you keep API keys safe? Security
Env vars / secret manager, never in code or git, least-privilege scopes, rotate on leak, .env in .gitignore.
What is an embedding? LLM
A vector representing meaning; used for semantic search, clustering, recommendations and RAG retrieval.
Temperature 0 vs 1 — when? LLM
0 for extraction/classification (deterministic). Higher for creative brainstorming.
How do you monitor an automation in production? Monitoring
Structured logs, success/failure metrics, alerts on failure rate & queue depth, cost dashboard, runbook.
Tell me about a project you're proud of. Behavioral
Use STAR: Situation, Task, Action (tools, APIs, LLM role), Result (numbers if possible) + what you'd improve.
Flashcards
Quick-fire quiz
1. A provider retries a webhook 5 times. First check?
2. Best temperature for invoice extraction?
3. Private company docs must inform answers. First choice?
0 / 0 correct