Beginner track · project

Build an AI helper that actually knows your stuff

You build Scout, a support assistant for a fictional outdoor-gear store, and you write the code. Scout answers from a 30-page handbook, says which page each answer came from, looks up a customer's order, resists tricks, and comes with numbers that show how well it works. It runs on your own computer, with Claude Code, or your own Claude or OpenAI key.

What you end up with

  • A keyword search engine (BM25) and a handbook chunker that you wrote yourself.
  • Search by meaning, built from a model and a keyword shortlist, and a way to combine searches.
  • An assistant that answers only from the passages it was given, names its sources, and rejects an answer that cites one it was never shown.
  • An order tool that returns nothing unless the order number and the email on it both match, and a loop that uses it.
  • A way to measure how often it is right, with five test questions of your own, and protections tested against ten tricks.
  • A cost per question at three sizes of business, a memo on which model to use, and a scorecard you can show someone.

What you need

  • Python 3.10 or later, and a terminal. You will write Python: functions of 5 to 30 lines, each described in plain words.
  • One model, your choice: Claude Code (signed in, no key to enter), your own Claude API key, or your own OpenAI API key.
  • About 15 to 20 hours, over as many sittings as you like. Your progress is saved.
  • The checks on your code are free and instant. The live checks cost roughly $1 to $2 in model calls at API prices; with Claude Code they count against your plan instead. The kit shows an estimate and asks before it spends anything.

How a module works

  • Each module tells you which file in scout/ to open. The file explains each function you must write.
  • You write it. Then you run the module.
  • The module first runs checks on your code. They use a fake model, so they are instant and free, and each says in plain words what it expected and what it got.
  • When every check passes, the module tries your code against your real model, shows the result, and counts the module as done when the numbers clear the bar.

The ten modules

Each module is one guided page and one run of the project. The pages are part of the track: they open once you own it.

  1. Module 0 Check that everything works
  2. Module 1 Decide what the assistant should and should not handle
  3. Module 2 See what a model actually reads
  4. Module 3 Split the handbook into chunks and search it by keyword
  5. Module 4 Search by meaning and combine the searches
  6. Module 5 Make it answer from the passages and name its source
  7. Module 6 Let it look up a customer's order
  8. Module 7 Measure how often it is right
  9. Module 8 Protect it from tricks and from private data
  10. Module 9 Work out what it costs, put it together and see your results

Also in the track

Setup, troubleshooting, a worked example to compare your results with, and a guide to running a model on your own computer with Ollama.

Back to the Beginner track