Catching Bad Changes Concepts › Why you can't unit-test this, and what you can do instead
01 · part 1, conceptsconcept

Why you can't unit-test this, and what you can do instead

An ordinary program gives the same output for the same input, so a test that passes today passes tomorrow. A system built around a language model doesn’t have that property, and the reason isn’t only that the model is random.

Four things decide an answer

what decides an answer, over one week prompt your team, when it releases code your team, when it releases index a nightly job, every night model the provider, whenever it decides Two of the four change without anyone on your team doing anything.
Four things decide an answer, and each changes on its own schedule. Your team changes two of them.
Nothing in this scene is measured.
partwho changes itdo you hear about it?
promptyour teamyes: you released it
codeyour teamyes: you released it
indexa scheduled job, every nightonly if the job reports what it changed
modelthe provideroften not, if you call it by name without a version

The index, the documents the system retrieves from, is rebuilt on a schedule whether or not anything went wrong. The model is whatever the provider currently serves under the name you asked for. A test you ran before your last release says nothing about a change that arrived after it, from someone who isn’t on your team.

What you can do instead

practicewhat it gives youstep
record what produced every answerthe ability to explain and reproduce a bad answer2, 3
test before releasinga chance to catch your own changes before anyone sees them4, 5
watch signals all the timea chance to notice changes nobody released6
keep a way back for each partthe ability to undo each kind of change7
The rest of this volume takes those four in order. The first one, recording what produced every answer, is the one every other practice depends on: without it a problem can be noticed and still not explained.
Try it. In the labs folder, run:
python3 run.py

The three incidents, when each starts and how many answers it broke, then how many users got a broken answer as each practice is added.