Genius or vending machine
Ask an AI the same question twice and you get two answers. Everyone has done it, and everyone has drawn the same conclusion: if it can't even repeat itself, how can I trust it with anything real?
Fair. A single run does vary. But ask a colleague the same question on Monday and on Thursday. Different sentences, probably a different order, maybe a different example. Nobody calls the colleague unreliable.
AI runs on a computer, so we test it like software: same input, same output, or it's broken. But it answers like a person: same question, different words. We're using the machine test on something that behaves like a colleague. That's the whole confusion, and it's a stubborn one.
The test you lose #
You want a genius that tests like a vending machine. Genius: hand it the question, let it find what it needs, analyse it, and solve the thing. Vending machine: press B4, get B4, every time, so you can check it with one line.
You can't have both. Put the genius behind a form and you get the vending machine, minus the genius. Let it loose and you get the genius, minus the one thing engineers trust most: the simple test. Same input, expected output, green or red. Gone.
Everyone else lives without it #
People have solved this before, many times. Nobody judges a drug by one patient. A production line doesn't ask for identical parts; it sets a tolerance and counts how many land inside it. A weather forecast says seventy per cent chance of rain, not "rain". Even machine learning judges a model on thousands of examples, never on one. None of these expects the same result twice, and none of them minds.
So it's less a problem of the technology than of the field it landed in. Software is the one field built on a promise: same input, same output, every time. Now there's something inside the software that doesn't keep that promise, and a test is no longer a yes or a no. It's a whole evaluation: many runs, a range, a rate. That's where most people's trouble starts.
And the first instinct is the wrong one: how do I make it predictable again? You can, and you get a smaller product. The better question is what can the product do now that it couldn't before, and then measure that.
We see it with every client. Everyone wants the report to look the same every time, because that's what reports were always good at: same headings, same tables, same order. Fair. But where does it go when the agent finds something nobody asked for: a new risk, an odd number, a problem nobody knew existed? A fixed template has no box for it. So it gets squeezed into the wrong section or left out, and the most valuable finding of the run is the one the report drops.
What replaces it #
Our field is data science and the interpretation of data, and two good analysts don't write the same paragraph either. So we ask whether the answer is inside the tolerance. Ours has three parts, and the order matters.
Facts first: every finding points at evidence the run actually found, and a wrong fact is out, however nicely it's worded. Then completeness: everything the question asked, answered, and every fact that bears on it, found. Short and complete passes; long with a gap doesn't. Form last: how it's written matters least, unless another program has to read it.
We test the same way. Many scenarios, each run several times, often on different model setups, and every fact checked and every gap counted. An improvement has to show up in every repeat, across all of that. Once is a lucky run, and a lucky run doesn't ship. Do the arithmetic: ten scenarios, three repeats, ten model setups, and one change costs three hundred runs to judge. That's what a test is now.
The fair objection: three hundred runs to judge one change sounds slow and expensive. It is slower than one demo. It's still cheaper than shipping a change that only worked once.
Most of our time goes on the top two: teaching the agent to search wider than the first good answer, and picking better between the answers it comes back with. It's slower than pressing B4. It's also the only way we know to tell a genius from a lucky run.
