Almost every ML team we've worked with shares the same pathology: they have a model that's 6% better than the baseline in evaluation, and zero models in production. We've helped a dozen teams fix this. The fix is always the same, and it always feels wrong to the team for the first week.
The fix is to ship the worst-performing model that meets the absolute minimum bar of usefulness, route real traffic to it, and treat everything you learn next as the actual project. Eighteen months later, the teams that did this have working ML platforms. The teams that didn't are still polishing notebooks.
What 'embarrassingly simple' looks like
For most production ML tasks, your first model should be one of: a simple logistic regression, a small XGBoost ensemble, an off-the-shelf LLM with a basic prompt, or — and this is the one people resist most — a hand-written rule. Yes, a rule. If your task is 'flag the suspicious transactions,' a hand-written rule that flags transactions over a certain amount from a new device is a perfectly reasonable first model. It will catch more fraud than zero models do.
The point isn't accuracy. The point is to get the operational layer in place: data pipelines that move features to the model, observability that catches drift, a retraining loop you can trigger on demand, a rollback path, a way to compare versions. None of this exists in a notebook. All of it has to exist before any model is genuinely 'in production.'
What you learn that the notebook can't tell you
Real traffic teaches you what your offline evaluation can't: which features have lower quality than you expected, which inputs your model has never seen before, which users are gaming whatever signal you're using, where the long tail of weird-but-legitimate behavior lives. Every one of these is invisible in a curated eval set.
Your first model exists to surface these surprises while the team's emotional investment in the model is still low. When the surprises are bigger than your model can handle — and they will be — you can swap the model out without anyone getting precious about it. After the third or fourth model, your team has internalized the operational discipline. That's when you start chasing accuracy.
The trap of waiting
The strongest case against shipping early is 'we'll embarrass ourselves with a bad model.' This is almost always a misread of the audience. Internal users tolerate a bad model that's improving more than they tolerate a long silence. External users almost never notice the accuracy of a model — they notice the latency, the explanation, and whether it's useful for their task. None of those are improved by another week in a notebook.
We have one rule for any ML engagement: a model serving real traffic by week four, behind a feature flag, observable, and rollbackable. Not better than the alternative. Just serving. Everything else flows from that constraint.
Want this kind of work on your project?
One conversation, honest pricing, recommended engagement model — within a business day.
Keep reading
All articlesHow we estimate software projects (and why most estimates are wrong)
Every estimate is wrong. The useful ones are wrong in known ways. A practical method we've refined across 50+ engagements — and the failure modes we learned to spot first.
The unreasonable effectiveness of plain SQL
ORMs are fine. Until they aren't. After a decade of building data layers on every framework we could find, we keep coming back to plain SQL — and the patterns that make it pleasant.
Why we don't write microservices anymore
We split into microservices because it was 2019. Then we spent three years feeling the cost. Here's what we learned, and how we structure backends in 2025.