An MLOps layer around the used-car price model (project A3): versioned data, MLflow tracking and registry, a measured drift detector, and evidence-based champion/challenger promotion.
Accuracy is only one axis of a production decision. Each candidate is measured on a frozen holdout split alongside its operational cost — artifact size, prediction latency, and batch throughput — so the trade-off is explicit rather than assumed.
| Model | Holdout MAE | Artifact | Predict p50 | Batch throughput |
|---|---|---|---|---|
| Ridge | 15,422 PLN | 0.0 MB | 5.1 ms | 728k rows/s |
| LightGBM | 9,278 PLN | 3.3 MB | 12.9 ms | 119k rows/s |
| RandomForest | 8,908 PLN | 338.5 MB | 47.1 ms | 148k rows/s |
The random forest achieves the lowest error but ships a 338 MB artifact and the slowest single prediction; LightGBM reaches within a few percent of that error at roughly one hundredth of the size — the kind of trade-off a registry should record, not hide.
Weekly production snapshots are replayed against six scenarios. The detector is not trusted on faith: it is itself measured for false alarms and detection power, and its population-stability thresholds are calibrated on the project's own data rather than borrowed from the textbook 0.2 rule, which does not hold here.
| Scenario | What changes | MAE change | Alert |
|---|---|---|---|
| stable | no change (control) | −0.9% | no |
| price_shock | prices inflate, features unchanged | +135.2% | yes |
| fuel_mix_shift | more electric and hybrid cars | +2.1% | yes |
| mileage_shift | higher mileage across the week | +24.5% | yes |
| unseen_makes | makes absent from training | +7.5% | yes |
| missing_engine_volume | engine volume arrives empty | +26.1% | yes |
A challenger replaces the champion only when the improvement is shown to be real rather than
fortunate. The decision is made by a paired bootstrap on the frozen holdout: a small nominal gain
is promoted only if its confidence interval clears zero. The winner is registered in MLflow and
served through a stable champion alias, so deployment follows evidence, not a single
lucky evaluation.
mlops-car-price keeps the A3 model alive: reproducible data versioning, MLflow experiment tracking and a model registry, a drift monitor with quantified error rates, bootstrap-based promotion, and an API that serves the current champion. The engineering decisions are recorded as ADRs in the repository.