The €2.1M inspection bottleneck a model didn't fix
The brief we were given "Our scrap rate is too high. Build us an AI that spots defects on the line."
Inspection wasn't inaccurate — it was a queue. And the expensive failures were escaping into warranty claims nobody could trace back to a batch.
- Customer
- A Mittelstand precision-components supplier
- Sector
- Automotive tier-2 manufacturing
- Location
- Germany
- Size
- ≈480 employees, three plants in Baden-Württemberg
- Engagement
- 18 weeks — 6 on site, the rest remote
The situation
The customer makes precision machined components for automotive tier-1 suppliers — the kind of parts where a dimension out by forty microns is scrap, and one that escapes to a customer is a warranty claim with a long tail.
Three plants, twenty-two production cells, end-of-line visual and dimensional inspection done by trained operators. Quality data lived in three places that did not talk to each other: an MES that recorded cell and batch, an SAP module that held orders and shipments, and an eleven-year-old quality database where inspectors typed free-text defect notes in German.
They had already bought a camera system from a vendor. It had been switched off for eight months because it flagged too much and nobody trusted it.
How it was approached
The engagement did not start with a model. It started with two days standing next to the inspection station with a notebook, because the stated problem and the real one are almost never the same thing — and on this line they were not even about the same process step.
What followed ran in three overlapping tracks: make the humans faster first, build the model properly second, and only then automate the analysis nobody had asked about but that turned out to carry most of the money.
The engagement, as it happened
Each step carries what the engineer did and, separately, what a Forward Deployed Engineer should take from it.
-
Week 0 Discovery
The brief was an answer, not a question
The request arrived fully formed: build a defect-detection model, here is the camera system, here are the images. The plant manager had a budget, a vendor quote and a deadline.
The first thing the engineer did was decline to accept the framing. Not by arguing — by asking for numbers that would make the framing checkable:
- What is the scrap rate, split by cell and by defect class?
- What does a warranty claim cost, all-in, including the engineering hours to trace it?
- How long does a part wait at inspection at shift change?
Nobody could answer the third. That question turned out to be the engagement.
What an FDE takes from this The customer states the solution they have imagined. Your first job is to convert it back into a problem statement, using questions whose answers can be checked.
-
Weeks 1–2 Discovery
Two days on the shop floor, no laptop
Shadowing three shifts, including the 22:00 handover, produced the finding the data could not have:
Inspection was not inaccurate. It was a queue. Operators were good — measured against a re-inspection sample, they were right about 97% of the time. But at shift change and during the last hour before a shipping cut-off, parts backed up. Under that pressure the rational move was to pass borderline parts, because stopping the line was visible and a marginal part was not.
The defects that escaped were not the ones a model would find easily. They were the ones a tired human waved through at 05:40.
Second finding: when an inspector did stop on something unusual, they walked to a filing cabinet. Eleven years of resolved defect reports, searchable only by date. The median hunt was eleven minutes. It happened maybe fifteen times a shift.
What an FDE takes from this Go and watch. The constraint is often a moment in the day, not a step in the process — and it will not appear in any table you are given.
-
Week 3 Discovery
Mapping the data as it actually is
Three systems, no shared key. The MES knew cell and batch. SAP knew order and customer. The quality database knew defects, and referenced batches by a string operators typed by hand — which meant
CH-2291,CH 2291andch2291were all in there.The image archive was better than expected and worse than hoped: roughly 340,000 images from the switched-off camera system, almost none labelled. But the quality database held free-text German notes, timestamped, often within minutes of an image.
The engineer wrote a one-page data reality memo and walked the quality manager through it. Three things were now obvious that had not been before:
- The labels existed — they were in prose, in German, in a different system.
- Batch identity was recoverable by fuzzy matching, but it needed a decision about how wrong a match could be before a human looked at it.
- Nobody had ever joined a warranty claim to a production batch. Not once.
What an FDE takes from this Write the data reality down and make the customer read it with you. Half the value of discovery is the moment a stakeholder sees their own systems described plainly.
-
Weeks 4–5 Build
The first thing shipped had no model in it
Eleven minutes, fifteen times a shift, three shifts, twenty-two cells. That is the cheapest large number on the site, and it needed retrieval, not machine vision.
Nine days of work: OCR the archive, embed the German defect notes, hybrid search over text plus structured filters for cell, material and defect class, and a single search box on a tablet mounted at the inspection station. Type what you see; get the five most similar past cases with what was done about them.
It went in on a Tuesday. By Friday it was being used without prompting, which is the only adoption metric that means anything. Median lookup went from eleven minutes to under forty seconds.
It also bought the thing every FDE needs and few ask for early enough: credibility to disagree later.
What an FDE takes from this Ship something useful inside two weeks, even if it is not the thing you were hired for. Trust is the currency you spend on every hard conversation that follows.
-
Weeks 6–9 Build
Building the inspection model, labels first
Now the model — but the interesting work was labelling, not architecture.
The German defect notes were turned into weak labels: a language model classified each note into the plant's existing eleven defect classes, and the timestamp joined it to nearby images. That produced roughly 90,000 noisily-labelled images in a week, against a hand- labelling estimate of four months.
Two inspectors then hand-labelled 4,000 images chosen by an active-learning loop — the ones the model was least sure about — in sessions of ninety minutes, because attention degrades and mislabels are worse than no labels.
The architecture was deliberately unexciting: a fine-tuned convolutional backbone, exported to ONNX, one model per defect family rather than one heroic multi-class model, because per-family thresholds could then be tuned to what each defect actually costs.
What an FDE takes from this Most "we have no labels" problems are really "our labels are in prose, in another system". Look before you commission a labelling project.
-
Weeks 10–11 Integration
Getting it onto the line without touching the PLC
This is the part that is invisible in a demo and decides whether anything ships.
Constraints, all non-negotiable, all discovered by asking rather than assuming:
- No cloud round trip. The cell's cycle time left a 180ms budget for a verdict.
- No modification to PLC logic — the machine builder's warranty forbade it. The system could read over OPC UA and raise an advisory signal, nothing more.
- Plant network is air-gapped from the internet. Model updates travel on a signed
artefact and a scheduled sync, not
git pull.
So: an industrial edge box per cell, ONNX Runtime, 47ms median inference, verdicts written to the existing andon display the operators already watched. The system advises; the human decides and the decision is recorded. That last point mattered enormously three weeks later.
What an FDE takes from this Integration constraints are requirements, not obstacles. Ask about latency budgets, warranties and network posture in week one — each one can invalidate an architecture.
-
Weeks 12–13 Evaluation
Publishing the number that wasn't good
Evaluation was per defect family against a held-out set the inspectors had labelled independently, not against the weak labels.
Ten families landed between 94% and 99% recall. One — hairline surface cracking under certain lighting — reached 71% and would not move. Three weeks of work would perhaps have bought 80%.
The engineer put that in the report as the headline, not a footnote, and proposed routing that family to mandatory human inspection with the model used only to flag candidates. The quality manager's response was the sentence that made the rest of the project possible: "This is the first vendor report I have believed."
An honest 71% that everyone plans around is worth more than a claimed 95% that quietly fails on the night shift.
What an FDE takes from this Report the class your system is bad at, loudly, with a plan for it. Credibility compounds; an unreported weakness detonates later.
-
Weeks 14–15 Automation
The automation nobody asked for, where the money was
Back to week three's finding: no warranty claim had ever been joined to a production batch.
Claims arrived as PDFs and emails from tier-1 customers, in German and English, with part numbers in four different formats. An engineer spent, on average, six hours per claim reconstructing which batch it came from — reading shipment records backwards. About 40 claims a month. Roughly 1.4 full-time engineers doing archaeology.
The pipeline built: extract structured fields from the claim document, resolve the part and date to candidate shipments, walk back through SAP to orders and through the MES to batches, and score the candidates. Above a confidence threshold it proposed a batch with its evidence chain; below it, it handed a ranked shortlist to a human instead of guessing.
Six hours became eleven minutes for 78% of claims, and about ninety minutes for the rest. And because claims were now joined to batches, a question that had been unanswerable became routine: which cell, tool and shift do our field failures actually come from?
That analysis found a single fixture on one cell responsible for 31% of a year's warranty cost. It was corrected in a day.
What an FDE takes from this The highest-value automation is usually adjacent to the brief, not inside it. Follow the expensive manual work, wherever it sits.
-
Weeks 16–17 Adoption
The Betriebsrat, the AI Act, and why adoption is engineering
In Germany a system that observes work has to be agreed with the works council. This is not a formality and it is not a legal footnote — it is a design constraint, and finding it in week sixteen would have been fatal. It was raised in week two.
Three commitments shaped the build itself:
- No individual performance measurement. Verdicts are recorded against cell and shift, never against a named operator. Enforced in the schema, not in policy — the operator column does not exist.
- The human overrides, always, without justifying it. Overrides are counted in aggregate to improve the model, never reviewed individually.
- Written explanation on every flag — defect family, confidence, and the nearest historical cases. An operator can see why.
Under the EU AI Act the system was assessed as limited-risk: it advises on product quality, not on people, and the works council agreement plus the no-operator-column decision were what kept it there. That assessment was documented, because the next auditor will ask.
Handover was two days of training, a runbook covering the three failure modes that had actually occurred, a Grafana dashboard the quality manager helped design, and a defined retraining trigger: drift beyond a threshold on the weekly evaluation set.
What an FDE takes from this Adoption constraints are architecture. "Works council approval" and "regulatory class" change your schema, not just your paperwork — and they are cheap in week two and ruinous in week sixteen.
-
Week 18 → Outcome
What it earned, measured six months later
The customer's own controlling department produced these figures at the six-month mark, which is the only version worth quoting.
The split is the point: the inspection model — the thing actually requested — accounts for roughly a third of the benefit. Retrieval and claims automation, neither of which appeared in the original brief, account for the rest.
The capacity number is the one the board cared about. Removing the shift-change queue released throughput on a line that was turning away orders, and that showed up as revenue rather than cost avoidance.
What an FDE takes from this Measure with the customer's own finance function, not your own spreadsheet. A number they produced is a number they will defend.
Measured outcomes
Before and after, so each claim is checkable rather than an adjective.
| Measure | Before | After | Change |
|---|---|---|---|
|
Historical defect lookup
Speed
Retrieval over 11 years of German defect reports. Shipped in week 5.
|
11 min | < 40 sec | −94% |
|
Warranty claim → batch trace
Speed
For the 78% resolved above the confidence threshold; the rest take ~90 min.
|
6 hrs | 11 min | −97% |
|
Inspection recall, 10 of 11 defect families
Accuracy
The eleventh family reached 71% and is routed to mandatory human inspection.
|
≈97% (human, degrading under queue pressure) | 94–99%, stable across shifts | Consistent |
|
Escaped defects reaching customers
Accuracy
Largest single contributor was removing the shift-change queue, not the model.
|
Baseline | −63% | −63% |
|
Engineering hours on claim archaeology
Capacity
Equivalent to 1.2 engineers returned to process improvement.
|
≈240 hrs/month | ≈35 hrs/month | −85% |
|
Warranty cost, annualised
Cost
A single fixture on one cell, found only once claims could be joined to batches.
|
Baseline | −31% | −31% |
|
Scrap and rework, annualised
Cost
Customer controlling department figure at six months.
|
Baseline | −€610k | −€610k |
|
Released line capacity
Revenue
Booked as revenue, not cost avoidance — the line was turning orders away.
|
Queue-limited at shift change | +4.2% throughput | +€1.5M |
The reading
The inspection model matters, but it is not where the value concentrated. Two thirds of the measured benefit came from work nobody requested in the original brief: connecting warranty claims back to production batches, and giving inspectors instant access to how similar defects had been resolved before.
That is the pattern this case exists to teach. A Forward Deployed Engineer is paid to find the expensive problem, not to build the requested artefact.
Learn to work like this
The AI Forward Deployed Engineer Program is twenty weeks of exactly this — discovery, honest evaluation, integration into systems you didn’t build, and automating the expensive work nobody put in the brief.