The Problem: The Bench Test That Worked Fine Until It Didn't
Here's a pattern that's common at hardware startups. A hardware startup gets through EVT with 50 boards and a bench test: a bench supply, a DMM, a USB cable, and a printed checklist. One engineer runs it, and it takes about six minutes per DUT. Nobody worries about it.
Then PVT lands with 2,000 units. The test moves to the contract manufacturer, and two operators on two shifts run it. Six minutes per unit becomes 200 hours of test time. Two operators read "LED is green" two different ways. Somebody skips the sleep current check because the DMM needs a range change. Three weeks after shipping, field returns start coming back, and nobody can say which test step should have caught them, because the logs say PASS and nothing else.
The usual reaction is "we need to automate the test station." That's right, but it's too vague to act on. Automate what, in what order, and how much fixture can you afford before the design freezes?
Why It Happens: The Spec Was Written for the Wrong Reader
Most startup test specs start life as design verification documents. They answer "does this design work?" A production test has to answer a different question: "was this unit built correctly?" So the spec says things like "verify 3.3V rail" with no limits, no measurement point, no load condition, and no idea how long the step takes or what failure it's supposed to catch.
A spec like that can't tell you where automation pays off. But each line can be read again with manufacturing questions in mind, and once you do that, the order to automate in becomes fairly obvious.
The Solution: A Four-Lens Triage of Every Spec Line
Put every test step in a spreadsheet, one row each, and score it on four lenses.
1. Frequency × time
Multiply the step's duration by the number of units you'll build this year. Illustrative numbers: a 90-second visual check of four LEDs across 2,000 units is 50 operator-hours. A 5-second firmware version readback is under 3 hours. Cycle-time cost tells you where your money is going, and it's usually not where people guess. The slowest step also sets your UPH (units per hour), so find it first.
2. Operator judgment
Flag every step that depends on a person's eyes, ears or fingers: "LED looks right," "buzzer sounds OK," "button feels fine." Judgment calls cause both escapes and false failures, and you can't analyze them later because nothing was measured. These steps go first in line to be replaced with a real measurement, such as a light sensor on the LED, a microphone with an FFT on the buzzer, or a current signature on the actuator. That holds even when they're quick.
3. Defect coverage
For each step, write down which failure modes it actually detects: opens and shorts, wrong part placed, part missing, bad solder on a connector, firmware not loaded, calibration not stored. Then check what your CM already covers with AOI or ICT. You'll often find two functional steps that catch the same thing as AOI, and a whole class of defects, like a reversed polarity cap that still powers up, that nothing catches. Coverage gaps matter more than slow steps.
4. Automation difficulty
Some steps can be automated with a serial command and a programmable supply. Others need a bed-of-nails fixture, test pads that don't exist on the current revision, or an RF enclosure. Be honest about this column. It's where schedules die.
Put It on a 2×2
Combine lenses 1–3 into a rough ROI score, then plot it against difficulty. Here's where steps commonly land on a typical connected device:
| Low difficulty | High difficulty | |
|---|---|---|
| High ROI | Do now: power rail voltages and current draw under load, sleep current, firmware flash plus version and checksum readback, writing serial number and calibration constants, LED/buzzer checks with simple sensors | Plan for it: full functional test that needs a bed-of-nails, sensor calibration that needs a reference stimulus, boundary scan if the parts support it |
| Low ROI | Automate when convenient: label checks, steps already covered by AOI/ICT | Defer or dedicate: cosmetic inspection, conducted/radiated RF performance (usually a separate station or sampling plan) |
The top-left box nearly always pays back within the first production build, and it needs little more than a programmable supply, a DMM or SMU, a USB/serial link to the DUT, and a script that logs real numbers.
Mistakes I See Teams Make When Scaling
- Automating everything before the design freezes. Every board spin breaks the fixture. Automate what's stable and keep the rest manual but measured.
- Building the fixture before the DUT has test points. Testability is a layout decision. If the pads aren't on the fab drawing at DVT, there's no fixture at PVT.
- Logging pass/fail instead of values. A 3.29V reading against a 3.20–3.40V limit is a pass, but a population drifting toward the limit is a yield problem coming your way. If you only store
PASS, you'll never see it. Log the measured value, the limits, the serial number, the station ID and the firmware version, every time. - Setting limits from the datasheet alone. Use golden units and limit samples, then tighten from real distribution data once you have it. Limits set too loose let defects escape. Limits set too tight wreck your first-pass yield (FPY).
- Never checking the station itself. If two stations disagree on the same unit, which one is right? A station self-test and a basic GR&R study answer that before the CM asks.
Where to Start Tomorrow
Take your current spec and add four columns: units × seconds, judgment call?, failure modes caught, what it takes to automate. Fill them in honestly. In an afternoon you'll have a ranked automation backlog and a list of coverage gaps, and that's a better station plan than most teams have going into PVT.
If you want a more structured version, I put together a free Pre-Production Test Checklist: 12 questions to answer before you build a test station.
Planning a production test station? Send me your test spec and I'll tell you exactly where automation will pay off — no obligation, just a straight answer from someone who's built these systems.