From recall to on-time launch: coordinating hardware and software after a field escape

A flagship mobility product with an electric propulsion system reached the market and was recalled shortly after launch. The escape shut down a production line at a cost of approximately $9M, and cost the company customer confidence that then had to be rebuilt.

Critical issues surfaced in waves and the organization moved into reactive mode under investor scrutiny, with a re-launch date that could not move again. Software shipped continuously. Hardware moved through sequential build, test and release cycles that could not match that pace. Integration failures reached production, and priorities churned as the team tried to respond to each one.

A failure in the electric propulsion system. Behind it, the supplier risk assessment had gaps. Failure modes that mattered were never carried into the supplier’s analysis and were never independently challenged from the customer side.

Underneath that sat an integration assumption nobody owned. Hardware and software ran on different release clocks, and no one held the interface between them. What “ready” meant was not shared across the two organizations, and the validation gates were not tied to real-world failure modes. Customers experienced one product; the teams shipped on two schedules.

After launch, through the recall.

The gaps in the supplier risk assessment, however, existed and were reviewable well before that. Nothing about them required a field failure to become visible. They required someone outside the supplier to read the analysis and ask what was missing from it.

Two tracks in parallel: contain what was already in the field, and rebuild the release system that let it get there.

• Containment. Launch-blocking issues isolated and a containment plan deployed to stop further field damage, with a shared triage rhythm so decisions could be made quickly.
• One release cadence. A coordinated hardware and software cadence with explicit integration checkpoints, so hardware constraints and software velocity could coexist without collisions.
• Bench validation. A bench-testing protocol built to detect hardware and software incompatibilities before release, converting tribal knowledge into repeatable gates tied to release readiness.
• Shared definitions. Engineering and product leadership aligned on what done, ready, validated and shippable actually meant, so decisions anchored to release criteria rather than to urgency.

Containment deployed immediately and the product stabilized in the field. Critical defects began being caught before release rather than by customers. Hardware and software releases were synchronized within roughly four months, and the product shipped to the customer on the committed date, under investor scrutiny and with no further slip.

A supplier’s risk assessment is not evidence until someone outside the supplier has challenged it. Gaps in that analysis do not announce themselves. They sit quietly until the failure mode they omitted reaches a customer.

And when hardware and software run on different release clocks with nobody owning the interface, integration failures are not accidents. They are scheduled, just not written down anywhere.

Recovery after an escape is possible. It is also the most expensive place to resolve a product risk, and the hardest place to rebuild confidence.