By clicking “Accept All Cookies”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Privacy Policy for more information.
All posts

Car manufacturer recalls: an engineer’s look under the hood

The true cost of a recall is hard to calculate, but the root cause is almost always the same

min read
If you’re asking ‘how do we build a safer car?,’ you’re asking the wrong question.

Cars today are extraordinary and complex products.

They’re made up of 30,000 mechanical parts, 6km of electrical wiring, 100 million lines of code, and there are 2ˆ40 possible variants for each manufactured model.

Getting all those elements working together is an incredible feat of synchronized engineering. The more cars become like moving supercomputers, the more complex this synchronization process becomes.

The alignment it takes to create a car (or any connected hardware product, for that matter) is very unlike engineering software-only products. With software, engineering teams can work on their tasks separately, and then put it all together at the push of a button.

With connected hardware, the complexity comes not only from the difficulty of the engineering task, but from all the dependencies between the mechanical, electrical, and software pieces of the puzzle working together. Simply put: a car is more than just the sum of its parts.

Imagine you’re an engineer. One of the 10,000 engineers working on getting a car from concept to consumer.

First came your childhood obsession with LEGOs. Then, you started taking apart and rebuilding your own computers. This led you to study electrical engineering at one of the elite universities in your country, where you graduated at the top of your class. Thanks to the internship you completed, you seamlessly transitioned to your first real job at one of the leading automotive manufacturers in the world.

It was the proverbial dream job — you got to work on cars. Not just any cars, but the most innovative, best performing, must-have models that won’t yet be in the market for another four years.

You knew that the degree of complexity would keep the job interesting, and that working alongside some of the smartest engineers in the world would challenge you every day. What you didn’t anticipate, though, was how little time and flexibility you would have to do actual engineering work.

You and your fellow engineers spend around 70% of your time on documenting and sharing engineering knowledge.

It’s a lot. That’s because this type of highly specialized information is incredibly hard to transfer. It involves engineering principles, minute details, physical properties, and years of hands-on experience that is nearly impossible to express in any format or medium. It typically lives in Excel sheets, distributed databases, CAD files, 2D illustrations, BOMs, and extensive instruction manuals in PDF.

But to understand the engineering logic behind all of that information, it’s not enough to just review the CAD model or read through a 26,800-page manual. You have to connect the dots and grasp the dependencies between the massive amounts of specialized information in the context of the actual product.

To get even a nugget of applicable insight in an overwhelming sea of data, you must go straight to the source. You must get to the people who created the information. In your experience working with thousands of engineers, each holding just one specialized piece of an ultra-complex puzzle, that’s easier said than done. In fact, it’s damn near impossible.

Grasping the engineering logic often involves navigating around different teams, multiple levels of hierarchy, and several time zones just to get to the right person.

It’s exhausting. And as you can imagine, in large engineering organizations like yours, this manual way of knowledge sharing from person to person is essentially a high-stakes game of telephone. It can lead to critical insights getting stuck in silos, slower innovation cycles, and a higher likelihood of errors and wasted resources.

You and your colleagues are frustrated that you spend the majority of your time on this process. But given the lack of better tools for your specific engineering needs, you learn to live with it.

Until one day, boom! Your worst nightmare has come true: they announce safety recalls of the exact car model you’ve spent several years working on :(

Your day — and your plans to chill at home and watch Brooklyn 99 — are ruined. 350,000 cars of the vehicle you’ve (metaphorically) poured your blood sweat and tears into have been recalled. Even worse, the recalls were due to an unknown electronics failure — exactly your domain.

What could have caused it? While you allow yourself to wallow in self-pity, 350,000 vehicles are being brought to various repair shops all over the world to figure out what happened and how to fix it.

Your colleagues reassure you that recalls are almost a rite of passage for any automotive engineer. They can happen to anyone and they’re not a signifier of your engineering ability, but they still suck so hard. They’re expensive and public and awful and, of course, absolutely necessary for the safety of the public.

Recalls are a symptom of a larger complexity issue

In fact, recalls happen a lot more than we realize. In the last decade, they outnumbered car sales by 201%

They’re a huge pain point for every automaker around the world. The true cost of a recall is hard to calculate — it goes beyond the direct financial impact, the risk of lawsuits, and the negative press.

For one, nowadays car brands aren’t as close to their customers as they used to be, so brand perception is much more important than ever before. A major recall can potentially affect public trust in that brand’s safety, which will impact its sales numbers. But we must also consider the impact they have on the environment.

Given the inertia of engineering knowledge, it often means that local engineers might not be able to fix the problem onsite.

The affected cars are taken to the nearest repair location, to which the engineers responsible for the malfunctioning parts must travel to educate the local experts.

Then, the vehicles are parked awaiting to be fixed before they can be taken back to the customer. If the error cannot be identified, thousands of cars would have to be shipped to the company’s regional engineering hub or be scrapped entirely — an ecological nightmare you want to avoid at all costs.

If engineers can prevent errors from happening in the first place, OEMs would also significantly lower their environmental footprint. A win for all.

So, in the interest of public safety, customer satisfaction, and minimizing the financial and environmental cost of recalls, you must find and fix the failure that caused them.

By now, you know what happened. Your cars were recalled because of a recurring error in the automated parking function, which affects various peripheral systems — something you were heavily involved in engineering.

After investigating some of the affected cars and running some tests, you have narrowed it down to a list of suspects. It seems some of the electrical control units (ECUs) get overloaded by redundancies in the communication signals triggered by a specific, yet common combo of functions. To fix this, you have to map the communication paths in the wiring harness and restructure the redundant connections to balance the ECUs.

Knowledge flows in the automotive product lifecycle

57 phone calls, 294 emails, and 42 meetings later, you and your team identified the root cause of the failure and were able to correct it in all 350,000 cars.

This involved you going to the Projektbeauftragter through your team lead’s boss’s boss, who gave you the name of a colleague in Detroit, who identified the correct person in the Shanghai office that had worked directly with the ECU supplier, who was finally able to give you the information you needed.

You made sure they were up to the highest safety standards, so the vehicles can finally go back to their respective owners. You can breathe a sigh of relief. All’s well that ends well, right? Not exactly.

For automakers, safety recalls are just the tip of the iceberg.

The growing quality problems are widespread. They’re a symptom of a bigger issue that impacts all types of complex connected hardware products — not just cars. For one, there’s increased cost pressure and competition in the market, which drives manufacturers to accelerate development cycles.

This means products only reach maturity when they’re at the customer. In the past, there was a larger safety buffer before new products were brought to market. Nowadays, this buffer has shrunk to the point where potential errors come to light much faster and more publicly than before.

The lack of transparency also leads to a lot of waste. Wasted engineering potential, wasted opportunities to improve products, wasted money, wasted material, and a shorter product lifecycle. With environmental legislation, the automotive industry risks catastrophic & expensive consequences if we don’t meet sustainability targets.

The transportation sector accounts for 1/5 of worldwide emissions and for each car, 80% of its total emissions are decided in the design phase.

Lifecycle assessments (LCA) typically calculate vehicle emissions in two phases — during production and during use. To give a very rough illustration:

For every 1kg of car weight, 4.25kg CO2 are emitted during production and 30kg CO2 during use. The average car weighs about 1.5 metric tons, which means 51.4 tons CO2 will be released into the atmosphere throughout its lifecycle.

The global annual vehicle production output is 78 million vehicles. If we can optimize materials in the design phase to make each car just 1kg lighter (without compromising quality or functionality, of course) it would mean saving around 2.73 billion kg CO2. That’s the equivalent of 6 million flights from Berlin to London and back.

But to uncover these potentials for optimization, we first need to deal with the system complexity and knowledge inertia in the product lifecycle.

The product complexity will continue to steadily increase, while the need for products to be safe remains a constant. How do we solve this?

Hire top talent and skillset, check.

Get the best materials and suppliers, check.

Allocate sufficient resources & prioritize, check.

Put strict regulation and safety measures in place, check.

You and the people you work with are highly intelligent and accomplished people with millennia of experience combined. It’s not for lack of resources or the will to address this pervasive issue — it’s in everyone’s interest to do everything they can to prevent recalls.

How big is your organization’s knowledge footprint?

The root cause is a small knowledge footprint in the product lifecycle.

According to Nick Milton who coined the term, a ‘knowledge footprint’ refers to how far knowledge spreads within an organization. The way knowledge flows at engineering organizations of this scale means there’s a 10:1 ratio of insights generated to insights applied — that’s an astronomical waste of potential.

Knowledge is stagnated. Siloed. Squandered. And automakers pay the price with recalls.

Blame it on the toolset from last century — there’s no transparency, traceability, collaboration, insight, or contextualization of product data.

PLM systems are designed as repositories for data from across the entire lifecycle. They don’t qualify or contextualize the data as it relates to the product. They were not built for engineers to interact with the data, so teams have to find other ways for engagement in their day to day. They don’t provide insights or analyses, so they don’t influence your engineering decisions.

Think about it like this: if you want to get from São Paulo to Manchester, the tools at your disposal would give you a globe, 2D maps of both cities, and separate, alphabetized lists of transportation methods and locations along the way.

What you really need is an intuitive step-by-step guide that suggests the most efficient route with accurate geotags and lets you buy the tickets directly on the platform while automatically syncing your itinerary on your calendar and suggesting what to pack based on the weather.

On the flip side, there are many domain-specific engineering tools that are excellent at solving specific problems for different use cases.

They’re all the rage with your peers — the latest technologies for generative engineering, geometric search, simulation, diagnostics, predictive maintenance, etc. They’re cutting-edge solutions, intuitive and delightful to use, and you can actually justify implementing them because they show tangible benefits!

The problem with these targeted applications is that, by definition, they’re separate. They don’t help you break out of the knowledge silos in their target domains or functions. Those solutions might work really well for a team of 10, but they don’t scale the benefits for an organization of 10,000.

They won’t help with managing system complexity or recalls; the biggest problems in engineering are cross-functional.

It’s like that axiom that humans only utilize 10% of our brains: today, engineers can only tap into 10% of the intelligence generated along the product lifecycle.

But what if you could access 100%? If you and your 9,999 colleagues could instantly tap into a single comprehensive, objective, shared engineering brain?

One way to prevent recalls from happening in the first place is to collect and connect all the engineering intelligence needed in the process of creating a car. Make it intuitive, interactive, and available to every engineer working on it. A single engineering intelligence network to connect the entire product lifecycle.

SPREAD’s engineering intelligence network

The best engineers deserve the best toolset — a new medium for engineering intelligence.

That’s our vision at SPREAD: to power the exponential growth of the world’s engineering intelligence.

A single source of objective, intuitive product intelligence that’s made with engineers like you in mind. For you to create and share your engineering experience through product information. To engage with your 9,999 colleagues. To instantly access their collective intelligence and augment your own. To free your time and budget from chasing down information halfway around the world.

A platform that gives you domain-specific insights at the micro-level, with contextual, universal system understanding at the macro-level. Direct feedback loops that notify you of a recurring need for rework in Production, so you can prevent it from happening in R&D. Intelligent models that continuously learn from each interaction to help you identify inefficiencies and remove redundancies at each step.

Engineering intelligence to simplify complexity management. Prevent vs react to errors. Repair a small part vs scrap an entire vehicle. Identify opportunities to improve the product and reduce waste. Instantly share engineering knowledge at scale. More intelligence with less effort. More collaboration, less synchronization.

More engineering, fewer recalls.

More from our stories

Read more
Pauline Müller

CUBE'23 recap: how to master product complexity on a whole new level

min read
Read more
Pauline Müller

How to break knowledge silos for innovation: empowering engineers through access to product data

min read
Read more
Philipp Noll

A few thoughts about SPREAD’s recent $16 million Series A round

min read