Why Your Product Analytics Should Work Like Your Car

Note: This article is the first in a 2-part series. The second article in the series is “How to Set Up Your Analytics Like a Car’s Immune System”.

What can your car teach you about product analytics?

That’s what we’ll discover in this article.

You know the saying, “what gets measured, gets managed”.

Metrics allow you to keep your finger on the pulse of what’s happening. They give the signal for when and where you should step in, to ask questions or add value.

But as you scale as a Product Leader, your scope grows. You can’t keep track of everything. Nor can you speak to every team, every day.

Think about the work you put in now to monitor your product.

If you had to take on two or three times more scope, would your current way of staying on top of things be scalable?

I’m guessing no.

You could put in more and more hours…

But you don’t need to burn out.

There’s a smarter approach: set up analytics in a way that empowers you to scale without overworking.

Why Most PMs Fail at Product Analytics

It’s easy to fail at analytics.

PMs know how important analytics are, so they put in the time and effort. Still, they usually fail for one of these reasons:

1. PMs jump straight into collecting data and defining metrics. They’re not critical enough on the what.

   Consequence: months, quarters, or even years instrumenting code, resulting in slow time-to-value.

2. PMs decide to monitor a large number of metrics, and receive an overwhelming number of alerts.

   Consequence: people get desensitized to the alerts, and they lose their effectiveness.

3. PMs create elaborate dashboards full of graphs and numbers, that quickly get overwhelming.

   Consequence: people don’t know how to make sense of this information. Or they don’t build a habit of visiting the dashboards regularly. After a while, they stop consulting them.

4. PMs and organizations eventually decide that current metrics aren’t working. This results from the points above.

   Consequence: they launch yet another project, the “one dashboard to rule them all”. The cycle starts all over again.

The answer boils down to the classic PM best practice: start with pain points, not the solution.

PMs know this when working on external products. But for some reason, they tend to forget it in their internal work.

Metrics, dashboards, and alerts are solutions. But what are the pain points you’re trying to fix? Define these pain points and then – only then – work on the solutions.

So how do you design an effective analytics system?

We can take inspiration from an unlikely source – automobiles.

What Cars Can Teach You About Designing Great Analytics

How do you monitor the health of your car?

The car’s “immune system” is made up of three parts:

1. The car dashboard provides an early warning system.
   When something is wrong, a light goes off. It doesn’t tell you what the exact problem is, but it gives you a general sense of where to start looking. For example, if the engine icon lights up, you start investigating there.
2. Technicians can run comprehensive diagnostics.
   Once the Early Warning System gets triggered, you take the car to the shop. There, the technician plugs it into a computer that identifies the exact issue. Running diagnostics on every system would take too long. But because of the first step, the mechanic knows where to start looking.
3. Car dashboards show you preventive alerts.
   Messages like “change oil soon” or “service due” remind you to take preventive actions to avoid further issues.

Together, the car’s “immune system” looks like this:

Think about this framework for a moment.

It addresses the same pain points you face when monitoring the health of your products!

You need:

1. Early warnings when things go wrong in your product. These signs should indicate where to look to further diagnose the issue.
2. A quick way to diagnose the exact problem. Once you get past the first step, your team needs to identify the root cause of the issue.
3. Continuous prevention of issues. You need to be alerted not just when there is a problem, but when there is an opportunity to prevent a problem.

Taking inspiration from your car’s immune system, you can design your product analytics with the same approach:

1. Start by defining a small set of key metrics to monitor the overall health of your product. Like lights on a car dashboard, these metrics won’t tell you exactly what’s wrong but will give you a sense of direction.
2. Define more detailed, diagnostic metrics. These will help you quickly drill into the problem once the first level of metrics is triggered.
3. Set up alerts to help prevent problems from occurring.

Final Thoughts

Cars can teach us plenty about setting up effective product analytics.

The three levels of their “immune system” are powerful ways to monitor the health of your product.

In the next article, we’ll dive deeper into how to apply this framework to set up your analytics. I’ll share practical tips and best practices that have proven effective throughout my career.

Soon, you’ll be turning this theory into a reality for your team.