Tuesday Email: The Measurement Trap

Happy Tuesday!

Every Tuesday I'd like to offer strategies for the week ahead and a thought to fuel your action.

We all have KPIs, metrics, and data points we track to measure the health of our business.

A metric, or a group of metrics, drives decisions that empower us to improve what we care most about.

But here's the problem: as our businesses evolve, the metrics stay the same, and what we actually want to measure turns out to be nearly unmeasurable. Client satisfaction. Client empowerment. Clients feeling valued and feeling they're getting value.

I'm a metrics-analytical person. I love data, and I believe metrics matter.

But I started to see the other side.

Metrics alone are driving decisions, and they're crowding out gut judgment. What I've realized is that having numbers is like being the pessimist in the room: you sound the smartest, and you're hard to argue with. When you use your gut, you're the optimist. It doesn't sound as intelligent, doesn't sound as feasible, and can always be rebutted by data.

Some of the best companies, the best ideas, the most impactful things in our world have been built on gut. On having an idea.

What this research has taught me is that I don't have a clean answer. This is a journey I'm on to challenge my own analytical instincts, because what I've found is that measuring what you actually want is always harder than you think.

That tension is what I want to explore today.

There's a concept called managerialism: the belief that organizational complexity can be reduced to standardized numbers, replacing subjective, experience-based judgment with scalable, repeatable decisions.

And then there's something called quantophrenic fervor, which makes a simpler point: we built dashboards to get smarter, and instead we got more data points and less understanding.

We always walk into metrics with a lens. A belief about what we want to see. And that lens shapes how we read every number.

If a metric is high, we say "look how high it is." If it's lower than we wanted, we say "look at the trend." If two out of three numbers are good, we highlight those two and rationalize the third.

Metrics and statistics are similar that way: we can let them tell whatever story we want to tell. That's why metrics alone can't be the sole driver of strategic decisions.

Accountability has been quietly redefined to mean demonstrating success through standardized measurement only. That which can be counted is what counts. But the thing we actually want to measure? It doesn't count that way.

Then there's Goodhart's Law: when a measure becomes a target, it ceases to be a good measure.

When you set something as a target, people figure out how to hit it. Say our target is 99% client retention. Sounds great. But watch another metric, and you'll likely see fee cuts start to rise.

Client comes in upset; advisor remembers the retention number; advisor cuts the fee or gives a year free. The metric corrupted the behavior. Good intent. Bad outcome. The metric became a system for gaming.

Peter Drucker is often quoted saying "what gets measured gets managed." Turns out that was misattributed.

What Drucker actually argued in The Effective Executive is that knowledge work cannot be measured like manual labor, and that by the time results show up in numbers, it's often too late to manage the behaviors that produced them.

Metrics work beautifully in a factory: increase this, get more of that. But knowledge work is different. A decision an advisor makes today, how does that impact a family five years from now? An advisor who bends the minimums to take on a client who needs help, what does that cost them in time and availability down the road?

We look at the number in the moment, not what the number means later. Instant analysis. Instant gratification.

Medicine shows this clearly.

Publishing surgical success rates sounds reasonable; patients should know how their surgeons perform. But the outcome wasn't the intention.

Surgeons started refusing the hardest cases to protect their rankings. Their livelihood depended on their ratings, so they stopped taking the risks where the gain wasn't worth the cost.

Well-intentioned policy. Seriously misaligned outcome. The advisory world isn't immune. If everything points to AUM and new clients, do we unconsciously prioritize those over existing clients? If the metric is retention, are we quietly putting fees at risk to hit the number?

No metric comes without a negative side effect. That's what makes this hard. And the typical solution is just to add more metrics, which makes everything more confusing without making anything clearer.

Now AI enters the picture.

And it looks like the answer. It is not.

AI can surface more metrics faster, and it feels like a solution, but Goodhart's Law doesn't disappear when you add AI. It accelerates. If you don't deeply understand why you're tracking something and how you'll interpret it before you start tracking it, AI just amplifies the problem you already have.

I'm living this on my own side. Trying to measure AI adoption at our firm.

The more tools we build, the more data we have, and the less I feel like I understand what's actually working. So I go partly on gut. But when leadership asks "how are we doing," they want a number. We try to define utilization and impact in a single metric and track usage. But is usage being used the right way?

There's another tension with AI worth naming.

If AI improves turnaround time and output volume, many argue it degrades depth of value and contextual accuracy. My challenge: how do you measure depth of value? Is it retention? That's easy to game. NPS? Recency bias is a real problem that AI will accelerate, and you don't solve it with more AI.

The core value of what we provide is relational capital. Relational capital cannot be captured in a metric. Metrics don't measure trust. They don't measure loyalty. They don't measure the mutual obligation built over years. Retention is a piece of it. AUM growth is a piece of it. Wallet share is a piece of it. But as Drucker said, knowledge work cannot be measured like factory work. And when we build a business dictated by metrics alone, we are quietly converting relational capital into factory output. That is not the business we are in.

We do an exercise in our internal leadership training every year.

There's a wheel with different leadership focus points: self-awareness, communication with peers and directs, and so on.

On the back of the wheel is a number line. Each of us marks a number for how we think we're performing as a leadership team, and as a company. Then we pair up.

The number isn't the point. When I put a 2 and my partner put a 4, what mattered was the conversation that followed. Why did we land differently? What are we each seeing? The number opened the door. The conversation was the value.

That's where I think metrics belong: not as determinants, but as discussion points. A metric being watched shouldn't determine strategy. It should start a conversation, because there still needs to be some gut behind the number itself.

As we head into an AI-driven future, I think we need a real reframe.

A shift from a findings mindset, where data tells us what happened, to an insights mindset, where data starts a conversation about what's actually going on.

There's no clean answer here. This is an analytical mind (me) questioning my own assumptions about data.

I'm not offering a solution in this newsletter.

I'm offering a reframe. A check. One question to ask yourself before you step into the jet stream: Am I using my metrics to provide me an answer or to generate a better question?

The best is ahead!

-Matt