Metrics versus experience →

A very good look at why measuring stuff matters:

You can’t really argue that you made something better if nothing ever changes with how people use your product. Conversely, if you make a change and people start to use your product less, I don’t care what you did, but the evidence is fairly clear that you messed something up.

It's hard to disagree with Julie on this, even while I had a fair share of cases where wrong parameters were measured or with wrong tools. She provides a few good examples of wrong things to measure and instances where metrics fail us. Two of my favorites:

Understanding the cost of complexity: each time you add a new feature to your app, it’s likely that the metrics you are tracking will turn up positive (after all, before nobody used X, and now more people use X, and people don’t seem to be using Y or Z any less, so overall this feels like a win.) However, if you keep adding features, at some point, you’ll end up with what’s perceived as a cluttered and bloated product. Then, suddenly, some shiny new competitor will gain fast traction because everybody’s like “I love Q! It’s just so simple.”

The power of big bets: no metric can tell you what the bold strokes needed to win the future are. Imagine 2008, when smartphones were just starting to emerge. If you looked at the metrics for your website, you would have seen a tiny sliver of traffic coming from smartphones. You may have concluded, very practically, that you shouldn’t really invest too much into building for mobile since it’s such a small part of your audience. Today, we realize the vision and foresight of those who did bet big on mobile and reaped huge rewards. No examination of current behavior can accurately tell which way you need to leap. Strategic, long-term planning still requires much of the same thing it always did: trusting your gut.