The Madness of Measuring Design
I get asked pretty often how I have—or would—measure design in an organization. I’ve had the chance over the years to experiment with this from a few angles and to see it in both positive and negative lights. And since this continues to be an incredibly relevant, albeit maddening, topic for my friends in and around design practices, I figured it was time to do some writing on it.
So let's start with the high level.
Note: I’m writing from the perspective of leading design teams largely in US-based startups. Other contexts certainly bring other factors to this that I likely won’t be exhaustive in addressing.
What does it mean to measure design?
What are we really trying to figure out here? At its core, the question is: How do we determine the value that a design practice brings to an organization? Sometimes that translates to, “How do we justify our existence?” or “How do we make a case for more of us or for being included where we need to be?” or "How effectively are we changing behaviors?" But those are just the big existential questions—underneath them lie practical ones, like:
How productive are we being?
What’s the quality of our work?
How has our presence impacted goals? Experience? Humanity? Revenue?
How well are we connecting with our colleagues and our users?
How good are our ideas on the whole?
The meaning of “value” here shifts depending on who’s doing the looking. Executives will think about these measurements in a very different way than, say, a project manager might. And since we represent a sometimes wholly mysterious discipline, it’s our job to articulate our value in terms each of these audiences can understand.
Why's it so hard to measure design?
In short, it’s because design rarely owns much—at least, not in the traditional sense. We’re a servant team, one that facilitates connections between the organization and its audience. We’re often facilitating the connection between teams too. We don’t press the buttons that launch things, we don’t write the code that materializes things. What we do is usually the precursor to other teams’ actions—marketing, product, operations, engineering, etc.—that get things to the endpoint.
This dependency means that we’re at the mercy of how these teams prioritize addressing our findings, recommendations, and artifacts. And that timeline can be long. It could be months before something ships and hits the market. By then, will our colleagues remember our contributions? Will they think to share back the outcomes of what design delivered?
To measure the value of design, we have to keep the conversation going even after we’re no longer in the room. We have to track down results, analyze outcomes, and sometimes just remind people that, hey, design was here. But we don’t own the final delivery, and we definitely don’t own most of the fixtures needed to instrument the methods for measuring our impact.
Design's value is somewhat intangible
If this sounds difficult, it’s because it is. A lot of what we do in design is intangible. We spend our days observing, empathizing, solving problems, and making sense of complex interactions. We create recommendations, insights, and guidelines that shape how things feel and function. We lead countless sessions that cultivate and shape ideas. We live in a world where “if it can’t be measured, it doesn’t exist” is a prevailing philosophy. That’s a tough standard to be held to when what we’re delivering isn’t always something you can hold or count or track with ease.
I worked for a fantastic Head of Product years ago who would speak at length on the dangers of conflating delivery and design. Design is the journey—a journey composed of discovery and connection. Sure, delivery happens all along the journey in many forms, but that's an outcome of the journey not the journey itself. When these two get conflated—when we’re seen only as “making things look nice” or “polishing up the product”—our real value is overlooked. We’re left with colleagues who look at us strangely when we tell them, actually, delivery and design are separate things. Design is a process. Delivery is an outcome.
So how do we measure design?
Here’s where the madness truly begins. There isn’t a single, foolproof metric that captures the value of design. Instead, I think of measuring design as an ecosystem—different metrics, layered together, telling different parts of the story. Some are nearby and immediate, like velocity or estimates vs actual times. Some are medium-distance, like key experience indicators and the accuracy of design hypotheses. And some are distant, like cost to serve and return on experience, which take time but reveal the deeper impact design has on an organization.
But let’s take it step-by-step. If this sounds messy and complex, it totally is! And it’s a challenge I know a lot of us wrestle with. So this article is the start of a series where we’ll break down the constraints, the metrics, and the mindset needed to truly measure design’s contributions. Because if we’re going to measure design, we need to understand what we’re up against and what success really looks like.
Next up, we’ll look at the constraints: the appetite for measuring design and the instrumentation we need to get it right. Because if we’re ever going to make design’s value tangible, we need to understand the roadblocks—and find the way around them.