Don't Measure Innovation the Easy Way

by Jeffrey Phillips

Don't Measure Innovation the Easy WayStop me if you’ve heard this joke before, but I think it is exceptionally relevant to measuring innovation.

A guy walks out of a bar one evening and sees a drunk searching for something on the sidewalk. He stops the drunk and asks him what he is doing. “Looking for my keys” the drunk responds. The guy decides to help the drunk and they explore the entire sidewalk under the streetlight. Finally, the guy turns to the drunk. “Are you sure you lost your keys here?” he asks. “No” the drunk replies “I lost them in the alley but there’s no light back there.”

In my way of thinking, measuring innovation, at the corporate level or at the national level is often like a drunk looking for his keys. Both are important missions, undertaken in earnest, but in the wrong places.

Innovation is often reported on using factors that are favored because they are measurable. Often a company or a university is measured by the number of patents it generates. Patents are easily measured and have some relationship to innovation, so reporting that a particular firm or region is “innovative” based on patent generation seems reasonable. States and countries are measured by intellectual property generated, patents or startup firms, which are all reasonably easy to measure and are aligned to innovation. While these measures seem valid and reasonable, this is really a case of looking for keys under the streetlight, because the real information about whether or not an entity is “innovative” is rarely quantifiable.

So we search under the streetlight called “quantifiable information” rather than search in the dark alley of qualitative factors like passion, culture, networks, engagement and so forth. In fact, I’d argue that patents and start-up firms are simply evidence of an environment that sponsors innovation, rather than good measurements of the innovation capability or potential. As an analogy, if I had a farm and great soil, which could be very productive, but planted crops that are ill-suited for the climate, I may appear to have an unproductive farm. If I have a farm with poor soil, but husband my resources, use appropriate fertilizers and carefully tend the appropriate crops, I may have a good yield. In the first case, a good infrastructure appears to be a poor producer, while in the second case poor infrastructure produces good results through hard work. Any in both cases, a disinterested observer would be misled about the actual potential of the farm.

Why might patents be a poor choice for innovation metrics? A good number of the patents generated are for new ideas, some of which may be converted into new products or services. Many patents, however, never leave the lab, so the ideas die on the vine or weren’t useful enough to be converted into products and services. Other ideas are protective or defensive patents, meant to defend an idea and make it more difficult for a competitor to enter a space. Many patents don’t add any new value to the market and actually make competition more difficult. Further, patents can be easily extended – look no further than the pharmaceutical industry which has mastered the ability to extend patents with new diagnosis or new delivery methods. This isn’t to say that patents are bad – quite the contrary, but they are not the perfect analogy for innovation that many people claim.

Further, consider startups. While a healthy entrepreneurial environment is critical for economic prosperity and growth, the number of startups in a country doesn’t say that much about how innovative the country is. If all the startups or entrepreneurial firms are grocers, dry cleaners and tailors, they are adding productivity and services but may not increase the standard of living or make the country or region more innovative.

Even other factors that we constantly use, like the distribution of education, can be misleading. Many Middle Eastern countries have high rates of college attendance yet exceptionally stagnant economies. It’s usually true that regions with higher education tend to be more innovative, but in many instances there are so many other factors at work that education by itself isn’t a determining factor either.

If these factors don’t necessarily indicate an innovative environment, why are they used so consistently? Because they are easy to measure, quantifiable and comparable across regions and countries. Other qualitative factors, like culture, employee engagement, passion, the ability to network, ability to fail and even go bankrupt without cultural penalties, rapid access to markets, tax policies etc are significant factors in whether or not a country or region is innovative, but they are much more difficult to measure, analyze and compare region to region. Further, many of these factors are “soft” factors and difficult to control, which means they take time to propagate in the region or society. So we, like the drunk, measure ourselves on indicators that give only a glimpse into the true reality, while the information we need lies in the dark corners where we refuse to tread.

What else are they saying about 'Stoking Your Innovation Bonfire'

Don’t miss a post (2,550+) – Subscribe to our RSS feed and join our Innovation Excellence group!

Jeffrey PhillipsJeffrey Phillips is a senior leader at OVO Innovation. OVO works with large distributed organizations to build innovation teams, processes and capabilities. Jeffrey is the author of “Make us more Innovative”, and

Leave a Reply