Globe with pictures of people, global communication network

Microsoft XC Research

Why and how to test your ideas to ensure success

Share this page

By John Westworth

people working as a team to solve a problem

Image credit: iStock 

In part one of this blog post, we introduced the concept that most new ideas, features, or products are destined to fail, and why. We’re going to continue on those thoughts with more reasons why, and how to mitigate them:

You’ve over engineered the solution:
The Roundabout PlayPump was initially met with excitement when it was announced – It was a merry go round that pumped water. The premise was that while the children were playing, they’d also be pumping water. However, the reality was different. It was too complex to maintain and difficult to repair, especially in the environments in Africa where it was being installed. A simple hand pump was far cheaper to install and a lot easier to fix if it stopped working.

While this is an extreme example, there are many examples where solution technically solved a problem but the trade offs didn’t make sense. They just ended up being too complex to use and to support.

Ask yourself and your team – “What’s the simplest way we can solve this problem?”

You listened to the Hippo
Hippo – Highest paid person in the Office. They’ve just read the latest buzzing business book, met one of their most vocal customers, are desperate to find ways to hit a metric that’s been imposed on them or just had a brainwave over the weekend. Either way they’ve got a brand new idea that’s got to be implemented.

It’s amazing how the necessity for being data-driven or data-informed can just get thrown out the window depending on the authority of whose idea it is. But this is is incredibly dangerous, In his paper “Power and Perspectives Not Taken” Aadam Galinsky notes that people with power find it more difficult to take someone elses perspective. Which means that their idea, while great to them, may not be so good for your users and customers.

Which leaves you in a bit of a pickle – how do you tell the emperor he has no clothes?

Either bring data, suggest additional research or testing the idea with customers or just go back to your companies or products mission statement. At Microsoft an easy question to ask is “How does this idea help our customers or users achieve more?” But please, don’t just go and implement it.

You’re solving a symptom, not the problem
Taking Ibuprofen may treat your toothache, but the underlying problem may be that you not taking care of your teeth as well as you should.

You can keep pumping are into your tire, but unless you fix the puncture it’s going to keep going flat.

And you can keep trying to solve the immediate problem of your users, without looking at the context, systems and incentives that are in place.

While you may treat the symptom may make the immediate issues, the root cause may still remain – and actually be getting worse if left untreated.

Use the 5 Whys to try and get to the root cause of the issue. Keep digging – you may be surprised at what you uncover.

You looked at the problem in isolation
Most problems aren’t caused by people in isolation – they’re caused by the context, environment, systems they inhabit and the way they’re incentivized.

If we just solve their problem in isolation we may cause further issues upstream or downstream of where the work that they’re doing.

For example, you may incentivize someone to do A and in the process they neglect B, C and D.

Map out your customer’s entire journey, not just the specific problem you’re looking at. Ask them what happens immediately before and after to get a sense of the context of the issue.

You solved one problem but created others
We’ve now democratized reduced the friction of creating presentations and holding meetings and democratized it. Now everyone can create fancy presentations and hold a meeting at moment’s notice. Hooray.

But wait a minute – The premise is that most presentations and meetings are good, therefore we should have more of them and invite more people.

But what if that premise is wrong? What if most presentations and meetings aren’t so good? What if making it easier and having more of them isn’t such a good idea? What if we’ve now made it easier for people to waste other people’s time?

Doing a premortem of your idea and taking on different roles is a great way of thinking through the unintended consequences of implementing your idea.

You’re solving your employer’s problem and rationalizing it as a customer problem
We humans are great at coming up with ideas then retrofitting the justification later. I bought this Tesla because it’s economical and good for the environment, definitely not because I wanted to have a cool car.

Sometimes a focus on our company’s strategy will make us come up with ideas that we then justify as being pro-user. A common example of this is having usage as your KPI or OKR or whatever the latest term you’re using for Goodhartesque(2) target is.

While you may hit the target, you may miss the point – Yes, forcing your users to use your product will drive metric X but will also increase resentment and disdain.

While company strategy is important, we need to balance this against customer outcomes.

You’re looking at the problem through the lens of your product or role
Sales of your product are down – Depending on your role it’s a research, design, development, sales or marketing problem. In reality, it’s probably more nuanced than that. It’s usually a combination of all of these. What tends to happen is instead of coming together to solve the issue each role will go away, look at it through the lens of their role , and try and solve the problem in isolation.

A similar thing happens if you have a portfolio of products. Each team will look at the same problem, in isolation, through the lens of their product. This leads to disjointed products, duplication of features, incomplete solutions and confusions for users.

Take a step back and get different perspectives on the issue. You’ll get a different, more coherent way of looking at the issue.

We need to take a good hard look at the problems we’re solving before we get into solutioning. Yes, coming up with solutions is the interesting and exciting piece but once you’re on this path it’s very hard to turn back. We fall in love with our idea, we fall for the sunk cost fallacy, we believe that A/B testing will fix the problem.

So yes, your solution may be incredibly innovative and completely cool – but does it make life better for the user? Is that increase worth the cost (both initial and lifecycle)?

No doubt you view your project differently – “of course there’s a market need for it” I hear you cry. Your confidence may be sky high because of one or more of the following:
• A customer asked for it.
• A competitor has it
• You ran a focus group and people loved it.

But beware – I can guarantee that someone on the team can use those very same reasons to justify their idea.

So before jumping to a solution, take some time to check you understand the problem. You’ll save yourself, your company and your customers time and money.

What do you think? How can these ideas help you design better products? Tweet us your thoughts at @MicrosoftRI or follow us on Facebook and join the conversation.

John Westworth is a Design Researcher in the Office Design and Research team. He is passionate about providing leadership to help companies implement change and manage transformation to use new technologies that impact how they work. Check out John’s other blog articles, “A responsible approach to innovation,”Conflict: The missing ingredient and biggest test of a growth mindset,” and “The true cost of your idea.

(2) Goodhart’s law – “When a measure becomes a target, it ceases to be a good measure”