A framework for insights
When developing products, we need basic insights and breakthrough insights. This framework breaks these down and highlights two common failure modes of UX research.
In a previous post, I explored Donald Campbell’s BVSR model of creativity and highlighted the importance of high-variance research for sparking breakthrough ideas. High-variance research focuses on behaviors and attitudes outside the norm and are critical for helping teams imagine breakthrough solutions, but of course this isn’t the only research we need.
For example, if a design change unexpectedly shifts a metric, usability testing can pinpoint straightforward fixes. Similarly, when building a new product, we need both baseline insights about common user needs and jobs to be done as well as some innovative ideas to differentiate our offering. Basic insights are crucial for the fundamentals, even as breakthrough insights are needed for differentiation.
A framework for insights
Zooming out a bit, we can organize all of the above along two key dimensions: how strategically relevant an insight is and how much behavioral variance is behind the insight:
Strategic relevance refers to how applicable a given insight is to whatever you’re working on. Low strategic relevance would be something that might be interesting but is not applicable to your project. High strategic relevance would be something that directly answers a key question your team is grappling with.
Behavioral variance refers to how much a behavior deviates from the norm. Low variance occurs when people act similarly, whereas high variance is when behaviors vary widely. Many tasks have both low-variance and high-variance components, and both are important. Take the example of choosing a restaurant: low variance behaviors include the act of considering prices and preferring a particular cuisine—most people engage in these behaviors. But there are also high-variance behaviors that go into this decision, such as prioritizing dietary needs, evaluating cleanliness, or even something like considering how Instagrammable the restaurant is. High-variance behaviors like these are more idiosyncratic.
We can put this into a 2x2 for thinking about how we can get both basic insights that consistently power the bulk of the products we build and the breakthrough insights that help products delight customers and stand out from competitors. And, importantly, we can also use this framework for seeing how to avoid some common research failure modes.
Breaking down the four types of insights
🤓 Basic insights
These fall into the top-left quadrant of the 2x2 and represent patterns of behavior or attitudes that are both highly consistent (low variance) and strategically important (high relevance). These are foundational insights that inform the product’s essential functionality, answering questions like: “What are the most common user needs?” or “What features must we prioritize to address the majority of use cases?”
These insights are the meat-and-potatoes of product development and reflect how most people behave or feel in common situations, which of course limits how helpful they can be in creating delight and differentiation. Common methods for discovering these insights include usability tests, simple intercept surveys, and AI-moderated interviews paired with AI-driven analysis.
🤯 Breakthrough insights
These are found in the top-right quadrant and arise from behaviors and attitudes with high variance. These insights capture the outlier behaviors that, while uncommon, hold the potential to transform a product or reveal opportunities for differentiation. They are “high-risk, high-reward” insights—digging into high-variance behaviors can lead to discovering something groundbreaking, but it also comes with the challenge of distinguishing what is strategically relevant from what’s merely “interesting.”
Breakthrough insights often feel surprising, sometimes even counterintuitive, but they can be the difference between a product people like and a product people love. Common methods for discovering these insights include 1:1 interviews, in-depth surveys, and AI-moderated interviews paired with human-driven analysis.
😴 “We already knew that”
The four most terrifying words for researchers are, “We already knew that.” Located in the lower-left quadrant, these “insights” represent one of the most common research failure modes, and it’s especially common when you’re new to a team. These insights are either inherently unremarkable (after all, they’re from the low-variance half) or reflect a lack of familiarity with the team’s current knowledge.
One way to combat this is to help your team document what they know before the research starts, which you could do through an insight mapping workshop or by documenting hypotheses and known behaviors in your research brief.
🤪 “Interesting but not relevant”
This is another dreaded phrase, although delivered in the form of a backhanded compliment. Insights that trigger such comments are in the lower-right quadrant and are interesting because they result from studying high-variance behavior.
As researchers, sometimes our curiosity gets the better of us and we assume that just because something is interesting, other people care. The truth is your team probably doesn’t care, unless it’s going to help them do their job, so when you find yourself in this quadrant, eliminate anything that doesn’t address what your team is working on.
An example
Imagine you’re building a system that helps restaurants build a website and online ordering system. You’ll need a ton of basic insights to build the essentials, such as what information people most value when considering a restaurant, how people prefer to pay, how important a loyalty program might be, etc.
But a breakthrough insight will be critical to go beyond the basics and build something that will make restaurant owners choose your platform over others. An example might be that while most people choose restaurants based on proximity or price (basic insights that can inform key product features), there are certain situations where users choose a restaurant based on its Instagrammability.
At first this might seem like a niche situation, but it has high strategic relevance because Instagram influencers can help a restaurant get discovered. Such an insight could lead to a feature that helps restaurants showcase gorgeous photos, social proof from other influencers, or Instagram-worthy features of the restaurant on their website.
Applying this framework
As a general rule of thumb, you probably want an average of 80% basic insights and 20% breakthrough insights. Of course, if you’re focused particularly on identifying differentiation opportunities, then those numbers might flip or you might even aim for 100% breakthrough insights. And if you’re operating in a well-known space and trying to help the team ship something as fast as possible, there might not be any appetite for breakthrough insights.
I’ve found this framework to be helpful both for making sure I’m targeting the right type of behavior(s)—usually a mix of low-variance and high-variance behaviors—and communicating insights that help move my team forward.
NOTE: This post was edited on 27-Oct-2024 to improve the definition of behavioral variance with clearer examples. This post was further edited on 1-Nov-2024 to change the label from “core insights” to “basic insights,” and again on 8-Nov-2024 to update the graphic to include color.