Get the Most out of Research: Choose the Right Scale and Method
Every organization can benefit from research that builds understanding of its constituents—customers, prospects, members, partners, etc. But there are different types of research that can answer different questions, from big-picture, strategic questions, to very focused questions about a product, service, or message.
It’s important to choose the right type of research for what your organization wants to learn. This post will help.
Research Granularity: Macro, Mid-Range, and Micro
It’s important to focus your research at the right ‘scale’ in order to understand the audiences you wish to reach and serve, and serve the customers/members you have.
There’s macro research—big-picture inquiries that answer strategic questions. Macro research can be large market studies to help an organization understand market forces, trends, and the competitive landscape so it can make informed business decisions.
There’s mid-range research, to help understand, or even define, an organization’s target audiences, the problems they want to solve, and what they think, need, and want in relation to the organization’s offerings. This type of research is often done as a survey, to reach a broad audience. It can also be a series of focus groups, interviews, or field studies that help an organization learn more about specific audiences.
Finally, there’s micro research, which answers narrow questions about how audiences will respond to particular concepts, designs, content, or interfaces. This type of research might be done as usability tests, in-depth interviews (IDIs with or without usability testing), card sorts, tree tests, eyetracking studies, etc.
Macro, Micro, or Mid-Range: How to Choose
If you generally understand your market but have big gaps in your understanding of target audiences, you’ll want to start with mid-range research into audiences.
On the other hand, if you have big strategic questions about your market that will affect the direction your company takes, you should be investing in macro research—or reviewing other companies’ research that pertains to your market, and then doing your own study to fill in any gaps.
If you want audience/user feedback on a website or app, an ad campaign, content, a design mockup, a proposed navigation structure, or another artifact, you’ll want to conduct micro research to gather feedback and answer pertinent questions.
Qual or Quant?
Once you’ve determined the focus of your research—macro, mid-range, or micro—you’ll want to determine whether you need a quantitative or qualitative study, or a mixed-method study.
Quantitative studies are for when you want concrete measures for comparison and understanding, and/or a scientific sample size. Surveys are a good example of quantitative studies, as are tree tests (a form of navigation testing).
Qualitative studies are for gathering subjective input--opinions, how and why people do something, what people think. Interviews, usability testing, and focus groups are examples of qualitative studies.
These types are not mutually exclusive:
You can get qualitative data from a largely quantitative study. For instance, we always include open-ended questions in surveys.
You can get quantitative data from a qualitative study. For instance, we gather empiric measures of task performance in our usability studies by putting a numeric rating on task performance, which allows us to compare task performance across tasks and audience groups. We also ask numeric scale questions in interviews, such as, "How favorable is your opinion of [company]?" This helps us understand and differentiate our participants.
Choosing Qualitative Research Methods
One mistake we often see is organizations choosing the wrong qualitative research method for the questions they need answered—for example, trying to get feedback on a website by using a focus group.
If you’re looking for personal perspectives on something specific, such as a website, app, physical product, TV show, etc., that’s a micro study in scale, and you’ll get the most benefit from interviews (possibly including usability testing).
Focus groups are great for getting input on the problems people want to solve, gathering sentiment on issues or messages, and getting feedback on general concepts, but they are not good for getting user feedback on websites, apps, or other products. One-on-one interviews are best for that.
Keep in mind that qualitative research typically will not give you a scientific data sample, unless you do a very large study. The value of qualitative research is that it gives you subjective feedback and input, and identifies strengths and weaknesses of your concept or product, from a “user” perspective. Qualitative research gets you into the mindset of your audiences and helps understand why they do what they do and want what they want.
Choosing Quantitative Research Methods
Quantitative research gives you empiric measures on which to make decisions, and can give you scientific data samples, if your study is large enough.
Methods such as surveys, card sorts, and tree tests are examples of quantitative research methods. Card sorts and tree tests help understand people’s “mental models” of an information space—that is, how they might group and label information your organization provides, and where they would look for specific content in a given navigation structure.
There are many other quantitative research tools—eyetracking, “heat maps,” feedback devices embedded on web pages, etc. The key is to choose the method appropriate to what you are trying to learn.
Keep in mind that even if you don’t get enough responses on a quantitative study to constitute a scientific sample, the study still provides ‘directional data’ that is useful, as long as you understand any limitations and blind spots.
Getting the Most out of Research
The keys to getting the most value out of your research are:
Define what you want to learn.
Choose the right type of research at the right time.
Do the research.
Communicate your findings throughout your organization.
The last point is an important one. If you do a strategic study and don’t communicate the results throughout the organization, it’s less likely that your staff will operate with a common sense of understanding and purpose. you want everyone to understand and own your strategic direction.
Similarly, if you do a usability study for one product and don’t communicate the results across product lines or up the chain of command, you risk replicating the same usability problems in future design efforts.
Final Thoughts
Research is valuable, and some is better than none. That means, even if you don't think you have the time or budget to conduct research, there are creative ways to do it.
Start early—don’t wait until you’ve invested so much time, effort, and money in a project or initiative that it would be costly and difficult to make changes if your research tells you something you didn’t know before.
Aim for diverse perspectives and experiences in your participant pool. Think about their career stage, job/role, organization type, race/ethnicity, gender, disability status, age, location, and other characteristics relevant to your organization and what you want to learn.
If you are facing pushback about doing research—for instance, maybe others in your organization have misconceptions about user research "taking too long" or "costing too much"—talk to them about the time, effort, and money it would cost to fix problems you could have avoided by doing research. We’ve seen organizations develop whole products that were not appealing to their target customers, all because they didn’t do the right research, early enough.
There’s a lot to know about research beyond what’s covered here—for instance, study design, objectivity, safety and consent, and other considerations. If you have questions or would like to discuss your research objectives, please get in touch.
