The document was a small piece I wrote for the junior designers in my team when I was in TuSimple

Use User Research 101 as part of your research checklist, as it is not a comprehensive list of what designers and product managers need to do. I recommend everyone to build their own ways of researching, yet keep the goals of research in mind.

Goals of Research

• For existing products: figure out the pain points and priorities of the pain points users have.
  • What pain points do users have?
  • What are the most pressing pain points that we need to solve for them?

• For new products: figure out the users, the scope of the product, and their priorities.
  • Who are the users?
  • What are their most pressing needs for the product and why?
  • What will the MVP look like?

• To identify potential product improvements that we can bring in the future:
  • What are the next steps for users? What are their plans?
  • What are the potential designs that we can prepare for the future?
  • What is the right order of priority for all potential product improvements?

Methodologies 

• Build your understanding of the product:
  • Understand the history of the product before conducting research.
  • Why did this tool not exist before?
  • How are engineers designing this platform before designers’ involvement?

• Understand the logic of the product:
  • Meet with current product owners for an overview.
  • Play with the tools by yourself.
  • Find out and read existing documents.

• Talk to the right person:
  • Most of the users in TuSimple can be divided into tech-users and non-tech users. Keep in mind the differences between them in terms of their skillsets, mindsets, and goals.
    • For example, PMs might only care about whether something has been finished, while engineers focus more on the reasons for system failures.
• Tech leads are not always the best person to approach. Usually, tech leads receive requirements from the users, but the actual developers are often the most familiar with the product.
  • Users, especially algorithm users, are usually the ones with the most complaints and pain points since they are the ultimate users for everything. They are also the ones that designers can ask for potential future requirements.

• Don’t forget to talk to product managers/project managers.
• Talk to the right amount of people:
  • Designers can talk to unlimited candidates and always have new discoveries. It’s important to build our own understanding about when we are ready to start the first draft of the design. Usually, it’s when:
    • You have talked to at least one user in each user group.
    • You already have some ideas on what you are going to design, and you know the new design will wow the users by improving their efficiency.
• Ask the right questions:
  • Observe first, before asking any questions.
    • Ask the user to show you how they would perform a task, while explaining their thoughts aloud. Take notes on where they get confused or frustrated (or say something positive).
    • It’s important to build your own understanding, not the understanding that users want you to build.
    • Not all users are good at summarizing and giving their product conclusions. Even with users who have good communication skills, what they summarized is second-hand information already.
    • Do not limit your observations to just the platform that you are designing. The tools we have are usually toolchains, meaning they usually have dependencies with other tools. Try to understand the whole picture during the interview.
• Use interviews to verify your assumptions, not just open-ended questions.
  • Use your judgment to evaluate the product, not users’. It’s easy to ask users direct questions on the product, such as ‘what are your biggest frustrations with the product,’ yet by doing so, we are handing our judgments to the users, and users don’t always have the right answer.
  • Instead, let’s try to ask more questions based on our observations and assumptions about the product, to find out what has blocked users and what has made users do extra operations.
• Look for the right information:
  • The only thing that we care about regarding the platforms and tools in TuSimple is whether they have boosted the efficiency of our users. In other words, we care about the overall efficiency of the system over the joy of users or the elegance of the design itself. I will talk about the design that should trigger your alarms when conducting research in the next section.

Evaluate the Product 

The following points are what I use for evaluating a product/design in TuSimple, and are also the key information that I always try to verify during research:

• Use cases
  • Does the system cover all the major use cases?
• User flow
  • Do users have a clear flow to follow? 
  • Do users receive clear guidance/cues on the next step? 
  • Do the flows align with the design? For example, do users have to jump back and forth to finish their tasks?
• Information/Status
  • Does the system indicate information clearly? Do they have proper hierarchies? For example, is the most important information the easiest one to find? 
  • Was the information designed based on certain logic, such as frequencies or priorities? 
  • Was the presentation of the information based on its nature? For example, are we using the best way to present data or just the easiest way? 
  • Does the system indicate changes of information clearly?
    • Can users be aware of changes of information/statuses easily? 
    • Can users differentiate between different information/statuses easily? 
• Has the system done enough to indicate the status of the system itself?
  • Are proper indicators provided for users to indicate what’s happening, such as whether the system is functioning or whether tasks will continue after users leave the current page? Are the wordings clear to everyone?
• Corner cases
  • Has the system thought enough about corner cases?
    • What are the potential errors the system might have, and how is the system addressing them?
    • Do users receive enough guidance on what they should do when errors happen?

Outcomes of Research

Before starting to summarize the research, remember:

• Provide conclusions. The purpose of summarizing is to give conclusions and judgments on the design plans and priorities, not just writing down everything users said.
• Keep the results short and clear. The quality of design research is evaluated based on the insights the research gave to design, not how long or how fancy the wordings are.

What must be included in a research report:

• Conclusions
  • The goal of the project/research, made simple
  • The challenge/problem, made clear
  • Key pain points that you discovered, ranked by priorities
  • Your plans and/or suggestions for the next steps, actionable
• Evidence:
  • The evidence that supports your conclusions on
  • The problems that you identified
  • The priorities that you made
  • The solutions that you are bringing up

• The evidence that you summarized helps you to design the product
  • User persona
  • Key use cases
  • User flow
  • Other design assumptions, such as preferences of users.

When providing conclusions:

• Keep the efficiency goal in mind.
  • Use the improvement of efficiencies as the principle to evaluate priorities.
• Think about ROIs.
  • Focus on design efforts that improve efficiency.
• Be sure they are actionable.
  • Align with engineers on their schedules.
  • Push the engineers to explore tech limitations.
  • Think about your own schedule.