Posted on : MAR 05, 2021
Are you working on a tight timeline with an even tighter budget? You can (and should!) still invest some of your energy into user interviews. It can be tempting to skip this step in product development when it feels like inviting more voices has the potential to derail plans to ship a product or feature. Research shows, however, that speaking to only 5 users can give you the best possible results from consultations.
At AMSD, we encourage our clients to make time for user feedback sessions whenever possible. To help them see the benefit of investing the time and energy for this critical step, we have streamlined our process into one that is organized and easy-to-ship. Getting the green light for conducting user interviews means flipping the switch and setting the process in motion within minutes. To help you create your own efficient and impactful procedures, we're sharing our user validation checklist and the inside scoop on how we ensure each session is the best it can be.
While your session may be adapted to include some and not all of these, a traditional user interview process consists of the following deliverables at AMSD:
- Study Guide -- a succinct guide outlining the motivation for conducting the research, the guiding research question, and an outline of the stakeholders involved and procedures in place
- Pre-Interview Survey -- results of a brief survey asking participants to share demographic info, relevant knowledge or experience, etc.
- Written Notes & Recorded Video -- interview notes and session recordings for each conversation with a participant
- Feedback Analysis -- a robust and actionable summary of recommendations drafted to present interview findings in the format of a spreadsheet
As mentioned earlier, one of the most effective ways that we can advocate for user testing is the promise of an efficient process. Some folks conduct formal studies which can take days to plan and an investment of tens of thousands of dollars. We have found that we can be effective and get good feedback with more flexible time and financial investments. With this approach, we can conduct research for our clients more often.
After collectively participating in hundreds of user interviews in various roles, our team has created a repeatable, templated, checklist for our process that allows us to focus on the joys of the conversations ahead while relying on our organization.
Here are the steps we follow for each project:
- Define roles + responsibilities
- Draft a study guide
- Create a pre-interview survey (if needed)
- Create a templated notes document
- Create a templated video consent form
- Customize an interview scheduling calendar with availabilities
- Refine the prototype to be tested
- Customize email templates to engage with participants
- Draft a templated analysis spreadsheet to be completed
Because the above materials have been drafted as templates, we can complete these steps and be prepared to reach out to participants in a matter of hours.
As our process has (and will continue to) evolve over time, we have introduced various tools to help us become increasingly efficient. If you're looking to establish a process or refine your own, we recommend the following tools:
- Google Suite -- we use a GDrive folder to store all documents in a central location we can easily share with our clients. We also use a Google Form to collect survey responses, Google Docs for collaborative note-taking, and a Google Sheet for organizing our results [💫Primary perks: all-in-one solution, collaborative and easy to share]
- ClickUp -- if you haven't already heard our team boosting ClickUp, you surely will. As our hub and central project management system it keeps us on track and organized [💫Primary perks: define project timelines, assign tasks, track progress]
- HelloSign -- we use HelloSign to securely process and store signed consent forms to ensure we remain professional and compliant [💫Primary perks: easy to use, central storage of documents]
- Calendly -- used to schedule our interviews, Calendly allows us to easily request bookings for interviews from our participants and manages our availability with ease. All AMSD team members can subscribe to our research calendar keeping us all in the loop on which sessions are scheduled. [💫Primary perks: free for one project, easily customizable, and directly integrated with Zoom]
- Figma -- ah Figma, our go-to design tool. Native prototyping capabilities make it super easy for us to create living designs to share with participants during our interviews. [💫Primary perks: where our designs already live, professional-looking prototypes, can design for multiple device types]
Alongside our checklist and our tools overview, we want to share with you how we have most notably optimized our user interviews over time. Our focus on effective moderation and refinement of our summary report have been two key facets in helping us reach new heights.
As we're about to conduct our interviews for a given project, we revisit our handy list of reminders and tips for hosting a smooth conversation. As interviews can be more art than science, it's always helpful for moderators to revisit this list:
- introduce yourself and the guiding research question to set the stage
- remind interviewees that your conversation is a test of the prototype and not of their abilities
- ask interviewees to share their thoughts as a stream of consciousness to capture their thinking in the greatest detail possible
- ask open-ended questions and avoid leading questions
- ask follow-up questions when possible, making the interviewee feel heard
- repeat complex feedback back to the interviewee when needed to ensure you've correctly captured their sentiments
- keep track of time
- avoid industry jargon
Refinement of our Summary Report --
Surely, each project will present unique requirements. Having said that, we have landed on a structure for organizing feedback that has been extremely well received across clients. Our summary reports contain the following:
- a summary of key insights
- a summary of actionable feedback
- a "quick win" proposed solution to accompany each of these points of feedback
- severity rating of UX impact and business impact
- a summary of language recommendations
Here is a deeper look at what is involved in each of these elements of the report:
- Key Insights
Our reports always start with an easy-to-digest summary of key insights observed throughout our discussions with potential users. Often, this is a bullet point list of 3-5 findings that we feel effectively summarize the sentiments of those with whom we spoke.
2. Actionable Feedback
This is the bulk of our report where we extract direct quotes from interviewees and list them for our client to reference in a single table. Here, we also group similar feedback and assign a label to identify who shared the feedback to provide additional context. (Where bias may skew the interpretation of feedback we may remove this label)
3. Proposed Solutions
Next to each piece of feedback, we offer an initial suggestion for how we might/where we might address the concern in the product. This is intended to spark conversation and guide conversation that may be picked up later on when features are considered. (Tip: don't spend too much time crafting an elegant solution with these. They are meant to capture initial ideas that are sparked during the time of the conversation, but may not hold as the product evolves)
4. Severity Rating
Next, we assign a UX impact rating and a business impact rating using this rubric to help our clients begin to prioritize feedback:
- Low - these items do not block the user from accomplishing the desired action
- Medium - these items somewhat block a user from accomplishing the desired action
- High - these items greatly impede a user's ability to complete the desired action
- Low - these items will not greatly impact the bottom line or affect the perception of the brand
- Medium - these items have the risk of impacting the bottom line for the better and could potentially increase brand adherence or loyalty
- High - these items have a high probability to positively impact the bottom line and create strong brand loyalty
These ratings often change as project priorities adjust over time, but they act as a helpful metric for identifying the greatest pain points and opportunities that surfaced during the interviews.
5. Language Recommendations
Interviews are an excellent source for identifying UX pitfalls and filling feature pipelines, but they are also a wonderful chance to hear how potential users choose to describe your product. Capturing language that is shared among interviewees to describe their journey throughout your product is a highly impactful way to meet your users where they are. Pay particular attention to the words they use to describe steps in a process, calls to action, etc.
We hope that you have found these tips, lists, and tools to be helpful in reflecting on your own research practices.
As user advocates, we have the responsibility to continue to provide users with a chance to have their voices heard in our product development. Speaking to even a handful of potential users can give you great, high-impact feedback.
When you're ready to jump into research or refine your current processes, consider using templated materials whenever possible. Also, explore how (often free!) tools can help you continue to build efficiencies. If there are two areas you might focus on optimizing first, consider repeatable and practiced moderation, and creating summary reports that will leave your team and/or your clients hoping they can make time for user testing with every feature release!
If you and your team are considering conducting user research of your own get in touch if we can support you in your efforts. We look forward to hearing from you!