Usability Testing in Web3 - Why it’s Crucial and How it’s Different
Posted on Dec 2, 2022
Disclaimer: This article shares best UX practices that we’ve seen over the years in designing for crypto & Web3 products. YOU are responsible for the safety of your funds, tokens, and assets. This should not be taken as financial or custodial advice, and we recommend that you consult with a financial advisor.
The Case for Testing –
When it comes to product design, usability testing is a great tool for unlocking the unique selling proposition of your solution. It’s a chance to test the assumptions that you’ve applied during your design phases. When performed well, it has the potential to steer your efforts, and maybe most importantly, your resources in a productive direction. This is why our team of Web3 UX/UI designers at Matcha Design Labs (MDL) swear by user testing when working on products like dApps, decentralized exchange(s), and web3 wallet(s) for clients.
The great thing about testing is that it comes in all shapes and sizes. So, whether you have endless resources or you’re a team of one - there is a version of user testing that can help you glean insights. Need more convincing? Check out this evidence to suggest that speaking to as little as 5 potential users could help guide you in the right direction.
As we consider how the design process applies to the wild west of Web3 product development - it remains that testing is an unparalleled way to validate and invalidate efficiently along your product roadmap. However, as we pave new roads through this uneven terrain we are recognizing that there are unique considerations when it comes to speaking to users about what they hope to see built in the space. Here are a few things that we are noticing as we take this journey with users and builders.
Providing Context vs. Leading the Witness –
In a user study we might moderate for Web2 products it’s crucial that we attempt to set clear context for study participants prior to our live conversation. Typically, this includes setting the scene for the conversation with a quick summary of the product’s goals followed by a snapshot of how the user has landed at the interaction we’ll be evaluating together. We may also kick off the conversation with a more general opener such as, “tell me the last time you performed X activity”, applying best practices for fruitful conversations as proven by resources like The Mom Test.
Jump to projects in Web3, this approach is challenging to replicate. Today, many users who we speak to have varying degrees of prior experience, it’s unreliable to assume that such conversations will begin with a common ground. While we can control this to a certain extent with pre-interview questionnaires and targeted recruiting - the truth is that the pool of “general adopters” is limited.
We find ourselves towing the line of providing useful context and leading the witness quickly. We often hear from teams that they have spoken to Degens and Crypto natives to get to where they are now, and they’re ready to speak to the masses who they hope to appeal to next. The challenge however, is to guide the rightfully curious newbies through “simple” workflows that have complex technical foundations. For example, we can evaluate whether a user can grasp performing a transaction using a wallet to send some ETH to their friend - but can we trust that they are actually comfortable with this type of transaction if let loose in the wild west on their own?
Continuing this example, what level of insight is important for the user to have about the construct of the Web3 wallet they’re using to send these funds within your prototype? If we show a newbie that during the process of creating a wallet they’ll create a password, a secret recovery phrase, and a private key will be generated - are we sure without us there that these technical components of the transaction might not quickly overwhelm them?
Together, with the intrepid UX designers on the ground today building this new world, we seek the sweet spot of empathetically hearing the concerns of the crypto-curious as well as the naysayers. It is our role to hold space for these uncomfortable growth periods where we sit as mediators between the trailblazing builders and their target audience. Our rule of thumb - sit in the discomfort a little longer, ask one more question to truly appreciate the concern of users and resist the urge to assert that, “this is just how it works”.
Hosting Great User Conversations
The great opportunity within this uncertainty is that as Web3 UX designers and researchers we have the chance to mine for the gold that will shape the world of Web3. Speaking to as many users as we can during the building process is our chance to mediate the conversation between the highly technical builders and the eager, curious markets.
Here is what we see as the best next steps we can all take to optimize user conversations across the ecosystem:
Keep listening, specifically noticing the words people are using. Today, we’re asking users to create Crypto wallets and transact using tokens, accept gas fees and collect NFTs - but it’s important to remember that this lexicon is as iterative as the products we’re building. As conversation facilitators, we’re on the frontlines of hearing how potential users break down these technical concepts for their own understanding. Take note of the language they choose, together this will help us continue to abstract the highly technical into universally adoptable concepts.
Get comfortable in the uncomfortable. Maybe you’ve heard the saying, “If you’re the smartest person in the room - you’re in the wrong room”. We think about this often on our team when it comes to participating in the Web3 space. It’s easy to feel overwhelmed by the technical developments building the scaffolding of the space, but it's important to stay in the conversation. Attend that webinar, go to that meetup, join that discord. It’s the mingling of perspectives and curiosity-driven conversations that will continue to enrich the world of Web3. Take this gained perspective into your user interviews, ask questions, share context - get uncomfy.
Craft strong opinions, loosely held. The power to learn and relearn is a key component to the productivity of user testing. It’s a practiced skill to moderate a user session keeping your personal bias at bay. Beyond avoiding leading questions and listening more than speaking - remain open to learning something new from anyone who you speak to. Remember that everyone, from technical founders and early adopters, to newbies and even our grandparents all have an important perspective to share as we co–create these new tools for our shared new world. Stay open to the idea that insight may come from where you least expect it, be prepared to advocate for any user, and be ready to change your mind!
Most importantly, keep hosting user conversations. Whether they are formal prototype-guided walkthroughs or casual coffee chats, these conversations will unlock the power of Web3.
At Matcha Design Labs, our team is constantly sharing examples, articles, and design ideas with each other so we can grow as a Web3 UX/Product Design Agency.
If you like the way we think and believe we could help your team, we’d love to chat! Drop us a note here :)