Usability Testing for Leafly.com
I performed usability testing for Leafly.com, the world’s most visited cannabis site. The research examined how usable the site was for first-time or infrequent users looking to address pain relief. I used both quantitative and qualitative UX research methods to deliver key findings and recommendations for the owners of Leafly.com
Project Timeline: 1 month, February 2019 - March 2019
My roles: UX Researcher: Research planning | User testing protocol | Stakeholder presentation
Methods: Card sorting | Clickstream analysis | Talk-aloud usability testing
Leafly is the world’s largest cannabis site, with 14 million monthly users and 4.9/5.0 stars on the app store.
The Problem
Leafly.com is the world’s most visited cannabis website, with over 14 million monthly visits. Its founders describe Leafly as “a hybrid of Yelp and Consumer Reports for cannabis products.”
As of 2019, cannabis sales are forecasted to double in 5 years, with users 65 and over being the fastest growing market.
At the time of testing, Leafly.com’s design was heavily organized around cannabis strains. My hypothesis was that the strain-based terminology and site organization might be confusing to first-time or infrequent cannabis users. Furthermore, I wondered how easy or difficult it would be for users to find cannabis products specifically meant for pain relief.
Research Questions
How usable is Leafly for first-time or infrequent (< once per year) users?
How well does Leafly help users find the range of products for pain-relief?
The Approach
Three methods were used that provided both qualitative and quantitative data:
Card sort
Clickstream analysis (unmoderated usability testing)
Talk-aloud testing (unmoderated talk-aloud usability testing)
Card Sort
Software: OptimalSort
Participants: 15; 8 <1 year cannabis users, 7 >1 year cannabis users
Task: Sort terminology found on Leafly.com
Why this method: This method focused on the first research question, “How usable is Leafly for first-time or infrequent (< once per year) users?” I wanted to know if infrequent cannabis users understood the terminology on Leafly’s site. Was there a difference between frequent cannabis users understanding of Leafly’s terms vs. those that consumed cannabis infrequently?
Participants used OptimalSort to help determine their familiarity with the terminology on Leafly.com. There were 15 participants who sorted 29 cards in a hybrid sort.
Data:
Card sort data:
Similarity Matrix
Time on task to sort cards
Post-study questionnaire:
Frequency of cannabis use
Daily | A few times per month | A few times per year | Less than once per year | Never | I prefer not to answer
How easy or difficult did they anticipate sorting would be? How easy or difficult was it?
Any additional comments they had about the task
Key Findings
First time or less than once-a-year cannabis users
took 33% longer to sort
were 2x more likely to report feelings stressed or that they made a mistake
Clickstream analysis
Software: Loop11
Participants: 30; range of cannabis users
Tasks:
Task 1: Use the site to find the name of a type of marijuana that purports to offer pain relief.
Task 2: Use the site to find the name of a topical cream that you think would help with pain relief.
Why this method: I wanted to know—how easy was it for users to find cannabis strains vs a topical cream product? Leafly’s information hierarchy is centered around flower strains. Would users take longer to find topicals? Was it clear to the user that a particular type of cannabis product was good for pain relief? How would they navigate the site to find it?
Data:
Clickstream data:
Time on task
Page views and sequence
Video showing the task
Task success or failure
Self-reported questionnaire data:
Frequency of cannabis use
How easy or difficult they found each task (Likert scale)
Why they found the task easy or difficult
Key Findings
The home page of Leafly.com prominently displays a search bar. The search suggestions center around marijuana flower strains.
When participants searched ‘cream’ to find a topical cream, only marijuana flower strains with cream in the name appeared on the first page of search results.
“There were some search results that came up that did not match what I wanted - i.e. tropical instead of topical!”
100% of users who used the menu to find a topical cream found it without backtracking. However, the menu’s content is hidden until the user clicks the menu icon. Only 1 in 10 users used the menu as their first strategy to find a topical cream.
Talk-Aloud usability Testing
Software: Usertesting.com
Participants: 3; all infrequent (less than once per year) users of cannabis products, age 21+
Tasks:
You’re evaluating different products for pain relief. Identify three different kinds of cannabis products (flower, balm, edibles, etc.) that purport to offer pain relief.
Questionnaire:
What new things did you learn about cannabis products during your exploration that you might not have known?
Would you recommend this site to those looking to use medicinal cannabis products for pain relief? Why or why not?
Why this method:
Learn what users think of the site, learn what terminology they find confusing, as well as what works well for this audience.
Learn participants’ emotions around the way leafly presents pain relief options.
Understand how non-cannabis users find information about cannabis for pain relief
Key Findings
First time or less than once-a-year cannabis users showed
unfamiliarity with language “strains”, “topicals, tinctures” “hybrid /sativa / indica”
satisfaction with Leafly’s Cannabis 101 blog as a learning resource
confusion when the “pain-relief strains” page didn’t load
a strong desire to recommend the site to a first-time user
Participants rated the Cannabis 101 blog highly across the board. Furthermore, all three talk-aloud participants who had never used cannabis used the blog to determine what kind of cannabis products were suitable for pain relief.
“I love this blog and think it’s helpful for someone unfamiliar with cannabis like me. I think I’m going to go and read more once this task is over!”
The Results
Summary of Findings
How usable is Leafly for first-time or infrequent (< once per year) users?
New users found the site beautiful and engaging.
New cannabis users relied heavily on the Cannabis 101 resource. All talk-aloud testers said the resource was “helpful” or “very helpful”.
How well does Leafly help users find the range of products for pain-relief?
Leafly does a great job of listing the attributes and effects for marijuana strains, but could do more to apply these attributes to its other cannabis products.
The search bar quickly helped users find marijuana strains for pain relief, but hindered users finding other cannabis products. Furthermore, the lack of a clear navigation bar meant other cannabis products were not as easily discoverable as were the strains.
Bugs and slow loading confused some users, with pages sometimes erroneously showing “0 results” for a valid search.
Recommendations
How can we make Leafly more usable for first-time or infrequent (< once per year) users?
Continue to build out the Cannabis 101 resource | lower priority; medium effort
The Cannabis 101 resource is already very robust. This was the number one resource that helped users with the language barrier.
Make more information available in context (e.g. tooltips) | medium priority; medium effort
The 101 blog is great for users looking to get deep into learning, but once users are searching for a product, they should be able to quickly look up unfamiliar terms.
How can Leafly better help users find the range of products for pain-relief?
Fix “0 results” bug | medium priority; low effort
Many users were confused to find zero results in some searches.
Make menu options more visible from homepage | high priority; medium effort
Users who used the menu were able to find pain relief topicals without backtracking through the website, unlike those who relied on the prominent search bar. Further A/B testing could be done to validate this change.
Build out tags and taxonomy for non-strain products | high priority; high effort
Leafly’s stated mission on its site is to help its users find the right cannabis product for them. Currently, it falls short for people that are not looking for strains to smoke. Adding the wider range of products to Leafly’s search will help improve the fundability of its range of cannabis products, and cater to a larger set of cannabis users.
Read Other Case Studies
