Summative Usability Test: Evaluating the LGBTQ+ Experience
Sponsor: Independent Student Work (No Affiliation with Tinder)
Course: ISE 215 - Usability Evaluation and Testing
Team: 5 HFE (M.S.) Graduate Students
Contribution: Expert Heuristic Evaluation \ Research Design and Strategy \ Data Collection (Moderating, Notetaking and Data Logging, and A/V Control) and Analysis \ Preparation and Presentation of Deliverables
In the Spring 2019 semester, 5 HFE (M.S.) students formed a team to evaluate the usability of Tinder (mobile app v. 10.10.1), a requirement for a graduate course on Usability Evaluation and Testing (ISE 215) at SJSU. The study evaluated performance, behavioral and subjective usability parameters of the Tinder experience, through critical user flows/tasks, and with representative users.
Tinder mobile is often marketed as LGBTQ+ 'inclusive'
The application is optimized for non-LGBTQ+ users and experiences
Evaluate the usability of Tinder mobile version 10.10.1 with representative users (LGBTQ+ and non-LGBTQ+)
Test performance, behavioral and subjective usability parameters, through critical user flows/tasks:
What is the overall usability (system-user performance, behavior, and subjective perceptions) of Tinder mobile version 10.10.1 for LGBTQ+ and non-LGBTQ+ users?
What is the usability of critical user flows/tasks?
How does the usability experience of LGBTQ+ users compare to the usability experience of non-LGBTQ+ users?
Quota Sample Matrix (N=20).
A total of 20 representative users were solicited, screened, scheduled for a 60-minute, one-on-one, in-lab usability session. Participants were selected to approximate data saturation across the following user categories:
Novice: < 6 months or a lapse longer than 6 months
Expert: > 6 months of regular use without a lapse longer that 6 months
LGBTQ+: Lesbian, Gay, Bisexual, Trans, Queer, +
non-LGBTQ+: Cisgender, Straight
In usability testing, data saturation refers to the stage in data analysis where no new information is discovered by the researchers, signaling the ability to cease additional data collection and analysis. A sample size of five (n = 5) in each user category is typically required to approximate data saturation.
Expert Heuristic Evaluation \ Usability Testing \ Interview \ Survey
iPhone X \ Samsung Galaxy with Tinder installed
Fictitious Tinder Accounts (Representative of Quota Sample User Categories)
Paper, Clipboard, and Pen/Pencil (For Notetaking)
Laptop with Google Sheets (For Data Logging)
Study Room with Glass Windows (SJSU MLK Library) and a Reliable Internet Connection (WiFi)
Chair and Table
A heuristic evaluation was conducted for researchers to familiarize themselves with Tinder mobile version 10.10.1 and to identify potential usability hotspots.
Sample (Local) Heuristic Violation: Visibility of System Status
A system should always keep the user informed of what is going on, through appropriate feedback within a reasonable time.
Task: 'Edit [Bio] Info'
Description of (Local) Violation: Users can edit their profiles (including photos, age, gender, and sexual preference) by tapping 'Edit Info' from the profile screen and scrolling through the 'Edit Info' wizard. Because the 'Photos' wizard element occupies the entire screen, users do not receive the feedforward required to scroll down for access to additional edit entry fields in the wizard. Instead, the user is directed to tap the 'Done' CTA before completing the entire 'Edit [Bio] Info' task flow.
Recommendation for (Local) Violation: Removing a row or two of photo placeholders from the 'Photos' wizard element and moving additional edit entry fields up will give users the necessary feedforward to complete the entire task flow.
Sample (Global/Universal) Heuristic Violation: Consistency
Users should not have to wonder whether different words, situations, or actions mean the same thing. It is best to follow platform conventions.
Task: Review User Photos
Description of (Global/Universal) Violation: To review a user's photos from the 'Matching' screen, users have to tap the left and right edges of the photo deck. To review photos from the 'Profile' screen, users have to swipe right and left through the photo deck. This inconsistency inherently changes one of Tinder's most popular interactions (i.e. swipe-to-match), which may confuse and frustrate users navigating back-and-forth between the 'Matching' and 'Profile' screens.
Recommendation for (Global/Universal) Violation: Tinder is well known for its 'swipe-to-match' interaction. The interaction mechanic used to review Photos from the Profile screen must match the mechanics of the 'Matching' screen (the main screen).
Lab-based, usability testing was conducted to evaluate the overall usability of Tinder mobile (v. 10.10.1).
Usability Parameters: Performance, Behavioral and Subjective Perceptions
User Groups: LGBTQ+ and non-LGBTQ+ Users
Sample User Goal: 'Match' with a local Tinder user that matches your sexual preference
Sample User Flow/Task: Create and complete a new Tinder account (Onboarding)
The following user flow visualizes the initial Onboarding task for Tinder mobile version 10.10.1:
10/10 non-LGBTQ+ participants were able to complete their Tinder account through the onboarding process
10/10 LGBTQ+ participants were unable able to set their sexual preference during the Onboarding process, extending the time it takes to complete the Onboarding task by an average of 98 seconds
A participant who identifies as straight, cisgender female expressed happiness/joy as they began swiping for potential 'Matches' immediately after completing the initial onboarding process.
A participant who identifies as gay, cisgender male displayed expressions of confusion/worry immediately after completing the initial onboarding process and initializing the 'Matching' process with female users.
10/10 LGBTQ+ participants rated the onboarding experience a 1/7, where 1 is Very Poor and 7 is Very Excellent
7/10 LGBTQ+ participants rated their overall experience a 2-or-below/7, where 1 is Very Poor and 7 is Very Excellent
During a post-interview, one LGBTQ+ participant (Cisgender Female, 23) reported the following about their experience with Tinder, specifically referring to the onboarding process:
"I think it's a slap in the face to our [the LGBTQ+] community"
By triangulating and converging on the analyzed data, the following insights were concluded by our team:
Tinder version 10.10.1 is solely optimized for straight, cis-gender, non-LGBTQ+ users
This sole prioritization of straight, cis-gender, not-LGBTQ+ experiences early in the onboarding process means that Tinder is not LGBTQ+ accessible
A report detailing and prioritizing usability issues and design recommendations was presented to the ISE 215 Usability Evaluation and Testing course at SJSU.
Critical Onboarding Recommendation:
Feature Gender and Sexual Preferences directly in the initial onboarding process.
This study provided insights into the overall usability of Tinder mobile (version 10.10.1) with representative LGBTQ+ and non-LGBTQ+ users, through main user flows/tasks. The priority given to the design of LGBTQ+ experiences must match non-LGBTQ+ experiences and be reflected in the UI.
User Experience Researcher | Google
"Victor was extremely easy to work with and always very open when communicating with the team. He does a good job aligning with stakeholders using his active listening skills to understand core usability issues and research questions. I was always impressed by the solutions he proposed, and any feedback given to him was incorporated thoughtfully."
Design Researcher | IBM
"During this project, Victor did an excellent job of leveraging both the needs of the user with the needs of product stakeholders in mind. He brought well thought out recommendations and insights to our data synthesis sessions. Victor craftfully executed this difficult research project by being highly organized and collaborative on the team. It was a pleasure to work with Victor."