SUMMATIVE USABILITY TEST: EVALUATING THE TINDER LGBTQ+ EXPERIENCE
THIS PROJECT IN THE DESIGN THINKING PROCESS.
Sponsor: Independent Student Work. No Affiliation with Tinder.
Course: ISE 215 - User Research and Usability Testing
Team: 5 HFE Graduate Students
Contribution: Expert Heuristic Evaluation \ Research Design \ Data Collection (Moderating, Notetaking and Data Logging, and A/V Control) and Analysis \ Preparation and Presentation of Deliverables
In the Spring 2019 semester, 5 HFE students formed a team to evaluate the usability of Tinder (mobile app v. 10.10.1), a requirement for a course in Usability Testing (ISE 215) at San Jose State University. The study measured performance, behavioral and subjective usability parameters of the Tinder experience, through critical tasks/user flows and with representative users.
Tinder mobile is often marketed as 'inclusive'
The application is optimized for non-LGBTQIA users and experiences
Evaluate the usability of Tinder mobile version 10.10.1 with LGBTQ+ users
Measure performance, behavioral and subjective usability parameters
Test critical user flows using task-based scenarios
What is the overall usability of Tinder mobile version 10.10.1 for LGBTQ+ users, across all critical tasks/user flows?
How do LGBTQ+ users perform and respond to their experiences through individual tasks and goals?
How does the usability experience of LGBTQ+ users compare with the usability experience of cisgender, straight users?
SAMPLE MATRIX (N = 20).
Novice: < 6 months or a lapse longer than 6 months
Expert: > 6 months of regular use without a lapse
LGBTQ+: Lesbian, Gay, Bisexual, Trans, Queer, +
non-LGBTQ+: Cisgender, Straight
Heuristic Evaluation \ Usability Testing \ Interview \ Survey
iPhone X \ Samsung Galaxy with Tinder and Screen Recording Installed
Fictitious Tinder Accounts (Representative of Quota Sample)
Paper, Clipboard, and Pen/Pencil (For Notetaking)
Laptop with Google Sheets (For Data Logging)
Study Room with Glass Windows (SJSU MLK Library) and a Reliable Internet Connection (WiFi)
Chair and Table
A heuristic evaluation was conducted for researchers to familiarize with Tinder version 10.10.1 and identify potential usability hotspots.
Sample Heuristic Violation I: Visibility of System Status
A system should always keep the user informed of what is going on, through appropriate feedback within a reasonable time
Task: 'Edit [Bio] Info'
Description of Violation I: Users can edit their profiles (including photos, age, gender, and sexual preference) by tapping 'Edit Info' on their profile screen and scrolling through the wizard. The 'Photos' uploader element is positioned to fit the entire screen, giving the user no indication to scroll down for access to additional entry fields. Instead, the user is directed to tap the red, 'Done' CTA.
Recommendation for Violation I: Removing a row or two of photo placeholders and moving entry fields up will give users feedforward to scroll down before tapping 'Done.'
Sample Heuristic Violation II: Consistency
Users should not have to wonder whether different words, situations, or actions mean the same thing. It is best to follow platform conventions.
Task: Review User Photos
Description of Violation II: To review a user's photos from the 'Matching' screen, the user has to tap the edges of the Profiles deck. To review photos from the 'Profile' screen, the users has to swipe through the photo deck. This inherently changes one of Tinder's most popular interactions, which can confuse and frustrate users.
Recommendation for Violation II: Tinder is well known for its 'swipe-to-match' system. The interaction mechanic used to review Photos from a Profile should match the mechanics of the 'Matching' screen (the main screen).
Usability testing was conducted to evaluate the usability of Tinder mobile (v. 10.10.1).
Usability Parameters: Performance, Behavioral & Subjective Responses
User Group: LGBTQ+
User Goal: Find a local 'Match'
Tasks: Create a new Tinder account and set your sexual preference
The following experience flow visualizes the Onboarding process for Tinder version 10.10.1 (screen stills above chart).
Although 20/20 participants were able to complete their Tinder account through the onboarding process, not all participants were able to set their sexual preference
10/10 LGBTQ+ participants were not able to set their sexual preference in the onboarding process, extending the time it takes to complete the actual 'onboarding' task by an average of 132 seconds for LGBTQ+ participants
A participant who identifies as a straight, cis-gen male was able to begin 'Matching' with their sexual preference immediately after completing the onboarding process.
A participant who identifies as a gay male expressed confusion when they noticed only potential suitors who identify as female were being shown to them in the 'Matching' screen.
It took the participant an additional 98 seconds to find and set their sexual preference in the 'Settings' screen, after completing the onboarding process.
10/10 LGBTQ+ participants rated the onboarding experience a 1/5, where 1 is Very Poor and 5 is Very Excellent
7/10 LGBTQ+ participants rated their overall experience a 2 or below on a Likert Scale of 1 to 5
During a post-interview, one LGBTQ+ participant (F, 23) reported the following about their experience with Tinder, specifically referring the onboarding process:
"I think it's a slap in the face to our [the LGBTQ+] community"
By triangulating and converging on the analyzed data, the following insights were concluded by our team:
Tinder version 10.10.1 is optimized for straight, cis-gender users
The prioritization of straight, cis-gender experiences early in the onboarding process blocks LGBTQ+ experiences, turning these users off
A report detailing and prioritizing usability issues and recommendations was presented to the ISE 215 class at San Jose State University.
Critical Onboarding Recommendation:
Integrate a Sexual Preference option directly into the onboarding process.
This study provided insights into the overall usability of Tinder mobile version 10.10.1 with representative LGBTQ+ and non-LGBTQ+ users, through main user flows and tasks. The priority given to the design of LGBTQ+ experiences must increase and be reflected in the UI.
Lih Seng Goh
User Experience Researcher | Google Stadia
"Victor was extremely easy to work with and always very open when communicating with the team. He does a good job aligning with stakeholders using his active listening skills to understand core usability issues and research questions. I was always impressed by the solutions he proposed, and any feedback given to him was incorporated thoughtfully."
Design Researcher | IBM
"During this project, Victor did an excellent job of leveraging both the needs of the user with the needs of product stakeholders in mind. He brought well thought out recommendations and insights to our data synthesis sessions. Victor craftfully executed this difficult research project by being highly organized and collaborative on the team. It was a pleasure to work with Victor."