Summative Usability Test: Evaluating the LGBTQ+ Experience

Tinder UI 2 Mockup (Side).png

Summary.​

Sponsor: Independent Student Work (No Affiliation with Tinder)

Course: ISE 215 - Usability Evaluation and Testing

Team: 5 HFE (M.S.) Graduate Students

Timeline: 3-Weeks

Contribution: Expert Heuristic Evaluation \ Research Design and Strategy \ Data Collection (Moderating, Notetaking and Data Logging, and A/V Control) and Analysis \ Preparation and Presentation of Deliverables

In the Spring 2019 semester, 5 HFE (M.S.) students formed a team to evaluate the usability of Tinder (mobile app v. 10.10.1), a requirement for a graduate course on Usability Evaluation and Testing (ISE 215) at SJSU. The study evaluated performance, behavioral and subjective usability parameters of the Tinder experience, through critical user flows/tasks, and with representative users.

Problem.​

  • Tinder mobile is often marketed as LGBTQ+ 'inclusive'

  • The application is optimized for non-LGBTQ+ users and experiences

 
 

Objectives.

  1. Evaluate the usability of Tinder mobile version 10.10.1 with representative users (LGBTQ+ and non-LGBTQ+)

  2. Test performance, behavioral and subjective usability parameters, through critical user flows/tasks:

    • Onboarding ​

    • "Matching"

    • Messaging

Research Questions.​

  1. What is the overall usability (system-user performance, behavior, and subjective perceptions) of Tinder mobile version 10.10.1 for LGBTQ+ and non-LGBTQ+ users?

  2. What is the usability of critical user flows/tasks?

  3. How does the usability experience of LGBTQ+ users compare to the usability experience of non-LGBTQ+ users?

 

Quota Sample Matrix (N=20).

A total of 20 representative users were solicited, screened, scheduled for a 60-minute, one-on-one, in-lab usability session. Participants were selected to approximate data saturation across the following user categories:

Experience Level

  • Novice: < 6 months or a lapse longer than 6 months

  • Expert: > 6 months of regular use without a lapse longer that 6 months

Sexual Identity

  • LGBTQ+: Lesbian, Gay, Bisexual, Trans, Queer, +

  • non-LGBTQ+: Cisgender, Straight

In usability testing, data saturation refers to the stage in data analysis where no new information is discovered by the researchers, signaling the ability to cease additional data collection and analysis. A sample size of five (n = 5) in each user category is typically required to approximate data saturation.

Victor Alejandro Cortes User Experience UX Research Portfolio

Methods.​

Expert Heuristic Evaluation \ Usability Testing \ Interview \ Survey

Tools.

  • iPhone X \ Samsung Galaxy with Tinder installed

  • Fictitious Tinder Accounts​ (Representative of Quota Sample User Categories)

  • Paper, Clipboard, and Pen/Pencil (For Notetaking)

  • Laptop with Google Sheets (For Data Logging)

 

Lab Set-up.

  • Study Room with Glass Windows (SJSU MLK Library) and a Reliable Internet Connection (WiFi)

  • Chair and Table

  • A/V Equipment

 
Victor Alejandro Cortes User Experience UX Research Portfolio
Victor Alejandro Cortes User Experience UX Research Portfolio
 

Process.

 

Findings.

Heuristic Evaluation

​A heuristic evaluation was conducted for researchers to familiarize themselves with Tinder mobile version 10.10.1 and to identify potential usability hotspots.

Sample (Local) Heuristic Violation: Visibility of System Status

A system should always keep the user informed of what is going on, through appropriate feedback within a reasonable time.

Task: 'Edit [Bio] Info'

Heuristic 1 .png
Scroll.png

Description of (Local) Violation: Users can edit their profiles (including photos, age, gender, and sexual preference) by tapping 'Edit Info' from the profile screen and scrolling through the 'Edit Info' wizard. Because the 'Photos' wizard element occupies the entire screen, users do not receive the feedforward required to scroll down for access to additional edit entry fields in the wizard. Instead, the user is directed to tap the 'Done' CTA before completing the entire 'Edit [Bio] Info' task flow.

Recommendation for (Local) Violation: Removing a row or two of photo placeholders from the 'Photos' wizard element and moving additional edit entry fields up will give users the necessary feedforward to complete the entire task flow. 

Sample (Global/Universal) Heuristic Violation: Consistency

Users should not have to wonder whether different words, situations, or actions mean the same thing. It is best to follow platform conventions.

Task: Review User Photos

Heuristic 2a.png
Tap.png
Heuristic 2b.png
Swipe.png

Description of (Global/Universal) Violation: To review a user's photos from the 'Matching' screen, users have to tap the left and right edges of the photo deck. To review photos from the 'Profile' screen, users have to swipe right and left through the photo deck. This inconsistency inherently changes one of Tinder's most popular interactions (i.e. swipe-to-match), which may confuse and frustrate users navigating back-and-forth between the 'Matching' and 'Profile' screens. 

Recommendation for (Global/Universal) Violation: Tinder is well known for its 'swipe-to-match' interaction. The interaction mechanic used to review Photos from the Profile screen must match the mechanics of the 'Matching' screen (the main screen).

Usability Test

Lab-based, usability testing was conducted to evaluate the overall usability of Tinder mobile (v. 10.10.1). 

 

Usability Parameters: Performance, Behavioral and Subjective Perceptions

 

User Groups: LGBTQ+ and non-LGBTQ+ Users 

 

Sample User Goal: 'Match' with a local Tinder user that matches your sexual preference

Sample User Flow/Task: Create and complete a new Tinder account (Onboarding)

The following user flow visualizes the initial Onboarding task for Tinder mobile version 10.10.1:

Victor Alejandro Cortes User Experience UX Research Portfolio

Performance:

  • 10/10 non-LGBTQ+ participants were able to complete their Tinder account through the onboarding process

  • 10/10 LGBTQ+ participants were unable able to set their sexual preference during the Onboarding process, extending the time it takes to complete the Onboarding task by an average of 98 seconds

Behavioral Responses:

Behavior 1.png

A participant who identifies as straight, cisgender female expressed happiness/joy as they began swiping for potential 'Matches' immediately after completing the initial onboarding process.

Behavior 2.png

A participant who identifies as gay, cisgender male displayed expressions of confusion/worry immediately after completing the initial onboarding process and initializing the 'Matching' process with female users.

Subjective Perceptions:

  • 10/10 LGBTQ+ participants rated the onboarding experience a 1/7, where 1 is Very Poor and 7 is Very Excellent

  • 7/10 LGBTQ+ participants rated their overall experience a 2-or-below/7, where 1 is Very Poor and 7 is Very Excellent

During a post-interview, one LGBTQ+ participant (Cisgender Female, 23) reported the following about their experience with Tinder, specifically referring to the onboarding process:

"I think it's a slap in the face to our [the LGBTQ+] community"

 

Insights.

​By triangulating and converging on the analyzed data, the following insights were concluded by our team:

  1. Tinder version 10.10.1 is solely optimized for straight, cis-gender, non-LGBTQ+ users

  2. This sole prioritization of straight, cis-gender, not-LGBTQ+ experiences early in the onboarding process means that Tinder is not LGBTQ+ accessible

 

Deliverables.​

A report detailing and prioritizing usability issues and design recommendations was presented to the ISE 215 Usability Evaluation and Testing course at SJSU.

Critical Onboarding Recommendation: 

Feature Gender and Sexual Preferences directly in the initial onboarding process.

Next Steps.​

This study provided insights into the overall usability of Tinder mobile (version 10.10.1) with representative LGBTQ+ and non-LGBTQ+ users, through main user flows/tasks. The priority given to the design of LGBTQ+ experiences must match non-LGBTQ+ experiences and be reflected in the UI.

 
 

Team Testimonials.

Lih Seng Goh, M.S.

User Experience Researcher | Google

"Victor was extremely easy to work with and always very open when communicating with the team. He does a good job aligning with stakeholders using his active listening skills to understand core usability issues and research questions. I was always impressed by the solutions he proposed, and any feedback given to him was incorporated thoughtfully."

Theresa Faught, M.S.

Design Researcher | IBM

"During this project, Victor did an excellent job of leveraging both the needs of the user with the needs of product stakeholders in mind. He brought well thought out recommendations and insights to our data synthesis sessions. Victor craftfully executed this difficult research project by being highly organized and collaborative on the team. It was a pleasure to work with Victor."