NCSU / IBM

Here-U Wearable

2018
Role
Team
UI/UX Designer
Shadrick Addy
Matt Lemmond
Krithika Sathyamurthy
The Brief

In collaboration with IBM Watson's health team, my Master of Graphic Design studio was tasked to design a conversational user interface for deaf, hard-of-hearing users utilizing machine learning technology.

The Solution

Over the course of six weeks, my team members and I created Here-U, a wearable system that helps DHH users stay present in a conversation by blurring background noises and clarifying sound direction through a glanceable interface.​

PART 1
Process
Screen Shot 2019-01-11 at 3.32_edited.jp
Interviews

After interviewing two individuals, one who is profoundly deaf and one who is moderately hard-of-hearing, we observed that many of their frustrations revolve around office workspaces - particularly developing professional and social relationships through conversation.

Personas

Andrea is a 23 year-old recent graduate who has just started a job as an HR Assistant. She is profoundly deaf and has cochlear implants in both ears. She’s also the oldest of three siblings and friendly, confident, perceptive, empathetic, and organized.

User Journey Mapping

After several rounds of critique, we selected a specific moment in Andrea’s day that our design would address. Both interviewees brought up that conversing in restaurants can be difficult - particularly participating in relationship-building conversations, because background noise is high and acoustics can be poor.

DHH User Journey Maps-04.jpg
Identified Pain Points
  1. Actively participating in many forms of interpersonal communication (for example, one-to-many and many-to-one conversations) without feeling excluded

  2. Reading the server’s lips when she speaks quickly and the restaurant is noisy

  3. Actively listening and conversing with friends while reading
    her menu

Rough Concepts
Augmented reality glasses
DHH Rough Mockup-02.jpg
DHH Rough Mockup-04.jpg
Video conference that visualizes paint points as experienced by DHH users
User Testing
Screen Shot 2019-01-11 at 5.08.49 AM.png
Screen Shot 2019-01-11 at 5.09.18 AM.png
Screen Shot 2019-01-11 at 5.09.37 AM.png
Paper prototype testing with interviewee
PART 2
Final Interface

We wanted to give the UI an organic feel, to further emphasize that
it is a stand-alone device that is non-intrusive and non-distracting. We color contrast tested our palette to make sure users with vision impairments would be able to utilize HERE-U’s visualizations, as well. The user is at the center of the screen and the UI will always adjust so that the top of the quadrants faces the same direction as the user. The bubbles represent sounds in the surrounding area. When someone speaks directly to the user, the speaker’s aura lights up and the other bubbles not in the radial vicinity fade to avoid confusion. These forms represent the shapes of their respective microphones.

DHHUI-2 (1)-03.png

No talking, no blurs

DHHUI-2 (1)-12.png

Waitress speaking, no blurs

DHHUI-2 (1)-09.png

Bubble Blur activated

DHHUI-2 (1)-04.png

Friend to right speaking

Portrait Blur activated

DHHUI-2 (1)-10.png

Bubble Blur activated - sustained vibration feedback

DHHUI-2 (1)-05.png

Moment flagged

DHHUI-2 (1)-08.png

Many-to-one conversation

Portrait Blur activated