Usability Case Study

The product

The product is a proprietary metrics analysis tool, specially designed to analyze logs from inbound phone systems. It’s a GUI over a database.

Defining and recruiting the user archetypes

We determined a range of users, from the non-technical director who only wants the bottom line, to the experienced Speech Scientist who is trying to solve a deep data problem. Then, we found each of these users incarnate.

Designing the test

It was important to us to test a range of skill sets, so we designed a range of problems with varying difficulty. Some of the tests were just to evaluate discoverability, while others were to complete specific filtering tasks. The last scenario was incredibly generic, but typical of real life: People are having problems. There's no specific complaint. What can you find that might be making people complain?

Moderating the test

I worked with the participants over a 2-day period, monitoring their behavior from the adjacent viewing room and taking detailed notes on their behavior. When they got stuck, or otherwise reached an unrecoverable point, I would regroup with them to get feedback. We found everything from major bugs to an unfortunate kerning problem.

Writing up the recommendations

It became immediately clear what needed to be fixed, and in what priority. I categorized my notes into a deliverable that made it easy to track the progress and disposition of each item.