This is the second post in a series about what we learned while developing Shortcut. Today, we have a look at usability testing.

When we designed the new user interface for Shortcut, we had several options for a new design. We wanted to test how users respond to the different interfaces, in particular how easy they were to navigate.

How we tested

As with all things in a startup, we prefer to do things simple and efficient. And as it turns out, even experts agree that you need only a few users and simple tests to find most of the problems with the user experience of your prototypes: Jakob Nielsen says 5 users are sufficient.

In his “Joel Test” Joel Spolsky mentions the same and calls it hallway usability testing. He suggests to pick a few random people from the hallway and let them test your prototype.

Introducing Starbucks Usability Testing

We decided to build on the same idea, but locate the tests to the closest Starbucks, rather than the hallway. Why Starbucks? We think Starbucks is the ideal location for testing mobile apps, because:

• there is WiFi
• people tend to have time
• the sample of users is less biased than the one composed from your average company hallway victims
• the target group matches quite well for mobile apps. Most people sitting in Starbucks seem to be smartphone owners

The Starbucks in Zürich where we conducted our usability tests

So two of us went to the closest Starbucks, equipped with our latest App prototypes and some pre-printed evaluation forms. We bought some Starbucks gift cards as rewards. And we were ready for testing.

We organized it the way, that one of us had to pick a potential participant from the guests and ask her/him if she was interested in participating in testing a new iPhone app, that it would take 20 minutes, and that they would get a gift card as a reward. This person then also navigated the test user through the tasks. The other person observed and took notes.

A surprising large fraction of people we asked agreed to take the test. And we got quite a variety of test users.

Honestly, at first it seemed a bit awkward to approach people with the task, but after a while it turned out to be a lot of fun!

What we tested

So, what did we test you may ask? We had two alternative designs for our result screen. A more “classic” version, and the “Path” version we described in our previous post. We knew that the Path version would solve some challenges we faced with the classic version, but we wanted to see how users react. The picture below shows the two alternatives.

The two user interface versions we tested. Left the "classic" version, right the "Path" version.

We had a simple test protocol which tested the usability some typical tasks on the result screen plus some additional general tasks of the app (such as if the introductory tour is understandable).

Without spending too many words on details: the tests confirmed that the “Path” version of the UI worked better, but that there were some common problems people ran into. These we could fix before our launch.

Conclusion

We believe that Starbucks usability testing is a simple and efficient way to test usability. In particular for mobile apps and for startups on a shoestring budget. Even for larger companies it can be highly efficient, as no time needs to be spent to arrange appointments with test persons. The tests could be extended with video recording from a laptop. The Starbucks environment is probably less suited to test complex applications or expert applications.