Heuristic evaluation is a very useful method to measure the usability of an interface in any system. In this article we will go through the pros and cons of using usability evaluation as well as a proposal for an approach on how to select the most appropriate heuristics for the system under evaluation.
Nielsen (usability expert and co-founder of Nielsen and Norman group with Donald Norman) explained why there should be at least 3 experts involved in this method. After conducting a quantitative experiment he also discovered the following facts:
- none evaluator discover all the issues in the system.
- some evaluators reported as issues false positives.
- the evaluators that discover less issues reported issues that others evaluators didn´t.
According to several studies no heuristic evaluation is fully reliable. In concrete Bailey (2001) reported in one of his studies that per each real issue discovered for each evaluator, they included about one false positive and missed slightly over half real usability issue.
All those facts could point out that expert evaluation is not a valid and reliable method on the other hand it shows a number of advantages when conducted thoughtfully:
- it is a cheap method compare to user testing and is less time consuming than log analysis.
- using the right guidelines help evaluators in focusing in the design.
- using a higher number of evaluators provide a more consistent and reliable result.
- also in very early stages of design is highly recommended to use evaluation instead of user testing, given that most of design issues can be discovered through this method.
But how can we ensure the principles we are following at the time of the evaluation are appropriate and helpful? Many authors had published their own principles from the ten principles proposed by Nielsen, going through the Golden rules proposed by Shneiderman and Tognazzini reviewed list. But how can we create our own guidelines and ensure that we are going in the right track?
Let’s in first place review the usability goals. In user-centered design usability goals are
- Effectiveness
- Efficient to use
- Safe to use
- Utility
- Learnability
- Memorability
Some other authors add to this list a more global goal: user satisfaction. This is just a subset of UX goals that has a very wide list that refers much more to create meaningful experiences that drives users to be engaged with the system. One critical goal that is rarely included in usability evaluation is aesthetics. As Norman (also author of the popular book The Design of Everyday Things has pointed out many times, if everyone follow those principles without bearing in mind aesthetics we would have highly usable and ugly products. With so many products in the market and many new start ups is required to stand out from competitors keeping in mind aspects as aesthetics, engagement and emotions.
But we will go back to this, let’s first look at how to integrate evaluation in each stage of the design process to ensure that we come up with the most appropriate questions when evaluating a system.
Getting inspiration from other authors is useful to stablish our higher level goals but we need to define custom subprinciples depending on the system and user needs.
Garrett’s elements of UX
Getting involve earlier in the creative process (scope and strategy), is critical to understand what our users’ needs are . Having a good understanding of product objectives and functional specification help out to come up with an appropiate guideline to evaluate wireframes and prototypes as well as understanding the system limitations. In subsequent stages we should apply these guidelines iteratively.
As per Garrets Elements of UX, when identifying an issue we should identify in which plane of the design process was introduced, otherwise we could be just detecting a false positive or even introducing new issues. It is equally important to identify which principle in your heuristic list is being violated.
Nielse remcommends during the evaluation to ponderate each issue found. Three key elements should be considered when prioritizing issues:
- how critical the issue is.
- likelihood to happen.
- and effort required to solve it.
Let’s bring back aesthetics up: usability evaluation is little concern about aesthetics and most of the guidelines proposed from different authors point out it shouldn’t compromise usability. In his book Seductive interaction design , Stephen P. Anderson explains in detail his user experience hierarchy model. As he states from bottom-top is a basic product maturity continuum and focus in usability first while a top to bottom approach is much more focus in the experience and standing out from competitors by approaching new and more innovative solutions.
Stephen P. Anderson UX hierarchy model
But as he also points out searching for beauty, engagement and innovation first it can also end up in a mess if we dismiss usability. Does it mean that usable systems has to be ugly or lack in aesthetics? Not at all. Good designers don’t just add aesthetics for the sake of it, there is always a good reason to use a color, a shade or a picture.
As mentioned above, our principles and goals should be inspired by our research and documents results from the processes in scope and strategy but there are some other ways to get good principles for our evaluation as using design guidelines, e.g.:
- simplicity doesn’t mean no aesthetics at all, it helps removing unuseful elements in the interface, and reuse existing ones to add value, e.g. using the pointer to show that the system is doing something. At the same time this principle helps in avoiding cluttered interfaces and free up some space helping in the space problem.
- Shadows and other 3d effects help to match users mental models, as for example adding shadows to buttons.
- Using transitions instead of showing and hiding elements abruptly helps in giving the user the feeling of control over the system.
As we can see there are several sources of inspiration to come up with a good list of key points to evaluate our systems at any stage of the design process, but to ensure a good quality evaluation is critical to keep track of the evolution of the design upfront.
Do you follow any methods or techniques to evaluate your designs? Which principles or guidelines do you follow? Which techniques would you use to engage the user with your system or website? Do you think we should be more concern about aesthetics during the evaluation process? Please leave a message with your opinion or contact us to get involve.
Where to learn more
The most important articles on how to conduct usability evaluation:
- the original article from Nielsen and Norman group site and how to prioritize problems found.
- and this article in interaction-design.org.
As we mentioned above using popular principles can be useful to apply them or to get some inspiration:
- Nielsen ten usability heuristic principles.
- Tognazzini principles.
- Schneiderman 8 golden rules.
- Emotional design.
For further information on how to conduct heuristic evaluation and other evaluation methods you can also check chapter 15 from Interaction design: beyond human-computer interaction.
For further readings on emotional design check “Seductive Interaction Design” by Stephen P. Anderson.
Leave a Reply