How to Conduct Heuristic Evaluation?

The usability testing is one of the essential stages of the user experience design process where the prototype is tested and evaluated to ensure that the final product is user-friendly in terms of usability factors such as functionality, user control, consistency, and error prevention. There are different usability testing methods that could be used in the testing and evaluation. One of the methods that have been part of a large debate is the heuristic evaluation. While many companies favour this tool because of its cheap cost and affordability compared with other testing methods, heuristic evaluation is suitable as an evaluation tool that needs to be conducted by a team of experts. Heuristic evaluation is closely linked to our understanding of ergonomics principles as they minimise these principles in digital applications.

Unlike the other usability testing methods that depend on a user sample to test the product, the heuristic evaluation depends on UX Design experts (evaluators) who review the product based on predefined characteristics. In this method, the evaluator checks the design based on an assessment document that evaluates different elements, such as the layout elements, function, flow, and system. The aim of heuristic evaluation is to identify the common usability problems in the design.

Related articles:

• Five Reasons Behind a Failed UX Design Process
• 5 Myths About The User Experience Design Process
• The Price of Not Doing a Proper UX Research

What is Heuristic Evaluation in Usability Testing?

The heuristic evaluation was first introduced by Jakob Nielsen and Rolf Molich in 1990. In their method, they suggested a number of guidelines that can be tested by a group of experts to evaluate the usability of the product. These guidelines act as a rule of thumb which can identify the common usability problems in the design. There are different guidelines that can be implemented in the heuristic evaluation, such as the following (Figure 1):

• Visibility of system status
• Match between the system and the real world
• User control and freedom
• Consistency and standards
• Error prevention
• Recognition rather than recall
• Flexibility and efficiency of use
• Aesthetic and minimalist design
• Help users recognize, diagnose and recover from errors
• Help and documentation

In the heuristic evaluation, a number of UX evaluators test the product based on the above guidelines to uncover usability problems during the testing process. The aim of the heuristic evaluation is to identify the common usability issues rather than the user-specific problems, as it doesn’t include the user sample in the testing process, which is the main reason behind the debate on the viability of the tool to use in the usability testing process.

The Advantages of Using the Heuristic Evaluation

Companies may depend on the heuristic evaluation rather than another usability testing because of the following advantages:

• It is an inexpensive usability testing method that can test the product based on the number of in-house UX experts
• It is a quick testing tool as it doesn’t require preparing a representative user sample to do the testing
• It can be used to element the common usability problems that don’t need feedback from the end user
• It can be used prior to other usability testing methods to focus on user-specific usability issues

The Disadvantage of Using the Heuristic Evaluation

In contrast to the above advantages, the heuristic evaluation comes with different disadvantages that may affect its viability and a trusted tool to use in the usability testing. These disadvantages may include the following:

• It doesn’t involve the user’s opinion in the testing. As the UX experts who do the testing are not the same as the end, the user should depend on more than one UX expert evaluator in order to ensure accurate results for the testing.
• It should depend on more than one UX expert evaluator in order to ensure accurate results for the testing. It may be difficult to find the experts who can do the evaluation in-house. Also, it may be expensive to hire external evaluators.
• It may be difficult to find the experts who can do the evaluation in-house. Also, it may be expensive to hire external evaluators. It only works to identify general usability testing issues. Some issues can be identified as a problem when it is not actually a problem for the end users.
• It only works to identify general usability testing issues. Some issues can be identified as a problem when it is not actually a problem for the end users.
• It can’t be used alone to end up with accurate usability testing. Other methods with user involvement should be used as well.

How to Conduct Heuristic Evaluation in Usability Testing?

Although there is a long debate about using the heuristic evaluation in usability testing, it can still use in an early stage of the testing phase before depending on other testing methods that involve the user sample. Using the heuristic evaluation can reduce the cost of the usability testing as well as the tested factors. As in-house UX evaluators can start by testing the product based on the guidelines highlighted above, then the company may use other external usability testing methods that involve the end user in the testing process.

In order to implement the heuristic evaluators, the company need to prepare the following:

• Between 3-5 UX evaluators
• The heuristic guidelines
• The user tasks that need to be tested
• The product prototype or wireframe that needs to be tested
• A form to collect testing results

The Heuristic Evaluation can be done either as an explorative tool at the Explore for the Double Diamond Design Thinking or during the testing (Develop) phase. To ensure a successful heuristic evaluation process, in particular, and a usability testing process in general, the company need to consider the following:

• A piece of good knowledge of the user behaviour, needs, and demographics in order to consider while doing the evaluation
• Clear definition of the user tasks and use cases that will be evaluated by the heuristic evaluation
• Depend on more than one evaluator to ensure accurate output for the usability testing. If one issue is missed by one evaluator, it can be highlighted in the other evaluators’ reports.
• Once the evaluators test the product and collect the results, Each evaluator analyzes the results and presents them in a slide presentation to discuss with the rest of the evaluators and stakeholders to reach final reports.

In this example, we will explore how to apply Heuristic Evaluation on the BBC Bitesize homepage and see how the ten criteria are applied to the webpage.

The visibility of the system status refers to keeping the user informed about the system and its current status or if there are any changes. The alerts at the top of the page provide updated information for the user. The Picks of the Week also informs the user about if there are added resources (Figure 2).

The match between the system and the real-world criterium is represented in the icons and symbols. The parents section is represented by the backpack and lunch. The teachers’ section is represented with the illustrations of the notebook and coffee (Figure 3).

The consistency and standards are represented in the whole design elements and structure. The links related to the whole BBC site are allocated in the top menu, while the links related to the education website are located under the logo. The layout of each learning section (i.e. the Learn & Revise, Extra resources and KS3 Biology) is having the same design style (Figure 4).

The user control and freedom factor allows the user to easily identify where they are and be able to return to the correct path. This factor is represented in the top menu, which allows the user to navigate between different sections of the website (Figure 5).

The Error prevention factor may not be applicable on this page as errors aren’t likely to happen.

Recognition rather than recall is applied by using different colours to differentiate the content, such as the primary, secondary and post-16 content. Also, the use of background colours to separate between sections such as KS3 Biology and The Regenerators (Figure 6).

Flexibility and efficiency of use are reflected in the All Bitesize link, which allows users to find links directly instead of using the links on each page. If the users are aware of the content of the website, they can simply jump to the content needed through this sitemap page (All Bitesize) (Figure 7).

Aesthetic and minimalist design appears on the usage of layout, colours, fonts and icons. The design layout is clear and easy to understand and navigate. The colours are used to separate between sections and identify each part of the website (Figure 4).

The next criterion is helping users to recognize, diagnose, and recover from errors. This applies to having the navigation system available for users to easily return to the main page (Figure 5).

The final criteria are Help and documentation. The system provides help for the users to know how they can use it (i.e. hoe to use the Bitesize website to learn at home) (Figure 8).

The above quick overview provides a brief about how to identify each criterion of the heuristic evaluation to assess a system. Evaluators can rate each of the above factors based on a scale from one to ten. You can use a heuristic evaluation template, which is available in the article URL in the video description. Finally, you can download our free Heuristic Evaluation Template below.

In addition to the above guidelines that need to be considered before doing the heuristic evaluation, other factors may depend on each project’s characteristics, For example, if the product requires specific skills or experience to test it. Then, the company may consider reviewing the evaluator’s resume to ensure they have the proper experience required to test the product prototype.

Although there is still a debate on the validity of using the heuristic evaluation method in the usability testing phase, it can be used in an early stage of the testing phase before moving with other user testing methods to reduce the external testing costs and the tested features. The heuristic evaluation tool can be very tricky, especially with its end results. Therefore, companies need to consider the above guidelines in order to ensure the accuracy of the results of the tool.

Bibliography 

Nielsen, J. (2020). 10 Usability Heuristics for User Interface Design. Access URL: https://www.nngroup.com/articles/ten-usability-heuristics/#poster. [Accessed on: 12/2021]

Nielsen, J. (1994). How to Conduct a Heuristic Evaluation. Access URL: https://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/. [Accessed on: 12/2021]

Nielsen, J., & Molich, R. (1990, March). Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 249-256).

Tan, W. S., Liu, D., & Bishu, R. (2009). Web evaluation: Heuristic evaluation vs. user testing. International Journal of Industrial Ergonomics, 39(4), 621-627.