You are here

Cognitive Walkthrough

Resource Type: 
Technique
Abstract: 

The cognitive walkthrough is a formalized way of imagining people's thoughts and actions when they use an interface for the first time. Cognitive walkthrough involves one or a group of evaluators inspecting a user interface by going through a set of tasks and evaluate its understandability and ease of learning. The user interface is often presented in the form of a paper mock-up or a working prototype, but it can also be a fully developed interface.

References

Developer and source: 

Wharton, C. Bradford, J. Jeffries, J. Franzke, M. Applying Cognitive Walkthroughs to more Complex User Interfaces: Experiences, Issues and Recommendations CHI ’92 pp381–388, 1992

C. Wharton et al. "The cognitive walkthrough method: a practitioner's guide" in J. Nielsen & R. Mack "Usability Inspection Methods" pp. 105-140, 1994

http://www.pages.drexel.edu/~zwz22/CognWalk.htm

http://hcibib.org/tcuid/chap-4.html#4-1

Følstad, A. (2007). Group-based Expert Walkthrough. Proc. Third COST294-MAUSE Int. Workshop, p. 58-60.

Year of development / publication, updates etc: 

1992

General Description

Purpose: 

Using a prototype, a conceptual design document, or the final product, a group of evaluator steps through tasks, evaluating at each step how difficult it is for the user to identify and operate the system element and how clearly the system provides feedback to that action. Cognitive walkthroughs take into consideration the user's thought processes that contribute to decision making, such as memory load and ability to reason.

Type (e.g. observation, questionnaire, interview, checklist, measurement instrument, etc.): 

Usability inspection technique

Technical description of method or tool etc

Description of the content/study: 

Cognitive walkthrough involves one or a group of evaluators inspecting a user interface by going through a set of tasks and evaluate its understandability and ease of learning. There are four inputs to a walkthrough:

(1) A description or a prototype of the interface. It doesn't have to be complete, but it should be fairly detailed. Things like exactly what words are in a menu can make a big difference.

(2) A task description. The task should usually be one of the representative tasks you're using for task-centred design, or some piece of that task.

(3) A complete, written list of the actions needed to complete the task with the interface.

(4) An idea of who the users will be and what kind of experience they'll bring to the job. This is an understanding you should have developed through your task and user analysis.

The evaluators may include human factors engineers, software developers, or people from marketing, documentation, etc. This technique is best used in the design stage of development. But it can also be applied during the code, test, and deployment stages.

Important issues to clarify before a walkthrough are:

Who will be the users of the system? What task(s) will be analyzed? What is the correct action sequence for each task? How is the interface defined?

Important issues to clarify during the walkthrough are:

Will the users try to achieve the right effect? Will the user notice that the correct action is available? Will the user associate the correct action with the effect to be achieved? If the correct action is performed, will the user see that progress is being made toward solution of the task?

The goal of a walkthrough is to improve a user interface. Thus, during the walkthrough focus should be not only on problems, but also on how the interface should be changed to remove or reduce the problems. The main follow-up of a walkthrough is to change the user interface.

Technical requirements for using the method, tool, etc: 

n/a

Measure/Response Type: 

n/a

Results obtained and interpretation: 

A set of identified problems and possible fixes to these.

Evaluation

Advantages: 

Requires fewer resources than user testing.

Because the evaluators can imagine the behaviour of entire classes of users, the walkthrough will often identify many more problems than one would find with a single, unique user in a single test session.

Good at assessing the effectiveness of an interface.

Disadvantages: 

Walkthroughs were designed evaluate simple, walk up and use interfaces, and according to Wharton et al. (1992), walkthroughs do not scale to more complex interfaces without being adjusted.

Not suited for identifying problems experienced by trained users.

Not so good at assessing the efficiency and satisfaction of an interface.

Alternative Methods: 

Heuristic evaluation, user testing.

Usability (ease of use, efficiency, effectiveness)

Ease of use: 
high
Efficiency: 
high
Effectiveness: 
medium
Constraints concerning conditions of use: 

n/a

Reliability: 

n/a

Validity: 

As the interfaces are evaluated by experts, the validity is not as good as for user tests.

Required effort (to conduct & to analyse): 

The efforts needed for doing a cognitive walkthrough is to a large extent proportional to the size and complexity of the interface being evaluated. Generally, cognitive walkthrough is a fairly low effort technique.

Level of HF expertise needed (required user qualification)

Medium: limited level of expertise required, some training required
Other expertise needed (required user qualification): 

n/a

Cost Information

Very low: (<100 €) low costs to purchase or free license, no special devices necessary
Experiences of use by SESAR partners (including references): 

Cognitive walkthrough is used occasionally by NATMIG partners in various application domains, but no known use in ATM. One NATMIG partner has adapted the technique to evaluating mobile user interfaces using domain experts (Følstad, 2007).

Reported and/or published experiences of use (including references): 

Wharton et al. (1992) reports on experiences in using cognitive walkthrough on complex user interfaces. Wharton et al. (1994) summarizes and discuss a number of experiences, along with additional examples of interface success and failure stories.

Applicability to lifecycle phase (E-OCVM): 

Probably best suited for use in V3.

Application Area: 

Cognitive walkthrough is independent of application area, and has been applied in different domains. Any use in aviation, ATM or transportation is not known.

Keywords: 

Usability, evaluation, expert evaluation, inspection, walkthrough

Short Description: 

The cognitive walkthrough is a formalized way of imagining people's thoughts and actions when they use an interface for the first time. Cognitive walkthrough involves one or a group of evaluators inspecting a user interface by going through a set of tasks and evaluate its understandability and ease of learning. The user interface is often presented in the form of a paper mock-up or a working prototype, but it can also be a fully developed interface.