You are here

NASA Task Load Index (NASA TLX)

HP Activity Categories: 
Resource Type: 
Method
Abstract: 

The NASA TLX is a multi-dimensional rating scale for operators to report their mental workload. It uses six dimensions of workload to provide diagnostic information about the nature and relative contribution of each dimension in influencing overall operator workload. Operators rate the contribution made by each of six dimensions of workload to identify the intensity of the perceived workload.

References

Developer and source: 

Hart, S. G. & Staveland, L. E. (1988) Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P. A. Hancock and N. Meshkati (Eds.) Human Mental Workload. Amsterdam: North Holland Press.

Hart, S. G. (2006). NASA-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting, 904-908. Santa Monica: HFES.

Year of development / publication, updates etc: 

1988

Update 2006

General Description

Purpose: 

The NASA Task Load Index (TLX) is a multidimensional subjective workload-rating method. In NASA TLX, workload is defined as the “cost incurred by human operators to achieve a specific level of performance.” The subjective experience of workload is defined as an integration of weighted subjective responses (emotional, cognitive, and physical) and weighted evaluation of behaviours. The behaviours and subjective responses, in turn, are driven by perceptions of task demand. Task demands can be objectively quantified in terms of magnitude and importance. An experimentally based process of elimination led to the identification of six dimensions for the subjective experience of workload: mental demand, physical demand, temporal demand, perceived performance, effort, and frustration level.

Type (e.g. observation, questionnaire, interview, checklist, measurement instrument, etc.): 

Questionnaire

Technical description of method or tool etc

Description of the content/study: 

The NASA TLX provides an overall score based on a weighted average of ratings the six subscales as described in the figure below:

Three dimensions relate to the demands imposed on the subject (Mental, Physical and Temporal Demands) and three to the interaction of a subject with the task (Effort, Frustration, and Performance). The degree to which each of the six factors contributes to the workload of the specific task to be evaluated from the rater’s perspective is determined by their responses to pair-wise comparisons among the six factors. Magnitude ratings on in each subscale are obtained after each performance of a task or task segment. Ratings of factors deemed most important in creating the workload of the task.

Technical requirements for using the method, tool, etc: 

Questionnaire applicable as paper and pencil version but also software tools are available.

Measure/Response Type: 

NASA Task Load Index (TLX) method assesses work load on six scales with 21 gradations each.

Results obtained and interpretation: 

Sources of load (Weights)

The NASA TLX is a two-part evaluation procedure consisting of both weights and ratings. The first requirement is for each rater to evaluate the contribution of each factor (its weight) to the workload of a specific task. These weights account for potential sources of between rater variability differences in workload definition, between rates within a task and differences in the sources of workload between tasks. In addition, the weights themselves provide diagnostic information about the nature of the workload imposed by the task.

There are 15 possible pair-wise comparisons of the six scales. Each pair is presented on a card. Subjects circle the member of each pair that contributed more to the workload of that task. The number of times of each factor is selected is tallied. The tallies can range from 0 (not relevant) to 5 (more important than any other factor).

A different set of weights is obtained for each distinctly different task or tasks element upon its completion. The same set of weights can be used for many different versions of the same task if the contribution of the six factors to their workload is fairly similar.

Magnitude of load (Ratings)

The second requirement is to obtain numerical ratings for each scale that reflect the magnitude of that factor in a given task. The scales are presented on a rating sheet. Subjects respond by margin each scale at the desired location. In operational situations, rating sheets or verbal responses are more practical, while a computerized version (available from NASA Ames Research Center) is more efficient for most simulation and laboratory settings. Ratings may be obtained either during a task, after task segments, or following an entire task. Each scale is presented as a 12 cm line divided into 20 equal intervals anchored by bipolar descriptors (high/ low). The 21 vertical tick marks on each scale divide the scale from 0 to100 in increments of 5. If a subject marks between two ticks, the value of the right tick is used (i.e. round up).

Weightings and Averaging Procedure

The overall workload score for each subject is composed by multiplying each rating by the weight given to that factor by that subject. The sum of the weighted ratings for each task is divided by 15 (the sum of the weights).

Evaluation

Advantages: 

NASA TLX is widely accepted in the research community.

Quick and easy method of estimating workload.

Flexible.

Well-established.

Easy to apply, software available.

Multi-dimensional approach.

Disadvantages: 

Can be intrusive and disruptive to primary task performance if applied online.

Participants may have forgotten relevant workload aspects, as it is filled out after the task.

Response bias (halo effect, primacy-recency effect).

Workload ratings may be correlated with task performance e.g. subjects who performed poorly on the primary task may rate their workload as very high and vice versa.

Sub scale weighting which is laborious and time consuming.

Sub-scale ratings could be repetitive, i.e. the same or similar rating applied to each sub-scale.

Alternative Methods: 

SWAT (Subjective Workload Assessment Technique);

WP (Workload Profile)

ISA (Instantaneous Self-Assessment of Workload)

DRAWS Workload Scales

VACP method (Visual, Auditory, Cognitive, Psychomotor)

Usability (ease of use, efficiency, effectiveness)

NASA TLX is reasonably easy to use and reliably sensitive to experimentally important manipulations over the past 20 years.

Ease of use: 
high
Efficiency: 
high
Effectiveness: 
high
Constraints concerning conditions of use: 

The questionnaire should be filled in immediately after each working session

Reliability: 

NASA-TLX has good reliability and validity and may be used as a valid tool to assess mental workload

The re-test reliability, split-half reliability, Cronbach's alpha coefficient and correlation coefficients between item score and total score were adopted to test the reliability.

The re-test reliability coefficients of these two scales and their items ranged from 0.516 to 0.753 (P < 0.01), indicating the two scales had good re-test reliability.

The split-half reliability of NASA-TLX and Cronbach's alpha coefficient were more than 0.80, the correlation coefficients between its items score and total score were all more than 0.60 (P < 0.01) except the item of performance. Both scales had good inner consistency.

The Pearson correlation coefficient between the two scales was 0.492 (P < 0.01), implying the results of the two scales had good consistency.

Validity: 

The test of validity included structure validity Factor analysis showed that the two scales had good structure validity and it resulted extensively validated.

Required effort (to conduct & to analyse): 

Due to the pair-wise comparisons the effort and necessary time to administer the NASA TLXis increased.

Level of HF expertise needed (required user qualification)

High: high level of expertise required, only for experts, lots of training required
Other expertise needed (required user qualification): 

HF Specialist and researcher

Cost Information

Very low: (<100 €) low costs to purchase or free license, no special devices necessary
Experiences of use by SESAR partners (including references): 

EUROCONTROL – ENAV following the European Operational Concept Validation

Methodology (E-OCVM) in the SESAR Concept of Operations (CONOPS)

Reported and/or published experiences of use (including references): 

In exercises with the BETA system and de-briefing sessions including questionnaires.

Standard methods like NASA TLX were applied.

Applicability to lifecycle phase (E-OCVM): 

It is applicable for lifecycle phase V3 as “pre-operational validation”.

Application Area: 

Aviation- Transport - Typically used for tasks involving human interaction with a machine

Keywords: 

Workload measurement

Short Description: 

The NASA TLX is a multi-dimensional rating scale for operators to report their mental workload. It uses six dimensions of workload to provide diagnostic information about the nature and relative contribution of each dimension in influencing overall operator workload.