HP Activity Categories
- Human performance related transition factors
- Design of working environment and human-machine interfaces
- Impact on human performance
- Technical system supporting human performance
You are here
NASA Task Load Index (NASA TLX)
The NASA TLX is a multi-dimensional rating scale for operators to report their mental workload. It uses six dimensions of workload to provide diagnostic information about the nature and relative contribution of each dimension in influencing overall operator workload. Operators rate the contribution made by each of six dimensions of workload to identify the intensity of the perceived workload.
Hart, S. G. & Staveland, L. E. (1988) Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P. A. Hancock and N. Meshkati (Eds.) Human Mental Workload. Amsterdam: North Holland Press.
Hart, S. G. (2006). NASA-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting, 904-908. Santa Monica: HFES.
The NASA Task Load Index (TLX) is a multidimensional subjective workload-rating method. In NASA TLX, workload is defined as the “cost incurred by human operators to achieve a specific level of performance.” The subjective experience of workload is defined as an integration of weighted subjective responses (emotional, cognitive, and physical) and weighted evaluation of behaviours. The behaviours and subjective responses, in turn, are driven by perceptions of task demand. Task demands can be objectively quantified in terms of magnitude and importance. An experimentally based process of elimination led to the identification of six dimensions for the subjective experience of workload: mental demand, physical demand, temporal demand, perceived performance, effort, and frustration level.
Technical description of method or tool etc
The NASA TLX provides an overall score based on a weighted average of ratings the six subscales as described in the figure below:
Three dimensions relate to the demands imposed on the subject (Mental, Physical and Temporal Demands) and three to the interaction of a subject with the task (Effort, Frustration, and Performance). The degree to which each of the six factors contributes to the workload of the specific task to be evaluated from the rater’s perspective is determined by their responses to pair-wise comparisons among the six factors. Magnitude ratings on in each subscale are obtained after each performance of a task or task segment. Ratings of factors deemed most important in creating the workload of the task.
Questionnaire applicable as paper and pencil version but also software tools are available.
NASA Task Load Index (TLX) method assesses work load on six scales with 21 gradations each.
Sources of load (Weights)
The NASA TLX is a two-part evaluation procedure consisting of both weights and ratings. The first requirement is for each rater to evaluate the contribution of each factor (its weight) to the workload of a specific task. These weights account for potential sources of between rater variability differences in workload definition, between rates within a task and differences in the sources of workload between tasks. In addition, the weights themselves provide diagnostic information about the nature of the workload imposed by the task.
There are 15 possible pair-wise comparisons of the six scales. Each pair is presented on a card. Subjects circle the member of each pair that contributed more to the workload of that task. The number of times of each factor is selected is tallied. The tallies can range from 0 (not relevant) to 5 (more important than any other factor).
A different set of weights is obtained for each distinctly different task or tasks element upon its completion. The same set of weights can be used for many different versions of the same task if the contribution of the six factors to their workload is fairly similar.
Magnitude of load (Ratings)
The second requirement is to obtain numerical ratings for each scale that reflect the magnitude of that factor in a given task. The scales are presented on a rating sheet. Subjects respond by margin each scale at the desired location. In operational situations, rating sheets or verbal responses are more practical, while a computerized version (available from NASA Ames Research Center) is more efficient for most simulation and laboratory settings. Ratings may be obtained either during a task, after task segments, or following an entire task. Each scale is presented as a 12 cm line divided into 20 equal intervals anchored by bipolar descriptors (high/ low). The 21 vertical tick marks on each scale divide the scale from 0 to100 in increments of 5. If a subject marks between two ticks, the value of the right tick is used (i.e. round up).
Weightings and Averaging Procedure
The overall workload score for each subject is composed by multiplying each rating by the weight given to that factor by that subject. The sum of the weighted ratings for each task is divided by 15 (the sum of the weights).
NASA TLX is widely accepted in the research community.
Quick and easy method of estimating workload.
Easy to apply, software available.
Can be intrusive and disruptive to primary task performance if applied online.
Participants may have forgotten relevant workload aspects, as it is filled out after the task.
Response bias (halo effect, primacy-recency effect).
Workload ratings may be correlated with task performance e.g. subjects who performed poorly on the primary task may rate their workload as very high and vice versa.
Sub scale weighting which is laborious and time consuming.
Sub-scale ratings could be repetitive, i.e. the same or similar rating applied to each sub-scale.
SWAT (Subjective Workload Assessment Technique);
WP (Workload Profile)
ISA (Instantaneous Self-Assessment of Workload)
DRAWS Workload Scales
VACP method (Visual, Auditory, Cognitive, Psychomotor)
Usability (ease of use, efficiency, effectiveness)
NASA TLX is reasonably easy to use and reliably sensitive to experimentally important manipulations over the past 20 years.
The questionnaire should be filled in immediately after each working session
NASA-TLX has good reliability and validity and may be used as a valid tool to assess mental workload
The re-test reliability, split-half reliability, Cronbach's alpha coefficient and correlation coefficients between item score and total score were adopted to test the reliability.
The re-test reliability coefficients of these two scales and their items ranged from 0.516 to 0.753 (P < 0.01), indicating the two scales had good re-test reliability.
The split-half reliability of NASA-TLX and Cronbach's alpha coefficient were more than 0.80, the correlation coefficients between its items score and total score were all more than 0.60 (P < 0.01) except the item of performance. Both scales had good inner consistency.
The Pearson correlation coefficient between the two scales was 0.492 (P < 0.01), implying the results of the two scales had good consistency.
The test of validity included structure validity Factor analysis showed that the two scales had good structure validity and it resulted extensively validated.
Due to the pair-wise comparisons the effort and necessary time to administer the NASA TLXis increased.
Level of HF expertise needed (required user qualification)
HF Specialist and researcher
EUROCONTROL – ENAV following the European Operational Concept Validation
Methodology (E-OCVM) in the SESAR Concept of Operations (CONOPS)
In exercises with the BETA system and de-briefing sessions including questionnaires.
Standard methods like NASA TLX were applied.
It is applicable for lifecycle phase V3 as “pre-operational validation”.
Aviation- Transport - Typically used for tasks involving human interaction with a machine
The NASA TLX is a multi-dimensional rating scale for operators to report their mental workload. It uses six dimensions of workload to provide diagnostic information about the nature and relative contribution of each dimension in influencing overall operator workload.