The Impact of Warning Location and Consequential Statements

Purpose: Graduating Master Project

Role: Researcher

Team: Solo

Duration: 6 months

A culminating graduation project was developed as a requirement for the completion of the SJSU Human Factors and Ergonomics Masters Program.

A copy of the report is available upon request.

Background and Goals

Studies dedicated to increasing user compliance and memory during and after the use of a product assist manufacturers in understanding the benefits and importance of designing effective warning statements. This study provides additional information relevant to the design of instructional warnings.

The goal of this study was to determine how the location of warnings and the presence of explicit consequential statements affect an individual’s compliance when using a product and their memory of warnings after use.

Hypotheses

Hypotheses: This study hypothesized there would be a difference in user compliance and memory of warnings between explicit and non-explicit consequential warning statements located either before or after a relevant line of instruction when using a product.

Process

An literature review was conducted to provide an overview of common topics in warning and instructional research over the past several decades, outline current literature on these topics, and describe the knowledge gaps in warning research.

User testing was conducted in person at San Jose State University with 32 participants using a placebo insectidce product.

Afterward, an analysis of study results was done to determine the effectiveness of each instruction type on user compliance and memory of warnings.

Experimental Design

This study implemented a 2x2 between-subjects design to reduce order and learning effects. This design included:

Independent variables: location of warning (before or after the related procedural step) and description of possible consequences of non-compliance (present or absent)
Dependent variables: performance with compliance of warnings and memory of the warning at the end of the session.

An insecticide product was chosen for this study because it presents many safety hazard warnings for users to comply with and instructions for insecticide products are simple to follow without extensive explanation of the instructions.

32 novice participants were presented with the same placebo insecticide product and supplies as well as one of four sets of instructions containing either

(a) warnings present before the line of instruction without a consequential statement

(b) warnings present before the line of instruction containing a consequential statement

d) warnings present after the line of instruction containing a consequential statement

Procedure

Participants were introduced to the placebo insecticide product and provided with the same scenario to consider throughout the study. They were informed their goal was to exterminate the insects from the provided plants using the materials provided. The true purpose of this study was not revealed to participants until the conclusion of the study to eliminate biases.

After completing the task, participants were asked to share their thoughts and experiences using the insecticide product as a distractor to allow for sufficient time to pass before answering questions aimed at evaluating participants’ memory of warnings imbedded in the instructions.

questions).

Example of instructions (warnings present before the line of instruction containing a consequential statement):

Results and Insights

Results of this study found that there was no significant difference in user compliance to and memory of warnings when presented with different instructional warning formats.

However, insights from the study include:

Participants felt that the location of warnings after the applicable line of instruction was out of place
Many participants corrected their non-compliant behavior when reading the warnings listed after the relevant line of instruction

Limitations of this study:

Small sample sizes for this study increased the chance of a Type 2 error
Convenience sampling resulted reduced representation primarily to female college-educated young adults, which is only a fraction of the possible user base
It was not an accurate representation of normal use cases for an insecticide product due to use of a testing laboratory and limited control over noises and other distractions
Factors such as material placement and the wording of warnings may have guided users to take certain actions or may have led to confusion that resulted in the participant's non-compliance

Future research should consider:

Testing longer or more in-depth instructions to understand how user compliance and memory of warnings differ from small instruction sets
Expanding research to other products, especially for products not typically studied in instructional warning research such as digital products
Testing novice, intermediate, and experienced users of a product to study how different warning formats impact users across all experience levels
Replicating this study to account for these constraints

My Learnings

What I learned from this project:

How to use Quadratics for participant requirement, qualification, and data needed for the study
The importance of creating a controlled and realistic testing environment, having a diverse participant pool, and having non-influential testing instructions and scripts to lower the chance of biases results
While significant results may not be found, there are many inferences and insights you can gather that can help future studies