Quality Assurance

Bias

20 min

Simple Explanation

Bias is systematic error that causes assessment results to be unfair to certain groups. Unlike random error, bias consistently pushes results in one direction for particular learners.

Professional Definition

Bias in assessment refers to systematic factors that distort measurement for particular groups. Types include cultural bias, gender bias, linguistic bias, and construct-irrelevant variance. Bias threatens validity and fairness.

Explain It to a 10-Year-Old

Imagine a weighing scale that always adds 2kg when a tall person steps on it but works fine for everyone else. That is bias — a systematic error tied to a group characteristic, not random variation.

Why It Matters

Bias is often invisible to those not affected by it. A developer may not realise a context is culturally specific until someone from a different background points it out. Systematic bias review is essential.

Where It Fits in the Process

This topic belongs to Quality Assurance in the assessment development journey.

Accessibility Fairness Bias SME Management Quality Assurance Technical Review Version Control

Step-by-Step Process

1

Identify potential bias sources

What groups might be affected?

2

Review every item

Language, context, names, assumptions

3

Involve diverse reviewers

People from different backgrounds spot different issues

4

Check post-assessment data

Do group results show systematic differences?

5

Revise and re-check

Bias fixes should be re-reviewed

Worked Example

Bias review examples:
Cultural bias: 'Explain how a mortgage works.' — assumes knowledge of UK housing market.
Fix: Provide sufficient context or use a universal scenario.
Gender bias: All managers referred to as 'he'.
Fix: Use 'they' or vary gender representation.
Linguistic bias: 'Notwithstanding the aforementioned...' — unnecessarily complex.
Fix: 'Despite what was said earlier...'

Common Mistakes

Mistake	Problem	Better Approach
Reviewing alone	One person cannot spot all potential bias	Involve multiple reviewers from diverse backgrounds
Treating all group differences as bias	Some differences may reflect genuine differences in attainment	Investigate, do not assume either explanation

Checklist

Cultural assumptions checked?
Contexts, scenarios, names

Gender representation varied?
Pronouns, roles, examples

Language complexity appropriate?
Not testing vocabulary above construct level

Diverse review panel involved?
Different perspectives included

of 4 complete

Related Tool

Apply what you learned with a practical tool.

Open Blueprint Builder

Interview Connection

You may be asked about this topic in assessment developer interviews.

Practice Interview Questions

Knowledge Check (3 Questions)

You scored %

Summary

Bias is systematic, not random, error tied to group membership
Cultural, gender, linguistic, and socioeconomic bias can all occur
Bias is often invisible to the assessment developer
Bias review should involve diverse reviewers
Bias found after delivery must be addressed for future versions