Fair and unfair data

Years 7 to 13

An AI learns from its data, so missing or lopsided data can make it work worse for some people.

900 training photos of group A but only 60 of group B: the model has little to learn from for group B.

For learners

A computer can only learn from the examples it is shown. If almost all the examples are about one group, it may work well for that group and badly for others.

This is called bias. It isn't the computer being mean. It's the data being unfair or incomplete.

A good question to ask about any data is: who or what is missing?

A worked example

A weather app is trained on summer data only. When might it be wrong, and why?

It has never seen winter data.
So it may get winter forecasts wrong.
The fix is to include data from the whole year.

For grown-ups

Real systems have been less accurate for some groups because those groups were under-represented in the training data. Surveys have the same problem: asking only your friends, or only people at a bus station, does not give a fair picture of everyone. Checking the sample is as important as checking the maths.

Common mistakes

Assuming a computer's answer is automatically fair.
Using a big sample that is all of the same kind.
Judging a model by its overall accuracy without checking each group.

Why it matters

AI now helps decide things like what you see online, so it matters that it works fairly for everyone.

Topics in quarter that use this

Is the data fair?
Odd one out in data
Data detective