AI Learning Lab · supplied records

What accuracy can hide

Year 7 · Lesson 3 · Content 2026.11.02.1

All supplied-record lessons · Open interactive lesson

Supplied-record alternative. Enter your responses in your school assignment. These supplied results support learning but do not show that you ran an algorithm.

Your goal

Use correct and wrong answers to explain who a total score can hide.

To be able to use test results to explain why overall accuracy can hide errors for a smaller group.

Before you start

Eight of ten answers are correct. Select the fraction.

Choices: 8/10 / 2/10

Bridge feedback after trying

Eight correct out of ten is 8/10 × 100 = 80%. The other two are wrong.

Words for this lesson

Accuracy
The percentage of tested answers that are correct.
Error rate
The percentage of tested answers that are wrong.
Subgroup
A smaller group within the whole test.

The idea

A total accuracy score combines all tested examples. It can hide a small group that gets more errors. First count correct answers out of the number tested. Then look at each group separately. An error rate counts wrong answers instead. In this activity every answer is right or wrong, so accuracy and error rate add to 100%. A confidence score is a model score for a prediction, not proof it is correct. Our table uses checked answers.

A completed example

Practice: 3 correct answers out of 5 means 3 / 5 x 100 = 60% accuracy. Two are wrong, so the error rate is 40%. This does not tell us which users received the errors.

Check your understanding

18 correct out of 20 is ...

  1. 90%
  2. 18%
  3. 10%
Practice feedback after trying

90%. Divide correct by tested and multiply by 100.

Does 90% overall mean every group scored 90%?

  1. No
  2. Yes
  3. Only for small groups
Practice feedback after trying

No. The total combines groups with different counts.

P got 2 of 4 noisy requests correct. Use denominator ...

  1. 4
  2. 20
  3. 16
Practice feedback after trying

4. The question asks only about the noisy group.

Task 1 · Make a prediction

Same total means same noisy-group result

Options: Yes / No / Not sure

Write your prediction, or state that you skipped it.

Try it using supplied records

  1. Predict whether two models with the same total score must treat the noisy group equally.
  2. Reveal 20 request results for P, then Q. Both have 18 correct.
  3. Press Show quiet group and Show noisy group. Count each group's correct results.
  4. Use the fraction helper to record the group comparisons.

Calculated fractions from supplied outcomes

Complete data and source records

Supplied library request results. Fictional teaching data.
ModelGroupTestedCorrectWrong
PQuiet16160
PNoisy422
QQuiet16142
QNoisy440
Read as stacked records
Model
P
Group
Quiet
Tested
16
Correct
16
Wrong
0
Model
P
Group
Noisy
Tested
4
Correct
2
Wrong
2
Model
Q
Group
Quiet
Tested
16
Correct
14
Wrong
2
Model
Q
Group
Noisy
Tested
4
Correct
4
Wrong
0

Expected results · supplied

These are supplied reference results. They are separate from your own explanation. No code needs to run to read them.

1. Render fixed example IDs A1 to A16 and B1 to B4. P: A1-A16 correct, B1-B2 correct, B3-B4 wrong. Q: A1-A14 correct, A15-A16 wrong, B1-B4 correct. Never randomise these outcomes.

2. P and Q overall 18/20=90%; error 2/20=10%. P quiet 100%, noisy 50%. Q quiet 87.5%, noisy 100%. Group filtering changes the denominator, not the underlying predictions.

3. The fraction helper asks which numerator and denominator fit before revealing the percentage. Support use is recorded as help used, never deducted from conceptual evidence.

Help with this step

Hint 1

Which group does the question name?

Hint 2

For noisy accuracy use only the four noisy requests, not all 20.

Hint 3

Practice: 1 correct noisy request out of 2 is 50%. An overall total uses the correct counts from all groups.

Task 2 · Use the evidence

  1. Complete overall accuracy and error rate for P and Q. Use the same denominator of 20.
  2. Complete quiet and noisy group accuracy, using each group's own total. The calculator may supply the arithmetic after you select the fraction.
  3. Explain why 90% accurate is not enough information. Recommend one further test and explain who it includes.

My explanation

Write in your school assignment. Keep these answers in your own words.

Model P

Model Q

Your explanation

Explain a fresh example

A poster says 90% accurate so suitable for everyone. Use one result from this test to explain why you would ask another question.

My confidence

I need another explanation / I can explain with help / I can explain with my own example

My next step

Write what you will do next.

Optional depth

Why is 4/4 still limited evidence about all future noisy requests? Compare error consequences in a library and an urgent warning service.

What this activity can show

These are invented test outcomes. Subgroup accuracy is one fairness clue. It does not measure all fairness or establish a universal safe percentage.

This is a student learning record. Submission is completed through the school assignment system.