AI Learning Lab · supplied records

Choosing useful metrics

Year 8 · Lesson 4 · Content 2026.11.02.1

All supplied-record lessons · Open interactive lesson

Supplied-record alternative. Enter your responses in your school assignment. These supplied results support learning but do not show that you ran an algorithm.

Your goal

Use the right group of books to measure correct decisions, useful flags and damage found.

To be able to calculate precision and recall and justify which errors matter for a stated task.

Before you start

Staff flag five books; three really need repair. Which fraction describes useful flags?

Choices: 3/5 / 3/all books

Bridge feedback after trying

Use all five flagged books as the denominator.

Three damaged books were found. If six books were actually damaged, which fraction describes damage found?

Choices: 3/6 / 3/5

Bridge feedback after trying

Recall uses all actual damaged books: 3/6.

Words for this lesson

Accuracy
Correct decisions out of all tested decisions.
Precision
Real damaged books out of all flagged books.
Recall
Damaged books found out of all actual damaged books.

The idea

Our positive label means needs repair. A false alarm flags a sound book. A miss leaves a damaged book unflagged. Accuracy looks at all correct decisions. Precision asks how many flagged books really need repair. Recall asks how much of the actual damage was found. Different tasks make different mistakes and staff workload matter. No single score tells us everything.

A completed example

Practice: 3 real damaged books among 5 flags gives precision 3/5=60%. If there were 6 damaged books altogether, recall is 3/6=50%. The same 3 successes use different denominators.

Check your understanding

A sound book flagged for repair is ...

  1. A false alarm
  2. A missed damaged book
  3. A correct non-flag
Practice feedback after trying

A false alarm. It was flagged but did not need repair.

Precision uses denominator ...

  1. All flagged books
  2. All actual damaged books
  3. All books in school
Practice feedback after trying

All flagged books. Precision asks how useful the flags were.

Finding most actual damage puts particular emphasis on ...

  1. Recall
  2. Output length
  3. Model size
Practice feedback after trying

Recall. Recall measures actual damaged books found, with workload also checked.

Task 1 · Make a prediction

Prediction for each A–D book

Options: Flag / Leave unflagged

Write your prediction, or state that you skipped it.

Try it using supplied records

  1. At cut off 0.5, predict which of A to D will be flagged. Reveal the flags and actual labels.
  2. Count false alarms and missed damage.
  3. Switch only the cut off to 0.8 and predict again.
  4. Compare the two error counts. The underlying scores and actual conditions stay fixed.

Calculated decisions from fixed scores · supplied evaluation results

Complete data and source records

Four scored books. Fictional teaching data.
ItemScoreActual condition
A0.9Damaged
B0.7Damaged
C0.6Sound
D0.2Sound
Read as stacked records
Item
A
Score
0.9
Actual condition
Damaged
Item
B
Score
0.7
Actual condition
Damaged
Item
C
Score
0.6
Actual condition
Sound
Item
D
Score
0.2
Actual condition
Sound
Supplied results on the same 20 new books. Fictional teaching data.
SettingRight flags TPFalse alarms FPMissed damage FNRight non-flags TN
S6248
T9614
Read as stacked records
Setting
S
Right flags TP
6
False alarms FP
2
Missed damage FN
4
Right non-flags TN
8
Setting
T
Right flags TP
9
False alarms FP
6
Missed damage FN
1
Right non-flags TN
4

Expected results · supplied

These are supplied reference results. They are separate from your own explanation. No code needs to run to read them.

1. Flag if score >= cut off. At 0.5 flags A,B,C: FP=1,FN=0. At 0.8 flags A: FP=0,FN=1. Scores are invented ranking scores, not verified probabilities.

2. Accuracy=(TP+TN)/20*100; precision=TP/(TP+FP)*100; recall=TP/(TP+FN)*100; flags=TP+FP. S gives 70%,75%,60%,8. T gives 65%,60%,90%,15. T is defensible under the stated find-most-damage and capacity-15 brief, while one miss and six false alarms remain.

3. Show one fraction question at a time with a named population. Zero denominator returns Not defined and an explanation, never 0% or 100%. F1 is More detail: 2PR/(P+R), consistent units, S about 66.7%, T 72%; not needed for the exit objective.

Help with this step

Hint 1

Read which group the measure asks about.

Hint 2

Precision uses all flags. Recall uses all actual damage. Accuracy uses all tested books.

Hint 3

Practice: 2 correct flags among 4 flags is 50% precision. Finding 2 of 5 damaged books is 40% recall.

Task 2 · Use the evidence

  1. For S and T, select the correct fraction for accuracy, precision and recall. Use calculator support after selecting the group.
  2. Count the books staff must inspect. Compare against capacity of 15 in this batch.
  3. Recommend a setting for finding most damage, using recall, workload and a remaining weakness.

My explanation

Write in your school assignment. Keep these answers in your own words.

Cut off 0.5

Cut off 0.8

Setting S

Setting T

Your recommendation

Explain a fresh example

S finds 6 of 10 damaged books and creates 8 flags. T finds 9 of 10 and creates 15 flags. Staff can inspect 15. Recommend a setting and explain what still goes wrong.

My confidence

I need another explanation / I can explain with help / I can explain with my own example

My next step

Write what you will do next.

Optional depth

Open F1 in More detail. Why does combining precision and recall still omit response time, access and some harms?

What this activity can show

A cut off changes decisions, not the true labels. Precision does not universally improve when a threshold rises. All results describe the supplied test, not deployment safety.

This is a student learning record. Submission is completed through the school assignment system.