Read AI Classification Reports Through Counts, Precision and Recall

A classification report becomes useful when a reader can reconstruct which cases were counted, what each label meant, and which kinds of disagreement remain. A single accuracy percentage cannot carry all of that information. Before using an AI-assisted classification report to discuss a pilot, ask for the underlying comparison between the proposed labels and the reference labels. Then keep the statistical description separate from the decision about what anyone should do next.

This guide uses six original fictional examples. Their counts and labels were deliberately written for teaching; they are not observed model outputs, client records, or measured business results. The imagined task is routing an incoming item to a review queue. Positive means route to review, while negative means do not route to that queue. Those names describe categories in these examples. They do not mean that an item is good, bad, safe, approved, or ready for automatic action.

Start with the question that the labels answer

Write the classification question as a sentence before calculating anything. In the fictional routing task, the question is whether an item belongs in the specified review queue under the supplied reference rule. That is narrower than asking whether the item is important or whether a customer should receive a response. A different question needs different labels, even when both tasks use the same words yes and no. Reusing familiar category names does not establish that two evaluations are comparable.

The reference label also needs a stated origin. For a teaching exercise, an author can supply the expected category. For actual work, the team needs an account of the labeling instructions, who could resolve uncertainty, and how disagreements were handled. A reference column is evidence about the evaluation design; its presence does not prove that every real-world judgment is correct. Preserve unresolved items explicitly rather than turning missing information into the negative category because a spreadsheet formula expects a number.

Give every case an identifier that survives sorting, filtering, and correction. An item labeled positive in one file must be matched to the proposed label for that same item in the comparison file. Equal list lengths are insufficient. A shifted row can compare one item's reference with another item's proposal and still produce a neat matrix. Keep the original inputs, the aligned comparison, and the resulting counts available as separate reviewable materials.

For this guide, the comparison is binary and each resolved case has exactly one reference label and one proposed label. Multiple simultaneous categories, missing proposals, and abstentions require additional definitions. Do not squeeze them into this two-category example by silently assigning a default. An abstention can be an intentional workflow outcome, but a report must explain how it participates in the reported denominator and who handles the unresolved item afterward.

Read the four cells before reading a percentage

A confusion matrix groups aligned comparisons by reference category and proposed category. The scikit-learn confusion-matrix documentation uses reference categories for rows and proposed categories for columns, with an explicit label order. The practical implication is to label both axes when moving a matrix into a presentation. A bare square of numbers loses the context needed to tell a missed positive from an extra positive. Other presentations may use another orientation, so inspect the labels instead of assuming the layout.

In our declared order, the four counts are true negative, false positive, false negative, and true positive. A true negative is reference negative and proposed negative. A false positive is reference negative and proposed positive. A false negative is reference positive and proposed negative. A true positive is reference positive and proposed positive. The word true describes agreement with the supplied reference in this comparison. It is not independent verification of the underlying business situation.

Before reviewing any rate, sum the four cells and reconcile that total with the number of resolved aligned cases. Then reconcile each reference-row total and each proposed-column total with the corresponding lists. These checks answer different questions. The overall total catches an omitted count; the marginals can expose a transposition or labeling mistake that leaves the overall total unchanged. Keep a note of excluded cases because the matrix total alone cannot tell a reader what was excluded.

Case A: reconstruct a modest routing example

Case A contains twelve invented comparisons: six true negatives, two false positives, one false negative, and three true positives. There are four reference positives, eight reference negatives, five proposed positives, and seven proposed negatives. Nine of the twelve proposed labels match their references. The correct-count fraction is therefore nine divided by twelve, or three quarters. That arithmetic describes this authored example only; it is not an estimate of any actual system's performance.

Look at the five items proposed for the review queue. Three have positive references and two have negative references. The proportion of proposed positives that match positive references is three fifths. Now look at the four reference positives. Three were proposed positive and one was proposed negative. The proportion of reference positives found is three quarters. The two fractions answer different questions because one starts with what was proposed and the other starts with what the reference required.

A useful review note can state all three results together: nine of twelve comparisons match; three of five proposed positives match positive references; three of four reference positives were found. This wording exposes the denominators without requiring the reader to remember a metric name. It also makes the two extra queue entries and the one missed reference-positive item visible. An AI-generated summary that says only mostly accurate leaves out the very disagreements a workflow owner needs to examine.

For an actual pilot, the next step would be to inspect the disputed items against the agreed labeling instructions and the intended workflow. This fictional example does not tell us the effort of reviewing an extra entry or the consequence of missing an item. It also does not establish a release threshold. A team could reasonably care about both types of disagreement while assigning them different operational significance. Those judgments need evidence beyond the four counts.

Name precision and recall without swapping their questions

The scikit-learn precision and recall documentation defines precision using true positives over all proposed positives, and recall using true positives over all reference positives. Support refers to the number of reference occurrences for a class. Preserve these names alongside the plain-language denominators. A report that labels three fifths as recall in Case A changes what the result means, even though three fifths is a correctly calculated fraction for a different question.

In a review conversation, ask the presenter to identify which cases belong in each denominator. For precision in Case A, point to the five proposed positives. For recall, point to the four reference positives. This is often more effective than asking someone to repeat a formula. If the underlying list cannot be produced or the category definitions changed during the discussion, pause the interpretation until the counted population is clear.

Avoid treating either measure as a universal preference. A crowded review queue and a missed reference-positive item create different kinds of work. Their consequences depend on the task, available review capacity, fallback process, and actual handling rules. A high recall value does not establish that the queue is manageable. A high precision value does not establish that enough relevant items are being found. Both descriptions should remain attached to the same case population and comparison date.

Case B: a high accuracy fraction can hide an empty queue

Case B deliberately changes the population. Its one hundred invented comparisons contain ninety reference negatives and ten reference positives. Every proposed label is negative. The four cells are ninety true negatives, zero false positives, ten false negatives, and zero true positives. Ninety of one hundred comparisons match, giving an accuracy fraction of nine tenths. However, none of the ten reference positives is found. The positive-class recall is zero divided by ten, which is zero.

There are no proposed positives in this example. Positive-class precision would require division by the number of proposed positives, which is zero. We therefore record that precision as undefined in the teaching data. This is different from observing a nonempty proposed-positive group with no matching references. Reporting a numerical zero may be a declared software convention, but the report must not conceal that the denominator was empty. Software settings and mathematical interpretation belong in the note.

Case B is useful when a summary claims that ninety percent accuracy proves a routing process works. The invented system has found none of the items that the positive reference category identifies. That does not by itself settle whether any real application should be deployed; it shows why the correct-count fraction is incomplete evidence for this task. Ask for the category mix and the missed-positive count before accepting a headline comparison between two reports.

Do not compare Case B's nine tenths directly with Case A's three quarters as though they were runs on the same evaluation set. Their sizes and reference proportions differ. Case B has more negative references, making the all-negative proposal agree more frequently overall. An actual comparison should identify the common evaluation cases, reference version, and handling rules. If populations differ, describe that difference rather than attributing every percentage change to an improved classifier.

Case C: changing the positive category changes the metric question

Case C uses the same twelve comparisons as Case A but declares the other category positive. This is a deliberate relabeling exercise, not a new run. The four counts become three true negatives, one false positive, two false negatives, and six true positives. Nine comparisons still match, so the overall correct-count fraction remains three quarters. There are now seven proposed positives and eight reference positives for the newly named category.

For this relabeled category, precision is six sevenths and recall is six eighths, or three quarters. Case A's three-fifths precision and Case C's six-sevenths precision are not contradictory. They describe different selected classes. The identical recall fractions in these two particular examples are an authored coincidence, not a rule that relabeling preserves recall. Always carry the category name with a metric rather than leaving a detached percentage on a slide.

A plain-language comparison would say that the original review category has three matches among five proposals, while the other category has six matches among seven proposals. This makes the changed question visible. When a report uses numeric labels such as zero and one, retain the mapping back to meaningful category names. Neither a library's default ordering nor an AI assistant's preferred wording should silently determine which class a business reader thinks the report describes.

Case D: compare two proposals on the same reference population

Case D supplies two invented sets of proposed labels for one fixed population of twelve cases. Both use four reference positives and eight reference negatives. The stricter proposal has seven true negatives, one false positive, two false negatives, and two true positives. The broader proposal has five true negatives, three false positives, one false negative, and three true positives. These count sets were authored for comparison; no model or threshold experiment produced them.

The stricter proposal has accuracy nine twelfths, precision two thirds, and recall one half. The broader proposal has accuracy eight twelfths, precision one half, and recall three quarters. Moving from the stricter proposal to the broader proposal finds one additional reference-positive item while adding two additional reference-negative items to the proposed-positive queue. The queue grows from three items to six. These specific count changes explain the tradeoff more clearly than a claim that one proposal performs better overall.

A reviewer can use the example to formulate questions without inventing operational conclusions. Could the queue handle three additional items in the actual workflow? What evidence would be needed about the newly found item? Would a human review all proposed positives before any action? Are there cases where the fallback must detect a missed item? The teaching counts do not answer these questions, and they do not establish that either proposal is suitable for an actual business.

An actual threshold comparison also needs a record of how the alternatives were chosen and evaluated. If someone repeatedly adjusts a rule after inspecting evaluation examples, the resulting report must explain that development history. Keep examples used for designing a proposal distinguishable from examples reserved to assess it. This guide's two authored alternatives demonstrate arithmetic and reporting boundaries; they do not supply a validation design, a generalization estimate, or a threshold recommendation.

Case E: keep group support and empty denominators visible

Case E divides ten fictional comparisons into Cedar and Maple groups. Cedar has six true negatives, zero false positives, one false negative, and one true positive. Maple has one true negative, one false positive, zero false negatives, and zero true positives. Cedar therefore has two reference-positive items and finds one. Maple has no reference-positive items in this supplied population. These group names identify invented example subsets and have no demographic or client meaning.

Cedar's positive-class recall is one half. Maple's positive-class recall is undefined because it has no reference positives. The pooled counts are seven true negatives, one false positive, one false negative, and one true positive. Pooled recall is one divided by two, or one half. It is not an equally weighted average of two observed recall fractions: the second fraction does not exist under the mathematical definition used here. A software convention that substitutes zero needs explicit disclosure.

Suppose a summary averages Cedar's one half with a substituted Maple zero and announces one quarter. The arithmetic is correct for that declared substitution rule, but the number answers a different question from pooled recall. Keep the chosen averaging method in the report heading, including how undefined group values were handled. An aggregate alone cannot establish that both groups were evaluated meaningfully. It also cannot support a fairness conclusion, because this exercise supplies neither a fairness definition nor relevant real-world evidence.

When a group is small or has no support for a class, ask what the evaluation actually observed. Maple's example contains two reference-negative items, so it can describe their proposed labels. It cannot describe how a reference-positive Maple item was handled because no such item is present. More decimal places will not fill that evidence gap. Preserve the group count and the missing class support so that readers can identify the unanswered question directly.

Case F: unresolved references remain outside a completed score

Case F contains four invented items. F1 has reference positive and proposal positive. F2 has reference negative and proposal negative. F3 has reference positive and proposal negative. F4 has an unresolved reference and proposal positive. Three references are resolved, and two of those three comparisons match. The resolved-subset accuracy is two thirds. The reference-resolution coverage is three quarters of the supplied four items. These are different fractions with different purposes.

The full four-item accuracy remains unresolved. If F4 later receives a positive reference, three of four comparisons would match. If it receives a negative reference, two of four would match. Neither possibility may be presented as the observed full score while the reference is missing. Likewise, F4 cannot be placed in the false-positive cell simply because its reference is blank. Missing reference information is not evidence for the negative class.

Keep a clear boundary between a score on resolved cases and a score on every supplied case. A report can be honest and useful while incomplete: identify F4, explain which question remains open, state the resolved-subset result, and refrain from a full-population conclusion. Resolution coverage does not measure semantic coverage of the task, and a completed reference column would still not establish representativeness. It only changes which supplied comparisons can be counted under the declared rules.

Prepare an AI-assisted report that a person can check

Give an assistant the aligned comparison table, the category definitions, the positive category, the axis convention, and the inclusion rules. Ask it to return the four counts before any summary. Require every fraction to show its numerator and denominator, and require undefined values to retain their reason. Request separate statements for resolved-subset results, reference-resolution coverage, and unanswered cases. These are original suggested instructions for an evidence review, not a tested prompt or a promise of accurate model behavior.

A useful requested output has three connected paragraphs. The first describes the supplied population and exclusions. The second explains the counts and rates with category names. The third identifies unresolved operational questions, including consequences and review capacity. Ask for direct references to case identifiers when discussing disagreements. Keep a human check against the supplied rows before circulating the summary, since a fluent explanation can still transpose cells or silently change the denominator.

Store the comparison with its reference version and a brief correction history. If a reference changes, recompute the affected counts and explain which case changed rather than quietly replacing a previous slide. Retain both the original and corrected interpretation when the distinction matters to a review. The goal is a report whose statements can be traced back to supplied evidence, with a separate decision process for any proposed workflow change.

Frequently asked questions

Does ninety percent accuracy mean an AI classifier is ready to use?

It does not establish readiness by itself. Case B's invented all-negative proposals match ninety of one hundred references while finding none of the ten reference positives. Actual readiness requires an explicit task, credible evaluation evidence, consequences, fallback arrangements, and an authorized decision. This guide provides arithmetic examples rather than a recommendation to release any system.

Which denominator belongs to precision and which belongs to recall?

For the selected positive category, precision uses all proposed positives, while recall uses all reference positives. In Case A, three matching positives among five proposed positives gives three fifths precision. Three found among four reference positives gives three quarters recall. Carry the category name and the case population with both results so a reader can reconstruct them.

Should an empty denominator be reported as zero?

State that the mathematical ratio is undefined and disclose any software convention used to display a number. Case B has no proposed positives, so positive-class precision has an empty denominator. That differs from a nonempty proposed-positive group with no matching references. An undisclosed substitution can change an average and conceal what the evaluation never observed.

Can I average the recall numbers for several groups?

Only after defining the intended aggregate and handling missing or undefined values explicitly. Pooled recall combines relevant counts; an equal-group average weights groups differently. Case E also has a group with no reference positives, so its recall is undefined under the stated rule. Preserve group supports and avoid presenting a substitution-based average as the pooled rate.

How should I handle a case whose reference label is unresolved?

Keep its reference state explicit and report which results use only the resolved subset. Case F has two matches among three resolved references and three resolved references among four supplied items. Full-population accuracy remains unresolved. Do not treat a blank reference as negative or replace an unknown result with the more favorable of two hypothetical resolutions.

Were these examples produced by a model or checked with scikit-learn?

No model produced these teaching examples, and scikit-learn was not executed for them. The counts, case labels, and expected fractions are original authored fixtures checked with narrow independent local arithmetic and consistency assertions. Primary documentation supplies limited metric terminology. Those checks establish the stated fictional arithmetic, not actual model accuracy, professional review, business value, fairness, or release approval.