Read Multiclass Macro, Micro and Support-Weighted Metrics Before Comparing AI Reports

A classification report can contain several correct averages that answer different questions. Macro averaging gives each selected class the same weight. Support weighting gives more influence to classes with more reference examples. Micro averaging pools the selected true-positive, false-positive and false-negative counts before calculating a ratio. An AI summary that calls all three the overall score can hide the distinction that a reader needs to understand the evaluation.

This guide develops an original reading workflow for single-label multiclass reports. Each example assigns one reference class and one predicted class to a resolved case. The six practice groups use invented labels and supplied counts, not outputs from a model or records from a client. Python checks use exact fractions and explicit arithmetic. No scikit-learn function, classifier, statistical test, professional review or operating approval was executed or obtained.

Specify the task before reading the average

Begin with the classification unit and allowed labels. Is each row one document, message, passage or another declared object? Does every resolved row have exactly one reference category and one prediction? This guide uses that single-label setting. A multilabel task, in which one row can carry several labels, needs a different interpretation of counts and some averages. Do not transfer an identity from this guide to another task merely because its report uses the same metric names.

Also separate a supplied reference label from verified truth. The calculations below take the authored labels as inputs so that their arithmetic is inspectable. They do not establish that a real reviewer chose the correct category, that the categories capture the business question or that the cases represent future traffic. The existing reviewer-agreement and evaluation-dataset guides address related evidence questions. An average cannot repair an unsuitable reference definition or a population that was selected for another purpose.

The official scikit-learn precision, recall and F-score documentation distinguishes per-label results, global count pooling, unweighted label means and reference-support weighting. It also documents choices for undefined ratios. That limited definition and interface context is the primary-source basis here. The examples implement their own small arithmetic exercise rather than claiming library conformance, installed-version behavior or a complete treatment of multilabel evaluation.

Record the label set explicitly. A report about Cedar, Maple and Birch has three named categories even if one candidate never predicts one of them. An additional category with no references or predictions creates a separate undefined-value question. If a tool infers the categories from whichever values happen to appear, two reports may average different sets. Read the selected labels before comparing their displayed numbers.

Build the per-class counts from one declared matrix

Fictional case A supplies a matrix whose rows are reference classes and columns are predicted classes, both ordered Cedar, Maple, Birch. The rows are [6,1,1], [1,1,0] and [0,1,1]. Reference supports are eight, two and two; predicted counts are seven, three and two. The diagonal totals eight matching labels among twelve authored cases. The matrix is original teaching data, not a confusion matrix observed from any model.

For each selected class, temporarily ask whether that class is present versus all other classes. Cedar has six true positives, one false positive and two false negatives. Maple has one true positive, two false positives and one false negative. Birch has one true positive, one false positive and one false negative. These are three different views of the same supplied cases, not three independent evaluations or thirty-six independent observations.

Precision divides a class's true positives by its predicted count. Recall divides its true positives by reference support. Case A therefore gives Cedar precision six-sevenths and recall three-quarters; Maple precision one-third and recall one-half; Birch precision and recall one-half. Keep the class name and denominator with each value. A statement that Maple scores one-third without naming precision leaves the reader unable to tell which question the ratio answers.

Use the F-score formula only after the underlying count convention is clear. Here the per-class F1 score is twice true positives divided by twice true positives plus false positives plus false negatives. Case A gives Cedar four-fifths, Maple two-fifths and Birch one-half. These are unweighted binary one-versus-rest calculations for each named class. They are not costs, probabilities of business success or severity-adjusted outcomes.

Read macro averaging as equal label influence

The macro F1 score in case A is the mean of four-fifths, two-fifths and one-half, giving seventeen-thirtieths. Each of the three classes contributes one-third of the average despite reference supports of eight, two and two. That equal class influence is the declared aggregation rule. It does not establish that each class has equal practical importance or that the smaller class results are estimated with adequate evidence.

Macro precision and macro recall are calculated separately from their per-class values. Case A's macro precision is seventy-one over 126, and macro recall is seven-twelfths. Macro F1 is the mean of the three class F1 values. It is not defined here by taking the harmonic mean of macro precision and macro recall. A report can display all three averages while still requiring their separate construction to be explained.

Equal influence can make a weak smaller class more visible than an accuracy figure does. That is a descriptive property of the weighting rule, not automatic evidence that macro averaging is the right release criterion. A business decision needs the task, consequences, uncertainty and operational constraints that matter to that actual setting. The guide recommends making the chosen question visible; it does not choose a universal score or threshold for deployment.

When an AI draft says the macro result proves balanced performance, ask for the class table. Equal weighting in a formula is not evidence that performance is similar across classes. In case A the class F1 values differ. The average compresses those differences into one number. Retain the detailed results if the reader's question depends on which categories were confused rather than merely how a chosen summary was computed.

Read support weighting without changing the meaning of support

For case A, support-weighted F1 multiplies the three F1 values by reference counts eight, two and two, then divides by twelve. The result is forty-one-sixtieths. It is a weighted mean of class scores. It is neither the unweighted class mean nor the result of pooling the F1 denominators first. Keep the class support column with the calculation so another person can reproduce the contribution of each category.

Fictional case E returns to the same per-class recalls and makes the weight choice explicit. Reference-support weighting gives recall two-thirds. Substituting predicted counts seven, three and two as the weights gives thirty-one over forty-eight instead. That alternative number uses different weights; it is not the support-weighted recall defined in this guide. The arithmetic is deliberate teaching data, not an actual software bug or a finding that a report used the wrong implementation.

Support means the number of supplied true instances for a label in this setting. It does not mean confidence, importance, predicted frequency or the number of records a reviewer happened to inspect. If the report uses sample weights, the relevant definition can change and must be stated. This article uses no sample weights and makes no claim about an unseen tool's handling of them. Do not infer a weighted calculation from the word weighted alone.

A class with more supplied references has more influence on a support-weighted mean. That may describe the authored evaluation mixture while differing from future traffic or an intended service population. Keep evaluation composition separate from deployment composition. Changing the mix of cases can change an aggregate without changing any individual class ratio, so a comparison should preserve both the per-class values and the weights used to combine them.

Read micro averaging as count pooling

Micro averaging sums the selected class true positives, false positives and false negatives before calculating precision, recall or F1. In case A those totals are eight, four and four. Micro precision, recall and F1 are each two-thirds. This equality follows for the declared complete single-label setting with all three classes included. It is not a universal identity for every classification task or every selected subset of labels.

Each mismatched single-label case contributes one false positive to its predicted class and one false negative to its reference class. With all labels included, the pooled predicted and reference totals both equal the case count. That explains why the three micro ratios match label accuracy here. It does not mean the metrics are interchangeable when the task allows several labels, abstentions, omitted classes or another counting convention.

Do not call the pooled count an independent sample size. Summing one-versus-rest counts reorganizes the same twelve cases rather than creating more observations. The dependence and sampling questions remain attached to the original evaluation units. This guide computes no confidence interval or statistical significance. A more elaborate table does not by itself create more evidence than the underlying supplied cases contain.

Compare candidates without letting accuracy erase the class pattern

Fictional case B keeps reference supports eight, two and two but predicts Cedar for every case. Eight labels match among twelve, so accuracy and full-label micro F1 remain two-thirds, just as in case A. The class behavior is different: Maple and Birch receive no predictions. Equal accuracy therefore does not establish equal category behavior, identical output text or an equivalent operational effect.

For the all-Cedar candidate, Cedar precision is two-thirds, recall one and F1 four-fifths. Maple and Birch precision are undefined because their predicted counts are zero; their recalls are zero. For this authored comparison only, the case declares zero substitution for their undefined precision and uses zero F1. Macro F1 becomes four-fifteenths, and support-weighted F1 eight-fifteenths. The replacement convention must travel with those numbers.

Those results show why a comparison should name its criterion. Case A and case B match on one aggregate while differing on others. They do not establish which real candidate should be deployed, how costly the mistakes are or whether a missed category has been judged appropriately. An assistant should describe the supplied tradeoff and open questions rather than convert a higher average into an unsupported recommendation or claim of business improvement.

Keep a selected label subset separate from row filtering

Fictional case C selects only Maple and Birch for micro aggregation within case A's full matrix. Their pooled true positives total two, false positives total three and false negatives total two. The resulting precision is two-fifths, recall one-half and F1 four-ninths. The calculation retains errors involving Cedar that contribute to the selected classes. It is not the all-label micro score of two-thirds.

Selecting labels for reporting is different from discarding every row whose reference is Cedar. A Cedar-reference case predicted Maple contributes a false positive for Maple in the label-selected calculation. Removing that case first would change the question and the counts. State whether a report selects columns of class results, filters cases, or does both in a declared order. A shared subset name does not establish equivalent populations.

A subset can be useful when a reader has a specific category question. Preserve its label membership and reason for selection instead of calling the result overall accuracy. If another report includes all categories, present the scope difference before comparing values. The guide supplies no fairness judgment, priority classification or authorization to exclude difficult cases from an actual evaluation. Its purpose is to make the arithmetic scope visible.

Make absent classes and undefined values explicit

Fictional case D adds an Ash label to the reporting set but supplies no Ash references and no Ash predictions. Both precision and recall for Ash have zero denominators; its F1 also has a zero denominator under the stated count formula. The example leaves those raw ratios undefined. It does not silently treat the absence of cases as perfect or poor observed performance, because neither claim follows from the authored inputs.

If a declared reporting policy inserts a zero Ash F1 before macro averaging, the four-class mean becomes seventeen-fortieths. If the undefined class is excluded, the three-class mean remains seventeen-thirtieths. These are different declared summaries, not evidence that either convention measures the absent class. The official library documents undefined-value options, but no library function is executed here and the example does not prescribe a universal policy.

A report should retain the reason a value is missing. No references, no predictions and an unavailable source label are different situations. A displayed dash can conceal that difference. Ask an assistant to preserve undefined states alongside any replacement summary so that a reader can see which values were observed ratios and which were inserted under a reporting convention. A cleaner table should not turn missing evidence into a measured result.

Separate unavailable references from mismatches

Fictional case F supplies three reference entries: Cedar, Maple and unknown, with predictions Cedar, Birch and Maple. The first two references are resolved and one prediction matches, so resolved-subset accuracy is one-half. Reference availability is two-thirds of the three authored cases. Those denominators answer different questions. Neither number establishes the full-set accuracy while the third reference remains unknown.

If the unknown reference were Maple, the full-set matching count would become two of three. If it were Cedar or Birch, the count would remain one of three. These are explicitly hypothetical completions, not imputed labels or observed outcomes. Keep the complete class report unresolved until the missing input is actually supplied. A missing reference should not automatically become a correct prediction, a mismatch or a new category.

Record how case identifiers connect references and predictions before aggregating. A positionally aligned table can appear valid while pairing different objects. The separate paired-outcome and record-list guides explain related alignment questions. This guide assumes the authored pairing supplied in its cases and verifies no real identity resolution. Class arithmetic can be correct for a table that still answers the wrong business question if its rows were assembled incorrectly.

Ask AI for a reproducible interpretation

A bounded prompt can supply the label order, matrix orientation, selected classes, undefined-value convention and exact aggregation rule. Ask the assistant to show per-class counts before the chosen average, name the denominator and keep unsupported operational conclusions open. These are original suggested instructions, not an observed model response or a tested prompt. Verify its arithmetic against the supplied records before relying on the explanation.

Keep the report's practical question in view. You may want to understand a category that is rarely predicted, compare two candidate patterns or explain why an average changed after the case mixture changed. A well-scoped explanation can answer that question without claiming that an average proves readiness, fairness or real-world value. The linked evaluation-dataset guide helps keep evidence design separate from a calculation on the examples already supplied.

Explain what changed between two reports

Before interpreting a new average, compare the report definitions in a fixed order: case population, reference version, label set, pairing, counting rule, undefined-value policy and aggregation weights. A change at any of those levels can alter the number without establishing that a candidate changed its behavior. Keep a small description of the evaluation version with the result so an assistant can identify a definition change before narrating a performance trend.

Then compare the per-class contributions. In the support-weighted case A calculation, Cedar contributes eight times four-fifths, Maple two times two-fifths and Birch two times one-half to the numerator before division by twelve. Seeing those separate terms helps explain why the larger class influences the result. It also makes a transcription mistake easier to locate than a report containing only the final decimal. The contribution is arithmetic influence, not a claim that one class deserves greater practical priority.

Preserve exact fractions or enough underlying counts to reproduce a rounded display. Two rounded values may look equal while their exact ratios differ, and repeated rounding can create apparent inconsistencies between a table and its summary. This guide retains exact fractions in the checks rather than treating a displayed decimal as the input for another calculation. It does not prescribe the number of decimal places suitable for every report or infer statistical precision from formatting.

Finally distinguish a numerical difference from an explanation for it. A lower Maple recall can be described in terms of its supplied true positives and support. Explaining why the candidate made those mistakes requires different evidence, such as actual cases and a defensible review process. No such investigation occurs here. Keep a proposed cause as a question rather than making a class-average table stand in for an analysis of language, data collection or operating conditions.

Frequently asked questions

Are macro F1 and micro F1 the same score?

No. Macro F1 averages per-class F1 values with equal label influence. Micro F1 pools selected class counts first. Case A gives seventeen-thirtieths and two-thirds respectively from the same authored matrix. Neither establishes business value, reference correctness or suitability for a real release.

What weights does support-weighted averaging use here?

It uses reference support for each class. Case A's supports are eight, two and two, producing weighted F1 forty-one-sixtieths. Case E deliberately substitutes prediction counts for recall weights and obtains a different result. That alternative is not the stated support-weighted calculation and is not an observed tool failure.

Why can two candidates have equal accuracy but different macro scores?

Accuracy counts matching cases without preserving every category pattern. Case B predicts Cedar for all twelve cases and matches eight, like case A, while never predicting Maple or Birch. Their macro results differ under the explicitly stated undefined-value convention. This does not determine which actual candidate should be used.

Does selecting two labels mean deleting all other reference rows?

Not automatically. Case C selects Maple and Birch for reporting while retaining their false positives from other reference classes. Its micro precision is two-fifths and F1 four-ninths. A row-filtered evaluation would be a different task whose population and counting rules must be stated separately.

What should a report do with a class that has no cases?

Show the undefined state and any declared replacement or exclusion rule. Case D's absent Ash class yields different macro means when zero is inserted versus the class omitted. Neither summary measures performance on Ash. No universal policy or actual library behavior is certified by this exercise.

Do the cases prove model quality or fairness?

No. They are six original fictional groups checked for supplied counts, exact ratios and declared averaging conventions only. No classifier, client evaluation, statistical inference, reference validation, fairness assessment, professional review, endorsement or operating approval was performed. Their CSV is teaching data rather than an executable utility.