P e x c e r a

Mean Average Precision (mAP) Evaluation

Evaluating object detection models requires a metric that balances classification accuracy and localization precision. Mean Average Precision (mAP) solves this by averaging performance across classes and IoU thresholds using the precision-recall curve.

Precision-Recall Curve in Detection

In object detection, a prediction is labeled a True Positive (TP) if its IoU with a ground truth box of the same class exceeds a predefined threshold (usually 0.5); otherwise, it is a False Positive (FP).

Classifying Predictions

Predictions are sorted by confidence score. As we traverse this list, we calculate cumulative precision and recall to plot a precision-recall curve. The Area Under this Curve (AUC) represents the Average Precision (AP) for a single class.

Calculating Average Precision

AP is computed by interpolating the precision at different recall levels, typically using the COCO 101-point interpolation or the area under the interpolated curve: AP = \\sum (r_{k+1} - r_k) p_{interp}(r_{k+1}).

Mean Average Precision (mAP)

mAP aggregates AP scores across all categories. In benchmarks like COCO, it is further averaged across multiple IoU thresholds to encourage tight localization.

mAP@0.5 and mAP@0.5:0.95

mAP@0.5 evaluates detection at a loose 50% IoU threshold. mAP@0.5:0.95 computes the average of mAP across 10 IoU thresholds (from 0.50 to 0.95 with steps of 0.05), penalizing loose bounding boxes.

Evaluating with TorchMetrics

TorchMetrics provides a standard implementation of the COCO mAP metric for PyTorch.

<pre><code class="language-python">from torchmetrics.detection import MeanAveragePrecision import torch metric = MeanAveragePrecision(box_format="xyxy") preds = [ dict( boxes=torch.tensor([[10, 20, 110, 120]], dtype=torch.float32), scores=torch.tensor([0.9]), labels=torch.tensor([0]) ) ] targets = [ dict( boxes=torch.tensor([[10, 20, 100, 100]], dtype=torch.float32), labels=torch.tensor([0]) ) ] metric.update(preds, targets) print(metric.compute()) # Computes map, map_50, map_75 etc.</pre>