Landmark 5 stops to get here · leads to 1

Intersection over Union

Also called Jaccard Index

A score from 0 to 1 for how well a predicted region matches the true one: the area they share divided by the area they cover together.

Your route here

5 stops · basics first
  1. Machine Learning ✓ understood

    Building systems that learn patterns from data instead of following hand-written rules, getting better at a task as they see more examples.

  2. Neural Network ✓ understood

    A model built from layers of simple units, each taking a weighted sum of its inputs and applying a nonlinearity; training adjusts the weights so the whole stack maps inputs to useful outputs.

  3. Deep Learning ✓ understood

    A subset of machine learning that uses neural networks with multiple layers (deep neural networks) to learn hierarchical representations of data.

  4. Computer Vision ✓ understood

    The field of AI that gets computers to extract meaning from images and video: what is in them, where it is, and how it moves.

  5. Bounding Box ✓ understood

    A rectangular box defined by coordinates that localizes an object in an image, used in object detection.

  6. Intersection over Union · you are here ✓ understood

Picture it

ground truthpredictionoverlapIoU = overlap ÷ union= 13,200 ÷ 54,000= 0.24
Notice IoU is the shaded overlap divided by the total area both boxes cover: 1 means a perfect match, 0 means no overlap.

Intersection over Union, or IoU, measures how well two shapes line up. In object detection the two shapes are the model’s predicted bounding box and the hand-labelled ground truth box. Take the area where they overlap and divide it by the total area covered by either one. Identical boxes score 1. Boxes that don’t touch score 0.

The same ratio is known in statistics as the Jaccard index. Computer vision uses it because it’s easy to read, and it doesn’t depend on scale: a box that’s 10 pixels off matters a lot for a small object and very little for a large one, and IoU reflects that.

Why the union matters

Dividing by the union, as the diagram shades it, is what stops a detector from cheating. If you only measured how much of the true box the prediction covered, a huge box drawn over the whole image would score perfectly. The union grows with every stray pixel, so both overshooting and undershooting lower the score.

Turning a score into right or wrong

Detection benchmarks use IoU as a pass mark. The PASCAL VOC challenge counted a prediction as correct only when its IoU with a true box exceeded 0.5, a threshold its authors set deliberately low because hand-drawn boxes are imprecise. Anything below becomes a false positive. The COCO benchmark tightened this: its headline average precision is averaged over ten thresholds from 0.50 to 0.95, which rewards detectors that place boxes precisely, not just roughly.

IoU does other jobs too. Non-maximum suppression uses it to spot duplicate boxes around the same object, and semantic segmentation applies the same ratio to pixel masks instead of rectangles, usually reported as mean IoU across classes.

The catch

IoU is flat at zero for every pair of boxes that don’t overlap, whether they’re nearly touching or on opposite sides of the image. That makes it a poor training signal on its own: a model with a wildly wrong box gets no hint which way to move. Generalized IoU, proposed in 2019, fixes this by also penalizing the empty space inside the smallest box that encloses both. The related Dice coefficient weights the overlap more generously and is often used for medical image segmentation.

Where it sits

Before this

Bounding Box
Intersection over Union

Explore nearby

Sources

  1. Everingham et al., "The PASCAL Visual Object Classes (VOC) Challenge" . International Journal of Computer Vision 88, 2010; section 4.2, the 50% overlap criterion
  2. COCO, "Detection Evaluation" . COCO dataset documentation
  3. Rezatofighi et al., "Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression" . 2019