# BUTom21

**A fully hand-annotated still-image tomato dataset for detection and instance segmentation.**

## Download \& extract

The dataset is distributed as a single compressed archive, "BUTom21.tar.gz".
After downloading, extract it with:

"""
tar -xzf BUTom21.tar.gz

"""

## Folder structure

"""
BUTom21/
├── CKA\_Tomato\_2021\_images/
│   ├── 2021\_08\_18/                 # capture day 1
│   │   ├── row\_1/
│   │   │   ├── rgb/                # RGB images,   <frame\_id>.png
│   │   │   └── depth/              # depth images, <frame\_id>.png
│   │   ├── row\_2/
│   │   └── ...                     # rows 1–6
│   └── 2021\_09\_10/                 # capture day 2
│       └── ...
├── CKA\_tomato\_2021.json            # COCO-format annotations
├── CKA\_tomato\_2021.yaml            # class definitions + train/valid/eval splits + statistics
├── image\_camera\_row\_identity.yaml  # which image came from which camera, per row
└── BUTom21\_structure.md
"""

Images were captured using the Intel RealSense D435i. Each RGB image has a matching depth image
with the same file name (<frame\_id>.png) in the parallel "depth/" folder. All images are
"720 × 1280" (width × height, portrait).

The "<frame\_id>" is the capture timestamp and is the unique identifier used throughout the dataset
(it matches the RGB file name, the depth file name, and the entries in "image\_camera\_row\_identity.yaml").

\---

## Files

### "CKA\_tomato\_2021.json" — annotations

A standard COCO JSON with three top-level keys: "images","categories", and "annotations".

"images" — one entry per image. Key fields:

|field|description|
|-|-|
|"id"|unique **image\_id**. This links to the split lists in the YAML (below).|
|"file\_name"|"<frame\_id>.png"|
|"path"|relative path to the RGB image inside the dataset, e.g. "/BUTom21/CKA\_Tomato\_2021\_images/2021\_09\_10/row\_5/rgb/<frame\_id>.png"|
|"width", "height"|"720", "1280"|



"categories" — the five ripeness subclasses (see the class table below). Each entry has an "id","name", and display "color".

"annotations" — one entry per annotated tomato instance. Key fields:

|field|description|
|-|-|
|"image\_id"|the image this instance belongs to (matches "images\[].id")|
|"category\_id"|ripeness subclass id ("0"–"4")|
|"segmentation"|**polygon** mask in COCO format: "\[\[x1, y1, x2, y2, ...]]" (image pixel coordinates)|
|"bbox"|bounding box "\[x, y, width, height]"|
|"area"|mask area in pixels|
|"iscrowd"|"false" (all instances are individually labelled)|



### "CKA\_tomato\_2021.yaml" — classes, splits, and statistics

* "image\_sets" — the official split. Three lists ("train", "valid", "eval") of image\_ids.
These ids correspond directly to "images\[].id" in the JSON. Splits are row-disjoint to guarantee a held-out evaluation set:

  * "train" — 123 images
  * "valid" — 72 images
  * "eval"  — 98 images
* "class\_ids" / "class\_labels" — parallel lists defining the classes and their order.
* "class\_areas", "class\_instances", "class\_weights", "idf"\*\* — per-class statistics, in the same order as "class\_ids".
* "img\_size" — "height: 1280", "width: 720".
* "img\_mu", "img\_std"\*\* — per-channel (RGB) mean and standard deviation for normalisation.



### "image\_camera\_row\_identity.yaml" — camera identity

This file records which camera each image came from, so users who want to work with a single camera can filter accordingly.
Structure is "day → row → camera → \[frame\_id, ...]":

"""
2021\_08\_18:
row\_1:
cam\_1: \['1629277558049203873', ...]
cam\_2: \['1629277563049440622', ...]
row\_2:
...
2021\_09\_10:
...
"""

"frame\_id"s are listed as strings and sorted, and match the RGB/depth file names exactly.

\---

## Categories (ripeness subclasses)

|id|name|color|
|-|-|-|
|0|"red"|"#e31c1c"|
|1|"mixed\_red"|"#d34c03"|
|2|"green"|"#5fad00"|
|3|"orange"|"#cd9418"|
|4|"mixed\_orange"|"#dbc58a"|
|255|"bg"|—|

"bg" (background) exists only in the YAML statistics; it is not an annotated category in the JSON.

\---

## Quick start

A minimal PyTorch-style dataloader is provided in the repository:
https://github.com/Agricultural-Robotics-Bonn/BUTom21-ST21

\---

## Citation

If you use this dataset, please cite both the associated paper and the dataset itself as below:

## Paper citation

@misc{halstead2026imagespatialtemporaltomatodata,
title={Still image and spatial-temporal tomato data enabling detection, segmentation, tracking, and video-instance segmentation using strong and weak labels},
author={Michael Halstead and Esra Guclu and Mohamed Farag and Enrico Pallotta and Christian Hund and Ribana Roscher and Maren Bennewitz and Juergen Gall and Cyrill Stachniss and Chris McCool},
year={2026},
eprint={2607.14934},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2607.14934},
}

## Dataset citation

@data{FK2/DHTEH1\_2026,
author    = {Halstead, Michael and Guclu, Esra and Farag, Mohamed and Pallotta, Enrico and Hund, Christian and Roscher, Ribana and Bennewitz, Maren and Gall, Juergen and McCool, Chris},
publisher = {bonndata},
title     = {BUTom21: Still image tomato dataset for detection and segmentation,
year      = {2026},
version   = {V1},
doi       = {10.60507/FK2/DHTEH1},
url       = {https://doi.org/10.60507/FK2/DHTEH1}
}



\---