This file was generated on 2026-07-14 by Esra Guclu A GENERAL INFORMATION 1. Title of the dataset: BUTom-ST21: Spatial-temporal tomato data enabling detection, segmentation, tracking, and video-instance segmentation using strong and weak labels 2. Brief description of the research project and its aims: The BUTom-ST21 alleviates the data paucity issue within the horticulture and tomato domain. We employed BUTom21 still image dataset and generated spatial-temporal tomato dataset using a neural-radiance field based approach. In total we have 217 video sequences (123 train/ 72 valid/ 22 eval) can be used for image-based/video-based detection and segmentation, and multi object tracking. The train and validation sets are weakly labeled(pseudo-labeled), and evaluation set is hand-labeled ground-truth. BUTom-ST21 has 7749, and 4536 pseudo-labeled images across the training and validation set, and 1386 images hand-labeled in the evaluation set. Additionally it has camera poses per sequence, enabling the use of 3D reconstruction purposes in horticultural field. 3. Author Information A. Investigator Contact Information Name: Michael Halstead Institution: Institute of Agricultural Engineering Email: michael.halstead@uni-bonn.de Name: Esra Guclu Institution: Institute of Agricultural Engineering Email: egueclue@uni-bonn.de B. Project Supervisor (Principal Investigator) Contact Information Name: Chris McCool Institution: CSIRO Address: Australia Email: Chris.Mccool@csiro.au Name: Cyrill Stachniss Institution: University of Bonn Address: Bonn Email: cyrill.stachniss@igg.uni-bonn.de Name: Juergen Gall Institution: University of Bonn Address: Bonn Email: gall@cs.uni-bonn.de Name: Maren Bennewitz Institution: University of Bonn Address: Bonn Email: maren@cs.uni-bonn.de Name: Ribana Roscher Institution: University of Bonn Address: Bonn Email: ribana.roscher@uni-bonn.de C. In case of questions related to this dataset, please contact: Name: Esra Guclu Institution: Institute of Agricultural Engineering Email: egueclue@uni-bonn.de 4. Date of data collection: 2021-08-18 - 2021-09-10 5. Information about funding sources that supported the collection of the data: - 6. Language of the dataset: English 7. Geographic location of data collection: Commercial Glasshouse at Campus Klein-Altendorf (CKA), University of Bonn B DATA & FILE OVERVIEW 1. File List: - `rgb/` : RGB images for all sequences (png format, grouped by video) - `depth/` : Depth images matching the RGB frames (png format, grouped by video) - `poses/` : pose information from COLMAP and Hloc (one per sequence) - `pseudo_labels_m2f/` : Pickle files containing bounding boxes, masks, and instance IDs based on Mask2Former - `pseudo_labels_yolo26/` : Pickle files containing bounding boxes, masks, and instance IDs based on yolov26 - `eval_annotations/` : Pickle files containing bounding boxes, masks, and instance IDs for evaluation set - `camera_parameters.yaml` : Camera intrinsics used during recording - `train_valid_eval_splits.yaml` : Dataset split configuration (train / validation / evaluation) - `annotated_seq_frame_list.yaml` : The list of annotated frame ids. - `pagnerf_seq_frames.yaml` : The list of the "83" frame ids present in rgb,depth and posesv - `image_camera_row_identity.yaml` : Records which camera each sequence was captured with - `README_BUTom-ST21.md` : Explains how to work with the dataset, annotations, pose, and camera parameters 2. Are there multiple versions of the dataset?: No 3. Relationship between files: frame ID 4. Additional related data collected that was not included in the current data package: - C SHARING/ACCESS INFORMATION 1. Was data derived from another source?: No 2. Licenses/restrictions placed on the data: CC BY 4.0 3. Links to publications that cite or use the data: If you use this dataset, please cite both the associated paper and the dataset itself as below: ## Paper citation ## @misc{halstead2026imagespatialtemporaltomatodata, title={Still image and spatial-temporal tomato data enabling detection, segmentation, tracking, and video-instance segmentation using strong and weak labels}, author={Michael Halstead and Esra Guclu and Mohamed Farag and Enrico Pallotta and Christian Hund and Ribana Roscher and Maren Bennewitz and Juergen Gall and Cyrill Stachniss and Chris McCool}, year={2026}, eprint={2607.14934}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2607.14934}, } ## Dataset citation ## @data{FK2/TTPCNV_2026, author = {Halstead, Michael and Guclu, Esra and Farag, Mohamed and Pallotta, Enrico and Hund, Christian and Roscher, Ribana and Bennewitz, Maren and Gall, Juergen and McCool, Chris}, publisher = {bonndata}, title = {BUTom-ST21: Spatial-temporal tomato data enabling detection, segmentation, tracking, and video-instance segmentation using strong and weak labels, year = {2026}, version = {V1}, doi = {10.60507/FK2/TTPCNV}, url = {https://doi.org/10.60507/FK2/TTPCNV} } 4. Links to other publicly accessible locations of the data : 5. Links/relationships to ancillary datasets: BUTom21 and BUP-ST20 ## BUTom21 ## Halstead, Michael; Guclu, Esra; Farag, Mohamed; Pallotta, Enrico; Hund, Christian; Roscher, Ribana; Bennewitz, Maren; Gall, Juergen; McCool, Chris, 2026, "BUTom21: Still image tomato dataset for detection and segmentation", https://doi.org/10.60507/FK2/DHTEH1, bonndata ## BUP-ST20 ## Guclu, Esra; Halstead, Michael; Denman, Simon; McCool, Chris, 2025, "BUP-ST20: Weakly Labelled Spatial Temporal Sweet Pepper Data", https://doi.org/10.60507/FK2/NUMVO1, bonndata D METHODOLOGICAL INFORMATION 1. Description of methods used for collection/generation of data: The dataset was collected using a mobile robotic platform equipped with RGB-D cameras in a commercial glasshouse. Images were recorded while the robot moved row-by-row through the crop field, capturing synchronized RGB and depth images across multiple sessions. 2. Methods for processing the data: The raw image data was processed using a NeRF-based panoptic segmentation pipeline (PAg-NeRF - doi:10.1109/LRA.2023.3338515), which enabled weak labelling for training/validation frames. Ground-truth annotations for the evaluation set were manually corrected. Camera calibration were synchronized with visual data during processing. 3. Instrument- and/or software-specific information needed to interpret the data: The dataset was processed using Python-based workflows, including PyTorch for segmentation model 4. People involved in sample collection, processing, analysis and/or submission: Michael Halstead, Esra Guclu, Chris McCool, Patrick Zimmer, Christian Hund, Yan Wang, Sina Raufi, Efe Incir, Nelson Pinheiro, Julian Rosbach, Rafay Aamir, Moein Taherkhani, Sicong Pan, Gokul Krishna Gandhi Chenchani 5. Describe any quality-assurance procedures performed on the data: Annotations were generated via the NeRF-based panoptic segmentation approach, PAg-NeRF (doi:10.1109/LRA.2023.3338515). The evaluation set annotations were manually checked and verified by human annotators.