-------------------------------------------------------------------------------------------------------------

This file was generated on 2025-09-29 by Esra Guclu


A GENERAL INFORMATION

1. Title of the dataset: BUP-ST20 (Weakly Labelled Spatial Temporal Sweet Pepper Data)

2. Brief description of the research project and its aims: 

	Accurate monitoring of crop phenotypic traits is essential for efficient farm management and automation in agriculture. Multi-object tracking (MOT) and video instance segmentation (VIS) offer promising approaches to enhance agricultural robotic vision systems, yet a major limitation is the scarcity of high-quality spatial-temporal datasets. We introduce BUP-ST20, a novel weakly labelled spatial-temporal dataset for sweet pepper tracking and segmentation captured on a robotic platform. BUP-ST20 contains 16,240 images from 275 sequences, each with bounding boxes, instance segmentation masks, and temporal identities.The dataset has weakly labelled training and validation sets, while the evaluation set includes 3810 frames with hand-labelled ground truth annotations.

3. Author Information
	A. Investigator Contact Information
		Name: Esra Guclu
		Institution: Institute of Agricultural Engineering
		Email: egueclue@uni-bonn.de
		
		Name: Michael Halstead
		Institution: Institute of Agricultural Engineering 
		Email: michael.halstead@uni-bonn.de

	B. Project Supervisor (Principal Investigator) Contact Information
		Name: Chris McCool
		Institution: Institute of Agricultural Engineering
		Email: cmccool@uni-bonn.de

	C. In case of questions related to this dataset, please contact: 
		Name: Esra Guclu
		Institution: Institute of Agricultural Engineering
		Email: egueclue@uni-bonn.de

4. Date of data collection: 2020-09-24 - 2020-10-01

5. Information about funding sources that supported the collection of the data: -

6. Language of the dataset: English

7. Geographic location of data collection: Commercial Glasshouse at Campus Klein-Altendorf (CKA), University of Bonn


B DATA & FILE OVERVIEW

1. File List: 

	- `rgb/` : RGB images for all sequences (TIFF format, grouped by video)
	- `depth/` : Depth images matching the RGB frames (TIFF format, grouped by video)
	- `odoms/` : Wheel odometry CSV files (one per sequence)
	- `annotations/` : Pickle files containing bounding boxes, masks, and instance IDs
	- `cam_params.yaml` : Camera intrinsics and extrinsics used during recording
	- `train_valid_eval_splits.yaml` : Dataset split configuration (train / validation / evaluation)
	- `dataset_structure.md` : Describes folder and file structure of the dataset
    	- `how_to_use_BUPST20.md` : Explains how to work with the dataset, annotations, odometry, and camera parameters


2. Are there multiple versions of the dataset?: No

3. Relationship between files: frame ID

4. Additional related data collected that was not included in the current data package: -


C SHARING/ACCESS INFORMATION

1. Was data derived from another source?: No

2. Licenses/restrictions placed on the data: CC BY 4.0

3. Links to publications that cite or use the data: If you use this dataset, please cite both the associated paper and the dataset itself as below:

## Paper citation ## 
@Article{2025bupst20,
  author   = {Guclu, Esra and Halstead, Michael and Denman, Simon and McCool, Chris},
  title    = {Weakly Labelled Spatial-Temporal Sweet Pepper Data: enabling higher quality detection, segmentation, and tracking.},
  journal  = {The International Journal of Robotics Research},
  year     = {2025},
  volume   = {},
  pages    = {},
  month    = {},
  doi      = {10.1177/02783649251379093},
  publisher= {SAGE Publications Sage UK: London, England}
}

## Dataset citation ##  
@data{FK2/NUMVO1_2025,  
  author    = {Guclu, Esra and Halstead, Michael and Denman, Simon and McCool, Chris},
  publisher = {bonndata},  
  title     = {{BUP-ST20: Weakly Labelled Spatial Temporal Sweet Pepper Data}},  
  year      = {2025},  
  version   = {V2},  
  doi       = {10.60507/FK2/NUMVO1},  
  url       = {https://doi.org/10.60507/FK2/NUMVO1}  
}


4. Links to other publicly accessible locations of the data: 


D METHODOLOGICAL INFORMATION

1. Description of methods used for collection/generation of data: The dataset was collected using a mobile robotic platform equipped with RGB-D cameras in a commercial sweet pepper glasshouse. Images were recorded while the robot moved row-by-row through the crop field, capturing synchronized RGB and depth images across multiple sessions.

2. Methods for processing the data: The raw image data was processed using a NeRF-based panoptic segmentation pipeline (PAg-NeRF - doi:10.1109/LRA.2023.3338515), which enabled weak labelling for training/validation frames. Ground-truth annotations for the evaluation set were manually corrected. Wheel odometry and camera calibration were synchronized with visual data during processing.

3. Instrument- and/or software-specific information needed to interpret the data: The dataset was processed using Python-based workflows, including PyTorch for segmentation model

4. People involved in sample collection, processing, analysis and/or submission: Esra Guclu, Michael Halstead, Claus Smitt, Patrick Zimmer, Chris McCool

5. Describe any quality-assurance procedures performed on the data: Annotations were generated via the NeRF-based panoptic segmentation approach, PAg-NeRF (doi:10.1109/LRA.2023.3338515).The evaluation set annotations were manually checked and verified by human annotators.