Datasets For Scene Analysis and Reconstruction
Object Detection and Tracking Datasets
| Dataset | Year | Link | Dataset Size | Type of Access |
|---|---|---|---|---|
| MIT pedestrian dataset | 2000 | MIT pedestrian | 22MB | Public |
| Caltech Pedestrian Dataset | 2009 | Caltech | 11,9 GB | Public |
| INRIA Pedestrian Dataset | 2005 | INRIA | 1,1 GB | Public |
| PASCAL Visual Object Classes 2007 | 2007 | Pascal VOC 2007 | 880MB | Public |
| PASCAL Visual Object Classes 2012 | 2012 | Pascal VOC 2012 | 2GB Train/Validation Data | Register Required for Test Data |
| ImageNet Large Scale Visual Recognition Challenge (ILSVRC) | 2015 | ILSVRC | 167.62 GB | Available on Kaggle. Additional data requires registration. |
| Microsoft COCO (MS-COCO) | 2014 | MS-COCO | 62GB all labeled images | Public |
| Open Images | 2017 | Open Images | 561GB | Public |
Visual Relationships and Scene Graph Generation
| Dataset | Year | Link | Dataset Size | Type of Access |
|---|---|---|---|---|
| Real-World Scene Graphs Dataset | 2015 | Real-World SG | 2GB | Public |
| Visual Relationship Dataset (VRD) | 2016 | VRD | 1.86GB | Public |
| Visual Genome Dataset (VGD) | 2017 | Visual Genome | 15.7GB | Public |
| HCVRD Dataset | 2018 | HCVRD | 15.7GB | Public |
| ImageNet-VidVRD | 2017 | ImageNet-VidVRD | 4.2GB | Public |
| VidOR | 2019 | VidOR | 27.4GB | Public |
| ActionGenome | 2020 | ActionGenome | 13GB | Public |
Action Recognition
| Dataset | Year | Link | Dataset Size | Type of Access |
|---|---|---|---|---|
| HMDB51 | 2011 | HMDB51 | 2GB | Public |
| Sports1M | 2014 | Sports1M | 1.13m youtube videos | Public |
| YouTube8M | 2016 | YouTube8M | 31GB for video features, 1.53TB for frame features | Public |
| Charades | 2016 | Charades | 55 GB | Private? |
| Kinetics | 2017 | Kinetics | 650k youtube videos | Public |
| AVA Actions Datase | 2018 | AVA | 430 15-minute youtube videos | Public |
| Moments in Time | 2019 | MiT | 1m 3s videos | Request Required |
| HACS | 2019 | HACS | 50k youtube videos | Public |
| Holistic Video Understanding | 2020 | HVU | 577k youtube videos | Public for train set, Request for test and missing videos |
| BABEL | 2021 | BABEL | 43 hours of MOCAP sequences from the AMASS dataset | Registration Required |
Human Attributes Recognition
| Dataset | Year | Link | Dataset Size | Type of Access |
|---|---|---|---|---|
| PETA | 2014 | PETA | 224MB | Public |
| Parse 27K | 2015 | Parse 27K | 3.7GB | Public |
| RAP-2.0 | 2018 | RAP 2.0 | 84928 images of pedestrians | Registration Required |
| Human Attributes (HAT) | 2011 | HAT | 9344 images of pedestrians | Contact Required |
| Berkeley-Attributes of People (BAP) | 2011 | BAP | 671MB | Public |
| PA-100K | 2017 | PA 100k | 430MB | Public |
Pose Estimation
| Dataset | Year | Link | Dataset Size | Type of Access |
|---|---|---|---|---|
| Leeds Sports Pose (LSP) | 2010 | LSP | 163MB | Public |
| MPII Dataset | 2014 | MPII | 12.9GB | Public |
| CrowdPose | 2019 | CrowdPose | 2.2GB | Public |
| Joint-annotated HMDB (J-HMDB) | 2013 | J-HMDB | 928 clips with 21 action categories | Registration Required |
| PoseTrack | 2018 | PoseTrack | offline | |
| Human-in-Events (HiEve) | 2020 | HiEVE | 1M+ poses and 56k+ action labels | Public |
| Human3.6M | 2013 | Human3.6M | 3.6 million human poses and corresponding images | Account creation required |
| MoVi | 2020 | MoVi | 1056 files with MoCap and video data | Access request required |
| AMASS | 2019 | AMASS | 40 hours of motion data | Registration required |
| 3DPW | 2018 | 3DPW | 60 video sequences with 2D and 3D poses | Public |
| CMU Panoptic | 2015 | CMU Panoptic | Offline | |
| Joint Track Auto (JTA) | 2018 | JTA | 12.1GB | Public |
Gait Estimation
| Dataset | Year | Link | Dataset Size | Type of Access |
|---|---|---|---|---|
| CASIA | 2006 | CASIA | 10GB | Registration Required |
| TUM GAID | 2012 | TUM GAID | 51.5GB | Public |
| OU-ISIR | 2012 | OU-ISIR | 1.7GB | Public |
| OU-MVLP | 2018 | OU-MVLP | 23GB | Public |
| OUMVLP-Pose | 2020 | OUMVLP-Pose | 12GB | Public |
| GREW | 2021 | GREW | 233 857 sequences | Registration Required |
Scene Reconstruction
| Dataset | Year | Link | Dataset Size | Type of Access |
|---|---|---|---|---|
| Sun RGB-D | 2015 | SUN RGB-D | 6.4GB | Public |
| Scan Net | 2017 | Scan-net | 1500 scans | Access request required |
| UP3D | 2017 | UP3D | 57.7GB | Public |
| SURREAL | 2017 | Surreal | 86GB | Accept the license terms |
| Scan2CAD | 2019 | Scan2CAD | 1506 scans | Registration Required |
| Kitti-360 | 2021 | Kitti-360 | 320k images and 100k laser scans in a driving distance of 73.7km | Registration Required |
| RELLIS-3D | 2020 | RELLIS-3D | 13,556 LiDAR scans and 6,235 images | Public |
| Hypersim | 2021 | Hypersim | 1.9TB | Public, but the ground truth triangle meshes for each scene needs to be purchased |