Awesome-Thermal-Deep-3D-Learning

A curated list of resources for deep 3D vision from thermal image

Table of contents

• 1. Dataset for Thermal 3D Vision
• 2. Deep 3D Vision from Thermal Image
  • 2.1 Supervised 3D Vision
  • 2.2 Self-supervised 3D Vision
• 3. Deep 3D Vision from Thermal and Additional Sensor Fusion
• 4. Non-Deep 3D Vision from Thermal and Other Sensor

1. Dataset for Thermal 3D Vision

Dataset List

Publication	Title	Sensor	Platform	Scene Types	Dataset Link	Highlight
CVPR 17	CATS: A Color And Thermal Stereo Benchmark	Stereo RGB, Stereo Thermal, Lidar	Handheld	In+Outdoor	Here
ITS 19	KAIST Multi-Spectral DayNight Data Set for Autonomous and Assisted Driving	Co-aligned RGB/Thermal, Stereo RGB, Lidar, GPS/IMU	Vehicle	Outdoor	Here	Large-scale outdoor dataset
ICRA 21	A Multi-spectral Dataset for Evaluating Motion Estimation Systems	RGB, Thermal, Kinect, IMU	Handheld	In+Outdoor	Here	Large-scale indoor datase
RA-L 22	ViViD++: Vision for Visibility Dataset	RGB-D, Thermal, Lidar, GPS/IMU, Event	Handheld, Vehicle	In+Outdoor	Here	Large-scale outdoor dataset
Arxiv 22	OdomBeyondVision; An Indoor Multi-modal Multi-platform Odometry Dataset Beyond the Visible Spectrum	RGB, Thermal, Lidar, IMU, Radar	Handheld, UGV, UAV	Indoor	Here	Large-scale indoor dataset

---

2. Deep 3D Vision from Thermal Image

2.1 Supervised 3D Vision from Thermal Image

Paper List

Publication	Title	Task	Dataset	Scene Types	Code	Highlight

---

2.2 Self-Supervised 3D Vision from Thermal Image

Paper List

Publication	Title	Task	Dataset	Scene Types	Code	Highlight
AAAI 18	Multispectral Transfer Network: Unsupervised Depth Estimation for All-Day Vision	MonoDepth	KAIST Multi-Spectral Dataset	Outdoor	Here	Spatial RGB, thermal image reconstruction
WACV 21	An Alternative of LiDAR in Nighttime: Unsupervised Depth Estimation Based on Single Thermal Image	MonoDepth	Own dataset	In+Outdoor	here	Spatial thermal image reconstruction
RA-L 21	SuperThermal: Matching Thermal as Visible Through Thermal Feature Exploration	Feature Matching	CSS, KAIST pedestrian dataset	Outdoor	Here	TBA
RA-L 22& ICRA 22	Self-Supervised Depth and Ego-Motion Estimation for Monocular Thermal Video Using Multi-Spectral Consistency Loss	MonoDepth, Pose	ViViD dataset	In+Outdoor	Here	Temporal RGB, thermal image reconstruction loss
RA-L 22& IROS 22	Maximizing Self-supervision from Thermal Image for Effective Self-supervised Learning of Depth and Ego-motion	MonoDepth, Pose	ViViD dataset	In+Outdoor	Here	Temporal thermal image reconstruction only

---

3. Deep 3D Vision from Thermal and Additional Sensor Fusion

Paper List

Publication	Title	Task	Sensor	Dataset	Scene Types	Code	Highlight
RA-L 20	DeepTIO: A Deep Thermal-Inertial Odometry With Visual Hallucination	Odometry	Thermal, IMU	Own dataset	Here	TBA	TBA
IROS 20	Tp-tio: A robust thermal-inertial odometry with deep thermalpoint	Odometry	Thermal, IMU	TBA	TBA	TBA	TBA
TRO 21	Graph-Based Thermal–Inertial SLAM With Probabilistic Neural Networks	Odometry	Thermal, IMU	TBA	TBA	TBA	TBA

---

4. Non-Deep 3D Vision from Thermal and Other Sensor

Paper List

Publication	Title	Task	Sensor	Dataset	Scene Types	Code	Highlight
IROS 21	Radar Visual Inertial Odometry and Radar Thermal Inertial Odometry: Robust Navigation even in Challenging Visual Conditions	Odometry	Thermal, IMU, Radar	TBA	TBA	TBA	TBA
CVPR 21	Shape from Thermal Radiation:Passive Ranging Using Multi-spectral LWIR Measurements	MonoDepth	Thermal	Own dataset	In+Outdoor	Here	TBA
RA-L 22	Thermal-Inertial SLAM for the Environments With Challenging Illumination	Odometry	Thermal, IMU	TBA	TBA	TBA	TBA

---