SceneFlowGAN

This repository contains Keras implementation of our paper

A Conditional Adversarial Network for Scene Flow Estimation (RO-MAN 2019)

Ravi Kumar Thakur and Snehasis Mukherjee

Abstract

The problem of Scene flow estimation in depth videos has been attracting attention of researchers of robot vision, due to its potential application in various areas of robotics. The conventional scene flow methods are difficult to use in reallife applications due to their long computational overhead. We propose a conditional adversarial network SceneFlowGAN for scene flow estimation. The proposed SceneFlowGAN uses loss function at two ends: both generator and descriptor ends. The proposed network is the first attempt to estimate scene flow using generative adversarial networks, and is able to estimate both the optical flow and disparity from the input stereo images simultaneously. The proposed method is experimented on a large RGB-D benchmark sceneflow dataset

Requirements

The code has been tested on Ubuntu 16.04 with CUDA 9.0. Python2 and Keras are required. Relevant python libraries can installed (inside virtual environment) using:

pip3 install -r requirements.txt

Alternatively, a docker image can be created for running SceneFlowGAN inside a container. All the dependencies are included in the Dockerfile.

Acknowledgement

We would like to thank Soumen Ghosh and Shiv Ram Dubey for providing feedback and insightful discussion. The work was supported by Department of Science and Technology, Government of India under Project ECR/2016/00652.

Reference

Please use the following for citation purpose

@inproceedings{thakur2019conditional,
  title={A conditional adversarial network for scene flow estimation},
  author={Thakur, Ravi Kumar and Mukherjee, Snehasis},
  booktitle={2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)},
  pages={1--6},
  year={2019},
  organization={IEEE}
}

Note

In case of difficulty in running the code, please post your question by filing an issue. To suggest any improvement to make the code more readable or optimized, open a pull request.