StretchBEV: Stretching Future Instance Prediction Spatially and Temporally (ECCV 2022)

StretchBEV: Stretching Future Instance Prediction Spatially and Temporally,
Adil Kaan Akan,
Fatma Guney,
European Conference on Computer Vision (ECCV), 2022

Features

StretchBEV is a future instance prediction network in Bird's-eye view representation. It earns temporal dynamics in a latent space through stochastic residual updates at each time step. By sampling from a learned distribution at each time step, we obtain more diverse future predictions that are also more accurate compared to previous work, especially stretching both spatially further regions in the scene and temporally over longer time horizons

Requirements

All models were trained with Python 3.7.10 and PyTorch 1.7.0

A list of required Python packages is available in the environment.yml file.

Datasets

For preparations of datasets, we followed FIERY. Please follow this link below if you want to construct the datasets.

Training

To train the model on NuScenes:

• First, you need to download static_lift_splat_setting.ckpt and copy it to this directory.
• Run python train.py --config fiery/configs/baseline.yml DATASET.DATAROOT ${NUSCENES_DATAROOT}.

This will train the model on 4 GPUs, each with a batch of size 2. To train on single GPU add the flag GPUS 1, and to change the batch size use the flag BATCHSIZE ${DESIRED_BATCHSIZE}.

Evaluation

To evaluate a trained model on NuScenes:

• Download pre-trained weights.
• Run python evaluate.py --checkpoint ${CHECKPOINT_PATH} --dataroot ${NUSCENES_DATAROOT}.

Pretrained weights

You can download the pretrained weights from the releases of this repository or the links below.

Normal setting weight

Fishing setting weight

How to Cite

Please cite the paper if you benefit from our paper or the repository:

@InProceedings{Akan2022ECCV,
            author    = {Akan, Adil Kaan and G\"uney, Fatma},
            title     = {StretchBEV: Stretching Future Instance Prediction Spatially and Temporally},
            journal = {European Conference on Computer Vision (ECCV)},
            year      = {2022},
            }

Acknowledgments

We would like to thank FIERY and SRVP authors for making their repositories public. This repository contains several code segments from FIERY's repository and SRVP's repository. We appreciate the efforts by Berkay Ugur Senocak for cleaning the code before release.