A Comprehensive Framework for Visual SLAM Baselines and Datasets

Alejandro Fontan · Tobias Fischer · Nicolas Marticorena

Somayeh Hussaini · Ted Vanderfeen · Javier Civera · Michael Milford

Introduction

VSLAM-LAB is designed to simplify the development, evaluation, and application of Visual SLAM (VSLAM) systems.
This framework enables users to compile and configure VSLAM systems, download and process datasets, and design, run, and
evaluate experiments — all from a single command line!

Why Use VSLAM-LAB?

• Unified Framework: Streamlines the management of VSLAM systems and datasets.
• Ease of Use: Run experiments with minimal configuration and single command executions.
• Broad Compatibility: Supports a wide range of VSLAM systems and datasets.
• Reproducible Results: Standardized methods for evaluating and analyzing results.

Getting Started

To ensure all dependencies are installed in a reproducible manner, we use the package management tool pixi. If you haven't installed pixi yet, please run the following command in your terminal:

curl -fsSL https://pixi.sh/install.sh | bash 

After installation, restart your terminal or source your shell for the changes to take effect. For more details, refer to the pixi documentation.

If you already have pixi remember to update: pixi self-update

Clone the repository and navigate to the project directory:

git clone https://github.com/alejandrofontan/VSLAM-LAB.git && cd VSLAM-LAB

Quick Demo

You can now execute any baseline on any sequence from any dataset within VSLAM-LAB using the following command:

pixi run demo <baseline> <dataset> <sequence>

For a full list of available systems and datasets, see the VSLAM-LAB Supported Baselines and Datasets.
Example commands:

pixi run demo mast3rslam eth table_3
pixi run demo droidslam euroc MH_01_easy
pixi run demo orbslam2 rgbdtum rgbd_dataset_freiburg1_xyz

To change the paths where VSLAM-LAB-Benchmark or/and VSLAM-LAB-Evaluation data are stored (for example, to /media/${USER}/data), use the following commands:

pixi run set-benchmark-path /media/${USER}/data
pixi run set-evaluation-path /media/${USER}/data

VSLAM-LAB Info Functions

pixi run baseline-info <baseline>
pixi run print-baselines
pixi run print-datasets

Configure your own experiments

With VSLAM-LAB, you can easily design and configure experiments using a YAML file and run them with a single command.
To run the experiment demo, execute the following command:

pixi run vslamlab configs/exp_vslamlab.yaml (--overwrite)

Experiments in VSLAM-LAB are sequences of entries in a YAML file (see example ~/VSLAM-LAB/configs/exp_vslamlab.yaml):

exp_vslamlab:
  Config: config_vslamlab.yaml  # YAML file containing the sequences to be run 
  NumRuns: 1                    # Maximum number of executions per sequence
  Parameters: {verbose: 1}      # Vector with parameters that will be input to the baseline executable 
  Module: droidslam             # droidslam/monogs/orbslam2/mast3rslam/dpvo/...                    

Config files are YAML files containing the list of sequences to be executed in the experiment (see example ~/VSLAM-LAB/configs/config_vslamlab.yaml):

rgbdtum:
  - 'rgbd_dataset_freiburg1_xyz'
hamlyn:
  - 'rectified01'
7scenes:
  - 'chess_seq-01'
eth:
  - 'table_3'
euroc:
  - 'MH_01_easy'
monotum:
  - 'sequence_01'

For a full list of available VSLAM systems and datasets, refer to the section VSLAM-LAB Supported Baselines and Datasets.

VSLAM-LAB Pipeline Functions

Instead of running the full VSLAM-LAB pipeline, you can interact with datasets and baselines using the commands below:

pixi run validate-experiment-yaml <exp_yaml>             # Example: pixi run validate-experiment-yaml configs/exp_vslamlab.yaml
pixi run overwrite-exp <exp_yaml>                        # Example: pixi run overwrite-exp configs/exp_vslamlab.yaml
pixi run update-experiment-csv-logs <exp_yaml>           # Example: pixi run update-experiment-csv-logs configs/exp_vslamlab.yaml

pixi run check-experiment-resources <exp_yaml>           # Example: pixi run check-experiment-resources configs/exp_vslamlab.yaml
pixi run get-experiment-resources <exp_yaml>             # Example: pixi run get-experiment-resources configs/exp_vslamlab.yaml

pixi run check-experiment-state <exp_yaml>               # Example: pixi run check-experiment-state configs/exp_vslamlab.yaml

pixi run install-baseline <baseline>                     # Example: pixi run install-baseline droidslam
pixi run install-baselines <baseline1> <baseline2> ...   # Example: pixi run install-baselines droidslam orbslam2

pixi run download-sequence <dataset> <sequence>          # Example: pixi run download-sequence eth table_3
pixi run download-sequences <dataset1> <sequence1> <dataset2> <sequence2> ... \
                                                         # Example: pixi run download-sequences eth table_3 rgbdtum rgbd_dataset_freiburg1_xyz
pixi run download-dataset <dataset>                      # Example: pixi run download-dataset eth
pixi run download-datasets <dataset1> <dataset2>         # Example: pixi run download-datasets eth rgbdtum

pixi run run-exp <exp_yaml>                              # Example: pixi run run-exp configs/exp_vslamlab.yaml
pixi run evaluate-exp <exp_yaml>                         # Example: pixi run evaluate-exp configs/exp_vslamlab.yaml
pixi run compare-exp <exp_yaml>                          # Example: pixi run compare-exp configs/exp_vslamlab.yaml

Add a new dataset

Datasets in VSLAM-LAB are stored in a folder named VSLAM-LAB-Benchmark, which is created by default in the same parent directory as VSLAM-LAB.

1. To add a new dataset, structure your dataset as follows:

~/VSLAM-LAB-Benchmark
└── YOUR_DATASET
    └── sequence_01
        ├── rgb_0
            └── img_01
            └── img_02
            └── ...
        ├── calibration.yaml
        ├── rgb.csv
        └── groundtruth.csv
    └── sequence_02
        ├── ...
    └── ...   

2. Derive a new class dataset_{your_dataset}.py for your dataset from ~/VSLAM-LAB/Datasets/Dataset_vslamlab.py, and create a corresponding YAML configuration file named dataset_{your_dataset}.yaml.

3. Include the call for your dataset in function def get_dataset(...) in ~/VSLAM-LAB/Datasets/get_dataset.py

 from Datasets.dataset_{your_dataset} import {YOUR_DATASET}_dataset
    ...
 def get_dataset(dataset_name, benchmark_path)
    ...
    switcher = {
        "rgbdtum": lambda: RGBDTUM_dataset(benchmark_path),
        ...
        "dataset_{your_dataset}": lambda: {YOUR_DATASET}_dataset(benchmark_path),
    }

License

VSLAM-LAB is released under a LICENSE.txt. For a list of code dependencies which are not property of the authors of VSLAM-LAB, please check docs/Dependencies.md.

Citation

If you're using VSLAM-LAB in your research, please cite. If you're specifically using VSLAM systems or datasets that have been included, please cite those as well. We provide a spreadsheet with citation for each dataset and VSLAM system for your convenience.

@article{fontan2025vslam,
  title={VSLAM-LAB: A Comprehensive Framework for Visual SLAM Methods and Datasets},
  author={Fontan, Alejandro and Fischer, Tobias and Civera, Javier and Milford, Michael},
  journal={arXiv preprint arXiv:2504.04457},
  year={2025}
}

VSLAM-LAB Supported Baselines and Datasets

We provide a spreadsheet with more detailed information for each baseline and dataset.