Pre-training a Density-Aware Pose Transformer for Robust LiDAR-based 3D Human Pose Estimation (AAAI 2025)

Xiaoqi An¹ Lin Zhao¹ Chen Gong¹ Jun Li¹ Jian Yang¹

¹PCA Lab, Nanjing University of Science and Technology

We propose a simple yet powerful framework, which provides insights both on modeling and augmentation of point clouds.
Specifically, we first propose a concise and effective density-aware pose transformer (DAPT) to get stable keypoint representations. By using a set of joint anchors and a carefully designed exchange module, valid information is extracted from point clouds with different densities. Then 1D heatmaps are utilized to represent the precise locations of the keypoints.
Secondly, a comprehensive LiDAR human synthesis and augmentation method is proposed to pre-train the model, enabling it to acquire a better human body prior. We increase the diversity of point clouds by randomly sampling human positions and orientations and by simulating occlusions through the addition of laser-level masks.

DAPT Framework

Environment

1. Clone the project

git clone --recursive https://github.com/AnxQ/dapt

2. Create environment

Please make sure your connection to anaconda and pypi is stable.

conda env create -f environment.yml
conda activate dapt

3. Configure Huggingface Accelerate

accelerate config

For details, please refer to Accelerate documentation.
Note that differences in computing facilities can lead to some differences in training results.

4. Configure Wandb for logging (Optional)

pip install wandb
wandb login

Data

Please refer to DATA.md for data download and preparation.

Test

Pre-trained checkpoints can be downloaded from [OneDrive/GoogleDrive/BaiduNetdisk].

accelerate launch main.py configs/dapt-waymo-50b64-finetune.py --test \
    --ckpt checkpoints/dapt-waymo-50b64-finetune.safetensors

Change the config name for evaluation on different dataset .

Train

# Pre-Training
accelerate launch main.py configs/dapt-synpretrain-50b64.py
# Finetuning
accelerate launch main.py configs/dapt-waymo-50b64-finetune.py \
    --options model.pretrained=work_dir/dapt-synpretrain-50b64/checkpoints/checkpoint_4/model.safetensors

Change the config name for training on different dataset.

Acknowledgement

We thank the following open source projects for helping us implement our idea.

Citation

@article{an2025dapt,
  title={Pre-training a Density-Aware Pose Transformer for Robust LiDAR-based 3D Human Pose Estimation},
  volume={39},
  doi={10.1609/aaai.v39i2.32169},
  number={2},
  journal={Proceedings of the AAAI Conference on Artificial Intelligence},
  author={An, Xiaoqi and Zhao, Lin and Gong, Chen and Li, Jun and Yang, Jian},
  year={2025},
  pages={1755-1763}
}

AutoCONFIG/dapt