HPGO

Hybrid Parallelism Global Orchestration

This project is in Alpha Stage: The functionality provided should be considered research grade and the name "HPGO" is expected to change upon stable release

Announcement

Repository temporarily moved to Lab: https://lab.rongyi.io/research/ml/HPGO until further notice. Previous archive can be downloaded here and here.

Get Started

Install

The binary package of this library is provided for every release.

From Rust Cargo, as a Rust crate

(deprecated)

cargo install HPGO

From Python PyPI, as a Python3 package

(release only)

pip3 install HPGO

Use

# Import HPGO Python API from HPGO.so
import HPGO
# Construct the Conductor object
c = HPGO.conductor_from_torch_graph_and_seps("./profiles/xlnet/graph.txt", 64, 512, [8, 16])
res = c.py_orchestrate()
print(res)

Contribution

License

This project is open sourced under the terms of BSD-3-Clause, details of which can be found in the LICENSE file

If you apply this library to any project and research, please cite this code:

@misc{HPGO,
  author       = {Yi Rong},
  title        = {HPGO, [{Hybrid Parallelism} Global Orchestration]},
  howpublished = {\url{https://github.com/LER0ever/HPGO}},
  year         = {2020}
}

The project contains source code from PipeDream, a Microsoft Research project licensed under the MIT License. Code taken from Pipedream includes a Rust file src/input/torch_graph_py.rs, several profiling data files under the profiles directory, and the contrib directory.

This project also contains profiling data provided by the AlibabaPAI/DAPPLE project. The development of the DAPPLE planner continues here.

LER0ever/HPGO