HPGO

Hybrid Parallelism Global Orchestration

(Project Name TBD)

Get Started

Install

From Rust Cargo, as a Rust crate

cargo install HPGO

From Python PyPI, as a Python3 package

pip3 install HPGO

Build from source

rustup default nightly
cargo build --release
# library produced under targets/release/libHPGO.so

Use

# Import HPGO Python API from HPGO.so
import HPGO
# Construct the Conductor object
c = HPGO.conductor_from_torch_graph_and_seps("./profiles/xlnet/graph.txt", 64, 512, [8, 16])
res = c.py_orchestrate()
print(res)

License

This project is open sourced under the terms of BSD-3-Clause, details of which can be found in the LICENSE file

If you apply this library to any project and research, please cite this code:

@misc{HPGO,
  author       = {Yi Rong},
  title        = {HPGO, [{Hybrid Parallelism} Global Orchestration]},
  howpublished = {\url{https://github.com/LER0ever/HPGO}},
  year         = {2020}
}

The project contains source code from PipeDream, a Microsoft Research project licensed under the MIT License. Code taken from Pipedream includes a Rust file src/input/torch_graph_py.rs, several profiling data files under the profiles directory, and the contrib directory.

This project also contains profiling data provided by the AlibabaPAI/DAPPLE project. The development of the DAPPLE planner continues here.