4 releases (2 breaking)
new 0.3.1 | Oct 29, 2024 |
---|---|
0.3.0 | Oct 29, 2024 |
0.2.0 | Oct 26, 2024 |
0.1.0 | Oct 24, 2024 |
#26 in Machine learning
221 downloads per month
79KB
2K
SLoC
tensor-man
is a utility to inspect, validate, sign and verify machine learning model files.
Supported Formats
[!IMPORTANT] PyTorch models are loaded and inspected in a networkless Docker container in order to prevent unintended code execution on the host machine.
Install with Cargo
cargo install tensor-man
Build Docker image
To build the Docker image for the tool, run:
docker build . -t tman
Build from source
Alternatively you can build the project from source, in which case you'll need to have Rust and Cargo installed on your system.
Once you have those set up, clone the repository and build the project:
cargo build --release
The compiled binary will be available in the target/release
directory. You can run it directly or add it to your system's PATH:
# Run directly
./target/release/tman
# Or, copy to a directory in your PATH (e.g., /usr/local/bin)
sudo cp target/release/tman /usr/local/bin/
Usage
Inspect
Inspect a file and print a brief summary:
tman inspect /path/to/whatever/llama-3.1-8b-instruct.safetensors
Print detailed information about each tensor:
tman inspect /path/to/whatever/llama-3.1-8b-instruct.safetensors --detail full
Filter by tensor name:
tman inspect /path/to/whatever/llama-3.1-8b-instruct.onnx -D full --filter "q_proj"
Save the output as JSON:
tman inspect /path/to/whatever/llama-3.1-8b-instruct.gguf -D full --to-json output.json
Sign and Verify
The tool allows you to generate an Ed25519 key pair to sign your models:
tman create-key --private-key private.key --public-key public.key
Then you can use the private key to sign a model (this will automatically include and sign external data files if referenced by the format):
# this will generate the tinyyolov2-8.signature file
tman sign /path/to/whatever/tinyyolov2-8.onnx -K /path/to/private.key
# you can provide a safetensors index file and all files referenced by it will be signed as well
tman sign /path/to/whatever/Meta-Llama-3-8B/model.safetensors.index.json -K /path/to/private.key
And the public one to verify the signature:
# will verify the signature in tinyyolov2-8.signature
tman verify /path/to/whatever/tinyyolov2-8.onnx -K /path/to/public.key
# will verify with an alternative signature file
tman verify /path/to/whatever/tinyyolov2-8.onnx -K /path/to/public.key --signature /path/to/your.signature
Inference Graph
Generate a .dot file for the execution graph of an ONNX model:
tman graph /path/to/whatever/tinyyolov2-8.onnx --output tinyyolov2-8.dot
More
For the full list of commands and options, run:
tman --help
# get command specific help
tman inspect --help
License
This tool is released under the GPL 3 license. To see the licenses of the project dependencies, install cargo license with cargo install cargo-license
and then run cargo license
.
Dependencies
~14–26MB
~485K SLoC