Lib.rs

› Users #tokenize #word-piece

n1t0

Anthony MOI

Joined crates-io 5 years ago. Joined GitHub 13 years ago.

Member of GitHub orgs

totemstech, huggingface

Shares crates with

ArthurZucker, Narsil

n1t0 created 1 crate

unicode-normalization-alignments

298K Sole owner since Dec 2019. Last updated 5 years ago.

Co-owns 1 crate

tokenizers

232K Member since Dec 2019 (5 years)