PyDigger - unearthing stuff about Python


NameVersionSummarydate
PyTokenCounter 1.8.2 A Python library for tokenizing text and counting tokens using various encoding schemes. 2025-08-22 03:20:04
tokker 0.3.9 Tokker: a fast local-first CLI tokenizer with all the best models in one place 2025-08-09 17:59:33
rs-bpe 0.1.0 A ridiculously fast Python BPE (Byte Pair Encoder) implementation written in Rust 2025-03-19 05:58:24
autotiktokenizer 0.2.1 🧰 The AutoTokenizer that TikToken always needed -- Load any tokenizer with TikToken now! ✨ 2024-11-11 21:16:02
hourdayweektotal
6413109584315456
Elapsed time: 2.58371s