Name | Version | Summary | date |
---|---|---|---|
tokker | 0.1.1 | A fast, simple CLI tool for tokenizing text using OpenAI's tiktoken library | 2025-07-28 20:19:10 |
rs-bpe | 0.1.0 | A ridiculously fast Python BPE (Byte Pair Encoder) implementation written in Rust | 2025-03-19 05:58:24 |
PyTokenCounter | 1.6.4 | A Python library for tokenizing text and counting tokens using various encoding schemes. | 2025-02-03 23:06:07 |
autotiktokenizer | 0.2.1 | 🧰 The AutoTokenizer that TikToken always needed -- Load any tokenizer with TikToken now! ✨ | 2024-11-11 21:16:02 |
hour | day | week | total |
---|---|---|---|
110 | 2171 | 10290 | 303881 |