| Name | Version | Summary | date |
| ultra-tokenizer |
0.1.3 |
Advanced tokenizer with support for BPE, WordPiece, and Unigram algorithms |
2025-08-16 09:58:51 |
| nfelib |
2.2.0 |
nfelib: electronic invoicing library for Brazil |
2025-08-06 02:13:20 |
| tokenizers |
0.21.4 |
None |
2025-07-28 13:22:43 |
| smoltoken |
0.1.3 |
A light-weight & fast library for Byte Pair Encoding (BPE) tokenization. |
2025-02-06 04:43:52 |
| kitoken |
0.10.1 |
Fast and versatile tokenizer for language models, supporting BPE, Unigram and WordPiece tokenization |
2024-12-20 03:07:14 |
| bpeasy |
0.1.3 |
Fast bare-bones BPE for modern tokenizer training |
2024-08-23 10:47:52 |
| pyregtokenizer |
0.0.2 |
A BPE Tokenizer using regex |
2024-04-21 07:30:19 |