| Name | alphacodings JSON |
| Version |
0.2.0
JSON |
| download |
| home_page | None |
| Summary | base26 ([A-Z]) and base52 ([A-Za-z]) encodings |
| upload_time | 2024-12-09 03:04:43 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >3.9 |
| license | MIT License Copyright (c) 2024 Federico Minutoli Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
gmp
nlp
chunking
encodings
tokenization
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
<div align="center">
# alphacodings
<img src="resources/alphacodings.png" width="256" height="256">
base26 ([A-Z]) and base52 ([A-Za-z]) encodings
</div>
## 🌟 overview
transform any string to alphabetic-only with base26 ([A-Z]) and base52 ([A-Za-z]) lossless encodings; useful for transmitting textual data over restrictive channels or for training AI models and tokenizers on simpler vocabularies.
**alphacodings** is a fast and lightweight library using [GMP arithmetic](https://gmplib.org).
## ⚙️ installation
```python
python -m pip install alphacodings
```
## 🚀 usage
```python
from alphacodings import base26_encode, base26_decode, base52_encode, base52_decode
string = """\
<!DOCTYPE html>
<html>
<head>
<title>sample page</title>
</head>
<body>
<h1>welcome!</h1>
<p>you are reading a sample HTML string.</p>
</body>
</html>
"""
if __name__ == "__main__":
encoding_base26 = base26_encode(string)
print(encoding_base26)
# >>> ["YBPNLKVNQWZQCMDHMLNDTVQCCRKQLNCFGMQPNGQCIXHUUPHFUNKUFEPDLKIGARFOKTDEZKQHXGCPYHDZKKVIUDNFOAYYAUOQFBJFFGSTKAXNWGDPVUJNBARPNXBASHZBXIBSSEFTAIQRPEADSOVVNXUMQXVDWTAIVCIVWQZAHAGYAVZYKGMETJOOUQNOEXMSOOGSKVMFBYZIBZDAITICYVXMJTTCCHPMSCABLYUMFDUNLVSLNKHSBPKCGASXJSFYDHZFAOEQTUACEBIFKQGYC"]
encoding_base52 = base52_encode(string)
print(encoding_base52)
# >>> ["EgcgYRPxckylMQWRLDADNZxPJiJcHaVwYHLnicahBgaotGGANZuvsvcpSSOJFLXvKPjRlNQCJqqdviiIdtnwJyDOnWojsrpkWSTZFHbMIREvREjpsODtSxoLlLjQZOoehsGFzawGQecyuomgpZQNyFnZQLWPiDhzClwxBFCCwdqduGJoshrwFdwHWMtJpSTmjxzaYmNvzOIOwLkJvyQHCaFtrODPhbhBpPBmC"]
assert base26_decode(encoding_base26) == string
assert base52_decode(encoding_base52) == string
```
## 🧠 motivation
The library is inspired by [R. Heaton](https://github.com/robert)'s base26 implementation and his story of manipulating data transmission in restrictive network channels on long-distance flights using alphabetic-only encodings and tokenization.
have a look at the original [repository](https://github.com/robert/pyskywifi) and story [blog post](https://robertheaton.com/pyskywifi) and show him some love.
## 📊 benchmarking
our implementation is orders of magnitude more efficient on 100k+ strings:
<div align="center">
<img src="resources/benchmark.png" alt="benchmarking">
*Figure 1: runtime and memory usage performance against Heaton's original implementation with and without automatic chunking and SIMD on variable-length strings with a strict 60-second timeout; average over 5 trials.*
</div>
## 🤝 contributing
contributions to **alphacodings** are welcome!
feel free to submit pull requests or open issues on our repository.
## 📄 license
see the [LICENSE](LICENSE) file for more details.
Raw data
{
"_id": null,
"home_page": null,
"name": "alphacodings",
"maintainer": null,
"docs_url": null,
"requires_python": ">3.9",
"maintainer_email": null,
"keywords": "GMP, NLP, chunking, encodings, tokenization",
"author": null,
"author_email": "Federico Minutoli <fede97.minutoli@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/c4/61/d7ded79cb9515c70d056bb2b2d5fc654a9c3ffad3f57df59d5adfa2f9eaf/alphacodings-0.2.0.tar.gz",
"platform": null,
"description": "<div align=\"center\">\n\n# alphacodings\n\n<img src=\"resources/alphacodings.png\" width=\"256\" height=\"256\">\n\nbase26 ([A-Z]) and base52 ([A-Za-z]) encodings\n</div>\n\n## \ud83c\udf1f overview\n\ntransform any string to alphabetic-only with base26 ([A-Z]) and base52 ([A-Za-z]) lossless encodings; useful for transmitting textual data over restrictive channels or for training AI models and tokenizers on simpler vocabularies.\n\n**alphacodings** is a fast and lightweight library using [GMP arithmetic](https://gmplib.org).\n\n## \u2699\ufe0f installation\n\n```python\npython -m pip install alphacodings\n```\n\n## \ud83d\ude80 usage\n\n```python\nfrom alphacodings import base26_encode, base26_decode, base52_encode, base52_decode\n\n\nstring = \"\"\"\\\n<!DOCTYPE html>\n<html>\n<head>\n <title>sample page</title>\n</head>\n<body>\n <h1>welcome!</h1>\n <p>you are reading a sample HTML string.</p>\n</body>\n</html>\n\"\"\"\n\n\nif __name__ == \"__main__\":\n encoding_base26 = base26_encode(string)\n print(encoding_base26)\n # >>> [\"YBPNLKVNQWZQCMDHMLNDTVQCCRKQLNCFGMQPNGQCIXHUUPHFUNKUFEPDLKIGARFOKTDEZKQHXGCPYHDZKKVIUDNFOAYYAUOQFBJFFGSTKAXNWGDPVUJNBARPNXBASHZBXIBSSEFTAIQRPEADSOVVNXUMQXVDWTAIVCIVWQZAHAGYAVZYKGMETJOOUQNOEXMSOOGSKVMFBYZIBZDAITICYVXMJTTCCHPMSCABLYUMFDUNLVSLNKHSBPKCGASXJSFYDHZFAOEQTUACEBIFKQGYC\"]\n\n encoding_base52 = base52_encode(string)\n print(encoding_base52)\n # >>> [\"EgcgYRPxckylMQWRLDADNZxPJiJcHaVwYHLnicahBgaotGGANZuvsvcpSSOJFLXvKPjRlNQCJqqdviiIdtnwJyDOnWojsrpkWSTZFHbMIREvREjpsODtSxoLlLjQZOoehsGFzawGQecyuomgpZQNyFnZQLWPiDhzClwxBFCCwdqduGJoshrwFdwHWMtJpSTmjxzaYmNvzOIOwLkJvyQHCaFtrODPhbhBpPBmC\"]\n\n assert base26_decode(encoding_base26) == string\n assert base52_decode(encoding_base52) == string\n```\n\n## \ud83e\udde0 motivation\n\nThe library is inspired by [R. Heaton](https://github.com/robert)'s base26 implementation and his story of manipulating data transmission in restrictive network channels on long-distance flights using alphabetic-only encodings and tokenization.\n\nhave a look at the original [repository](https://github.com/robert/pyskywifi) and story [blog post](https://robertheaton.com/pyskywifi) and show him some love.\n\n## \ud83d\udcca benchmarking\n\nour implementation is orders of magnitude more efficient on 100k+ strings:\n\n<div align=\"center\">\n<img src=\"resources/benchmark.png\" alt=\"benchmarking\">\n\n*Figure 1: runtime and memory usage performance against Heaton's original implementation with and without automatic chunking and SIMD on variable-length strings with a strict 60-second timeout; average over 5 trials.*\n</div>\n\n## \ud83e\udd1d contributing\n\ncontributions to **alphacodings** are welcome!\n\nfeel free to submit pull requests or open issues on our repository.\n\n## \ud83d\udcc4 license\n\nsee the [LICENSE](LICENSE) file for more details.\n",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2024 Federico Minutoli Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.",
"summary": "base26 ([A-Z]) and base52 ([A-Za-z]) encodings",
"version": "0.2.0",
"project_urls": null,
"split_keywords": [
"gmp",
" nlp",
" chunking",
" encodings",
" tokenization"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "c0b191861670659888fc655cc96cfae324a9413eb4fe0610a5f709090e55c51f",
"md5": "f7d53af9a7b7fd97bbe000a1a088a248",
"sha256": "efa215f5dca2d5c3b2e67e6c1bc8daa4404c07f11c41d2c3731a28b6f3430e22"
},
"downloads": -1,
"filename": "alphacodings-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f7d53af9a7b7fd97bbe000a1a088a248",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">3.9",
"size": 6579,
"upload_time": "2024-12-09T03:04:42",
"upload_time_iso_8601": "2024-12-09T03:04:42.849732Z",
"url": "https://files.pythonhosted.org/packages/c0/b1/91861670659888fc655cc96cfae324a9413eb4fe0610a5f709090e55c51f/alphacodings-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "c461d7ded79cb9515c70d056bb2b2d5fc654a9c3ffad3f57df59d5adfa2f9eaf",
"md5": "609af95b0d2a065b6b550514acfca817",
"sha256": "00af7678a8d6699614b75d4c59353c7ce82769ff565e219b95b63f62595a9c9d"
},
"downloads": -1,
"filename": "alphacodings-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "609af95b0d2a065b6b550514acfca817",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">3.9",
"size": 1759366,
"upload_time": "2024-12-09T03:04:43",
"upload_time_iso_8601": "2024-12-09T03:04:43.761090Z",
"url": "https://files.pythonhosted.org/packages/c4/61/d7ded79cb9515c70d056bb2b2d5fc654a9c3ffad3f57df59d5adfa2f9eaf/alphacodings-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-09 03:04:43",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "alphacodings"
}
