# Rune-512: Compact Binary Encoding
[](https://badge.fury.io/py/rune-512)
[](https://opensource.org/licenses/MIT)
**Rune-512** is a binary-to-text encoding scheme designed to safely and compactly embed arbitrary binary data in environments with strict character limits but also support a wide range of Unicode characters, such as social media bios (like Bluesky, Twitter).
It uses a carefully selected 512-character symbolic unicode alphabet that is not visually distracting and can represent data more densely than traditional encodings like Base64, packing 9 bits of data into a single character.
For example, here's 32 random bytes:
```
⣣⡳⣜▣╎⡇◡━┉◳⠢╖⠿⣺⢔▶⢎⡝╺⡂╍╞▨╿□⣼⣆⢼▤⡖⢀
```
Here's the string `"the fox jumped over the lazy dog"`:
```
⣟▴⣨□┩⠆⣍◞⠐⡪⣪▵▃⡖⢄⠛▻⡥⣤⢁▣⢆⢤⠛⠰╺⣲⢁┣⣶⣠
```
## Features
- **Compact:** Encodes 9 bits per character, offering significant space savings over Base64.
- **Reliable:** Uses a 17-bit checksum derived from the SHA-256 hash of the payload to detect data corruption.
- **Safe:** The alphabet consists of Unicode codepoints with wide compatibility across common platforms.
- **Easy to Use:** Provides a simple command-line interface and a straightforward Python library.
## Installation
Install `rune-512` from PyPI:
```bash
pip install rune-512
```
## Usage
### Command-Line Interface
The package provides a CLI for easy encoding and decoding from your terminal.
#### Encoding
To encode a string:
```bash
python -m rune_512 encode "hello world"
# Output: ⣦◩⣐▕╣⣆◤⠝▷╲⣘▐
```
To encode a hex string, use the `--hex` flag:
```bash
python -m rune_512 encode --hex "deadbeef"
# Output: ⢜╓▽⢶◷⣰
```
You can also pipe data from stdin:
```bash
echo "some data" | python -m rune_512 encode
# Output: ⠘⡴◍╻⣖⢤⠙⠰╴⣂
```
#### Decoding
To decode a `rune-512` string:
```bash
python -m rune_512 decode "⣦◩⣐▕╣⣆◤⠝▷╲⣘▐"
# Output: hello world
```
To decode to a hex string, use the `--hex` flag:
```bash
python -m rune_512 decode --hex "⢜╓▽⢶◷⣰"
# Output: deadbeef
```
### Library
You can also use `rune-512` as a library in your Python projects.
#### Encoding
To encode a byte string:
```python
from rune_512 import encode
payload = b'hello world'
encoded_string = encode(payload)
print(encoded_string)
# Output: ⣦◩⣐▕╣⣆◤⠝▷╲⣘▐
```
#### Decoding
To decode a string:
```python
from rune_512 import decode
encoded_string = '⣦◩⣐▕╣⣆◤⠝▷╲⣘▐'
try:
payload, codepoints_consumed = decode(encoded_string)
print(payload)
# Output: b'hello world'
print(f"Consumed {codepoints_consumed} codepoints.")
# Output: Consumed 12 codepoints.
except ValueError as e:
print(f"Decoding failed: {e}")
```
The `decode` function returns a tuple containing the decoded `bytes` and the number of Unicode codepoints consumed from the input string. This is useful for parsing data from streams or larger text blocks that may contain other information. Since the payload length is not encoded in the data, `rune-512` is designed for stream-based decoding. The decoder reads characters until it encounters one outside its alphabet, and the returned count helps you know how much of the input was part of the encoded data.
It is up to the user to decide on a scheme for indicating the start of a valid encoded payload inside of a larger body of text.
## How It Works
A `rune-512` encoded string consists of two parts:
1. **Header:** An 18-bit section containing a 17-bit checksum derived from the SHA-256 hash of the original payload and a parity bit for padding disambiguation.
2. **Payload:** The binary data, packed into 9-bit chunks.
Each 9-bit chunk is mapped to a character in the 512-character alphabet. This structure ensures that the data is both compact and verifiable.
## Limitations
`rune-512` is designed for encoding small to medium-sized binary payloads in text-based environments. It is not intended for all use cases. Please consider the following limitations:
* **Security:** The SHA-256-derived checksum only protects against accidental data corruption. **It does not provide cryptographic security.** Malicious actors can easily tamper with the data and forge a valid checksum. For applications requiring tamper-resistance, use a solution with cryptographic signatures or MACs (e.g., HMAC-SHA256).
* **Scalability:** The current implementations load the entire payload into memory. This makes them unsuitable for very large files, as it can lead to high memory usage and potential performance issues. In a server environment, processing excessively large inputs could pose a Denial of Service (DoS) risk. It is recommended to validate and limit input sizes before decoding.
## License
This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
Raw data
{
"_id": null,
"home_page": null,
"name": "rune-512",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "rune-512, encoding, decoding, base64, data encoding, binary encoding, codepoint encoding",
"author": null,
"author_email": "Lawrence Forman <me@merklejerk.com>",
"download_url": "https://files.pythonhosted.org/packages/21/2c/e72a26014d3cb555fbac1af229317892e463a627405f447c10336b76786f/rune_512-0.3.1.tar.gz",
"platform": null,
"description": "# Rune-512: Compact Binary Encoding\n\n[](https://badge.fury.io/py/rune-512)\n[](https://opensource.org/licenses/MIT)\n\n**Rune-512** is a binary-to-text encoding scheme designed to safely and compactly embed arbitrary binary data in environments with strict character limits but also support a wide range of Unicode characters, such as social media bios (like Bluesky, Twitter).\n\nIt uses a carefully selected 512-character symbolic unicode alphabet that is not visually distracting and can represent data more densely than traditional encodings like Base64, packing 9 bits of data into a single character.\n\nFor example, here's 32 random bytes:\n\n```\n\u28e3\u2873\u28dc\u25a3\u254e\u2847\u25e1\u2501\u2509\u25f3\u2822\u2556\u283f\u28fa\u2894\u25b6\u288e\u285d\u257a\u2842\u254d\u255e\u25a8\u257f\u25a1\u28fc\u28c6\u28bc\u25a4\u2856\u2880\n```\n\nHere's the string `\"the fox jumped over the lazy dog\"`:\n\n```\n\u28df\u25b4\u28e8\u25a1\u2529\u2806\u28cd\u25de\u2810\u286a\u28ea\u25b5\u2583\u2856\u2884\u281b\u25bb\u2865\u28e4\u2881\u25a3\u2886\u28a4\u281b\u2830\u257a\u28f2\u2881\u2523\u28f6\u28e0\n```\n\n## Features\n\n- **Compact:** Encodes 9 bits per character, offering significant space savings over Base64.\n- **Reliable:** Uses a 17-bit checksum derived from the SHA-256 hash of the payload to detect data corruption.\n- **Safe:** The alphabet consists of Unicode codepoints with wide compatibility across common platforms.\n- **Easy to Use:** Provides a simple command-line interface and a straightforward Python library.\n\n## Installation\n\nInstall `rune-512` from PyPI:\n\n```bash\npip install rune-512\n```\n\n## Usage\n\n### Command-Line Interface\n\nThe package provides a CLI for easy encoding and decoding from your terminal.\n\n#### Encoding\n\nTo encode a string:\n```bash\npython -m rune_512 encode \"hello world\"\n# Output: \u28e6\u25e9\u28d0\u2595\u2563\u28c6\u25e4\u281d\u25b7\u2572\u28d8\u2590\n```\n\nTo encode a hex string, use the `--hex` flag:\n```bash\npython -m rune_512 encode --hex \"deadbeef\"\n# Output: \u289c\u2553\u25bd\u28b6\u25f7\u28f0\n```\n\nYou can also pipe data from stdin:\n```bash\necho \"some data\" | python -m rune_512 encode\n# Output: \u2818\u2874\u25cd\u257b\u28d6\u28a4\u2819\u2830\u2574\u28c2\n```\n\n#### Decoding\n\nTo decode a `rune-512` string:\n```bash\npython -m rune_512 decode \"\u28e6\u25e9\u28d0\u2595\u2563\u28c6\u25e4\u281d\u25b7\u2572\u28d8\u2590\"\n# Output: hello world\n```\n\nTo decode to a hex string, use the `--hex` flag:\n```bash\npython -m rune_512 decode --hex \"\u289c\u2553\u25bd\u28b6\u25f7\u28f0\"\n# Output: deadbeef\n```\n\n### Library\n\nYou can also use `rune-512` as a library in your Python projects.\n\n#### Encoding\n\nTo encode a byte string:\n\n```python\nfrom rune_512 import encode\n\npayload = b'hello world'\nencoded_string = encode(payload)\nprint(encoded_string)\n# Output: \u28e6\u25e9\u28d0\u2595\u2563\u28c6\u25e4\u281d\u25b7\u2572\u28d8\u2590\n```\n\n#### Decoding\n\nTo decode a string:\n\n```python\nfrom rune_512 import decode\n\nencoded_string = '\u28e6\u25e9\u28d0\u2595\u2563\u28c6\u25e4\u281d\u25b7\u2572\u28d8\u2590'\ntry:\n payload, codepoints_consumed = decode(encoded_string)\n print(payload)\n # Output: b'hello world'\n print(f\"Consumed {codepoints_consumed} codepoints.\")\n # Output: Consumed 12 codepoints.\nexcept ValueError as e:\n print(f\"Decoding failed: {e}\")\n```\n\nThe `decode` function returns a tuple containing the decoded `bytes` and the number of Unicode codepoints consumed from the input string. This is useful for parsing data from streams or larger text blocks that may contain other information. Since the payload length is not encoded in the data, `rune-512` is designed for stream-based decoding. The decoder reads characters until it encounters one outside its alphabet, and the returned count helps you know how much of the input was part of the encoded data.\n\nIt is up to the user to decide on a scheme for indicating the start of a valid encoded payload inside of a larger body of text.\n\n## How It Works\n\nA `rune-512` encoded string consists of two parts:\n\n1. **Header:** An 18-bit section containing a 17-bit checksum derived from the SHA-256 hash of the original payload and a parity bit for padding disambiguation.\n2. **Payload:** The binary data, packed into 9-bit chunks.\n\nEach 9-bit chunk is mapped to a character in the 512-character alphabet. This structure ensures that the data is both compact and verifiable.\n\n## Limitations\n\n`rune-512` is designed for encoding small to medium-sized binary payloads in text-based environments. It is not intended for all use cases. Please consider the following limitations:\n\n* **Security:** The SHA-256-derived checksum only protects against accidental data corruption. **It does not provide cryptographic security.** Malicious actors can easily tamper with the data and forge a valid checksum. For applications requiring tamper-resistance, use a solution with cryptographic signatures or MACs (e.g., HMAC-SHA256).\n\n* **Scalability:** The current implementations load the entire payload into memory. This makes them unsuitable for very large files, as it can lead to high memory usage and potential performance issues. In a server environment, processing excessively large inputs could pose a Denial of Service (DoS) risk. It is recommended to validate and limit input sizes before decoding.\n\n## License\n\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.\n",
"bugtrack_url": null,
"license": null,
"summary": "A Python implementation of the Rune-512 encoding and decoding algorithm.",
"version": "0.3.1",
"project_urls": {
"Homepage": "https://github.com/merklejerk/rune-512",
"Issues": "https://github.com/merklejerk/rune-512/issues",
"Repository": "https://github.com/merklejerk/rune-512.git"
},
"split_keywords": [
"rune-512",
" encoding",
" decoding",
" base64",
" data encoding",
" binary encoding",
" codepoint encoding"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "593646416a5dcbc533a9e9e262f08b949561abc497c6350b34568b657cffe98f",
"md5": "f0405bf1da27d7028b5ec18bec55e331",
"sha256": "61482711568812eb92ce6bf39ae6c73fd942a31dbe698822464e92f0bad14c02"
},
"downloads": -1,
"filename": "rune_512-0.3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f0405bf1da27d7028b5ec18bec55e331",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 8560,
"upload_time": "2025-07-11T19:29:25",
"upload_time_iso_8601": "2025-07-11T19:29:25.865614Z",
"url": "https://files.pythonhosted.org/packages/59/36/46416a5dcbc533a9e9e262f08b949561abc497c6350b34568b657cffe98f/rune_512-0.3.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "212ce72a26014d3cb555fbac1af229317892e463a627405f447c10336b76786f",
"md5": "368157ab53b40577feb2d9bb07e35516",
"sha256": "b79bc832a1bc756186c5d9f201aaf128e338d3cfad507dbcec668dfaca80e980"
},
"downloads": -1,
"filename": "rune_512-0.3.1.tar.gz",
"has_sig": false,
"md5_digest": "368157ab53b40577feb2d9bb07e35516",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 11608,
"upload_time": "2025-07-11T19:29:26",
"upload_time_iso_8601": "2025-07-11T19:29:26.623230Z",
"url": "https://files.pythonhosted.org/packages/21/2c/e72a26014d3cb555fbac1af229317892e463a627405f447c10336b76786f/rune_512-0.3.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-11 19:29:26",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "merklejerk",
"github_project": "rune-512",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "rune-512"
}
JSON
