| Name | tally-token JSON |
| Version |
0.6.2
JSON |
| download |
| home_page | None |
| Summary | A library to split data into tokens |
| upload_time | 2025-01-31 17:40:40 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.9 |
| license | Copyright 2023-2025 Kitsu Yui
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# python-tally-token
[](https://badge.fury.io/py/tally-token)
[](https://pypi.python.org/pypi/tally-token/)
[](https://opensource.org/licenses/BSD-3-Clause)
[](https://codecov.io/gh/kitsuyui/python-tally-token)
## What is this?
tally-token is a Python library for split data into tokens with same length.
[Tally](https://en.wikipedia.org/wiki/Tally_stick) is a historical object for prove something by splitting wood into tokens and matching tokens.
<a href="https://en.wikipedia.org/wiki/Tally_stick#/media/File:Medieval_tally_sticks.jpg" target="_blank"><img src="https://github.com/kitsuyui/python-tally-token/assets/2596972/103a0184-c508-4ce6-9ed3-1b1eb58bc6cc" style="width:20rem" /></a>
> Medieval English split tally stick (front and reverse view). The stick is notched and inscribed to record a debt owed to the rural dean of Preston Candover, Hampshire, of a tithe of 20d each on 32 sheep, amounting to a total sum of £2 13s. 4d.
# Usage
## Install
```sh
$ pip install tally-token
```
## CLI Usage
```sh
$ tally-token --help
usage: tally-token [-h] {split,merge} ...
positional arguments:
{split,merge} Commands: split: split a file into multiple files merge: merge multiple files into a fileExample: tally-token split example.bin
example.bin.1 example.bin.2 example.bin.3 tally-token merge example-merged.bin example.bin.1 example.bin.2 example.bin.3
options:
-h, --help show this help message and exit
```
### split
You can use `split` to split a file into multiple files.
```sh
$ tally-token split something.bin split-1.bin split-2.bin split-3.bin
```
### merge
You can use `merge` to merge multiple files into a file.
```sh
$ tally-token merge merged.bin split-1.bin split-2.bin split-3.bin
```
### Large files
Nothing special. You can split and merge large file.
```sh
$ dd if=/dev/urandom of=original.1g.bin bs=1G count=1
$ tally-token split original.1g.bin split-1.bin split-2.bin split-3.bin
$ shasum -a 256 original.1g.bin
> 736a344d99d27e2dcdab8bc37ca94c83eda26f812a3dee87ac98989f89b3f965 original.1g.bin
$ tally-token merge recovery.1g.bin split-1.bin split-2.bin split-3.bin
$ shasum -a 256 recovery.1g.bin
> 736a344d99d27e2dcdab8bc37ca94c83eda26f812a3dee87ac98989f89b3f965 recovery.1g.bin
```
## Example
### split
You can use `split_text` to split text into tokens. `split_text` returns list of random bytes.
```python
>>> from tally_token import split_text
>>> split_text("Hello, World!")
[b'qQ\xa5\x97\x84\x88\xd7U%\xfb(k\xa1', b'94\xc9\xfb\xeb\xa4\xf7\x02J\x89D\x0f\x80']
```
### merge
You can use `merge_text` to merge tokens into text. `merge_text` returns cleartext.
```python
>>> from tally_token import merge_text
>>> merge_text([b'qQ\xa5\x97\x84\x88\xd7U%\xfb(k\xa1', b'94\xc9\xfb\xeb\xa4\xf7\x02J\x89D\x0f\x80'])
'Hello, World!'
```
### split with custom length
```python
>>> from tally_token import split_text, merge_text
>>> split_text("Hello, World!", 5)
[b'N&\xce\\\xbc6dxp\x87\xa8#z', b'\xa3D\\A\xf8\xd1KDX\x1cKx\x87', b'\xffZ\x03\xf5\x92Q\xf52\xc4\x1e\xf2\xf8\x06', b'\xaa\xdd:\x85F\xa1\xcdbp\xf3\xe6P\xe5', b'\xf0\x80\xc7\x01\xff;7;\xf3\x04\x9b\x97?']
>>> merge_text([b'N&\xce\\\xbc6dxp\x87\xa8#z', b'\xa3D\\A\xf8\xd1KDX\x1cKx\x87', b'\xffZ\x03\xf5\x92Q\xf52\xc4\x1e\xf2\xf8\x06', b'\xaa\xdd:\x85F\xa1\xcdbp\xf3\xe6P\xe5', b'\xf0\x80\xc7\x01\xff;7;\xf3\x04\x9b\x97?'])
'Hello, World!'
```
### split with custom encoding
```python
>>> from tally_token import split_text, merge_text
>>> split_text("こんにちは", encoding="CP932")
[b'g\xc3\x12\xeal?\xe5[\x03\xad', b'\xe5r\x90\x1b\xee\xf6g\xe4\x81`']
>>> merge_text([b'g\xc3\x12\xeal?\xe5[\x03\xad', b'\xe5r\x90\x1b\xee\xf6g\xe4\x81`'], encoding="CP932")
'こんにちは'
```
### bytes interface
You can use `split_bytes_into` and `merge_bytes_into` to split and merge bytes.
This is useful for split binary data.
```python
>>> from tally_token import split_bytes_into, merge_bytes_into
>>> split_bytes_into(b"Hello, World!", 5)
[b'\xc5b\xf4E)\xe1vO8\xff@\xf9\xdd', b'\x84\xb9X#\x85\xf5\xed\xbcM\xc4\xef\xf4\xd3', b'\xb47\xf6\xfa?\x14\xa8`\xc9\xe0\xe5\x87\x14', b'\x1cd\xb4o\xe8I:\xe5\xf6\x13\xe5\x93G', b'\xa1\xed\x82\x9f\x14e)!%\xba\xc3}|']
>>> merge_bytes_into([b'\xc5b\xf4E)\xe1vO8\xff@\xf9\xdd', b'\x84\xb9X#\x85\xf5\xed\xbcM\xc4\xef\xf4\xd3', b'\xb47\xf6\xfa?\x14\xa8`\xc9\xe0\xe5\x87\x14', b'\x1cd\xb4o\xe8I:\xe5\xf6\x13\xe5\x93G', b'\xa1\xed\x82\x9f\x14e)!%\xba\xc3}|'])
b'Hello, World!'
```
# Reference
- Tally stick https://en.wikipedia.org/wiki/Tally_stick
- Tessera or Symbolum (Hospitium token) https://en.wikipedia.org/wiki/Hospitium
- 割符 https://ja.wikipedia.org/wiki/%E5%89%B2%E7%AC%A6
- One-time pad https://en.wikipedia.org/wiki/One-time_pad
# LICENSE
BSD 3-Clause License
Raw data
{
"_id": null,
"home_page": null,
"name": "tally-token",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": null,
"author": null,
"author_email": "Yui KITSU <kitsuyui+github@kitsuyui.com>",
"download_url": "https://files.pythonhosted.org/packages/89/7f/b3023c07e0a753d7c5eca4f62aa7bfb2d7ce35a9bf3dff626e7200371c6d/tally_token-0.6.2.tar.gz",
"platform": null,
"description": "# python-tally-token\n\n[](https://badge.fury.io/py/tally-token)\n[](https://pypi.python.org/pypi/tally-token/)\n[](https://opensource.org/licenses/BSD-3-Clause)\n[](https://codecov.io/gh/kitsuyui/python-tally-token)\n\n## What is this?\n\ntally-token is a Python library for split data into tokens with same length.\n\n[Tally](https://en.wikipedia.org/wiki/Tally_stick) is a historical object for prove something by splitting wood into tokens and matching tokens.\n\n<a href=\"https://en.wikipedia.org/wiki/Tally_stick#/media/File:Medieval_tally_sticks.jpg\" target=\"_blank\"><img src=\"https://github.com/kitsuyui/python-tally-token/assets/2596972/103a0184-c508-4ce6-9ed3-1b1eb58bc6cc\" style=\"width:20rem\" /></a>\n> Medieval English split tally stick (front and reverse view). The stick is notched and inscribed to record a debt owed to the rural dean of Preston Candover, Hampshire, of a tithe of 20d each on 32 sheep, amounting to a total sum of \u00a32 13s. 4d.\n\n# Usage\n\n## Install\n\n```sh\n$ pip install tally-token\n```\n\n## CLI Usage\n\n```sh\n$ tally-token --help\nusage: tally-token [-h] {split,merge} ...\n\npositional arguments:\n {split,merge} Commands: split: split a file into multiple files merge: merge multiple files into a fileExample: tally-token split example.bin\n example.bin.1 example.bin.2 example.bin.3 tally-token merge example-merged.bin example.bin.1 example.bin.2 example.bin.3\n\noptions:\n -h, --help show this help message and exit\n```\n\n### split\n\nYou can use `split` to split a file into multiple files.\n\n```sh\n$ tally-token split something.bin split-1.bin split-2.bin split-3.bin\n```\n\n### merge\n\nYou can use `merge` to merge multiple files into a file.\n\n```sh\n$ tally-token merge merged.bin split-1.bin split-2.bin split-3.bin\n```\n\n### Large files\n\nNothing special. You can split and merge large file.\n\n```sh\n$ dd if=/dev/urandom of=original.1g.bin bs=1G count=1\n$ tally-token split original.1g.bin split-1.bin split-2.bin split-3.bin\n$ shasum -a 256 original.1g.bin\n> 736a344d99d27e2dcdab8bc37ca94c83eda26f812a3dee87ac98989f89b3f965 original.1g.bin\n$ tally-token merge recovery.1g.bin split-1.bin split-2.bin split-3.bin\n$ shasum -a 256 recovery.1g.bin\n> 736a344d99d27e2dcdab8bc37ca94c83eda26f812a3dee87ac98989f89b3f965 recovery.1g.bin\n```\n\n## Example\n\n### split\n\nYou can use `split_text` to split text into tokens. `split_text` returns list of random bytes.\n\n```python\n>>> from tally_token import split_text\n>>> split_text(\"Hello, World!\")\n[b'qQ\\xa5\\x97\\x84\\x88\\xd7U%\\xfb(k\\xa1', b'94\\xc9\\xfb\\xeb\\xa4\\xf7\\x02J\\x89D\\x0f\\x80']\n```\n\n### merge\n\nYou can use `merge_text` to merge tokens into text. `merge_text` returns cleartext.\n\n```python\n>>> from tally_token import merge_text\n>>> merge_text([b'qQ\\xa5\\x97\\x84\\x88\\xd7U%\\xfb(k\\xa1', b'94\\xc9\\xfb\\xeb\\xa4\\xf7\\x02J\\x89D\\x0f\\x80'])\n'Hello, World!'\n```\n\n### split with custom length\n\n```python\n>>> from tally_token import split_text, merge_text\n>>> split_text(\"Hello, World!\", 5)\n[b'N&\\xce\\\\\\xbc6dxp\\x87\\xa8#z', b'\\xa3D\\\\A\\xf8\\xd1KDX\\x1cKx\\x87', b'\\xffZ\\x03\\xf5\\x92Q\\xf52\\xc4\\x1e\\xf2\\xf8\\x06', b'\\xaa\\xdd:\\x85F\\xa1\\xcdbp\\xf3\\xe6P\\xe5', b'\\xf0\\x80\\xc7\\x01\\xff;7;\\xf3\\x04\\x9b\\x97?']\n>>> merge_text([b'N&\\xce\\\\\\xbc6dxp\\x87\\xa8#z', b'\\xa3D\\\\A\\xf8\\xd1KDX\\x1cKx\\x87', b'\\xffZ\\x03\\xf5\\x92Q\\xf52\\xc4\\x1e\\xf2\\xf8\\x06', b'\\xaa\\xdd:\\x85F\\xa1\\xcdbp\\xf3\\xe6P\\xe5', b'\\xf0\\x80\\xc7\\x01\\xff;7;\\xf3\\x04\\x9b\\x97?'])\n'Hello, World!'\n```\n\n### split with custom encoding\n\n```python\n>>> from tally_token import split_text, merge_text\n>>> split_text(\"\u3053\u3093\u306b\u3061\u306f\", encoding=\"CP932\")\n[b'g\\xc3\\x12\\xeal?\\xe5[\\x03\\xad', b'\\xe5r\\x90\\x1b\\xee\\xf6g\\xe4\\x81`']\n>>> merge_text([b'g\\xc3\\x12\\xeal?\\xe5[\\x03\\xad', b'\\xe5r\\x90\\x1b\\xee\\xf6g\\xe4\\x81`'], encoding=\"CP932\")\n'\u3053\u3093\u306b\u3061\u306f'\n```\n\n### bytes interface\n\nYou can use `split_bytes_into` and `merge_bytes_into` to split and merge bytes.\nThis is useful for split binary data.\n\n```python\n>>> from tally_token import split_bytes_into, merge_bytes_into\n>>> split_bytes_into(b\"Hello, World!\", 5)\n[b'\\xc5b\\xf4E)\\xe1vO8\\xff@\\xf9\\xdd', b'\\x84\\xb9X#\\x85\\xf5\\xed\\xbcM\\xc4\\xef\\xf4\\xd3', b'\\xb47\\xf6\\xfa?\\x14\\xa8`\\xc9\\xe0\\xe5\\x87\\x14', b'\\x1cd\\xb4o\\xe8I:\\xe5\\xf6\\x13\\xe5\\x93G', b'\\xa1\\xed\\x82\\x9f\\x14e)!%\\xba\\xc3}|']\n>>> merge_bytes_into([b'\\xc5b\\xf4E)\\xe1vO8\\xff@\\xf9\\xdd', b'\\x84\\xb9X#\\x85\\xf5\\xed\\xbcM\\xc4\\xef\\xf4\\xd3', b'\\xb47\\xf6\\xfa?\\x14\\xa8`\\xc9\\xe0\\xe5\\x87\\x14', b'\\x1cd\\xb4o\\xe8I:\\xe5\\xf6\\x13\\xe5\\x93G', b'\\xa1\\xed\\x82\\x9f\\x14e)!%\\xba\\xc3}|'])\nb'Hello, World!'\n```\n\n# Reference\n\n- Tally stick https://en.wikipedia.org/wiki/Tally_stick\n- Tessera or Symbolum (Hospitium token) https://en.wikipedia.org/wiki/Hospitium\n- \u5272\u7b26 https://ja.wikipedia.org/wiki/%E5%89%B2%E7%AC%A6\n- One-time pad https://en.wikipedia.org/wiki/One-time_pad\n\n# LICENSE\n\nBSD 3-Clause License\n",
"bugtrack_url": null,
"license": "Copyright 2023-2025 Kitsu Yui\n \n Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:\n \n 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.\n \n 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.\n \n 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.\n \n THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n ",
"summary": "A library to split data into tokens",
"version": "0.6.2",
"project_urls": {
"Homepage": "https://github.com/kitsuyui/python-tally-token"
},
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "aec7f9100237c79dfa6f8b33b2a623c6ee692ba0f5ef7e535151bca0ed8acad8",
"md5": "5a5f58b16db2c437f0d92ded0da65d18",
"sha256": "36467458d921bdb2e306a62d039607df06a4b886bb269e17a9bf6300ec240f4f"
},
"downloads": -1,
"filename": "tally_token-0.6.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "5a5f58b16db2c437f0d92ded0da65d18",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 7316,
"upload_time": "2025-01-31T17:40:38",
"upload_time_iso_8601": "2025-01-31T17:40:38.285946Z",
"url": "https://files.pythonhosted.org/packages/ae/c7/f9100237c79dfa6f8b33b2a623c6ee692ba0f5ef7e535151bca0ed8acad8/tally_token-0.6.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "897fb3023c07e0a753d7c5eca4f62aa7bfb2d7ce35a9bf3dff626e7200371c6d",
"md5": "930d6b2de319f7e60786c13ee69e44f7",
"sha256": "41d8e7f7b04821be328a61c2d33edd9cef2b88798c07eb97f51eb6e3d043bc57"
},
"downloads": -1,
"filename": "tally_token-0.6.2.tar.gz",
"has_sig": false,
"md5_digest": "930d6b2de319f7e60786c13ee69e44f7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 33931,
"upload_time": "2025-01-31T17:40:40",
"upload_time_iso_8601": "2025-01-31T17:40:40.102964Z",
"url": "https://files.pythonhosted.org/packages/89/7f/b3023c07e0a753d7c5eca4f62aa7bfb2d7ce35a9bf3dff626e7200371c6d/tally_token-0.6.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-31 17:40:40",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "kitsuyui",
"github_project": "python-tally-token",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "tally-token"
}
