| Name | hdict JSON |
| Version |
2.240217.0
JSON |
| download |
| home_page | |
| Summary | A versatile dictionary based on a novel paradigm useful for computing, caching, experiments, distributed data, among others. |
| upload_time | 2024-02-17 10:11:31 |
| maintainer | |
| docs_url | None |
| author | davips |
| requires_python | >=3.10,<4.0 |
| license | GPLv3 |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
[](https://codecov.io/gh/davips/hdict)
<a href="https://pypi.org/project/hdict">
<img src="https://img.shields.io/pypi/v/hdict.svg?label=release&color=blue&style=flat-square" alt="pypi">
</a>
[](https://www.gnu.org/licenses/gpl-3.0)
<!--- [](https://doi.org/10.5281/zenodo.5501845) --->
[](https://arxiv.org/abs/2109.06028)
[](https://davips.github.io/hdict)
[](https://hosh.page)
[](https://pepy.tech/project/hdict)
# hdict { A _unique_ data structure }
[Website](https://hosh.page) |
[Latest Release](https://pypi.org/project/hdict) |
[Current Code](https://github.com/davips/hdict) |
[API Documentation](https://davips.github.io/hdict)
Please see the [website](https://hosh.page) for more detailed usage information.
## Overview
Shortly, `hdict` is a data structure useful for frictionless experiments, distributed data, among others.
It can be more formally defined as a hosh¹-based cacheable lazy `dict` with predictable/deterministic universally unique² identifiers.
This library is stable for the most common scenarios.
However, we will wait for more use in the wild before reaching the major version 1.
See the Section Versioning below for more information.
¹ hosh: _Operable HaSH_
² unique: _probabilistically guaranteed_
### Context
* [hosh](https://pypi.org/project/hosh): identification engine at the core of `hdict`
* previous projects: [hoshmap](https://pypi.org/project/hoshmap), [idict](https://pypi.org/project/idict), [ldict](https://pypi.org/project/ldict), [garoupa](https://pypi.org/project/garoupa)
### The concept
`hdict` is like an ordinary `dict` with `str` keys.
Each entry, called _field_, and the `hdict` itself, are identified by 40-digit hashes (see [hosh](https://pypi.org/project/hosh)).
A `hdict` object (say `d`) provides two extra entries: `_id` (hdict identifier) and `_ids` (field identifiers),
accessible through square brackets or through the shortcuts `d.id` and `d.ids`.
**`hdict` creation**
<details>
<p>
```python3
from hdict import hdict
# From named arguments.
d = hdict(x=5, y=7, z=10)
# From a dict object.
d = hdict({"x": 5, "y": 7, "z": 10})
# From an empty 'hdict' object.
d = hdict() >> {"x": 5} >> {"y": 7, "z": 10}
# All three options have the same result.
d.show(colored=False)
"""
{
x: 5,
y: 7,
z: 10,
_id: BN-3Q3Z.2Q.9nsbIYnOI75HT7xhgjvF6wErwBPTn,
_ids: {
x: ecvgo-CBPi7wRWIxNzuo1HgHQCbdvR058xi6zmr2,
y: eJCW9jGsdZTD6-AD9opKwjPIOWZ4R.T0CG2kdyzf,
z: u-Yykj2nDtKaUMGzfqScX5Y14qC7eqJrO7lXrJ1m
}
}
"""
```
```python3
from hosh import setup
# For better integration within the documentation, we change the color theme.
setup(dark_theme=False)
d.show(colored=False)
"""
{
x: 5,
y: 7,
z: 10,
_id: BN-3Q3Z.2Q.9nsbIYnOI75HT7xhgjvF6wErwBPTn,
_ids: {
x: ecvgo-CBPi7wRWIxNzuo1HgHQCbdvR058xi6zmr2,
y: eJCW9jGsdZTD6-AD9opKwjPIOWZ4R.T0CG2kdyzf,
z: u-Yykj2nDtKaUMGzfqScX5Y14qC7eqJrO7lXrJ1m
}
}
"""
```
</p>
</details>
A field contains a value or a function application.
A field pointing to an application is only evaluated on demand, i.e., lazily.
<!-- ------------------------------------------------------------------------
Value objects can have custom identifiers as well, if provided whithin the entry `ids`.
Otherwise, identifiers for functions and values will be calculated through blake3 hashing of their content.
For functions, the bytecode is used as content.
For this reason, such functions should be simple, with minimal external dependencies or
with their import statements inside the function body.
This decreases the odds of using two functions with identical local code (and, therefore, identical identifiers)
performing different calculations.
transformation steps done through the operator `>>`, which symbolizes the ordering of the steps.
* **value insertion** - represented by dict-like objects
* **function application** - represented by ordinary Python functions
Functions, `hdict`s, and values have a deterministic UUID
(called _hosh_ - **o**perable **h**a**sh**).
Identifiers (hoshes) for `hdict`s and values are predictable through the
magic available [here](https://pypi.org/project/garoupa).
An `hdict` is completely defined by its key-value pairs so that
it can be converted from/to a built-in `dict`.
------------------------------------------------------------------------ -->
Please refer to our [website](https://hosh.page) for more examples.
## Installation
### ...as a standalone lib
```bash
# Set up a virtualenv.
python3 -m venv venv
source venv/bin/activate
# Install from PyPI...
pip install --upgrade pip
pip install -U hdict
# use the flag 'full' for extra functionality (recommended if working with persistence or some special values like DataFrames)
pip install -U hdict[full]
# ...or, install from updated source code.
pip install git+https://github.com/davips/hdict
```
### ...from source
```bash
git clone https://github.com/davips/hdict
cd hdict
poetry install
# use the flag 'full' for extra functionality (recommended if working with persistence or some special values like DataFrames)
poetry install -E full
```
## Examples
Please refer to our [website](https://hosh.page) for examples.
## Development
### Licensing
The initial license choosen is GPL. Please contact the developer for other licensing needs.
### Versioning
While the version scheme has a meaningful calendar component (`minor=yymmdd`), it is still compatible with semantic versioning.
For instance, the version `0.230215.1` means `major=0`, `minor=230215`, `micro/patch=1`. Notes:
* While `major=0`, some compatibility-breaking changes may occur.
* From `major=1` onwards, compatibility-breaking changes increment it, and update the minor version to reflect the release date.
* New (non breaking) features update only the minor version to reflect the release date.
* Bug fixes (including compatibility-breaking ones) increment only the micro version.
### Contributing
#### Collaboration
We have ongoing research applying this tool to machine learning and clinical academic experiments.
Although, the scope of application is broad as it encompasses software development in general.
Feel free to contact us if you are interested in the project/concept or have a suggestion/interesting problem to be solved.
#### Donation
Currently there are no established forms of donation.
Expenses:
* Programming hours
* Support
* Custom features
* Domain name maintenance yearly costs
### Acknowledgement
This work was based on a previous research supported by Fapesp under supervision of
Prof. André C. P. L. F. de Carvalho at CEPID-CeMEAI (Grants 2013/07375-0 – 2019/01735-0)
until 2021-03-31.
We would like to thank the providers of free tools that make this project feasible:
* github for hosting the code repository freely
* Oracle for hosting the website freely
* Developers of Mint, XFCE, and GNU/Linux for the operating system
* JetBrains for Intellij IDEA Community Edition
* Developers of Python packages listed in our project.toml
Raw data
{
"_id": null,
"home_page": "",
"name": "hdict",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10,<4.0",
"maintainer_email": "",
"keywords": "",
"author": "davips",
"author_email": "dpsabc@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/f4/e8/ebdf5c00084c270b39516baa07422bb391081e22e30538cbd871834b3964/hdict-2.240217.0.tar.gz",
"platform": null,
"description": "\n[](https://codecov.io/gh/davips/hdict)\n<a href=\"https://pypi.org/project/hdict\">\n<img src=\"https://img.shields.io/pypi/v/hdict.svg?label=release&color=blue&style=flat-square\" alt=\"pypi\">\n</a>\n\n[](https://www.gnu.org/licenses/gpl-3.0)\n\n<!--- [](https://doi.org/10.5281/zenodo.5501845) --->\n[](https://arxiv.org/abs/2109.06028)\n[](https://davips.github.io/hdict)\n[](https://hosh.page)\n[](https://pepy.tech/project/hdict)\n\n\n# hdict { A _unique_ data structure }\n[Website](https://hosh.page) | \n[Latest Release](https://pypi.org/project/hdict) |\n[Current Code](https://github.com/davips/hdict) |\n[API Documentation](https://davips.github.io/hdict)\n\nPlease see the [website](https://hosh.page) for more detailed usage information.\n\n## Overview\nShortly, `hdict` is a data structure useful for frictionless experiments, distributed data, among others.\nIt can be more formally defined as a hosh\u00b9-based cacheable lazy `dict` with predictable/deterministic universally unique\u00b2 identifiers.\n\nThis library is stable for the most common scenarios.\nHowever, we will wait for more use in the wild before reaching the major version 1.\nSee the Section Versioning below for more information.\n\n\u00b9 hosh: _Operable HaSH_\n\n\u00b2 unique: _probabilistically guaranteed_\n\n\n\n### Context\n\n * [hosh](https://pypi.org/project/hosh): identification engine at the core of `hdict`\n * previous projects: [hoshmap](https://pypi.org/project/hoshmap), [idict](https://pypi.org/project/idict), [ldict](https://pypi.org/project/ldict), [garoupa](https://pypi.org/project/garoupa)\n\n### The concept\n\n`hdict` is like an ordinary `dict` with `str` keys. \nEach entry, called _field_, and the `hdict` itself, are identified by 40-digit hashes (see [hosh](https://pypi.org/project/hosh)).\nA `hdict` object (say `d`) provides two extra entries: `_id` (hdict identifier) and `_ids` (field identifiers),\naccessible through square brackets or through the shortcuts `d.id` and `d.ids`.\n\n**`hdict` creation**\n<details>\n<p>\n\n```python3\nfrom hdict import hdict\n\n# From named arguments.\nd = hdict(x=5, y=7, z=10)\n\n# From a dict object.\nd = hdict({\"x\": 5, \"y\": 7, \"z\": 10})\n\n# From an empty 'hdict' object.\nd = hdict() >> {\"x\": 5} >> {\"y\": 7, \"z\": 10}\n\n# All three options have the same result.\nd.show(colored=False)\n\"\"\"\n{\n x: 5,\n y: 7,\n z: 10,\n _id: BN-3Q3Z.2Q.9nsbIYnOI75HT7xhgjvF6wErwBPTn,\n _ids: {\n x: ecvgo-CBPi7wRWIxNzuo1HgHQCbdvR058xi6zmr2,\n y: eJCW9jGsdZTD6-AD9opKwjPIOWZ4R.T0CG2kdyzf,\n z: u-Yykj2nDtKaUMGzfqScX5Y14qC7eqJrO7lXrJ1m\n }\n}\n\"\"\"\n```\n\n```python3\n\nfrom hosh import setup\n\n# For better integration within the documentation, we change the color theme.\nsetup(dark_theme=False)\n\nd.show(colored=False)\n\n\"\"\"\n{\n x: 5,\n y: 7,\n z: 10,\n _id: BN-3Q3Z.2Q.9nsbIYnOI75HT7xhgjvF6wErwBPTn,\n _ids: {\n x: ecvgo-CBPi7wRWIxNzuo1HgHQCbdvR058xi6zmr2,\n y: eJCW9jGsdZTD6-AD9opKwjPIOWZ4R.T0CG2kdyzf,\n z: u-Yykj2nDtKaUMGzfqScX5Y14qC7eqJrO7lXrJ1m\n }\n}\n\"\"\"\n```\n\n\n</p>\n</details>\n\nA field contains a value or a function application.\nA field pointing to an application is only evaluated on demand, i.e., lazily.\n\n<!-- ------------------------------------------------------------------------ \nValue objects can have custom identifiers as well, if provided whithin the entry `ids`. \nOtherwise, identifiers for functions and values will be calculated through blake3 hashing of their content.\nFor functions, the bytecode is used as content.\nFor this reason, such functions should be simple, with minimal external dependencies or\nwith their import statements inside the function body.\nThis decreases the odds of using two functions with identical local code (and, therefore, identical identifiers)\nperforming different calculations.\n\ntransformation steps done through the operator `>>`, which symbolizes the ordering of the steps.\n* **value insertion** - represented by dict-like objects\n* **function application** - represented by ordinary Python functions\n\nFunctions, `hdict`s, and values have a deterministic UUID\n(called _hosh_ - **o**perable **h**a**sh**). \nIdentifiers (hoshes) for `hdict`s and values are predictable through the\nmagic available [here](https://pypi.org/project/garoupa).\nAn `hdict` is completely defined by its key-value pairs so that\nit can be converted from/to a built-in `dict`.\n\n\n------------------------------------------------------------------------ -->\n\nPlease refer to our [website](https://hosh.page) for more examples.\n\n\n## Installation\n### ...as a standalone lib\n```bash\n# Set up a virtualenv. \npython3 -m venv venv\nsource venv/bin/activate\n\n# Install from PyPI...\npip install --upgrade pip\npip install -U hdict\n\n# use the flag 'full' for extra functionality (recommended if working with persistence or some special values like DataFrames)\npip install -U hdict[full]\n\n# ...or, install from updated source code.\npip install git+https://github.com/davips/hdict\n```\n\n### ...from source\n```bash\ngit clone https://github.com/davips/hdict\ncd hdict\npoetry install\n\n# use the flag 'full' for extra functionality (recommended if working with persistence or some special values like DataFrames) \npoetry install -E full\n```\n\n## Examples\n\nPlease refer to our [website](https://hosh.page) for examples.\n\n\n\n## Development\n### Licensing\nThe initial license choosen is GPL. Please contact the developer for other licensing needs.\n\n### Versioning\nWhile the version scheme has a meaningful calendar component (`minor=yymmdd`), it is still compatible with semantic versioning.\nFor instance, the version `0.230215.1` means `major=0`, `minor=230215`, `micro/patch=1`. Notes:\n * While `major=0`, some compatibility-breaking changes may occur.\n * From `major=1` onwards, compatibility-breaking changes increment it, and update the minor version to reflect the release date.\n * New (non breaking) features update only the minor version to reflect the release date.\n * Bug fixes (including compatibility-breaking ones) increment only the micro version.\n\n### Contributing\n#### Collaboration\nWe have ongoing research applying this tool to machine learning and clinical academic experiments.\nAlthough, the scope of application is broad as it encompasses software development in general.\nFeel free to contact us if you are interested in the project/concept or have a suggestion/interesting problem to be solved.\n\n#### Donation\nCurrently there are no established forms of donation.\nExpenses:\n * Programming hours\n * Support\n * Custom features\n * Domain name maintenance yearly costs\n\n### Acknowledgement\nThis work was based on a previous research supported by Fapesp under supervision of\nProf. Andr\u00e9 C. P. L. F. de Carvalho at CEPID-CeMEAI (Grants 2013/07375-0 \u2013 2019/01735-0)\nuntil 2021-03-31.\n\nWe would like to thank the providers of free tools that make this project feasible:\n * github for hosting the code repository freely\n * Oracle for hosting the website freely\n * Developers of Mint, XFCE, and GNU/Linux for the operating system\n * JetBrains for Intellij IDEA Community Edition\n * Developers of Python packages listed in our project.toml\n",
"bugtrack_url": null,
"license": "GPLv3",
"summary": "A versatile dictionary based on a novel paradigm useful for computing, caching, experiments, distributed data, among others.",
"version": "2.240217.0",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6295d490c15cc0a25d19d67fcdb44ac2c2191f28cbc93b9af836d5d070a68231",
"md5": "de4882c86391f92fafa96881ce242d66",
"sha256": "25df176f55d56584f747ff7019087bffed00731f3f518012d9a667fb23e6576c"
},
"downloads": -1,
"filename": "hdict-2.240217.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "de4882c86391f92fafa96881ce242d66",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10,<4.0",
"size": 77221,
"upload_time": "2024-02-17T10:11:28",
"upload_time_iso_8601": "2024-02-17T10:11:28.995803Z",
"url": "https://files.pythonhosted.org/packages/62/95/d490c15cc0a25d19d67fcdb44ac2c2191f28cbc93b9af836d5d070a68231/hdict-2.240217.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f4e8ebdf5c00084c270b39516baa07422bb391081e22e30538cbd871834b3964",
"md5": "9e2ea5e7ff00d5a5a7b12095112318b6",
"sha256": "17480434aae2a37f53bfa9c54f01165f13dd67891df32c0d154f297ec892fb4a"
},
"downloads": -1,
"filename": "hdict-2.240217.0.tar.gz",
"has_sig": false,
"md5_digest": "9e2ea5e7ff00d5a5a7b12095112318b6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10,<4.0",
"size": 57205,
"upload_time": "2024-02-17T10:11:31",
"upload_time_iso_8601": "2024-02-17T10:11:31.161047Z",
"url": "https://files.pythonhosted.org/packages/f4/e8/ebdf5c00084c270b39516baa07422bb391081e22e30538cbd871834b3964/hdict-2.240217.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-17 10:11:31",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "hdict"
}
