| Name | circular-dict JSON |
| Version |
1.9
JSON |
| download |
| home_page | https://github.com/Eric-Canas/CircularDict |
| Summary | CircularDict is a high-performance Python data structure that blends the functionality of dictionaries and circular buffers. Inheriting the usage of traditional dictionaries, it allows you to define constraints on size and memory usage. This way, the CircularDict will be always up-to-date with the last N added elements, ensuring that neither the maximum length nor the memory usage limit is exceeded. It is ideal for caching large data structures while maintaining control over memory footprint. |
| upload_time | 2024-05-12 20:40:34 |
| maintainer | None |
| docs_url | None |
| author | Eric-Canas |
| requires_python | >=3.6 |
| license | MIT |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# CircularDict
<img alt="CircularDict" title="CircularDict" src="https://raw.githubusercontent.com/Eric-Canas/CircularDict/main/resources/logo.png" width="20%" align="left"> **CircularDict** is a Python `dict` that acts as a **Circular Buffer**. This dictionary maintains a **controlled size**, limited either by a specified **number of items** (_maxlen_) or **total size in bytes** (_maxsize_bytes_). Upon reaching the defined limit, **CircularDict** automatically removes the oldest entries, maintaining **memory usage** under control.
Built upon Python's `OrderedDict`, **CircularDict** inherits all **standard dictionary usage** and operations, augmented by the capability of **memory management**. It's particularly useful in scenarios such as **caching**, where limiting **memory consumption** is crucial. The class combines _dictionary_ and _circular-queue_ behaviors, providing an efficient and scalable solution for various use cases.
## Installation
To install **CircularDict** simply run:
```bash
pip install circular-dict
```
## Usage
Working with **CircularDict** is as simple as using a standard Python `dict`, with additional parameters `maxlen` or `maxsize_bytes` on the initialization to control the buffer size. You can use one of them or both.
```python
from circular_dict import CircularDict
# Initialize a CircularDict with a maximum length of 3 items and a storage limit of 4Mb
my_dict = CircularDict(maxlen=3, maxsize_bytes=4*1024*1024)
```
### Example with `maxlen`
You can use `maxlen` to define the maximum amount of items that the dictionary can store. It is useful for defining fixed size buffers.
```python
from circular_dict import CircularDict
# Initialize a CircularDict with a maximum length of 3
my_buffer = CircularDict(maxlen=3)
# Fill it with 3 items
my_buffer['item1'] = 'value1'
my_buffer['item2'] = 'value2'
my_buffer['item3'] = 'value3'
print(f"When filling it: {circ_dict}")
# Add another item
my_buffer['item4'] = 'value4'
print(f"After adding an element beyond maxlen: {circ_dict}")
```
Output:
```bash
When filling it: {'item1': 'value1', 'item2': 'value2', 'item3': 'value3'}
After adding an element beyond maxlen: {'item2': 'value2', 'item3': 'value3', 'item4': 'value4'}
```
### Example with `maxsize_bytes`
You can use `maxsize_bytes` to define the maximum amount of memory that the `dict` can store. It is particularly beneficial when defining **caches**, to prevent **memory overflows**.
```python
from circular_dict import CircularDict
import numpy as np
import sys
# Initialize a CircularDict with a maximum length of 100KB
my_buffer = CircularDict(maxsize_bytes=100*1024)
# Add two arrays of ~40Kb (10*1024*4 bytes (int32) + 5 bytes (chars) + 100 bytes (numpy structure) + 50 bytes (str structure))
my_buffer['item1'] = np.zeros((10, 1024), dtype=np.int32)
my_buffer['item2'] = np.ones((10, 1024), dtype=np.int32)
print(f"{len(my_buffer)} Elements {tuple(my_buffer.keys())}. Dict size: {my_buffer.current_size/1024} Kb")
# Add a new element of ~32Kb will delete oldest elements ('item1') until fitting in the `dict`.
my_buffer['item3'] = np.ones((8, 1024), dtype=np.int32)
print(f"{len(my_buffer)} Elements {tuple(my_buffer.keys())}. Dict size: {my_buffer.current_size/1024} Kb")
# Create an element of ~160Kb (bigger than the defined maximum storage) to trigger a MemoryError
too_big_array = np.ones((40, 1024), dtype=np.int32)
try:
# Try to add it to the dict
my_buffer['item4'] = too_big_array
except MemoryError:
print(f"Cannot add an element with {sys.getsizeof(too_big_array)/1024}Kb in a dict with"\
f"maxsize_bytes of {my_buffer.maxsize_bytes/1024} Kb. Current elements {tuple(my_buffer.keys())}")
```
Output
```bash
2 Elements ('item1', 'item2'). Dict size: 80.35 Kb
2 Elements ('item2', 'item3'). Dict size: 72.35 Kb
Cannot add an element with 160.12Kb in a dict with maxsize_bytes of 100.0 Kb. Current elements ('item2', 'item3')
```
Please remember that the `maxsize_bytes` parameter considers the **total** memory footprint, including the sizes of _keys_ and _values_. If you try to add an item that exceeds the `maxsize_bytes`, a `MemoryError` will be raised.
Raw data
{
"_id": null,
"home_page": "https://github.com/Eric-Canas/CircularDict",
"name": "circular-dict",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": null,
"author": "Eric-Canas",
"author_email": "eric@ericcanas.com",
"download_url": "https://files.pythonhosted.org/packages/5c/dd/0c23c95fc6cc02e8566957d15063bea03d2422b4a095a544d4279b5f2373/circular_dict-1.9.tar.gz",
"platform": null,
"description": "# CircularDict\n<img alt=\"CircularDict\" title=\"CircularDict\" src=\"https://raw.githubusercontent.com/Eric-Canas/CircularDict/main/resources/logo.png\" width=\"20%\" align=\"left\"> **CircularDict** is a Python `dict` that acts as a **Circular Buffer**. This dictionary maintains a **controlled size**, limited either by a specified **number of items** (_maxlen_) or **total size in bytes** (_maxsize_bytes_). Upon reaching the defined limit, **CircularDict** automatically removes the oldest entries, maintaining **memory usage** under control.\n\nBuilt upon Python's `OrderedDict`, **CircularDict** inherits all **standard dictionary usage** and operations, augmented by the capability of **memory management**. It's particularly useful in scenarios such as **caching**, where limiting **memory consumption** is crucial. The class combines _dictionary_ and _circular-queue_ behaviors, providing an efficient and scalable solution for various use cases.\n\n## Installation\n\nTo install **CircularDict** simply run:\n\n```bash\npip install circular-dict\n```\n\n## Usage\n\nWorking with **CircularDict** is as simple as using a standard Python `dict`, with additional parameters `maxlen` or `maxsize_bytes` on the initialization to control the buffer size. You can use one of them or both.\n```python\nfrom circular_dict import CircularDict\n# Initialize a CircularDict with a maximum length of 3 items and a storage limit of 4Mb\nmy_dict = CircularDict(maxlen=3, maxsize_bytes=4*1024*1024)\n```\n\n### Example with `maxlen`\nYou can use `maxlen` to define the maximum amount of items that the dictionary can store. It is useful for defining fixed size buffers.\n\n```python\nfrom circular_dict import CircularDict\n\n# Initialize a CircularDict with a maximum length of 3\nmy_buffer = CircularDict(maxlen=3)\n\n# Fill it with 3 items\nmy_buffer['item1'] = 'value1'\nmy_buffer['item2'] = 'value2'\nmy_buffer['item3'] = 'value3'\n\nprint(f\"When filling it: {circ_dict}\")\n\n# Add another item\nmy_buffer['item4'] = 'value4'\n\nprint(f\"After adding an element beyond maxlen: {circ_dict}\")\n```\n\nOutput:\n```bash\nWhen filling it: {'item1': 'value1', 'item2': 'value2', 'item3': 'value3'}\nAfter adding an element beyond maxlen: {'item2': 'value2', 'item3': 'value3', 'item4': 'value4'}\n```\n\n### Example with `maxsize_bytes`\nYou can use `maxsize_bytes` to define the maximum amount of memory that the `dict` can store. It is particularly beneficial when defining **caches**, to prevent **memory overflows**.\n\n```python\nfrom circular_dict import CircularDict\nimport numpy as np\nimport sys\n\n# Initialize a CircularDict with a maximum length of 100KB\nmy_buffer = CircularDict(maxsize_bytes=100*1024)\n\n# Add two arrays of ~40Kb (10*1024*4 bytes (int32) + 5 bytes (chars) + 100 bytes (numpy structure) + 50 bytes (str structure))\nmy_buffer['item1'] = np.zeros((10, 1024), dtype=np.int32)\nmy_buffer['item2'] = np.ones((10, 1024), dtype=np.int32)\n\nprint(f\"{len(my_buffer)} Elements {tuple(my_buffer.keys())}. Dict size: {my_buffer.current_size/1024} Kb\")\n\n# Add a new element of ~32Kb will delete oldest elements ('item1') until fitting in the `dict`.\nmy_buffer['item3'] = np.ones((8, 1024), dtype=np.int32)\n\nprint(f\"{len(my_buffer)} Elements {tuple(my_buffer.keys())}. Dict size: {my_buffer.current_size/1024} Kb\")\n\n# Create an element of ~160Kb (bigger than the defined maximum storage) to trigger a MemoryError\ntoo_big_array = np.ones((40, 1024), dtype=np.int32)\ntry:\n # Try to add it to the dict\n my_buffer['item4'] = too_big_array\nexcept MemoryError:\n print(f\"Cannot add an element with {sys.getsizeof(too_big_array)/1024}Kb in a dict with\"\\\n f\"maxsize_bytes of {my_buffer.maxsize_bytes/1024} Kb. Current elements {tuple(my_buffer.keys())}\")\n```\n\nOutput\n\n```bash\n2 Elements ('item1', 'item2'). Dict size: 80.35 Kb\n2 Elements ('item2', 'item3'). Dict size: 72.35 Kb\nCannot add an element with 160.12Kb in a dict with maxsize_bytes of 100.0 Kb. Current elements ('item2', 'item3')\n```\n\nPlease remember that the `maxsize_bytes` parameter considers the **total** memory footprint, including the sizes of _keys_ and _values_. If you try to add an item that exceeds the `maxsize_bytes`, a `MemoryError` will be raised.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "CircularDict is a high-performance Python data structure that blends the functionality of dictionaries and circular buffers. Inheriting the usage of traditional dictionaries, it allows you to define constraints on size and memory usage. This way, the CircularDict will be always up-to-date with the last N added elements, ensuring that neither the maximum length nor the memory usage limit is exceeded. It is ideal for caching large data structures while maintaining control over memory footprint.",
"version": "1.9",
"project_urls": {
"Homepage": "https://github.com/Eric-Canas/CircularDict"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d87b9da30fa823f020f3dc39bdee7a381dd3cae877933100a75700edb630fa8b",
"md5": "883a3ae87d69eb9e2f349de0c5df1b91",
"sha256": "f7936bb7487c93fe478119e47164777af9bf95c8fa4788f20ea41d4c00cdcea9"
},
"downloads": -1,
"filename": "circular_dict-1.9-py3-none-any.whl",
"has_sig": false,
"md5_digest": "883a3ae87d69eb9e2f349de0c5df1b91",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 6573,
"upload_time": "2024-05-12T20:40:32",
"upload_time_iso_8601": "2024-05-12T20:40:32.830289Z",
"url": "https://files.pythonhosted.org/packages/d8/7b/9da30fa823f020f3dc39bdee7a381dd3cae877933100a75700edb630fa8b/circular_dict-1.9-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5cdd0c23c95fc6cc02e8566957d15063bea03d2422b4a095a544d4279b5f2373",
"md5": "e5b85e3b02d959971b079ed340187130",
"sha256": "dc93ea0b5a79de6235a63b734446f1c3fa57498ff9791e7763a232558baa2f7a"
},
"downloads": -1,
"filename": "circular_dict-1.9.tar.gz",
"has_sig": false,
"md5_digest": "e5b85e3b02d959971b079ed340187130",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 6178,
"upload_time": "2024-05-12T20:40:34",
"upload_time_iso_8601": "2024-05-12T20:40:34.592270Z",
"url": "https://files.pythonhosted.org/packages/5c/dd/0c23c95fc6cc02e8566957d15063bea03d2422b4a095a544d4279b5f2373/circular_dict-1.9.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-12 20:40:34",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Eric-Canas",
"github_project": "CircularDict",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "circular-dict"
}
