# marinate
`marinate` caches function calls to your disk. This lets you memoize operations _across runs_ of a program. So even if your program terminates, you can run it again without re-invoking slow or expensive functions.
```python
from marinate import marinate
@marinate
def my_slow_fn(input):
# Function that's slow or expensive
```
**Features:**
- Thread-safe
- Supports asynchronous functions
- Supports in-memory caching in addition to on-disk
- Uses Python's built-in `pickle` module under the hood
## Installation
```bash
> pip install marinate
```
## Usage
To marinate a function just add the `@marinate` decorator to it:
```python
from marinate import marinate
@marinate
def my_fn():
# Does something
```
If you later modify the function, you can invalidate its cache using the `overwrite` parameter:
```python
@marinate(overwrite=True)
def my_fn():
# Does something different
```
This will force a function call and overwrite whatever's in the cache.
### Cache location
By default, cached function calls are stored in the same directory as the files they're defined in. You'll find them in a folder called `.marinade`.
However, you can change where a function call is stored by setting the `cache_dir` parameter in the decorator:
```python
@marinate(cache_dir="~/pickle_jar")
def my_fn():
# ...
```
After running your program, you'll see pickle (`.pkl`) files appear in the directory you specified.
You can also specify a cache directory for _all_ marinated functions:
```python
from marinate import set_cache_dir
set_cache_dir("~/pickle_jar")
@marinate
def my_fn():
# Output will be stored in ~/pickle_jar
```
### Disk vs. RAM
You can also use `marinate` to cache functions in-memory rather than on-disk. This is preferred if you only care about memoizing operations _within_ a single run of a program, rather than _across_ runs.
```python
@marinate(store="memory")
def my_fn():
# Do things
```
In other words, `@marinate` is a drop-in replacement for Python's built-in `@cache` decorator.
## Limitations
Only certain functions can and should be marinated:
1. Functions that return an unpickleable object, e.g. sockets or database connections, cannot be cached.
2. Functions _must_ be pure and deterministic. Meaning they should produce the same output given the same input, and should not have side-effects.
3. Function arguments must be hashable.
4. Don't marinate functions that take less than a second. The disk I/O overhead will negate the benefits of caching.
## Authors
Created by [Paul Bogdan](https://github.com/paulcbogdan) and [Jonathan Shobrook](https://github.com/shobrook) to make our lives easier when iterating on data/training pipelines.
Raw data
{
"_id": null,
"home_page": "https://github.com/shobrook/marinate",
"name": "marinate",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3",
"maintainer_email": null,
"keywords": null,
"author": "shobrook",
"author_email": "shobrookj@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/90/c9/903932ed0b0bca114006c7dfdf675f43923d1df462821bf8cdfc1847546c/marinate-1.0.1.tar.gz",
"platform": null,
"description": "# marinate\n\n`marinate` caches function calls to your disk. This lets you memoize operations _across runs_ of a program. So even if your program terminates, you can run it again without re-invoking slow or expensive functions.\n\n```python\nfrom marinate import marinate\n\n@marinate\ndef my_slow_fn(input):\n # Function that's slow or expensive\n```\n\n**Features:**\n\n- Thread-safe\n- Supports asynchronous functions\n- Supports in-memory caching in addition to on-disk\n- Uses Python's built-in `pickle` module under the hood\n\n## Installation\n\n```bash\n> pip install marinate\n```\n\n## Usage\n\nTo marinate a function just add the `@marinate` decorator to it:\n\n```python\nfrom marinate import marinate\n\n@marinate\ndef my_fn():\n # Does something\n```\n\nIf you later modify the function, you can invalidate its cache using the `overwrite` parameter:\n\n```python\n@marinate(overwrite=True)\ndef my_fn():\n # Does something different\n```\n\nThis will force a function call and overwrite whatever's in the cache.\n\n### Cache location\n\nBy default, cached function calls are stored in the same directory as the files they're defined in. You'll find them in a folder called `.marinade`.\n\nHowever, you can change where a function call is stored by setting the `cache_dir` parameter in the decorator:\n\n```python\n@marinate(cache_dir=\"~/pickle_jar\")\ndef my_fn():\n # ...\n```\n\nAfter running your program, you'll see pickle (`.pkl`) files appear in the directory you specified.\n\nYou can also specify a cache directory for _all_ marinated functions:\n\n```python\nfrom marinate import set_cache_dir\n\nset_cache_dir(\"~/pickle_jar\")\n\n@marinate\ndef my_fn():\n # Output will be stored in ~/pickle_jar\n```\n\n### Disk vs. RAM\n\nYou can also use `marinate` to cache functions in-memory rather than on-disk. This is preferred if you only care about memoizing operations _within_ a single run of a program, rather than _across_ runs.\n\n```python\n@marinate(store=\"memory\")\ndef my_fn():\n # Do things\n```\n\nIn other words, `@marinate` is a drop-in replacement for Python's built-in `@cache` decorator.\n\n## Limitations\n\nOnly certain functions can and should be marinated:\n\n1. Functions that return an unpickleable object, e.g. sockets or database connections, cannot be cached.\n2. Functions _must_ be pure and deterministic. Meaning they should produce the same output given the same input, and should not have side-effects.\n3. Function arguments must be hashable.\n4. Don't marinate functions that take less than a second. The disk I/O overhead will negate the benefits of caching.\n\n## Authors\n\nCreated by [Paul Bogdan](https://github.com/paulcbogdan) and [Jonathan Shobrook](https://github.com/shobrook) to make our lives easier when iterating on data/training pipelines.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A Python package called marinate",
"version": "1.0.1",
"project_urls": {
"Homepage": "https://github.com/shobrook/marinate"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "90c9903932ed0b0bca114006c7dfdf675f43923d1df462821bf8cdfc1847546c",
"md5": "7aa1d837326d3119d29b58b4ada23a31",
"sha256": "a55c5f9583a9fb7598c368467741eb159b273eac0663c03baf3bcd134bde9b40"
},
"downloads": -1,
"filename": "marinate-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "7aa1d837326d3119d29b58b4ada23a31",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3",
"size": 4421,
"upload_time": "2024-12-22T07:47:12",
"upload_time_iso_8601": "2024-12-22T07:47:12.261288Z",
"url": "https://files.pythonhosted.org/packages/90/c9/903932ed0b0bca114006c7dfdf675f43923d1df462821bf8cdfc1847546c/marinate-1.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-22 07:47:12",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "shobrook",
"github_project": "marinate",
"github_not_found": true,
"lcname": "marinate"
}
JSON
