# Django EOSE
**Django Encrypted Object Search Engine**
An efficient search engine for encrypted or derived fields in Django querysets, with support for parallel processing, smart batching, and result caching.
`django-eose` is ideal for scenarios where you need to search data that is encrypted in the database or derived from other fields, providing high performance even on large datasets.
---
## Key Features
- **Parallel Search:** supports execution using processes, threads, or synchronous mode.
- **Smart Batching:** batch size automatically adapts to available memory.
- **Related Field Search:** search can be performed on fields of related objects, e.g., `order__client`.
- **Result Caching:** frequently searched results are cached for faster repeated queries.
- **Optimized for Large Datasets:** particularly useful for large datasets and encrypted fields.
---
## Installation
Install easily via `pip`:
```bash
pip install django-eose
```
## Requirements
- Python 3.10 or higher
- Django 5.2.5
- sqlparse==0.5.3
- asgiref==3.9.1
- psutil==7.0.0
## Model Configuration
To use django-eose, your encrypted fields must provide Getters so the package can access the decrypted data.
Example model using Fernet encryption:
```python
from django.db import models
from cryptography.fernet import Fernet
AES_KEY = b"<your_key_here>"
class Client(models.Model):
_encrypted_name = models.BinaryField()
_encrypted_email = models.BinaryField()
# Method to decrypt
def _decrypt_field(self, encrypted_value):
return Fernet(AES_KEY).decrypt(encrypted_value).decode()
# Method to encrypt
def _encrypt_field(self, value):
return Fernet(AES_KEY).encrypt(value.encode())
# Creates properties that handle encryption/decryption
@staticmethod
def _property(field_name):
def getter(self):
return self._decrypt_field(getattr(self, field_name))
def setter(self, value):
setattr(self, field_name, self._encrypt_field(value))
return property(getter, setter)
# Fields accessible as normal attributes
name = _property('_encrypted_name')
email = _property('_encrypted_email')
```
⚠️ You interact with name and email like regular fields, while encryption/decryption happens transparently.
## Usage
Import the search_queryset function and perform searches:
```python
from django_eose import search_queryset
from orders.models import OrderItem
# Example: search for "john" in related client fields
results = search_queryset(
search="john",
queryset=OrderItem.objects.all(),
related_field="order__client",
fields=("name", "email"),
only_fields=("_encrypted_name", "_encrypted_email"),
executor="processes",
max_batch_size=1_000_000
)
```
## `search_queryset` Parameters
- search: search term (case-insensitive).
- queryset: Django queryset to search in.
- related_field: path to a related object using __ notation.
- fields: fields of the related object to inspect.
- only_fields: fields to load with .only() for optimization (optional).
- executor: "processes", "threads", or "sync" (default: "processes").
- cache_timeout: cache duration in seconds (default: 600).
- imap_chunksize: chunk size per worker (default: 10240).
- memory_fraction: fraction of available memory for batching (default: 0.60).
- avg_obj_size_bytes: estimated average object size in bytes (optional).
- max_workers: maximum number of parallel workers (optional).
- max_batch_size: maximum number of objects per batch.
Refer to `search_queryset` for full parameter details.
## Default Settings
`django-eose` defines default settings in `django_eose/settings.py`:
- MEMORY_FRACTION
- IMAP_CHUNKSIZE
- EXECUTOR
- CACHE_TIMEOUT
- AVG_OBJ_SIZE_FALLBACK
- MIN_BATCH_SIZE
- MAX_BATCH_SIZE
## License
MIT © 2025 Paulo Otávio Castoldi
## Links
[Source](https://github.com/paulootaviodev/django-eose)
Raw data
{
"_id": null,
"home_page": null,
"name": "django-eose",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "django, search, multiprocessing, decrypted, cache",
"author": null,
"author_email": "Paulo Ot\u00e1vio Castoldi <paulootavio343@outlook.com>",
"download_url": "https://files.pythonhosted.org/packages/fa/fd/2564025555ad2bf10e1928e73292d6e9b87a93602c7c481fa83e9e865313/django_eose-0.2.2b1.tar.gz",
"platform": null,
"description": "# Django EOSE\n\n**Django Encrypted Object Search Engine** \nAn efficient search engine for encrypted or derived fields in Django querysets, with support for parallel processing, smart batching, and result caching.\n\n`django-eose` is ideal for scenarios where you need to search data that is encrypted in the database or derived from other fields, providing high performance even on large datasets.\n\n---\n\n## Key Features\n\n- **Parallel Search:** supports execution using processes, threads, or synchronous mode. \n- **Smart Batching:** batch size automatically adapts to available memory. \n- **Related Field Search:** search can be performed on fields of related objects, e.g., `order__client`. \n- **Result Caching:** frequently searched results are cached for faster repeated queries. \n- **Optimized for Large Datasets:** particularly useful for large datasets and encrypted fields.\n\n---\n\n## Installation\n\nInstall easily via `pip`:\n\n```bash\npip install django-eose\n```\n\n## Requirements\n\n- Python 3.10 or higher\n- Django 5.2.5\n- sqlparse==0.5.3\n- asgiref==3.9.1\n- psutil==7.0.0\n\n## Model Configuration\n\nTo use django-eose, your encrypted fields must provide Getters so the package can access the decrypted data.\n\nExample model using Fernet encryption:\n\n```python\nfrom django.db import models\nfrom cryptography.fernet import Fernet\n\nAES_KEY = b\"<your_key_here>\"\n\nclass Client(models.Model):\n _encrypted_name = models.BinaryField()\n _encrypted_email = models.BinaryField()\n\n # Method to decrypt\n def _decrypt_field(self, encrypted_value):\n return Fernet(AES_KEY).decrypt(encrypted_value).decode()\n\n # Method to encrypt\n def _encrypt_field(self, value):\n return Fernet(AES_KEY).encrypt(value.encode())\n \n # Creates properties that handle encryption/decryption\n @staticmethod\n def _property(field_name):\n def getter(self):\n return self._decrypt_field(getattr(self, field_name))\n \n def setter(self, value):\n setattr(self, field_name, self._encrypt_field(value))\n\n return property(getter, setter)\n \n # Fields accessible as normal attributes\n name = _property('_encrypted_name')\n email = _property('_encrypted_email')\n\n```\n\u26a0\ufe0f You interact with name and email like regular fields, while encryption/decryption happens transparently.\n\n## Usage\n\nImport the search_queryset function and perform searches:\n\n```python\nfrom django_eose import search_queryset\nfrom orders.models import OrderItem\n\n# Example: search for \"john\" in related client fields\nresults = search_queryset(\n search=\"john\",\n queryset=OrderItem.objects.all(),\n related_field=\"order__client\",\n fields=(\"name\", \"email\"),\n only_fields=(\"_encrypted_name\", \"_encrypted_email\"),\n executor=\"processes\",\n max_batch_size=1_000_000\n)\n```\n\n## `search_queryset` Parameters\n\n- search: search term (case-insensitive).\n- queryset: Django queryset to search in.\n- related_field: path to a related object using __ notation.\n- fields: fields of the related object to inspect.\n- only_fields: fields to load with .only() for optimization (optional).\n- executor: \"processes\", \"threads\", or \"sync\" (default: \"processes\").\n- cache_timeout: cache duration in seconds (default: 600).\n- imap_chunksize: chunk size per worker (default: 10240).\n- memory_fraction: fraction of available memory for batching (default: 0.60).\n- avg_obj_size_bytes: estimated average object size in bytes (optional).\n- max_workers: maximum number of parallel workers (optional).\n- max_batch_size: maximum number of objects per batch.\n\nRefer to `search_queryset` for full parameter details.\n\n## Default Settings\n\n`django-eose` defines default settings in `django_eose/settings.py`:\n\n- MEMORY_FRACTION\n- IMAP_CHUNKSIZE\n- EXECUTOR\n- CACHE_TIMEOUT\n- AVG_OBJ_SIZE_FALLBACK\n- MIN_BATCH_SIZE\n- MAX_BATCH_SIZE\n\n## License\n\nMIT \u00a9 2025 Paulo Ot\u00e1vio Castoldi\n\n## Links\n\n[Source](https://github.com/paulootaviodev/django-eose)\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Django Encrypted Object Search Engine.",
"version": "0.2.2b1",
"project_urls": {
"Homepage": "https://github.com/paulootaviodev/django-eose"
},
"split_keywords": [
"django",
" search",
" multiprocessing",
" decrypted",
" cache"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "39bb2d31a320bb3eca52be67ab0252feb8d2a7a43559d1dd911b9aa64776bac9",
"md5": "6cb7bdcbe226a5498fb66aa032133043",
"sha256": "c9f3abd5d824dc78620b4bdba091d52ccae5817f75218cb485d021072ba379e2"
},
"downloads": -1,
"filename": "django_eose-0.2.2b1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6cb7bdcbe226a5498fb66aa032133043",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 8701,
"upload_time": "2025-08-13T23:35:20",
"upload_time_iso_8601": "2025-08-13T23:35:20.878021Z",
"url": "https://files.pythonhosted.org/packages/39/bb/2d31a320bb3eca52be67ab0252feb8d2a7a43559d1dd911b9aa64776bac9/django_eose-0.2.2b1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "fafd2564025555ad2bf10e1928e73292d6e9b87a93602c7c481fa83e9e865313",
"md5": "1fc1700b13a6cb09be4b70c2087254d5",
"sha256": "0c282c23cf44267417bbbdb4fde8e727f8e5f98204aeefb423a7a9119a30b266"
},
"downloads": -1,
"filename": "django_eose-0.2.2b1.tar.gz",
"has_sig": false,
"md5_digest": "1fc1700b13a6cb09be4b70c2087254d5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 7574,
"upload_time": "2025-08-13T23:35:23",
"upload_time_iso_8601": "2025-08-13T23:35:23.586854Z",
"url": "https://files.pythonhosted.org/packages/fa/fd/2564025555ad2bf10e1928e73292d6e9b87a93602c7c481fa83e9e865313/django_eose-0.2.2b1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-13 23:35:23",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "paulootaviodev",
"github_project": "django-eose",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "asgiref",
"specs": [
[
"==",
"3.9.1"
]
]
},
{
"name": "Django",
"specs": [
[
"==",
"5.2.5"
]
]
},
{
"name": "psutil",
"specs": [
[
"==",
"7.0.0"
]
]
},
{
"name": "setuptools",
"specs": [
[
"==",
"78.1.1"
]
]
},
{
"name": "sqlparse",
"specs": [
[
"==",
"0.5.3"
]
]
},
{
"name": "wheel",
"specs": [
[
"==",
"0.45.1"
]
]
}
],
"lcname": "django-eose"
}
JSON
