[](https://github.com/jazzband/django-eav2/actions/workflows/test.yml)
[](https://codecov.io/gh/jazzband/django-eav2)
[](https://pypi.org/project/django-eav2/)
[](https://pypi.org/project/django-eav2/)
[](https://jazzband.co/)
## Django EAV 2 - Entity-Attribute-Value storage for Django
Django EAV 2 is a fork of django-eav (which itself was derived from eav-django).
You can find documentation <a href="https://django-eav2.rtfd.io">here</a>.
## What is EAV anyway?
> Entity–attribute–value model (EAV) is a data model to encode, in a space-efficient manner, entities where the number of attributes (properties, parameters) that can be used to describe them is potentially vast, but the number that will actually apply to a given entity is relatively modest. Such entities correspond to the mathematical notion of a sparse matrix. (Wikipedia)
Data in EAV is stored as a 3-tuple (typically corresponding to three distinct tables):
- The entity: the item being described, e.g. `Person(name='Mike')`.
- The attribute: often a foreign key into a table of attributes, e.g. `Attribute(slug='height', datatype=FLOAT)`.
- The value of the attribute, with links both an attribute and an entity, e.g. `Value(value_float=15.5, person=mike, attr=height)`.
Entities in **django-eav2** are your typical Django model instances. Attributes (name and type) are stored in their own table, which makes it easy to manipulate the list of available attributes in the system. Values are an intermediate table between attributes and entities, each instance holding a single value.
This implementation also makes it easy to edit attributes in Django Admin and form instances.
You will find detailed description of the EAV here:
- [Wikipedia - Entity–attribute–value model](https://en.wikipedia.org/wiki/Entity%E2%80%93attribute%E2%80%93value_model)
## EAV - The Good, the Bad or the Ugly?
EAV is a trade-off between flexibility and complexity. As such, it should not be thought of as either an amelioration pattern, nor an anti-pattern. It is more of a [gray pattern](https://wiki.c2.com/?GreyPattern) - it exists in some context, to solve certain set of problems. When used appropriately, it can introduce great flexibility, cut prototyping time or deacrease complexity. When used carelessly, however, it can complicate database schema, degrade the performance and make maintenance hard. **As with every tool, it should not be overused.** In the following paragraphs we briefly discuss the pros, the cons and pointers to keep in mind when using EAV.
### When to use EAV?
Originally, EAV was introduced to workaround a problem which cannot be easily solved within relational model. In order to achieve this, EAV bypasses normal schema restrictions. Some refer to this as an example of the [inner-platform effect](https://en.wikipedia.org/wiki/Inner-platform_effect#Examples). Naturally, in such scenarios RDMS resources cannot be used efficiently.
Typical application of the EAV model sets to solve the problem of sparse data with a large number of applicable attributes, but only a small fraction that applies to a given entity that may not be known beforehand. Consider the classic example:
> A problem that data modelers commonly encounter in the biomedical domain is organizing and storing highly diverse and heterogeneous data. For example, a single patient may have thousands of applicable descriptive parameters, all of which need to be easily accessible in an electronic patient record system. These requirements pose significant modeling and implementation challenges. [1]
And:
> [...] what do you do when you have customers that demand real-time, on-demand addition of attributes that they want to store? In one of the systems I manage, our customers wanted to do exactly this. Since we run a SaaS (software as a service) application, we have many customers across several different industries, who in turn want to use our system to store different types of information about _their_ customers. A salon chain might want to record facts such as 'hair color,' 'hair type,' and 'haircut frequency'; while an investment company might want to record facts such as 'portfolio name,' 'last portfolio adjustment date,' and 'current portfolio balance.' [2]
In both of these problems we have to deal with sparse and heterogeneous properties that apply only to potentially different subsets of particular entities. Applying EAV to a sub-schema of the database allows to model the desired behaviour. Traditional solution would involves wide tables with many columns storing NULL values for attributes that don't apply to an entity.
Very common use case for EAV are custom product attributes in E-commerce implementations, such as Magento. [3]
As a rule of thumb, EAV can be used when:
- Model attributes are to be added and removed by end users (or are unknowable in some different way). EAV supports these without ALTER TABLE statements and allows the attributes to be strongly typed and easily searchable.
- There will be many attributes and values are sparse, in contrast to having tables with mostly-null columns.
- The data is highly dynamic/volatile/vulnerable to change. This problem is present in the second example given above. Other example would be rapidly evolving system, such as a prototype with constantly changing requirements.
- We want to store meta-data or supporting information, e.g. to customize system's behavior.
- Numerous classes of data need to be represented, each class has a limited number of attributes, but the number of instances of each class is very small.
- We want to minimise programmer's input when changing the data model.
For more throughout discussion on the appropriate use-cases see:
1. [Wikipedia - Scenarios that are appropriate for EAV modeling](https://en.wikipedia.org/wiki/Entity%E2%80%93attribute%E2%80%93value_model#Scenarios_that_are_appropriate_for_EAV_modeling)
2. [StackOverflow - Entity Attribute Value Database vs. strict Relational Model E-commerce](https://stackoverflow.com/questions/870808/entity-attribute-value-database-vs-strict-relational-model-ecommerce)
3. [WikiWikiWeb - Generic Data Model](https://wiki.c2.com/?GenericDataModel)
## When to avoid it?
As we outlined in the opening section, EAV is a trade-off. It should not be used when:
##### 1. System is performance critical
> Attribute-centric query is inherently more difficult when data are stored in EAV form than when they are stored conventionally. [4]
In general, the more structured your data model, the more efficiently you can deal with it. Therefore, loose data storage such as EAV has obvious trade-off in performance. Specifically, application of the EAV model makes performing JOINs on tables more complicated.
##### 2. Low complexity/low maintenance cost is of priority
EAV complicates data model by splitting information across tables. This increases conceptual complexity as well as SQL statements required to query the data. In consequence, optimization in one area that also makes the system harder to understand and maintain.
However, it is important to note that:
> An EAV design should be employed only for that sub-schema of a database where sparse attributes need to be modeled: even here, they need to be supported by third normal form metadata tables. There are relatively few database-design problems where sparse attributes are encountered: this is why the circumstances where EAV design is applicable are relatively rare. [1]
## Alternatives
In some use-cases, JSONB (binary JSON data) datatype (Postgres 9.4+ and analogous in other RDMSs) can be used as an alternative to EAV. JSONB supports indexing, which amortizes performance trade-off. It's important to keep in mind that JSONB is not RDMS-agnostic solution and has it's own problems, such as typing.
## Installation
Install with pip
```bash
pip install django-eav2
```
## Configuration
Add `eav` to `INSTALLED_APPS` in your settings.
```python
INSTALLED_APPS = [
...
'eav',
]
```
Add `django.db.models.UUIDField` or `django.db.models.BigAutoField` as value of `EAV2_PRIMARY_KEY_FIELD` in your settings
``` python
EAV2_PRIMARY_KEY_FIELD = "django.db.models.UUIDField" # as example
```
### Note: Primary key mandatory modification field
If the primary key of eav models are to be modified (UUIDField -> BigAutoField, BigAutoField -> UUIDField) in the middle of the project when the migrations are already done, you have to change the value of `EAV2_PRIMARY_KEY_FIELD` in your settings.
##### Step 1
Change the value of `EAV2_PRIMARY_KEY_FIELD` into `django.db.models.CharField` in your settings.
```python
EAV2_PRIMARY_KEY_FIELD = "django.db.models.CharField"
```
Run the migrations
```bash
python manage.py makemigrations
python manage.py migrate
```
##### Step 2
Change the value of `EAV2_PRIMARY_KEY_FIELD` into the desired value (`django.db.models.BigAutoField` or `django.db.models.UUIDField`) in your settings.
```python
EAV2_PRIMARY_KEY_FIELD = "django.db.models.BigAutoField" # as example
```
Run again the migrations.
```bash
python manage.py makemigrations
python manage.py migrate
```
### Note: Django 2.2 Users
Since `models.JSONField()` isn't supported in Django 2.2, we use [django-jsonfield-backport](https://github.com/laymonage/django-jsonfield-backport) to provide [JSONField](https://docs.djangoproject.com/en/dev/releases/3.1/#jsonfield-for-all-supported-database-backends) functionality.
This requires adding `django_jsonfield_backport` to your `INSTALLED_APPS` as well.
```python
INSTALLED_APPS = [
...
'eav',
'django_jsonfield_backport',
]
```
## Getting started
**Step 1.** Register a model:
```python
import eav
eav.register(Supplier)
```
or with decorators:
```python
from eav.decorators import register_eav
@register_eav
class Supplier(models.Model):
...
```
**Step 2.** Create an attribute:
```python
Attribute.objects.create(name='City', datatype=Attribute.TYPE_TEXT)
```
**Step 3.** That’s it! You’re ready to go:
```python
supplier.eav.city = 'London'
supplier.save()
Supplier.objects.filter(eav__city='London')
# = <EavQuerySet [<Supplier: Supplier object (1)>]>
```
**What next? Check out the <a href="https://django-eav2.readthedocs.io/en/latest/#documentation">documentation</a>.**
---
### References
[1] Exploring Performance Issues for a Clinical Database Organized Using an Entity-Attribute-Value Representation, https://doi.org/10.1136/jamia.2000.0070475 <br>
[2] What is so bad about EAV, anyway?, https://sqlblog.org/2009/11/19/what-is-so-bad-about-eav-anyway <br>
[3] Magento for Developers: Part 7—Advanced ORM: Entity Attribute Value, https://devdocs.magento.com/guides/m1x/magefordev/mage-for-dev-7.html <br>
[4] Data Extraction and Ad Hoc Query of an Entity— Attribute— Value Database, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC61332/
Raw data
{
"_id": null,
"home_page": "https://github.com/jazzband/django-eav2",
"name": "django-eav2",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4.0",
"maintainer_email": "",
"keywords": "django,django-eav2,database,eav,sql",
"author": "Mauro Lizaur",
"author_email": "mauro@sdf.org",
"download_url": "https://files.pythonhosted.org/packages/58/56/7853c664b1fdd7319d8398922f8a41c989226cab18b9c696efa3c2225fe5/django_eav2-1.6.0.tar.gz",
"platform": null,
"description": "[](https://github.com/jazzband/django-eav2/actions/workflows/test.yml)\n[](https://codecov.io/gh/jazzband/django-eav2)\n[](https://pypi.org/project/django-eav2/)\n[](https://pypi.org/project/django-eav2/)\n[](https://jazzband.co/)\n\n## Django EAV 2 - Entity-Attribute-Value storage for Django\n\nDjango EAV 2 is a fork of django-eav (which itself was derived from eav-django).\nYou can find documentation <a href=\"https://django-eav2.rtfd.io\">here</a>.\n\n## What is EAV anyway?\n\n> Entity\u2013attribute\u2013value model (EAV) is a data model to encode, in a space-efficient manner, entities where the number of attributes (properties, parameters) that can be used to describe them is potentially vast, but the number that will actually apply to a given entity is relatively modest. Such entities correspond to the mathematical notion of a sparse matrix. (Wikipedia)\n\nData in EAV is stored as a 3-tuple (typically corresponding to three distinct tables):\n\n- The entity: the item being described, e.g. `Person(name='Mike')`.\n- The attribute: often a foreign key into a table of attributes, e.g. `Attribute(slug='height', datatype=FLOAT)`.\n- The value of the attribute, with links both an attribute and an entity, e.g. `Value(value_float=15.5, person=mike, attr=height)`.\n\nEntities in **django-eav2** are your typical Django model instances. Attributes (name and type) are stored in their own table, which makes it easy to manipulate the list of available attributes in the system. Values are an intermediate table between attributes and entities, each instance holding a single value.\nThis implementation also makes it easy to edit attributes in Django Admin and form instances.\n\nYou will find detailed description of the EAV here:\n\n- [Wikipedia - Entity\u2013attribute\u2013value model](https://en.wikipedia.org/wiki/Entity%E2%80%93attribute%E2%80%93value_model)\n\n## EAV - The Good, the Bad or the Ugly?\n\nEAV is a trade-off between flexibility and complexity. As such, it should not be thought of as either an amelioration pattern, nor an anti-pattern. It is more of a [gray pattern](https://wiki.c2.com/?GreyPattern) - it exists in some context, to solve certain set of problems. When used appropriately, it can introduce great flexibility, cut prototyping time or deacrease complexity. When used carelessly, however, it can complicate database schema, degrade the performance and make maintenance hard. **As with every tool, it should not be overused.** In the following paragraphs we briefly discuss the pros, the cons and pointers to keep in mind when using EAV.\n\n### When to use EAV?\n\nOriginally, EAV was introduced to workaround a problem which cannot be easily solved within relational model. In order to achieve this, EAV bypasses normal schema restrictions. Some refer to this as an example of the [inner-platform effect](https://en.wikipedia.org/wiki/Inner-platform_effect#Examples). Naturally, in such scenarios RDMS resources cannot be used efficiently.\n\nTypical application of the EAV model sets to solve the problem of sparse data with a large number of applicable attributes, but only a small fraction that applies to a given entity that may not be known beforehand. Consider the classic example:\n\n> A problem that data modelers commonly encounter in the biomedical domain is organizing and storing highly diverse and heterogeneous data. For example, a single patient may have thousands of applicable descriptive parameters, all of which need to be easily accessible in an electronic patient record system. These requirements pose significant modeling and implementation challenges. [1]\n\nAnd:\n\n> [...] what do you do when you have customers that demand real-time, on-demand addition of attributes that they want to store? In one of the systems I manage, our customers wanted to do exactly this. Since we run a SaaS (software as a service) application, we have many customers across several different industries, who in turn want to use our system to store different types of information about _their_ customers. A salon chain might want to record facts such as 'hair color,' 'hair type,' and 'haircut frequency'; while an investment company might want to record facts such as 'portfolio name,' 'last portfolio adjustment date,' and 'current portfolio balance.' [2]\n\nIn both of these problems we have to deal with sparse and heterogeneous properties that apply only to potentially different subsets of particular entities. Applying EAV to a sub-schema of the database allows to model the desired behaviour. Traditional solution would involves wide tables with many columns storing NULL values for attributes that don't apply to an entity.\n\nVery common use case for EAV are custom product attributes in E-commerce implementations, such as Magento. [3]\n\nAs a rule of thumb, EAV can be used when:\n\n- Model attributes are to be added and removed by end users (or are unknowable in some different way). EAV supports these without ALTER TABLE statements and allows the attributes to be strongly typed and easily searchable.\n- There will be many attributes and values are sparse, in contrast to having tables with mostly-null columns.\n- The data is highly dynamic/volatile/vulnerable to change. This problem is present in the second example given above. Other example would be rapidly evolving system, such as a prototype with constantly changing requirements.\n- We want to store meta-data or supporting information, e.g. to customize system's behavior.\n- Numerous classes of data need to be represented, each class has a limited number of attributes, but the number of instances of each class is very small.\n- We want to minimise programmer's input when changing the data model.\n\nFor more throughout discussion on the appropriate use-cases see:\n\n1. [Wikipedia - Scenarios that are appropriate for EAV modeling](https://en.wikipedia.org/wiki/Entity%E2%80%93attribute%E2%80%93value_model#Scenarios_that_are_appropriate_for_EAV_modeling)\n2. [StackOverflow - Entity Attribute Value Database vs. strict Relational Model E-commerce](https://stackoverflow.com/questions/870808/entity-attribute-value-database-vs-strict-relational-model-ecommerce)\n3. [WikiWikiWeb - Generic Data Model](https://wiki.c2.com/?GenericDataModel)\n\n## When to avoid it?\n\nAs we outlined in the opening section, EAV is a trade-off. It should not be used when:\n\n##### 1. System is performance critical\n\n> Attribute-centric query is inherently more difficult when data are stored in EAV form than when they are stored conventionally. [4]\n\nIn general, the more structured your data model, the more efficiently you can deal with it. Therefore, loose data storage such as EAV has obvious trade-off in performance. Specifically, application of the EAV model makes performing JOINs on tables more complicated.\n\n##### 2. Low complexity/low maintenance cost is of priority\n\nEAV complicates data model by splitting information across tables. This increases conceptual complexity as well as SQL statements required to query the data. In consequence, optimization in one area that also makes the system harder to understand and maintain.\n\nHowever, it is important to note that:\n\n> An EAV design should be employed only for that sub-schema of a database where sparse attributes need to be modeled: even here, they need to be supported by third normal form metadata tables. There are relatively few database-design problems where sparse attributes are encountered: this is why the circumstances where EAV design is applicable are relatively rare. [1]\n\n## Alternatives\n\nIn some use-cases, JSONB (binary JSON data) datatype (Postgres 9.4+ and analogous in other RDMSs) can be used as an alternative to EAV. JSONB supports indexing, which amortizes performance trade-off. It's important to keep in mind that JSONB is not RDMS-agnostic solution and has it's own problems, such as typing.\n\n\n## Installation\n\nInstall with pip\n\n```bash\npip install django-eav2\n```\n\n## Configuration\n\nAdd `eav` to `INSTALLED_APPS` in your settings.\n\n```python\nINSTALLED_APPS = [\n ...\n 'eav',\n]\n```\n\nAdd `django.db.models.UUIDField` or `django.db.models.BigAutoField` as value of `EAV2_PRIMARY_KEY_FIELD` in your settings\n\n``` python\nEAV2_PRIMARY_KEY_FIELD = \"django.db.models.UUIDField\" # as example\n```\n\n### Note: Primary key mandatory modification field\n\nIf the primary key of eav models are to be modified (UUIDField -> BigAutoField, BigAutoField -> UUIDField) in the middle of the project when the migrations are already done, you have to change the value of `EAV2_PRIMARY_KEY_FIELD` in your settings.\n\n##### Step 1\n Change the value of `EAV2_PRIMARY_KEY_FIELD` into `django.db.models.CharField` in your settings.\n\n ```python\n EAV2_PRIMARY_KEY_FIELD = \"django.db.models.CharField\"\n ```\n\n Run the migrations\n\n ```bash\n python manage.py makemigrations\n python manage.py migrate\n ```\n\n##### Step 2\n Change the value of `EAV2_PRIMARY_KEY_FIELD` into the desired value (`django.db.models.BigAutoField` or `django.db.models.UUIDField`) in your settings.\n\n ```python\n EAV2_PRIMARY_KEY_FIELD = \"django.db.models.BigAutoField\" # as example\n ```\n\n Run again the migrations.\n\n```bash\n python manage.py makemigrations\n python manage.py migrate\n ```\n\n### Note: Django 2.2 Users\n\nSince `models.JSONField()` isn't supported in Django 2.2, we use [django-jsonfield-backport](https://github.com/laymonage/django-jsonfield-backport) to provide [JSONField](https://docs.djangoproject.com/en/dev/releases/3.1/#jsonfield-for-all-supported-database-backends) functionality.\n\nThis requires adding `django_jsonfield_backport` to your `INSTALLED_APPS` as well.\n\n```python\nINSTALLED_APPS = [\n ...\n 'eav',\n 'django_jsonfield_backport',\n]\n```\n\n## Getting started\n\n**Step 1.** Register a model:\n\n```python\nimport eav\neav.register(Supplier)\n```\n\nor with decorators:\n\n```python\nfrom eav.decorators import register_eav\n\n@register_eav\nclass Supplier(models.Model):\n ...\n```\n\n**Step 2.** Create an attribute:\n\n```python\nAttribute.objects.create(name='City', datatype=Attribute.TYPE_TEXT)\n```\n\n**Step 3.** That\u2019s it! You\u2019re ready to go:\n\n```python\nsupplier.eav.city = 'London'\nsupplier.save()\n\nSupplier.objects.filter(eav__city='London')\n# = <EavQuerySet [<Supplier: Supplier object (1)>]>\n```\n\n**What next? Check out the <a href=\"https://django-eav2.readthedocs.io/en/latest/#documentation\">documentation</a>.**\n\n---\n\n### References\n\n[1] Exploring Performance Issues for a Clinical Database Organized Using an Entity-Attribute-Value Representation, https://doi.org/10.1136/jamia.2000.0070475 <br>\n[2] What is so bad about EAV, anyway?, https://sqlblog.org/2009/11/19/what-is-so-bad-about-eav-anyway <br>\n[3] Magento for Developers: Part 7\u2014Advanced ORM: Entity Attribute Value, https://devdocs.magento.com/guides/m1x/magefordev/mage-for-dev-7.html <br>\n[4] Data Extraction and Ad Hoc Query of an Entity\u2014 Attribute\u2014 Value Database, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC61332/\n\n",
"bugtrack_url": null,
"license": "GNU Lesser General Public License (LGPL), Version 3",
"summary": "Entity-Attribute-Value storage for Django",
"version": "1.6.0",
"project_urls": {
"Homepage": "https://github.com/jazzband/django-eav2",
"Repository": "https://github.com/jazzband/django-eav2"
},
"split_keywords": [
"django",
"django-eav2",
"database",
"eav",
"sql"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "51f2c106cf0724db1a4b1dc1d8f8e10e7b559b4e2de8637db5a777283ec0d448",
"md5": "4a08c2c95b9d05b6d247600f197d2a1b",
"sha256": "fe21438c1cce930737c3c147564cb0d80edb19f95beedd464e8961601c8b3ebb"
},
"downloads": -1,
"filename": "django_eav2-1.6.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4a08c2c95b9d05b6d247600f197d2a1b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4.0",
"size": 52105,
"upload_time": "2024-03-14T14:28:45",
"upload_time_iso_8601": "2024-03-14T14:28:45.272513Z",
"url": "https://files.pythonhosted.org/packages/51/f2/c106cf0724db1a4b1dc1d8f8e10e7b559b4e2de8637db5a777283ec0d448/django_eav2-1.6.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "58567853c664b1fdd7319d8398922f8a41c989226cab18b9c696efa3c2225fe5",
"md5": "6876797b986d52ae7acbf6c19030ee8e",
"sha256": "4a25bb8dd3fb90a653a3c52ae152633ead3f386a3f432f49fdb1d7de56bada2f"
},
"downloads": -1,
"filename": "django_eav2-1.6.0.tar.gz",
"has_sig": false,
"md5_digest": "6876797b986d52ae7acbf6c19030ee8e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4.0",
"size": 40879,
"upload_time": "2024-03-14T14:28:33",
"upload_time_iso_8601": "2024-03-14T14:28:33.771828Z",
"url": "https://files.pythonhosted.org/packages/58/56/7853c664b1fdd7319d8398922f8a41c989226cab18b9c696efa3c2225fe5/django_eav2-1.6.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-14 14:28:33",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "jazzband",
"github_project": "django-eav2",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "django-eav2"
}
JSON
