.. image:: https://i.imgur.com/ARniMyK.png
.. image:: https://img.shields.io/pypi/v/django-haul.svg
:target: https://pypi.python.org/pypi/django-haul
.. image:: https://readthedocs.org/projects/haul/badge/?version=latest
:target: https://haul.readthedocs.io/en/latest/?version=latest
:alt: Documentation Status
Object graph import/export framework for Django
* Free software: MIT license
* Documentation: https://haul.readthedocs.io.
* Experimental: active in production, but the API is subject to change.
About
-----
Haul allows you to add model export/import functionality to your Django app.
It can export some or all objects out of the Django ORM, store them in a file or a stream, and later import them back into the same or a *different* database / application instance.
When importing into a different database, you can customize how the imported objects are mapped against existing objects in the DB, and define what gets overwritten and what gets created anew.
Features
--------
* Automatically follows FK and M2M references
* Flexible serialization behaviours
* Flexible object relinking on import
* File attachments support
* Compressed and plaintext formats
Installation
------------
::
pip install django-haul
Example
-------
Consider following models:
.. code-block:: python
from django.db import models
class Tag(models.Model):
name = models.CharField(max_length=32)
class Author(models.Model):
name = models.CharField(max_length=100)
tags = models.ManyToManyField(Tag)
class Book(models.Model):
author = models.ForeignKey(Author, on_delete=models.CASCADE, related_name='books')
coauthor = models.ForeignKey(Author, on_delete=models.SET_NULL, null=True)
name = models.CharField(max_length=100)
isbn = models.CharField(max_length=100, null=True)
tags = models.ManyToManyField(Tag)
To expose models to Haul, you need to define ``Exporter`` classes:
.. code-block:: python
from haul import Exporter, ForeignKey, ReverseForeignKey, ManyToMany
class TagExporter(Exporter):
class Meta:
fields = '__all__'
model = Tag
class BookExporter(Exporter):
author = ForeignKey()
coauthor = ForeignKey(allow_null=True)
tags = ManyToMany()
class Meta:
fields = '__all__'
model = Book
class AuthorExporter(Exporter):
books = ReverseForeignKey()
tags = ManyToMany()
class Meta:
fields = '__all__'
model = Author
``Exporter`` base class is based on Django REST Framework's own ``ModelSerializer`` and will auto-discover non-relational fields.
Now, to export all books into a file, you can use:
.. code-block:: python
EXPORTERS = [BookExporter, AuthorExporter, TagExporter]
with open('export.haul', 'wb') as f:
c = ExportContainer(exporters=EXPORTERS)
c.export_objects(Book.objects.all())
c.write(f)
The output file will contain an object graph dump:
.. code-block:: yaml
---
_: header
metadata: null
object_kinds:
- test_app:book
- test_app:author
version: 1
---
_: object
attachments: []
data: !!omap
- books:
- !<ID>
kind: test_app:book
pk: 1
- tags: []
- name: '1'
id: !<ID>
kind: test_app:author
pk: 1
---
_: object
attachments: []
data: !!omap
- books: []
- tags: []
- name: '2'
id: !<ID>
kind: test_app:author
pk: 2
---
_: object
attachments: []
data: !!omap
- author: !<ID>
kind: test_app:author
pk: 1
- coauthor: null
- tags: []
- name: b1
- isbn: null
id: !<ID>
kind: test_app:book
pk: 1
You can also inspect the objects within the files with the CLI dump tool::
python -m haul.cli export.haul
Note how the ``Author`` objects related to the ``Book`` instances got picked up and exported automatically.
To import this data back into the database, you can simply feed it to an ``ImportContainer``:
.. code-block:: python
from haul import ImportContainer
c = ImportContainer(exporters=EXPORTERS)
with open('export.haul', 'rb') as f:
with c.read(f):
c.import_objects()
This, however, will unconditionally create new objects, even if books and authors with the same names already exist.
You can flexibly define how Haul should treat existing and duplicate objects. For example, let's prevent duplicate authors from being imported, but keep duplicate books and link them to the already existing authors:
.. code-block:: python
from haul import ImportPolicy, RelinkAction
class BookImportPolicy(ImportPolicy):
def relink_object(self, model_cls, obj):
if model_cls is Book:
# Unconditionally import as a new object
return RelinkAction.Create()
if model_cls is Author:
return RelinkAction.LinkByFields(
# Look up authors by their names
lookup_fields=('name',),
# Fall back to creation if not found
fallback=RelinkAction.Create(),
)
# Do not import other object types
return RelinkAction.Discard()
c = ImportContainer(exporters=EXPORTERS, policy=BookImportPolicy())
with open('export.haul', 'rb') as f:
with c.read(d):
c.import_objects()
See :mod:`haul.policy` for other relink actions.
Raw data
{
"_id": null,
"home_page": "https://github.com/elements-storage/haul",
"name": "django-haul",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "django, orm, export, import",
"author": "Eugene Pankov`",
"author_email": "e.pankov@elements.tv",
"download_url": "https://files.pythonhosted.org/packages/cd/e5/ad84a7fe8bae62e6fd4389b2b96962b03e3f904c1adc98479da5c91ebf95/django-haul-0.0.12.tar.gz",
"platform": null,
"description": ".. image:: https://i.imgur.com/ARniMyK.png\n\n.. image:: https://img.shields.io/pypi/v/django-haul.svg\n :target: https://pypi.python.org/pypi/django-haul\n\n.. image:: https://readthedocs.org/projects/haul/badge/?version=latest\n :target: https://haul.readthedocs.io/en/latest/?version=latest\n :alt: Documentation Status\n\nObject graph import/export framework for Django\n\n* Free software: MIT license\n* Documentation: https://haul.readthedocs.io.\n* Experimental: active in production, but the API is subject to change.\n\nAbout\n-----\n\nHaul allows you to add model export/import functionality to your Django app.\nIt can export some or all objects out of the Django ORM, store them in a file or a stream, and later import them back into the same or a *different* database / application instance.\n\nWhen importing into a different database, you can customize how the imported objects are mapped against existing objects in the DB, and define what gets overwritten and what gets created anew.\n\n\nFeatures\n--------\n\n* Automatically follows FK and M2M references\n* Flexible serialization behaviours\n* Flexible object relinking on import\n* File attachments support\n* Compressed and plaintext formats\n\nInstallation\n------------\n\n::\n\n pip install django-haul\n\n\nExample\n-------\n\nConsider following models:\n\n.. code-block:: python\n\n from django.db import models\n\n\n class Tag(models.Model):\n name = models.CharField(max_length=32)\n\n class Author(models.Model):\n name = models.CharField(max_length=100)\n tags = models.ManyToManyField(Tag)\n\n class Book(models.Model):\n author = models.ForeignKey(Author, on_delete=models.CASCADE, related_name='books')\n coauthor = models.ForeignKey(Author, on_delete=models.SET_NULL, null=True)\n name = models.CharField(max_length=100)\n isbn = models.CharField(max_length=100, null=True)\n tags = models.ManyToManyField(Tag)\n\n\nTo expose models to Haul, you need to define ``Exporter`` classes:\n\n.. code-block:: python\n\n from haul import Exporter, ForeignKey, ReverseForeignKey, ManyToMany\n\n class TagExporter(Exporter):\n class Meta:\n fields = '__all__'\n model = Tag\n\n\n class BookExporter(Exporter):\n author = ForeignKey()\n coauthor = ForeignKey(allow_null=True)\n tags = ManyToMany()\n\n class Meta:\n fields = '__all__'\n model = Book\n\n\n class AuthorExporter(Exporter):\n books = ReverseForeignKey()\n tags = ManyToMany()\n\n class Meta:\n fields = '__all__'\n model = Author\n\n``Exporter`` base class is based on Django REST Framework's own ``ModelSerializer`` and will auto-discover non-relational fields.\n\nNow, to export all books into a file, you can use:\n\n.. code-block:: python\n\n EXPORTERS = [BookExporter, AuthorExporter, TagExporter]\n\n with open('export.haul', 'wb') as f:\n c = ExportContainer(exporters=EXPORTERS)\n c.export_objects(Book.objects.all())\n c.write(f)\n\nThe output file will contain an object graph dump:\n\n.. code-block:: yaml\n\n ---\n _: header\n metadata: null\n object_kinds:\n - test_app:book\n - test_app:author\n version: 1\n\n ---\n _: object\n attachments: []\n data: !!omap\n - books:\n - !<ID>\n kind: test_app:book\n pk: 1\n - tags: []\n - name: '1'\n id: !<ID>\n kind: test_app:author\n pk: 1\n\n ---\n _: object\n attachments: []\n data: !!omap\n - books: []\n - tags: []\n - name: '2'\n id: !<ID>\n kind: test_app:author\n pk: 2\n\n ---\n _: object\n attachments: []\n data: !!omap\n - author: !<ID>\n kind: test_app:author\n pk: 1\n - coauthor: null\n - tags: []\n - name: b1\n - isbn: null\n id: !<ID>\n kind: test_app:book\n pk: 1\n\nYou can also inspect the objects within the files with the CLI dump tool::\n\n python -m haul.cli export.haul\n\nNote how the ``Author`` objects related to the ``Book`` instances got picked up and exported automatically.\n\nTo import this data back into the database, you can simply feed it to an ``ImportContainer``:\n\n.. code-block:: python\n\n from haul import ImportContainer\n\n c = ImportContainer(exporters=EXPORTERS)\n with open('export.haul', 'rb') as f:\n with c.read(f):\n c.import_objects()\n\nThis, however, will unconditionally create new objects, even if books and authors with the same names already exist.\n\nYou can flexibly define how Haul should treat existing and duplicate objects. For example, let's prevent duplicate authors from being imported, but keep duplicate books and link them to the already existing authors:\n\n.. code-block:: python\n\n from haul import ImportPolicy, RelinkAction\n\n class BookImportPolicy(ImportPolicy):\n def relink_object(self, model_cls, obj):\n if model_cls is Book:\n # Unconditionally import as a new object\n return RelinkAction.Create()\n\n if model_cls is Author:\n return RelinkAction.LinkByFields(\n # Look up authors by their names\n lookup_fields=('name',),\n # Fall back to creation if not found\n fallback=RelinkAction.Create(),\n )\n\n # Do not import other object types\n return RelinkAction.Discard()\n\n c = ImportContainer(exporters=EXPORTERS, policy=BookImportPolicy())\n with open('export.haul', 'rb') as f:\n with c.read(d):\n c.import_objects()\n\nSee :mod:`haul.policy` for other relink actions.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Structured object import/export for Django ORM",
"version": "0.0.12",
"project_urls": {
"Homepage": "https://github.com/elements-storage/haul"
},
"split_keywords": [
"django",
" orm",
" export",
" import"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "cde5ad84a7fe8bae62e6fd4389b2b96962b03e3f904c1adc98479da5c91ebf95",
"md5": "d28441746ba50c2b2ec403f0003509ec",
"sha256": "d5308c2a53305875a534ef55fec63856c7e1e098e04f2bc18284429220c956e2"
},
"downloads": -1,
"filename": "django-haul-0.0.12.tar.gz",
"has_sig": false,
"md5_digest": "d28441746ba50c2b2ec403f0003509ec",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 33930,
"upload_time": "2024-09-23T11:09:57",
"upload_time_iso_8601": "2024-09-23T11:09:57.722575Z",
"url": "https://files.pythonhosted.org/packages/cd/e5/ad84a7fe8bae62e6fd4389b2b96962b03e3f904c1adc98479da5c91ebf95/django-haul-0.0.12.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-23 11:09:57",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "elements-storage",
"github_project": "haul",
"github_not_found": true,
"lcname": "django-haul"
}
JSON
