# Django Gyro
A declarative system for importing and exporting CSV data with Django models. Django Gyro provides a clean, validation-rich way to map CSV columns to Django model fields with automatic foreign key resolution and intelligent data slicing capabilities.
## Features
- **Declarative Import System**: Define how CSV data maps to your Django models using simple class-based importers
- **Automatic Foreign Key Resolution**: Intelligently resolves relationships between models during import
- **Circular Dependency Handling**: Automatically resolves circular relationships (e.g., Customer ↔ CustomerReferral)
- **ID Remapping Strategies**: Multiple strategies for handling ID conflicts during imports
- **Data Slicing & Export**: Export subsets of your data with complex relationships intact
- **Multi-tenant Support**: Built-in support for multi-tenant architectures with tenant-aware remapping
- **PostgreSQL Bulk Loading**: High-performance imports using PostgreSQL COPY operations
- **Validation-Rich**: Comprehensive validation during import/export operations
- **Progress Tracking**: Built-in progress bars for long-running operations
## Quick Start
### Installation
```bash
pip install django-gyro
```
### Basic Usage
#### 1. Define Your Importers
```python
# myapp/importers.py
from django_gyro import Importer
from myapp.models import Tenant, Shop, Customer, Product, Order
class TenantImporter(Importer):
model = Tenant
class Columns:
pass
class ShopImporter(Importer):
model = Shop
class Columns:
tenant = Tenant
class CustomerImporter(Importer):
model = Customer
class Columns:
shop = Shop
tenant = Tenant
```
#### 2. Export Data
```python
from django_gyro import DataSlicer, ImportJob
# Define what data to export
tenant = Tenant.objects.filter(id=1)
shops = Shop.objects.filter(tenant=tenant)
customers = Customer.objects.filter(shop__in=shops)
# Export to CSV files
DataSlicer.run(
source=DataSlicer.Postgres(database_url),
target=DataSlicer.File('/path/to/export/'),
jobs=[
ImportJob(model=Tenant, query=tenant),
ImportJob(model=Shop, query=shops),
ImportJob(model=Customer, query=customers),
],
)
```
#### 3. Import Data
```python
# Import from CSV files
DataSlicer.run(
source=DataSlicer.File('/path/to/import/'),
target=DataSlicer.Postgres(database_url),
jobs=[
ImportJob(model=Tenant),
ImportJob(model=Shop),
ImportJob(model=Customer),
],
)
```
#### 4. Management Command for CSV Import
Django Gyro includes a management command for importing CSV data:
```bash
# Import CSV data with automatic ID remapping
python manage.py import_csv_data --source-dir /path/to/csv/files/
# Preview import without making changes
python manage.py import_csv_data --source-dir /path/to/csv/files/ --dry-run
# Use INSERT statements instead of PostgreSQL COPY (slower but more compatible)
python manage.py import_csv_data --source-dir /path/to/csv/files/ --use-insert
```
The command automatically handles dependency ordering and uses sequential ID remapping to avoid conflicts.
#### 5. ID Remapping Strategies (Python API)
For more control over ID remapping, use the Python API:
```python
from django_gyro.importing import (
ImportContext,
SequentialRemappingStrategy,
HashBasedRemappingStrategy,
TenantAwareRemappingStrategy,
NoRemappingStrategy
)
# Sequential remapping (assigns new IDs starting from MAX+1)
strategy = SequentialRemappingStrategy(model=Customer)
context = ImportContext(
source_directory=Path("/path/to/csv/files"),
id_remapping_strategy=strategy
)
# Hash-based remapping (stable IDs using business keys)
strategy = HashBasedRemappingStrategy(
model=Customer,
business_key="email" # Use email to generate stable IDs
)
# Tenant-aware remapping (works with any "tenant" model name)
strategy = TenantAwareRemappingStrategy(
tenant_model=Organization, # Your tenant model (any name)
tenant_mappings={1060: 10, 2000: 11} # staging -> local IDs
)
# Manual ID mappings
id_mappings = {
"myapp.Customer": {100: 200, 101: 201}, # old_id: new_id
"myapp.Order": {500: 600, 501: 601}
}
```
## Use Cases
- **Data Migration**: Move data between environments while preserving relationships
- **Selective Exports**: Export specific subsets of data for development or testing
- **Multi-tenant Data Management**: Handle complex tenant-based data relationships
- **CSV Import/Export**: Robust CSV handling with validation and error reporting
## Documentation
For detailed documentation, examples, and advanced usage, see [TECHNICAL_DESIGN.md](docs/TECHNICAL_DESIGN.md).
## Requirements
- Python 3.8+
- Django 3.2+
- PostgreSQL (for DataSlicer operations)
## Contributing
We welcome contributions! Please see our [Contributing Guidelines](CONTRIBUTING.md) for details.
1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
## Support
- 📖 [Documentation](docs/TECHNICAL_DESIGN.md)
- 🐛 [Issue Tracker](https://github.com/dev360/django-gyro/issues)
- 💬 [Discussions](https://github.com/dev360/django-gyro/discussions)
Raw data
{
"_id": null,
"home_page": "https://github.com/dev360/django-gyro",
"name": "django-gyro",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "django, gyro, data, import, export",
"author": null,
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/3e/88/fb752565b9ff5163ece3ef116a28b22a57dec5c6e945ec2bb1c7bc9b9a6e/django_gyro-0.3.4.tar.gz",
"platform": null,
"description": "# Django Gyro\n\nA declarative system for importing and exporting CSV data with Django models. Django Gyro provides a clean, validation-rich way to map CSV columns to Django model fields with automatic foreign key resolution and intelligent data slicing capabilities.\n\n## Features\n\n- **Declarative Import System**: Define how CSV data maps to your Django models using simple class-based importers\n- **Automatic Foreign Key Resolution**: Intelligently resolves relationships between models during import\n- **Circular Dependency Handling**: Automatically resolves circular relationships (e.g., Customer \u2194 CustomerReferral)\n- **ID Remapping Strategies**: Multiple strategies for handling ID conflicts during imports\n- **Data Slicing & Export**: Export subsets of your data with complex relationships intact\n- **Multi-tenant Support**: Built-in support for multi-tenant architectures with tenant-aware remapping\n- **PostgreSQL Bulk Loading**: High-performance imports using PostgreSQL COPY operations\n- **Validation-Rich**: Comprehensive validation during import/export operations\n- **Progress Tracking**: Built-in progress bars for long-running operations\n\n## Quick Start\n\n### Installation\n\n```bash\npip install django-gyro\n```\n\n### Basic Usage\n\n#### 1. Define Your Importers\n\n```python\n# myapp/importers.py\nfrom django_gyro import Importer\nfrom myapp.models import Tenant, Shop, Customer, Product, Order\n\nclass TenantImporter(Importer):\n model = Tenant\n\n class Columns:\n pass\n\nclass ShopImporter(Importer):\n model = Shop\n\n class Columns:\n tenant = Tenant\n\nclass CustomerImporter(Importer):\n model = Customer\n\n class Columns:\n shop = Shop\n tenant = Tenant\n```\n\n#### 2. Export Data\n\n```python\nfrom django_gyro import DataSlicer, ImportJob\n\n# Define what data to export\ntenant = Tenant.objects.filter(id=1)\nshops = Shop.objects.filter(tenant=tenant)\ncustomers = Customer.objects.filter(shop__in=shops)\n\n# Export to CSV files\nDataSlicer.run(\n source=DataSlicer.Postgres(database_url),\n target=DataSlicer.File('/path/to/export/'),\n jobs=[\n ImportJob(model=Tenant, query=tenant),\n ImportJob(model=Shop, query=shops),\n ImportJob(model=Customer, query=customers),\n ],\n)\n```\n\n#### 3. Import Data\n\n```python\n# Import from CSV files\nDataSlicer.run(\n source=DataSlicer.File('/path/to/import/'),\n target=DataSlicer.Postgres(database_url),\n jobs=[\n ImportJob(model=Tenant),\n ImportJob(model=Shop),\n ImportJob(model=Customer),\n ],\n)\n```\n\n#### 4. Management Command for CSV Import\n\nDjango Gyro includes a management command for importing CSV data:\n\n```bash\n# Import CSV data with automatic ID remapping\npython manage.py import_csv_data --source-dir /path/to/csv/files/\n\n# Preview import without making changes\npython manage.py import_csv_data --source-dir /path/to/csv/files/ --dry-run\n\n# Use INSERT statements instead of PostgreSQL COPY (slower but more compatible)\npython manage.py import_csv_data --source-dir /path/to/csv/files/ --use-insert\n```\n\nThe command automatically handles dependency ordering and uses sequential ID remapping to avoid conflicts.\n\n#### 5. ID Remapping Strategies (Python API)\n\nFor more control over ID remapping, use the Python API:\n\n```python\nfrom django_gyro.importing import (\n ImportContext,\n SequentialRemappingStrategy,\n HashBasedRemappingStrategy,\n TenantAwareRemappingStrategy,\n NoRemappingStrategy\n)\n\n# Sequential remapping (assigns new IDs starting from MAX+1)\nstrategy = SequentialRemappingStrategy(model=Customer)\ncontext = ImportContext(\n source_directory=Path(\"/path/to/csv/files\"),\n id_remapping_strategy=strategy\n)\n\n# Hash-based remapping (stable IDs using business keys)\nstrategy = HashBasedRemappingStrategy(\n model=Customer,\n business_key=\"email\" # Use email to generate stable IDs\n)\n\n# Tenant-aware remapping (works with any \"tenant\" model name)\nstrategy = TenantAwareRemappingStrategy(\n tenant_model=Organization, # Your tenant model (any name)\n tenant_mappings={1060: 10, 2000: 11} # staging -> local IDs\n)\n\n# Manual ID mappings\nid_mappings = {\n \"myapp.Customer\": {100: 200, 101: 201}, # old_id: new_id\n \"myapp.Order\": {500: 600, 501: 601}\n}\n```\n\n## Use Cases\n\n- **Data Migration**: Move data between environments while preserving relationships\n- **Selective Exports**: Export specific subsets of data for development or testing\n- **Multi-tenant Data Management**: Handle complex tenant-based data relationships\n- **CSV Import/Export**: Robust CSV handling with validation and error reporting\n\n## Documentation\n\nFor detailed documentation, examples, and advanced usage, see [TECHNICAL_DESIGN.md](docs/TECHNICAL_DESIGN.md).\n\n## Requirements\n\n- Python 3.8+\n- Django 3.2+\n- PostgreSQL (for DataSlicer operations)\n\n## Contributing\n\nWe welcome contributions! Please see our [Contributing Guidelines](CONTRIBUTING.md) for details.\n\n1. Fork the repository\n2. Create a feature branch (`git checkout -b feature/amazing-feature`)\n3. Commit your changes (`git commit -m 'Add amazing feature'`)\n4. Push to the branch (`git push origin feature/amazing-feature`)\n5. Open a Pull Request\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## Support\n\n- \ud83d\udcd6 [Documentation](docs/TECHNICAL_DESIGN.md)\n- \ud83d\udc1b [Issue Tracker](https://github.com/dev360/django-gyro/issues)\n- \ud83d\udcac [Discussions](https://github.com/dev360/django-gyro/discussions)\n",
"bugtrack_url": null,
"license": "BSD-3-Clause",
"summary": "Django Gyro",
"version": "0.3.4",
"project_urls": {
"Homepage": "https://github.com/dev360/django-gyro",
"Repository": "https://github.com/dev360/django-gyro"
},
"split_keywords": [
"django",
" gyro",
" data",
" import",
" export"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "18e2667de9f907140350e8b89c8e91544c500474178811f22fdb53f274e09cd2",
"md5": "e361b06d75304a2e6e16747bd80b3f8b",
"sha256": "7d597085ddb0cbbb85f8b2fb0905900314a7061c4a5ce7950de39176759b071c"
},
"downloads": -1,
"filename": "django_gyro-0.3.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e361b06d75304a2e6e16747bd80b3f8b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 31969,
"upload_time": "2025-07-14T13:01:59",
"upload_time_iso_8601": "2025-07-14T13:01:59.573245Z",
"url": "https://files.pythonhosted.org/packages/18/e2/667de9f907140350e8b89c8e91544c500474178811f22fdb53f274e09cd2/django_gyro-0.3.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3e88fb752565b9ff5163ece3ef116a28b22a57dec5c6e945ec2bb1c7bc9b9a6e",
"md5": "6dab0cb60b3cae9d62b6393701773b27",
"sha256": "c5cd84381bc8131a20cf14a1b5fc0ffb771661e514d31b425eae29181e8fbac7"
},
"downloads": -1,
"filename": "django_gyro-0.3.4.tar.gz",
"has_sig": false,
"md5_digest": "6dab0cb60b3cae9d62b6393701773b27",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 30945,
"upload_time": "2025-07-14T13:02:00",
"upload_time_iso_8601": "2025-07-14T13:02:00.698269Z",
"url": "https://files.pythonhosted.org/packages/3e/88/fb752565b9ff5163ece3ef116a28b22a57dec5c6e945ec2bb1c7bc9b9a6e/django_gyro-0.3.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-14 13:02:00",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "dev360",
"github_project": "django-gyro",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "django-gyro"
}
JSON
