# Django BDTOPO®
Import and manage all the IGN BDTOPO® data in one (customizable) app!
> Example of Commune object as seen in Django admin.
# Warnings
Importing all the data for all the apps is pretty heavy (time & size-wise):
- **~12Go** to download the three-part sql 7zipped file
- **~40+ Go** available space (extracted data)
- *Very big* tables with millions of entries
- *Huge* import time (like 2 hours on a somewhat beefy computer)
- *Huge* backups (raw .sql files are ~120GB)
You'll need the `7z` command line utility (tested on linux).
Each set of tables is imported in it's own app. Those apps are named after the folders the sql files came from:
- Administratif (`administratif`)
- Bâti (`bati`)
- Hydrographie (`hydrographie`)
- Lieux nommés (`lieux_nommes`)
- Occupation du sol (`occupation_du_sol`)
- Services et activités (`services_et_activites`)
- Transport (`transport`)
- Zones réglementées (`zones_reglementees`)
----
Some tables in `administratif` app have some speciel "foreign keys" that are only char fields (like `liens_vers_autorite_administrative`). Those chars are in fact references to the column `cleabs` in various other tables (`mairie`, `services_et_activites_zone_d_activite_ou_d_interet`). We have added those fields as foreignkeys to our models (`autorite_administrative`), and the foreign key are retrieved in the `create_models_links` management command.
You can find more info on [BD TOPO® Version 3 - Description du contenu](https://geoservices.ign.fr/sites/default/files/2021-07/DC_BDTOPO_3-0.pdf).
----
# Install
1) Install using pip (or poetry)
```sh
poetry add django-bdtopo
```
2) Add the apps you want in your `INSTALLED_APPS`
```py
INSTALLED_APPS = [
# ...
# Mandatory apps:
"django_bdtopo.apps.BDTOPOConfig",
"django_bdtopo.apps.services_et_activites.apps.BDTOPOServicesEtActivitesConfig", # Services et activités (objects are used in administratif & other apps)
"django_bdtopo.apps.lieux_nommes.apps.BDTOPOLieuxNommesConfig", # Lieux nommés (objects are used in administratif & other apps)
# Choose which app to add (or add them all):
"django_bdtopo.apps.administratif.apps.BDTOPOAdministratifConfig", # Administratif
"django_bdtopo.apps.bati.apps.BDTOPOBatiConfig", # Bâti
"django_bdtopo.apps.hydrographie.apps.BDTOPOHydrographieConfig", # Hydrographie
"django_bdtopo.apps.occupation_du_sol.apps.BDTOPOOccupationDuSolConfig", # Occupation du sol
"django_bdtopo.apps.transport.apps.BDTOPOTransportConfig", # Transport
"django_bdtopo.apps.zones_reglementees.apps.BDTOPOZonesReglementeesConfig", # Zones réglementées
```
3) Import data
Importing data is harder than on most other apps, since nothing in Django allow us to import data from a .sql, rename tables, and *automagically* create models.
So you'll need to import the data using our super cool script (it doesn't bite, but you'll need to define some env vars (in your website `settings.py` file)).
```py
# Add those vars in your settings.py:
# Django-bdtopo update_bdtopo script requirements
# --------------------------------------
DBTOPO_DB_NAME = DATABASES["default"]["NAME"]
DBTOPO_DB_HOST = DATABASES["default"]["HOST"]
DBTOPO_DB_PORT = DATABASES["default"]["PORT"]
DBTOPO_DB_USER = DATABASES["default"]["USER"]
```
```sh
# define your pg password like this to prevent being prompted for the password 180 times
PGPASSWORD="pg_password_here" poetry run python3 manage.py update_bdtopo
```
Answer (yes/no) to each import question (in order to import data to an app or another).
4) Run migrations (fake them)
```sh
# Fake all first migrations (tables already exist in db) - migrate only the apps you've imported
poetry run python3 manage.py migrate services_et_activites 0001 --fake
poetry run python3 manage.py migrate lieux_nommes 0001 --fake
poetry run python3 manage.py migrate administratif 0001 --fake
poetry run python3 manage.py migrate bati 0001 --fake
poetry run python3 manage.py migrate hydrographie 0001 --fake
poetry run python3 manage.py migrate occupation_du_sol 0001 --fake
poetry run python3 manage.py migrate transport 0001 --fake
poetry run python3 manage.py migrate zones_reglementees 0001 --fake
# Run "real" migration (add new fk fields)
poetry run python3 manage.py migrate administratif 0002
```
5) Run the script to create real ForeignKey to different objects.
```sh
poetry run python3 manage.py create_models_links
```
6) That's all folks!
----
# Update bdtopo
Updating bdtopo is no easy task, since it comes directly as .sql files.
We must:
- import the new data
- update the models based on this data (if it don't change too much)
- generate new migrations based on this data (if it don't change too much, or else we need to write migrations manually in order to preserve data)
- fake the migration
- launch tests to ensure that nothing broke
*We cannot be sure that the schema will stay the same, but we have no other choice than to hope that if change exist, they stay relatively small (big changes to the models may break the projects using this app).*
## Step by step update
1) Update source files url(s)
2) Import new data
*Move to a django project (in order to use manage.py).*
3) Generate new models (in new files in order to still have the old models in case of error)
```sh
poetry run python ./src/frontend/manage.py inspectdb administratif_arrondissement administratif_arrondissement_municipal administratif_collectivite_territoriale administratif_commune administratif_commune_associee_ou_deleguee administratif_condominium administratif_departement administratif_epci administratif_region > ../django-bdtopo/django_bdtopo/apps/administratif/models.new.py
poetry run python ./src/frontend/manage.py inspectdb bati_batiment bati_cimetiere bati_construction_lineaire bati_construction_ponctuelle bati_construction_surfacique bati_ligne_orographique bati_pylone bati_reservoir bati_terrain_de_sport bati_toponymie_bati > ../django-bdtopo/django_bdtopo/apps/bati/models.new.py
poetry run python ./src/frontend/manage.py inspectdb hydrographie_bassin_versant_topographique hydrographie_cours_d_eau hydrographie_detail_hydrographique hydrographie_limite_terre_mer hydrographie_noeud_hydrographique hydrographie_plan_d_eau hydrographie_surface_hydrographique hydrographie_toponymie_hydrographie hydrographie_troncon_hydrographique > ../django-bdtopo/django_bdtopo/apps/hydrographie/models.new.py
poetry run python ./src/frontend/manage.py inspectdb lieux_nommes_detail_orographique lieux_nommes_lieu_dit_non_habite lieux_nommes_toponymie_lieux_nommes lieux_nommes_zone_d_habitation > ../django-bdtopo/django_bdtopo/apps/lieux_nommes/models.new.py
poetry run python ./src/frontend/manage.py inspectdb occupation_du_sol_haie occupation_du_sol_zone_d_estran occupation_du_sol_zone_de_vegetation > ../django-bdtopo/django_bdtopo/apps/occupation_du_sol/models.new.py
poetry run python ./src/frontend/manage.py inspectdb services_et_activites_canalisation services_et_activites_erp services_et_activites_ligne_electrique services_et_activites_poste_de_transformation services_et_activites_toponymie_services_et_activites services_et_activites_zone_d_activite_ou_d_interet > ../django-bdtopo/django_bdtopo/apps/services_et_activites/models.new.py
poetry run python ./src/frontend/manage.py inspectdb transport_aerodrome transport_equipement_de_transport transport_itineraire_autre transport_non_communication transport_piste_d_aerodrome transport_point_d_acces transport_point_de_repere transport_point_du_reseau transport_route_numerotee_ou_nommee transport_section_de_points_de_repere transport_toponymie_transport transport_transport_par_cable transport_troncon_de_route transport_troncon_de_voie_ferree transport_voie_ferree_nommee transport_voie_nommee transport_voie_nommee_beta > ../django-bdtopo/django_bdtopo/apps/transport/models.new.py
poetry run python ./src/frontend/manage.py inspectdb zones_reglementees_foret_publique zones_reglementees_parc_ou_reserve zones_reglementees_toponymie_zones_reglementees > ../django-bdtopo/django_bdtopo/apps/zones_reglementees/models.new.py
```
*Move to django-bdtopo folder.*
4) Move old models to backup (& remove old files if they still exist)
```sh
rm django_bdtopo/apps/administratif/models.py.bak
mv django_bdtopo/apps/administratif/models.py django_bdtopo/apps/administratif/models.py.bak > /dev/null 2>&1
rm django_bdtopo/apps/bati/models.py.bak
mv django_bdtopo/apps/bati/models.py django_bdtopo/apps/bati/models.py.bak > /dev/null 2>&1
rm django_bdtopo/apps/hydrographie/models.py.bak
mv django_bdtopo/apps/hydrographie/models.py django_bdtopo/apps/hydrographie/models.py.bak > /dev/null 2>&1
rm django_bdtopo/apps/lieux_nommes/models.py.bak
mv django_bdtopo/apps/lieux_nommes/models.py django_bdtopo/apps/lieux_nommes/models.py.bak > /dev/null 2>&1
rm django_bdtopo/apps/occupation_du_sol/models.py.bak
mv django_bdtopo/apps/occupation_du_sol/models.py django_bdtopo/apps/occupation_du_sol/models.py.bak > /dev/null 2>&1
rm django_bdtopo/apps/services_et_activites/models.py.bak
mv django_bdtopo/apps/services_et_activites/models.py django_bdtopo/apps/services_et_activites/models.py.bak > /dev/null 2>&1
rm django_bdtopo/apps/transport/models.py.bak
mv django_bdtopo/apps/transport/models.py django_bdtopo/apps/transport/models.py.bak > /dev/null 2>&1
rm django_bdtopo/apps/zones_reglementees/models.py.bak
mv django_bdtopo/apps/zones_reglementees/models.py django_bdtopo/apps/zones_reglementees/models.py.bak > /dev/null 2>&1
```
5) Rename `models.new.py` into `models.py`
```sh
mv django_bdtopo/apps/administratif/models.new.py django_bdtopo/apps/administratif/models.py
mv django_bdtopo/apps/bati/models.new.py django_bdtopo/apps/bati/models.py
mv django_bdtopo/apps/hydrographie/models.new.py django_bdtopo/apps/hydrographie/models.py
mv django_bdtopo/apps/lieux_nommes/models.new.py django_bdtopo/apps/lieux_nommes/models.py
mv django_bdtopo/apps/occupation_du_sol/models.new.py django_bdtopo/apps/occupation_du_sol/models.py
mv django_bdtopo/apps/services_et_activites/models.new.py django_bdtopo/apps/services_et_activites/models.py
mv django_bdtopo/apps/transport/models.new.py django_bdtopo/apps/transport/models.py
mv django_bdtopo/apps/zones_reglementees/models.new.py django_bdtopo/apps/zones_reglementees/models.py
```
6) Remove header lines
```sh
sed -i '1,7d' django_bdtopo/apps/administratif/models.py
sed -i '1,7d' django_bdtopo/apps/bati/models.py
sed -i '1,7d' django_bdtopo/apps/hydrographie/models.py
sed -i '1,7d' django_bdtopo/apps/lieux_nommes/models.py
sed -i '1,7d' django_bdtopo/apps/occupation_du_sol/models.py
sed -i '1,7d' django_bdtopo/apps/services_et_activites/models.py
sed -i '1,7d' django_bdtopo/apps/transport/models.py
sed -i '1,7d' django_bdtopo/apps/zones_reglementees/models.py
```
7) Insert a new line at the beginning of the file to indicate that the file has been generated
```sh
today=$(date +'%Y-%m-%d')
sed -i "1i\# Original IGN django generated model as of $today" django_bdtopo/apps/administratif/models.py
sed -i "1i\# Original IGN django generated model as of $today" django_bdtopo/apps/bati/models.py
sed -i "1i\# Original IGN django generated model as of $today" django_bdtopo/apps/hydrographie/models.py
sed -i "1i\# Original IGN django generated model as of $today" django_bdtopo/apps/lieux_nommes/models.py
sed -i "1i\# Original IGN django generated model as of $today" django_bdtopo/apps/occupation_du_sol/models.py
sed -i "1i\# Original IGN django generated model as of $today" django_bdtopo/apps/services_et_activites/models.py
sed -i "1i\# Original IGN django generated model as of $today" django_bdtopo/apps/transport/models.py
sed -i "1i\# Original IGN django generated model as of $today" django_bdtopo/apps/zones_reglementees/models.py
```
8) Remove the lines containing managed = False
```sh
sed -i '/managed = False/d' django_bdtopo/apps/administratif/models.py
sed -i '/managed = False/d' django_bdtopo/apps/bati/models.py
sed -i '/managed = False/d' django_bdtopo/apps/hydrographie/models.py
sed -i '/managed = False/d' django_bdtopo/apps/lieux_nommes/models.py
sed -i '/managed = False/d' django_bdtopo/apps/occupation_du_sol/models.py
sed -i '/managed = False/d' django_bdtopo/apps/services_et_activites/models.py
sed -i '/managed = False/d' django_bdtopo/apps/transport/models.py
sed -i '/managed = False/d' django_bdtopo/apps/zones_reglementees/models.py
```
9) Rename models to remove the prefixes
```sh
sed -i 's/class Administratif/class /g' django_bdtopo/apps/administratif/models.py
sed -i 's/class Bati/class /g' django_bdtopo/apps/bati/models.py
sed -i 's/class Hydrographie/class /g' django_bdtopo/apps/hydrographie/models.py
sed -i 's/class LieuxNommes/class /g' django_bdtopo/apps/lieux_nommes/models.py
sed -i 's/class OccupationDuSol/class /g' django_bdtopo/apps/occupation_du_sol/models.py
sed -i 's/class ServicesEtActivites/class /g' django_bdtopo/apps/services_et_activites/models.py
sed -i 's/class Transport/class /g' django_bdtopo/apps/transport/models.py
sed -i 's/class ZonesReglementees/class /g' django_bdtopo/apps/zones_reglementees/models.py
```
10) Rename the Epci model to EPCI
```sh
sed -i 's/class Epci/class EPCI/g' django_bdtopo/apps/administratif/models.py
```
*Move to a django project.*
11) Generates the initial migration
```sh
run -frontend makemigrations administratif
run -frontend makemigrations bati
run -frontend makemigrations hydrographie
run -frontend makemigrations lieux_nommes
run -frontend makemigrations occupation_du_sol
run -frontend makemigrations services_et_activites
run -frontend makemigrations transport
run -frontend makemigrations zones_reglementees
```
12) Run pre-commit on the generated files
```sh
find django_bdtopo/*/migrations/ -name "*.py" -print | xargs pre-commit run --files
pre-commit run --files "django_bdtopo/*/models.py"
```
13) Fake the migrations
```sh
run -frontend migrate administratif --fake
run -frontend migrate bati --fake
run -frontend migrate hydrographie --fake
run -frontend migrate lieux_nommes --fake
run -frontend migrate occupation_du_sol --fake
run -frontend migrate services_et_activites --fake
run -frontend migrate transport --fake
run -frontend migrate zones_reglementees --fake
```
14) That's all folks!
----
# Todo
- update `create_models_links` to include other apps (now it creates fk only for administratif app)
- auto-detect which apps are installed before running the migrations
- meaningful errors messages in case of problem
- add tests
Raw data
{
"_id": null,
"home_page": "https://gitlab.com/kapt/open-source/django-bdtopo",
"name": "django-bdtopo",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "Dev Kapt",
"author_email": "dev@kapt.mobi",
"download_url": "https://files.pythonhosted.org/packages/29/77/afd10e200e42796ef183b3ff639571b3fb6f23d7fb8612f77669e33285e2/django_bdtopo-0.0.1.tar.gz",
"platform": null,
"description": "# Django BDTOPO\u00ae\n\nImport and manage all the IGN BDTOPO\u00ae data in one (customizable) app!\n\n\n\n> Example of Commune object as seen in Django admin.\n\n# Warnings\n\nImporting all the data for all the apps is pretty heavy (time & size-wise):\n- **~12Go** to download the three-part sql 7zipped file\n- **~40+ Go** available space (extracted data)\n- *Very big* tables with millions of entries\n- *Huge* import time (like 2 hours on a somewhat beefy computer)\n- *Huge* backups (raw .sql files are ~120GB)\n\nYou'll need the `7z` command line utility (tested on linux).\n\nEach set of tables is imported in it's own app. Those apps are named after the folders the sql files came from:\n- Administratif (`administratif`)\n- B\u00e2ti (`bati`)\n- Hydrographie (`hydrographie`)\n- Lieux nomm\u00e9s (`lieux_nommes`)\n- Occupation du sol (`occupation_du_sol`)\n- Services et activit\u00e9s (`services_et_activites`)\n- Transport (`transport`)\n- Zones r\u00e9glement\u00e9es (`zones_reglementees`)\n\n----\n\nSome tables in `administratif` app have some speciel \"foreign keys\" that are only char fields (like `liens_vers_autorite_administrative`). Those chars are in fact references to the column `cleabs` in various other tables (`mairie`, `services_et_activites_zone_d_activite_ou_d_interet`). We have added those fields as foreignkeys to our models (`autorite_administrative`), and the foreign key are retrieved in the `create_models_links` management command.\n\nYou can find more info on [BD TOPO\u00ae Version 3 - Description du contenu](https://geoservices.ign.fr/sites/default/files/2021-07/DC_BDTOPO_3-0.pdf).\n\n----\n\n# Install\n\n1) Install using pip (or poetry)\n ```sh\n poetry add django-bdtopo\n ```\n2) Add the apps you want in your `INSTALLED_APPS`\n ```py\n INSTALLED_APPS = [\n # ...\n # Mandatory apps:\n \"django_bdtopo.apps.BDTOPOConfig\",\n \"django_bdtopo.apps.services_et_activites.apps.BDTOPOServicesEtActivitesConfig\", # Services et activit\u00e9s (objects are used in administratif & other apps)\n \"django_bdtopo.apps.lieux_nommes.apps.BDTOPOLieuxNommesConfig\", # Lieux nomm\u00e9s (objects are used in administratif & other apps)\n\n # Choose which app to add (or add them all):\n \"django_bdtopo.apps.administratif.apps.BDTOPOAdministratifConfig\", # Administratif\n \"django_bdtopo.apps.bati.apps.BDTOPOBatiConfig\", # B\u00e2ti\n \"django_bdtopo.apps.hydrographie.apps.BDTOPOHydrographieConfig\", # Hydrographie\n \"django_bdtopo.apps.occupation_du_sol.apps.BDTOPOOccupationDuSolConfig\", # Occupation du sol\n \"django_bdtopo.apps.transport.apps.BDTOPOTransportConfig\", # Transport\n \"django_bdtopo.apps.zones_reglementees.apps.BDTOPOZonesReglementeesConfig\", # Zones r\u00e9glement\u00e9es\n ```\n3) Import data\n Importing data is harder than on most other apps, since nothing in Django allow us to import data from a .sql, rename tables, and *automagically* create models. \n So you'll need to import the data using our super cool script (it doesn't bite, but you'll need to define some env vars (in your website `settings.py` file)).\n ```py\n # Add those vars in your settings.py:\n # Django-bdtopo update_bdtopo script requirements\n # --------------------------------------\n DBTOPO_DB_NAME = DATABASES[\"default\"][\"NAME\"]\n DBTOPO_DB_HOST = DATABASES[\"default\"][\"HOST\"]\n DBTOPO_DB_PORT = DATABASES[\"default\"][\"PORT\"]\n DBTOPO_DB_USER = DATABASES[\"default\"][\"USER\"]\n ```\n ```sh\n # define your pg password like this to prevent being prompted for the password 180 times\n PGPASSWORD=\"pg_password_here\" poetry run python3 manage.py update_bdtopo\n ```\n Answer (yes/no) to each import question (in order to import data to an app or another).\n4) Run migrations (fake them)\n ```sh\n # Fake all first migrations (tables already exist in db) - migrate only the apps you've imported\n poetry run python3 manage.py migrate services_et_activites 0001 --fake\n poetry run python3 manage.py migrate lieux_nommes 0001 --fake\n poetry run python3 manage.py migrate administratif 0001 --fake\n poetry run python3 manage.py migrate bati 0001 --fake\n poetry run python3 manage.py migrate hydrographie 0001 --fake\n poetry run python3 manage.py migrate occupation_du_sol 0001 --fake\n poetry run python3 manage.py migrate transport 0001 --fake\n poetry run python3 manage.py migrate zones_reglementees 0001 --fake\n # Run \"real\" migration (add new fk fields)\n poetry run python3 manage.py migrate administratif 0002\n ```\n5) Run the script to create real ForeignKey to different objects.\n ```sh\n poetry run python3 manage.py create_models_links\n ```\n6) That's all folks!\n\n----\n\n# Update bdtopo\n\nUpdating bdtopo is no easy task, since it comes directly as .sql files.\n\nWe must:\n\n- import the new data\n- update the models based on this data (if it don't change too much)\n- generate new migrations based on this data (if it don't change too much, or else we need to write migrations manually in order to preserve data)\n- fake the migration\n- launch tests to ensure that nothing broke\n\n*We cannot be sure that the schema will stay the same, but we have no other choice than to hope that if change exist, they stay relatively small (big changes to the models may break the projects using this app).*\n\n## Step by step update\n\n1) Update source files url(s)\n\n2) Import new data\n\n*Move to a django project (in order to use manage.py).*\n\n3) Generate new models (in new files in order to still have the old models in case of error)\n```sh\npoetry run python ./src/frontend/manage.py inspectdb administratif_arrondissement administratif_arrondissement_municipal administratif_collectivite_territoriale administratif_commune administratif_commune_associee_ou_deleguee administratif_condominium administratif_departement administratif_epci administratif_region > ../django-bdtopo/django_bdtopo/apps/administratif/models.new.py\n\npoetry run python ./src/frontend/manage.py inspectdb bati_batiment bati_cimetiere bati_construction_lineaire bati_construction_ponctuelle bati_construction_surfacique bati_ligne_orographique bati_pylone bati_reservoir bati_terrain_de_sport bati_toponymie_bati > ../django-bdtopo/django_bdtopo/apps/bati/models.new.py\n\npoetry run python ./src/frontend/manage.py inspectdb hydrographie_bassin_versant_topographique hydrographie_cours_d_eau hydrographie_detail_hydrographique hydrographie_limite_terre_mer hydrographie_noeud_hydrographique hydrographie_plan_d_eau hydrographie_surface_hydrographique hydrographie_toponymie_hydrographie hydrographie_troncon_hydrographique > ../django-bdtopo/django_bdtopo/apps/hydrographie/models.new.py\n\npoetry run python ./src/frontend/manage.py inspectdb lieux_nommes_detail_orographique lieux_nommes_lieu_dit_non_habite lieux_nommes_toponymie_lieux_nommes lieux_nommes_zone_d_habitation > ../django-bdtopo/django_bdtopo/apps/lieux_nommes/models.new.py\n\npoetry run python ./src/frontend/manage.py inspectdb occupation_du_sol_haie occupation_du_sol_zone_d_estran occupation_du_sol_zone_de_vegetation > ../django-bdtopo/django_bdtopo/apps/occupation_du_sol/models.new.py\n\npoetry run python ./src/frontend/manage.py inspectdb services_et_activites_canalisation services_et_activites_erp services_et_activites_ligne_electrique services_et_activites_poste_de_transformation services_et_activites_toponymie_services_et_activites services_et_activites_zone_d_activite_ou_d_interet > ../django-bdtopo/django_bdtopo/apps/services_et_activites/models.new.py\n\npoetry run python ./src/frontend/manage.py inspectdb transport_aerodrome transport_equipement_de_transport transport_itineraire_autre transport_non_communication transport_piste_d_aerodrome transport_point_d_acces transport_point_de_repere transport_point_du_reseau transport_route_numerotee_ou_nommee transport_section_de_points_de_repere transport_toponymie_transport transport_transport_par_cable transport_troncon_de_route transport_troncon_de_voie_ferree transport_voie_ferree_nommee transport_voie_nommee transport_voie_nommee_beta > ../django-bdtopo/django_bdtopo/apps/transport/models.new.py\n\npoetry run python ./src/frontend/manage.py inspectdb zones_reglementees_foret_publique zones_reglementees_parc_ou_reserve zones_reglementees_toponymie_zones_reglementees > ../django-bdtopo/django_bdtopo/apps/zones_reglementees/models.new.py\n```\n\n*Move to django-bdtopo folder.*\n\n4) Move old models to backup (& remove old files if they still exist)\n```sh\nrm django_bdtopo/apps/administratif/models.py.bak\nmv django_bdtopo/apps/administratif/models.py django_bdtopo/apps/administratif/models.py.bak > /dev/null 2>&1\n\nrm django_bdtopo/apps/bati/models.py.bak\nmv django_bdtopo/apps/bati/models.py django_bdtopo/apps/bati/models.py.bak > /dev/null 2>&1\n\nrm django_bdtopo/apps/hydrographie/models.py.bak\nmv django_bdtopo/apps/hydrographie/models.py django_bdtopo/apps/hydrographie/models.py.bak > /dev/null 2>&1\n\nrm django_bdtopo/apps/lieux_nommes/models.py.bak\nmv django_bdtopo/apps/lieux_nommes/models.py django_bdtopo/apps/lieux_nommes/models.py.bak > /dev/null 2>&1\n\nrm django_bdtopo/apps/occupation_du_sol/models.py.bak\nmv django_bdtopo/apps/occupation_du_sol/models.py django_bdtopo/apps/occupation_du_sol/models.py.bak > /dev/null 2>&1\n\nrm django_bdtopo/apps/services_et_activites/models.py.bak\nmv django_bdtopo/apps/services_et_activites/models.py django_bdtopo/apps/services_et_activites/models.py.bak > /dev/null 2>&1\n\nrm django_bdtopo/apps/transport/models.py.bak\nmv django_bdtopo/apps/transport/models.py django_bdtopo/apps/transport/models.py.bak > /dev/null 2>&1\n\nrm django_bdtopo/apps/zones_reglementees/models.py.bak\nmv django_bdtopo/apps/zones_reglementees/models.py django_bdtopo/apps/zones_reglementees/models.py.bak > /dev/null 2>&1\n```\n\n5) Rename `models.new.py` into `models.py`\n```sh\nmv django_bdtopo/apps/administratif/models.new.py django_bdtopo/apps/administratif/models.py\nmv django_bdtopo/apps/bati/models.new.py django_bdtopo/apps/bati/models.py\nmv django_bdtopo/apps/hydrographie/models.new.py django_bdtopo/apps/hydrographie/models.py\nmv django_bdtopo/apps/lieux_nommes/models.new.py django_bdtopo/apps/lieux_nommes/models.py\nmv django_bdtopo/apps/occupation_du_sol/models.new.py django_bdtopo/apps/occupation_du_sol/models.py\nmv django_bdtopo/apps/services_et_activites/models.new.py django_bdtopo/apps/services_et_activites/models.py\nmv django_bdtopo/apps/transport/models.new.py django_bdtopo/apps/transport/models.py\nmv django_bdtopo/apps/zones_reglementees/models.new.py django_bdtopo/apps/zones_reglementees/models.py\n```\n6) Remove header lines\n```sh\nsed -i '1,7d' django_bdtopo/apps/administratif/models.py\nsed -i '1,7d' django_bdtopo/apps/bati/models.py\nsed -i '1,7d' django_bdtopo/apps/hydrographie/models.py\nsed -i '1,7d' django_bdtopo/apps/lieux_nommes/models.py\nsed -i '1,7d' django_bdtopo/apps/occupation_du_sol/models.py\nsed -i '1,7d' django_bdtopo/apps/services_et_activites/models.py\nsed -i '1,7d' django_bdtopo/apps/transport/models.py\nsed -i '1,7d' django_bdtopo/apps/zones_reglementees/models.py\n```\n7) Insert a new line at the beginning of the file to indicate that the file has been generated\n```sh\ntoday=$(date +'%Y-%m-%d')\nsed -i \"1i\\# Original IGN django generated model as of $today\" django_bdtopo/apps/administratif/models.py\nsed -i \"1i\\# Original IGN django generated model as of $today\" django_bdtopo/apps/bati/models.py\nsed -i \"1i\\# Original IGN django generated model as of $today\" django_bdtopo/apps/hydrographie/models.py\nsed -i \"1i\\# Original IGN django generated model as of $today\" django_bdtopo/apps/lieux_nommes/models.py\nsed -i \"1i\\# Original IGN django generated model as of $today\" django_bdtopo/apps/occupation_du_sol/models.py\nsed -i \"1i\\# Original IGN django generated model as of $today\" django_bdtopo/apps/services_et_activites/models.py\nsed -i \"1i\\# Original IGN django generated model as of $today\" django_bdtopo/apps/transport/models.py\nsed -i \"1i\\# Original IGN django generated model as of $today\" django_bdtopo/apps/zones_reglementees/models.py\n```\n\n8) Remove the lines containing managed = False\n```sh\nsed -i '/managed = False/d' django_bdtopo/apps/administratif/models.py\nsed -i '/managed = False/d' django_bdtopo/apps/bati/models.py\nsed -i '/managed = False/d' django_bdtopo/apps/hydrographie/models.py\nsed -i '/managed = False/d' django_bdtopo/apps/lieux_nommes/models.py\nsed -i '/managed = False/d' django_bdtopo/apps/occupation_du_sol/models.py\nsed -i '/managed = False/d' django_bdtopo/apps/services_et_activites/models.py\nsed -i '/managed = False/d' django_bdtopo/apps/transport/models.py\nsed -i '/managed = False/d' django_bdtopo/apps/zones_reglementees/models.py\n```\n\n9) Rename models to remove the prefixes\n```sh\nsed -i 's/class Administratif/class /g' django_bdtopo/apps/administratif/models.py\nsed -i 's/class Bati/class /g' django_bdtopo/apps/bati/models.py\nsed -i 's/class Hydrographie/class /g' django_bdtopo/apps/hydrographie/models.py\nsed -i 's/class LieuxNommes/class /g' django_bdtopo/apps/lieux_nommes/models.py\nsed -i 's/class OccupationDuSol/class /g' django_bdtopo/apps/occupation_du_sol/models.py\nsed -i 's/class ServicesEtActivites/class /g' django_bdtopo/apps/services_et_activites/models.py\nsed -i 's/class Transport/class /g' django_bdtopo/apps/transport/models.py\nsed -i 's/class ZonesReglementees/class /g' django_bdtopo/apps/zones_reglementees/models.py\n```\n\n10) Rename the Epci model to EPCI\n```sh\nsed -i 's/class Epci/class EPCI/g' django_bdtopo/apps/administratif/models.py\n```\n\n*Move to a django project.*\n\n11) Generates the initial migration\n```sh\nrun -frontend makemigrations administratif\nrun -frontend makemigrations bati\nrun -frontend makemigrations hydrographie\nrun -frontend makemigrations lieux_nommes\nrun -frontend makemigrations occupation_du_sol\nrun -frontend makemigrations services_et_activites\nrun -frontend makemigrations transport\nrun -frontend makemigrations zones_reglementees\n```\n\n12) Run pre-commit on the generated files\n```sh\nfind django_bdtopo/*/migrations/ -name \"*.py\" -print | xargs pre-commit run --files\npre-commit run --files \"django_bdtopo/*/models.py\"\n```\n\n13) Fake the migrations\n```sh\nrun -frontend migrate administratif --fake\nrun -frontend migrate bati --fake\nrun -frontend migrate hydrographie --fake\nrun -frontend migrate lieux_nommes --fake\nrun -frontend migrate occupation_du_sol --fake\nrun -frontend migrate services_et_activites --fake\nrun -frontend migrate transport --fake\nrun -frontend migrate zones_reglementees --fake\n```\n\n14) That's all folks!\n\n----\n\n# Todo\n\n- update `create_models_links` to include other apps (now it creates fk only for administratif app)\n- auto-detect which apps are installed before running the migrations\n- meaningful errors messages in case of problem\n- add tests\n",
"bugtrack_url": null,
"license": null,
"summary": "Import and manage all IGN BDTOPO\u00ae data in one (customizable) app!",
"version": "0.0.1",
"project_urls": {
"Homepage": "https://gitlab.com/kapt/open-source/django-bdtopo"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "bee296076a645a2d4e03abcfa92b4a1161945d179546c2a6e90acd1ff77901fa",
"md5": "5208ffac14cc0e14d1cc4929a3cca9e7",
"sha256": "b87f0cb490366bf42d9e11cb6a6594dd3b1c7f3b285eee82476bd8f606cb2182"
},
"downloads": -1,
"filename": "django_bdtopo-0.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "5208ffac14cc0e14d1cc4929a3cca9e7",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 27379,
"upload_time": "2024-10-29T16:12:27",
"upload_time_iso_8601": "2024-10-29T16:12:27.907547Z",
"url": "https://files.pythonhosted.org/packages/be/e2/96076a645a2d4e03abcfa92b4a1161945d179546c2a6e90acd1ff77901fa/django_bdtopo-0.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "2977afd10e200e42796ef183b3ff639571b3fb6f23d7fb8612f77669e33285e2",
"md5": "27ec413ceb443a128fbfbf1630716738",
"sha256": "a7715928339a7d2182af93b4ad9a74a3486dcb64597bac161cb13c32a4fbdc3e"
},
"downloads": -1,
"filename": "django_bdtopo-0.0.1.tar.gz",
"has_sig": false,
"md5_digest": "27ec413ceb443a128fbfbf1630716738",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 29029,
"upload_time": "2024-10-29T16:12:29",
"upload_time_iso_8601": "2024-10-29T16:12:29.513101Z",
"url": "https://files.pythonhosted.org/packages/29/77/afd10e200e42796ef183b3ff639571b3fb6f23d7fb8612f77669e33285e2/django_bdtopo-0.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-29 16:12:29",
"github": false,
"gitlab": true,
"bitbucket": false,
"codeberg": false,
"gitlab_user": "kapt",
"gitlab_project": "open-source",
"lcname": "django-bdtopo"
}
JSON