[`esy.osm.topology`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology.html) is a library to implement topology construction heuristics
from visual data like [OpenStreetMap](https://www.openstreetmap.org). It can be
used to construct workflows to extract infrastructure topologies like
electricity grids from OpenStreetMap data, for example.
As it is very unlikely that visual data is complete and coherent, the topology
construction is usually ambiguous. Missing data or inaccuracies must be fixed
case-by-case in individual workflows.
[`esy.osm.topology`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology.html) is designed for frictionless embedding into workflows and
tries to limit its dependencies to [numpy](https://numpy.org) for number
crunching, [scipy](https://scipy.org) for graph algorithms and
[shapely](https://shapely.readthedocs.io) for geometry processing.
[esy.osm.shape](https://dlr-ve-esy.gitlab.io/esy-osm-pbf/) (which pulls in
[Protocol Buffers](https://protobuf.dev) as transitive dependency) is required
to extract geometries from OpenStreetMap. Additionally,
[pyproj](https://pyproj4.github.io/pyproj/) is recommended for OpenStreetMap
data to convert coordinates to cartesian.
# Features
What it provides:
- Functions to construct graphs from [shapely](https://shapely.readthedocs.io)
geometry and vice-versa.
- Functions to reduce visual representations to topologies.
- Matplotlib graph visualization functions.
- GeoJSON output functions.
What it *doesn't* provide:
- Ready to use extraction of topology datasets from OpenStreetMap. For example
users need to:
- Assign topology attributes (like link capacities, ...)
- Merge multiple topologies (like electricity grid voltage levels, ...)
# Installation
Use `pip` to install [`esy.osm.topology`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology.html):
```
$ pip install esy-osm-topology
```
# Example workflow
[`esy.osm.topology`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology.html) provides functions to construct topologies from geometry
objects.
```python
>>> import numpy as np, scipy.sparse, matplotlib.pyplot as plt, shapely
>>> from esy.osm.topology.test import example
>>> import esy.osm.topology.graph, esy.osm.topology.mpl
>>>
>>> lines = [shapely.LineString(coords) for coords in [
... [[0, 1], [0, 0], [1, 0], [2, 0], [4, 0]],
... [[1, 0], [1, 1]],
... [[2, 0], [2, 1]],
... [[7, 0], [8, 0], [8, 2]],
... [[0, 0], [0, -1], [4, -1]],
... [[4, 0], [7, 0]],
... ]]
>>> terminals = [shapely.Polygon(coords) for coords in [
... [[-0.25, 0.75], [0.25, 0.75], [0, 1.25]],
... [[0.75, 0.75], [1.25, 0.75], [1, 1.25]],
... [[1.75, 0.75], [2.25, 0.75], [2, 1.25]],
... [[7.75, 1.75], [8.25, 1.75], [8, 2.25]],
... [[3.75, 0.25], [4.25, 0.25], [4.25, -1.25], [3.75, -1.25]],
... [[6.75, 0.25], [7.25, 0.25], [7.25, -0.25], [6.75, -0.25]],
... ]]
>>> with example('doc/figure/example/a.svg') as ax:
... _ = ax.add_collection(esy.osm.topology.mpl.patches(lines))
... _ = ax.add_collection(esy.osm.topology.mpl.patches(
... terminals, edgecolor='none', facecolor='lightgray')
... )
... ax.autoscale_view()
```
![](https://gitlab.com/dlr-ve/esy/esy.osm.topology/-/raw/main/doc/figure/example/a.svg)
## Construct graph from strings
Linestrings can be transformed to a spatial graph, that is an array of
coordinates and a sparse matrix of edges connecting those. The
[`esy.osm.topology.graph.from_string_coordinates()`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology/graph.html#from_string_coordinates) function constructs a spatial
graph from a list of shapely linestrings:
```python
>>> coords, graph, line_index = esy.osm.topology.graph.from_string_coordinates(
... [line.coords for line in lines]
... )
>>> coords.shape, graph.shape, line_index
((17, 2), (17, 17), array([0, 0, 0, 0, 1, 2, 3, 3, 4, 4, 5]))
>>> with example('doc/figure/example/b.svg') as ax:
... _ = ax.add_collection(esy.osm.topology.mpl.patches(
... terminals, edgecolor='none', facecolor='lightgray')
... )
... esy.osm.topology.mpl.annotate_graph(ax, coords, graph)
```
![](https://gitlab.com/dlr-ve/esy/esy.osm.topology/-/raw/main/doc/figure/example/b.svg)
[`esy.osm.topology.graph.from_string_coordinates()`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology/graph.html#from_string_coordinates) computes the coordinate array
and edges between those coordinates as well as an index to the originating
coordinate list.
As can be seen from the example, visual data may contain duplicate coordinates.
[`esy.osm.topology.graph.unique()`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology/graph.html#unique) can be used to detect duplicate coordinates
and remap edges.
```python
>>> graph, node_index, edge_index = esy.osm.topology.graph.unique(coords, graph)
>>> list(zip(graph.row.tolist(), graph.col.tolist()))
[(13, 14), (1, 13), (1, 2), (0, 1), (2, 6), (2, 3), (3, 8), (3, 4), (4, 9), (9, 10), (10, 11)]
>>> with example('doc/figure/example/c.svg') as ax:
... _ = ax.add_collection(esy.osm.topology.mpl.patches(
... terminals, edgecolor='none', facecolor='lightgray')
... )
... esy.osm.topology.mpl.annotate_graph(ax, coords, graph)
```
![](https://gitlab.com/dlr-ve/esy/esy.osm.topology/-/raw/main/doc/figure/example/c.svg)
## Identify terminal nodes
Terminal nodes of the infrastructure topologies might be visualized as geometry
shapes. The [`esy.osm.topology.graph.query_terminals()`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology/graph.html#query_terminals) function will identify
all edges that connect terminal nodes:
```python
>>> terminal_idx, terminal_map, edge_index = (
... esy.osm.topology.graph.query_terminals(coords, graph, terminals)
... )
>>> terminal_idx.tolist()
[0, 4, 6, 8, 9, 11, 14]
>>> graph = esy.osm.topology.graph.filter_edges(graph, edge_index)
>>> with example('doc/figure/example/d.svg') as ax:
... _ = ax.add_collection(esy.osm.topology.mpl.patches(
... terminals, edgecolor='none', facecolor='lightgray')
... )
... esy.osm.topology.mpl.annotate_graph(ax, coords, graph)
... esy.osm.topology.mpl.annotate_coords(
... ax, coords[terminal_idx], terminal_idx, facecolor='blue'
... )
```
![](https://gitlab.com/dlr-ve/esy/esy.osm.topology/-/raw/main/doc/figure/example/d.svg)
## Shortest path topology
Electricity grids tend to be mostly sequential line connections between terminal
nodes. A power line may be represented by a single linestring in the original
data, although it consists of multiple cables. The heuristic used to resolve
this ambiguity is the shortest path.
The [`esy.osm.topology.graph.terminal_topology()`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology/graph.html#terminal_topology) function computes the shortest
paths between all terminal nodes and collapses these into a topology graph.
```python
>>> topology, shapes, edge_paths = esy.osm.topology.graph.terminal_topology(
... coords, graph, terminal_idx, terminal_map
... )
>>> list(zip(topology.row.tolist(), topology.col.tolist(), topology.data.tolist()))
[(0, 14, 6.0), (0, 6, 3.0), (6, 8, 3.0), (8, 4, 3.0), (4, 9, 3.0), (9, 11, 3.0)]
>>> with example('doc/figure/example/e.svg') as ax:
... _ = ax.add_collection(esy.osm.topology.mpl.patches(
... terminals, edgecolor='none', facecolor='lightgray')
... )
... esy.osm.topology.mpl.annotate_graph(ax, coords, topology, facecolor='blue')
```
![](https://gitlab.com/dlr-ve/esy/esy.osm.topology/-/raw/main/doc/figure/example/e.svg)
# Powergrid example
See the [`esy.osm.topology.power`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology/power.html) module for an example based on real world data.
# Design & Development
Design and development notes are available in [`esy.osm.topology.test`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology/test.html).
# Contributions
We would be happy to accept contributions via merge request, but due to
corporate policy we can only accept contributions if you signed our
[contribution license agreement](CLA.md).
# License
[`esy.osm.topology`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology.html) is published under the
[BSD-3-Clause](https://spdx.org/licenses/BSD-3-Clause.html) license.
# The Team
[`esy.osm.topology`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology.html) is developed at the
[DLR](https://www.dlr.de/EN/Home/home_node.html) Institute of
[Networked Energy Systems](
https://www.dlr.de/ve/en/desktopdefault.aspx/tabid-12472/21440_read-49440/)
in the departement for [Energy Systems Analysis](
https://www.dlr.de/ve/en/desktopdefault.aspx/tabid-12471/21741_read-49802/).
# Acknowledgements
The authors would like to thank the Federal Government and the Heads of
Government of the Länder, as well as the Joint Science Conference (GWK), for
their funding and support within the framework of the NFDI4Ing consortium.
Funded by the German Research Foundation (DFG) - project number 442146713.
Raw data
{
"_id": null,
"home_page": null,
"name": "esy-osm-topology",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "OSM, OpenStreetMap, shapely, topology, graph",
"author": null,
"author_email": "Ontje L\u00fcnsdorf <ontje.luensdorf@dlr.de>",
"download_url": "https://files.pythonhosted.org/packages/41/0d/6d7a0c653e564e1ab6d14720f5f960a51dbd295ce28c5f421ce56776047d/esy_osm_topology-0.1.tar.gz",
"platform": null,
"description": "\n[`esy.osm.topology`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology.html) is a library to implement topology construction heuristics\nfrom visual data like [OpenStreetMap](https://www.openstreetmap.org). It can be\nused to construct workflows to extract infrastructure topologies like\nelectricity grids from OpenStreetMap data, for example.\n\nAs it is very unlikely that visual data is complete and coherent, the topology\nconstruction is usually ambiguous. Missing data or inaccuracies must be fixed\ncase-by-case in individual workflows.\n\n[`esy.osm.topology`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology.html) is designed for frictionless embedding into workflows and\ntries to limit its dependencies to [numpy](https://numpy.org) for number\ncrunching, [scipy](https://scipy.org) for graph algorithms and\n[shapely](https://shapely.readthedocs.io) for geometry processing.\n[esy.osm.shape](https://dlr-ve-esy.gitlab.io/esy-osm-pbf/) (which pulls in\n[Protocol Buffers](https://protobuf.dev) as transitive dependency) is required\nto extract geometries from OpenStreetMap. Additionally,\n[pyproj](https://pyproj4.github.io/pyproj/) is recommended for OpenStreetMap\ndata to convert coordinates to cartesian.\n\n# Features\n\nWhat it provides:\n\n- Functions to construct graphs from [shapely](https://shapely.readthedocs.io)\n geometry and vice-versa.\n- Functions to reduce visual representations to topologies.\n- Matplotlib graph visualization functions.\n- GeoJSON output functions.\n\nWhat it *doesn't* provide:\n\n- Ready to use extraction of topology datasets from OpenStreetMap. For example\n users need to:\n - Assign topology attributes (like link capacities, ...)\n - Merge multiple topologies (like electricity grid voltage levels, ...)\n\n# Installation\n\nUse `pip` to install [`esy.osm.topology`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology.html):\n\n```\n$ pip install esy-osm-topology\n```\n\n# Example workflow\n\n[`esy.osm.topology`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology.html) provides functions to construct topologies from geometry\nobjects.\n\n```python\n>>> import numpy as np, scipy.sparse, matplotlib.pyplot as plt, shapely\n>>> from esy.osm.topology.test import example\n>>> import esy.osm.topology.graph, esy.osm.topology.mpl\n>>> \n>>> lines = [shapely.LineString(coords) for coords in [\n... [[0, 1], [0, 0], [1, 0], [2, 0], [4, 0]],\n... [[1, 0], [1, 1]],\n... [[2, 0], [2, 1]],\n... [[7, 0], [8, 0], [8, 2]],\n... [[0, 0], [0, -1], [4, -1]],\n... [[4, 0], [7, 0]],\n... ]]\n>>> terminals = [shapely.Polygon(coords) for coords in [\n... [[-0.25, 0.75], [0.25, 0.75], [0, 1.25]],\n... [[0.75, 0.75], [1.25, 0.75], [1, 1.25]],\n... [[1.75, 0.75], [2.25, 0.75], [2, 1.25]],\n... [[7.75, 1.75], [8.25, 1.75], [8, 2.25]],\n... [[3.75, 0.25], [4.25, 0.25], [4.25, -1.25], [3.75, -1.25]],\n... [[6.75, 0.25], [7.25, 0.25], [7.25, -0.25], [6.75, -0.25]],\n... ]]\n>>> with example('doc/figure/example/a.svg') as ax:\n... _ = ax.add_collection(esy.osm.topology.mpl.patches(lines))\n... _ = ax.add_collection(esy.osm.topology.mpl.patches(\n... terminals, edgecolor='none', facecolor='lightgray')\n... )\n... ax.autoscale_view()\n \n```\n\n![](https://gitlab.com/dlr-ve/esy/esy.osm.topology/-/raw/main/doc/figure/example/a.svg)\n\n## Construct graph from strings\n\nLinestrings can be transformed to a spatial graph, that is an array of\ncoordinates and a sparse matrix of edges connecting those. The\n[`esy.osm.topology.graph.from_string_coordinates()`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology/graph.html#from_string_coordinates) function constructs a spatial\ngraph from a list of shapely linestrings:\n\n```python\n>>> coords, graph, line_index = esy.osm.topology.graph.from_string_coordinates(\n... [line.coords for line in lines]\n... )\n>>> coords.shape, graph.shape, line_index\n((17, 2), (17, 17), array([0, 0, 0, 0, 1, 2, 3, 3, 4, 4, 5]))\n>>> with example('doc/figure/example/b.svg') as ax:\n... _ = ax.add_collection(esy.osm.topology.mpl.patches(\n... terminals, edgecolor='none', facecolor='lightgray')\n... )\n... esy.osm.topology.mpl.annotate_graph(ax, coords, graph)\n\n```\n\n![](https://gitlab.com/dlr-ve/esy/esy.osm.topology/-/raw/main/doc/figure/example/b.svg)\n\n[`esy.osm.topology.graph.from_string_coordinates()`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology/graph.html#from_string_coordinates) computes the coordinate array\nand edges between those coordinates as well as an index to the originating\ncoordinate list.\n\nAs can be seen from the example, visual data may contain duplicate coordinates.\n[`esy.osm.topology.graph.unique()`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology/graph.html#unique) can be used to detect duplicate coordinates\nand remap edges.\n\n```python\n>>> graph, node_index, edge_index = esy.osm.topology.graph.unique(coords, graph)\n>>> list(zip(graph.row.tolist(), graph.col.tolist()))\n[(13, 14), (1, 13), (1, 2), (0, 1), (2, 6), (2, 3), (3, 8), (3, 4), (4, 9), (9, 10), (10, 11)]\n>>> with example('doc/figure/example/c.svg') as ax:\n... _ = ax.add_collection(esy.osm.topology.mpl.patches(\n... terminals, edgecolor='none', facecolor='lightgray')\n... )\n... esy.osm.topology.mpl.annotate_graph(ax, coords, graph)\n\n```\n\n![](https://gitlab.com/dlr-ve/esy/esy.osm.topology/-/raw/main/doc/figure/example/c.svg)\n\n## Identify terminal nodes\n\nTerminal nodes of the infrastructure topologies might be visualized as geometry\nshapes. The [`esy.osm.topology.graph.query_terminals()`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology/graph.html#query_terminals) function will identify\nall edges that connect terminal nodes:\n\n```python\n>>> terminal_idx, terminal_map, edge_index = (\n... esy.osm.topology.graph.query_terminals(coords, graph, terminals)\n... )\n>>> terminal_idx.tolist()\n[0, 4, 6, 8, 9, 11, 14]\n>>> graph = esy.osm.topology.graph.filter_edges(graph, edge_index)\n>>> with example('doc/figure/example/d.svg') as ax:\n... _ = ax.add_collection(esy.osm.topology.mpl.patches(\n... terminals, edgecolor='none', facecolor='lightgray')\n... )\n... esy.osm.topology.mpl.annotate_graph(ax, coords, graph)\n... esy.osm.topology.mpl.annotate_coords(\n... ax, coords[terminal_idx], terminal_idx, facecolor='blue'\n... )\n\n```\n\n![](https://gitlab.com/dlr-ve/esy/esy.osm.topology/-/raw/main/doc/figure/example/d.svg)\n\n## Shortest path topology\n\nElectricity grids tend to be mostly sequential line connections between terminal\nnodes. A power line may be represented by a single linestring in the original\ndata, although it consists of multiple cables. The heuristic used to resolve\nthis ambiguity is the shortest path.\n\nThe [`esy.osm.topology.graph.terminal_topology()`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology/graph.html#terminal_topology) function computes the shortest\npaths between all terminal nodes and collapses these into a topology graph.\n\n```python\n>>> topology, shapes, edge_paths = esy.osm.topology.graph.terminal_topology(\n... coords, graph, terminal_idx, terminal_map\n... )\n>>> list(zip(topology.row.tolist(), topology.col.tolist(), topology.data.tolist()))\n[(0, 14, 6.0), (0, 6, 3.0), (6, 8, 3.0), (8, 4, 3.0), (4, 9, 3.0), (9, 11, 3.0)]\n>>> with example('doc/figure/example/e.svg') as ax:\n... _ = ax.add_collection(esy.osm.topology.mpl.patches(\n... terminals, edgecolor='none', facecolor='lightgray')\n... )\n... esy.osm.topology.mpl.annotate_graph(ax, coords, topology, facecolor='blue')\n\n```\n\n![](https://gitlab.com/dlr-ve/esy/esy.osm.topology/-/raw/main/doc/figure/example/e.svg)\n\n# Powergrid example\n\nSee the [`esy.osm.topology.power`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology/power.html) module for an example based on real world data.\n\n# Design & Development\n\nDesign and development notes are available in [`esy.osm.topology.test`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology/test.html).\n\n# Contributions\n\nWe would be happy to accept contributions via merge request, but due to\ncorporate policy we can only accept contributions if you signed our\n[contribution license agreement](CLA.md).\n\n# License\n\n[`esy.osm.topology`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology.html) is published under the\n[BSD-3-Clause](https://spdx.org/licenses/BSD-3-Clause.html) license.\n\n# The Team\n\n[`esy.osm.topology`](https://dlr-ve.gitlab.io/esy/esy.osm.topology/esy/osm/topology.html) is developed at the\n[DLR](https://www.dlr.de/EN/Home/home_node.html) Institute of\n[Networked Energy Systems](\nhttps://www.dlr.de/ve/en/desktopdefault.aspx/tabid-12472/21440_read-49440/)\nin the departement for [Energy Systems Analysis](\nhttps://www.dlr.de/ve/en/desktopdefault.aspx/tabid-12471/21741_read-49802/).\n\n# Acknowledgements\n\nThe authors would like to thank the Federal Government and the Heads of\nGovernment of the L\u00e4nder, as well as the Joint Science Conference (GWK), for\ntheir funding and support within the framework of the NFDI4Ing consortium.\nFunded by the German Research Foundation (DFG) - project number 442146713.\n",
"bugtrack_url": null,
"license": "BSD-3-Clause",
"summary": "Construct topologies from visual data, like OpenStreetMap",
"version": "0.1",
"project_urls": {
"documentation": "https://dlr-ve.gitlab.io/esy/esy.osm.topology/",
"homepage": "https://gitlab.com/dlr-ve/esy/esy.osm.topology",
"tracker": "https://gitlab.com/dlr-ve/esy/esy.osm.topology/issues"
},
"split_keywords": [
"osm",
" openstreetmap",
" shapely",
" topology",
" graph"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "82599f34d734a819d972a29ad47d416732e145593d770eca4ac79d4819c5114d",
"md5": "924a365fc21c2d3267f85634b56475c2",
"sha256": "de33b8673d4bc2f7d48767366f1bc153eba483602d1a1f4ed5b3cae78d55d51c"
},
"downloads": -1,
"filename": "esy_osm_topology-0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "924a365fc21c2d3267f85634b56475c2",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 21308,
"upload_time": "2024-08-08T13:00:20",
"upload_time_iso_8601": "2024-08-08T13:00:20.914955Z",
"url": "https://files.pythonhosted.org/packages/82/59/9f34d734a819d972a29ad47d416732e145593d770eca4ac79d4819c5114d/esy_osm_topology-0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "410d6d7a0c653e564e1ab6d14720f5f960a51dbd295ce28c5f421ce56776047d",
"md5": "6c748cc3eea1d3daadca85f10613a0f8",
"sha256": "1d5f425c78da4f69557046a48fbd89f6f299259ce13b67947bde037f50f8db3d"
},
"downloads": -1,
"filename": "esy_osm_topology-0.1.tar.gz",
"has_sig": false,
"md5_digest": "6c748cc3eea1d3daadca85f10613a0f8",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 19485,
"upload_time": "2024-08-08T13:00:22",
"upload_time_iso_8601": "2024-08-08T13:00:22.967753Z",
"url": "https://files.pythonhosted.org/packages/41/0d/6d7a0c653e564e1ab6d14720f5f960a51dbd295ce28c5f421ce56776047d/esy_osm_topology-0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-08 13:00:22",
"github": false,
"gitlab": true,
"bitbucket": false,
"codeberg": false,
"gitlab_user": "dlr-ve",
"gitlab_project": "esy",
"lcname": "esy-osm-topology"
}