osmcha
=======
OSM Changeset Analyser, ``osmcha``, is a Python package to detect suspicious OSM changesets.
It was designed to be used with `osmcha-django <https://github.com/willemarcel/osmcha-django>`_,
but also can be used standalone or in other projects.
You can report issues or request new features in the the
`osmcha-frontend repository <https://github.com/mapbox/osmcha-frontend>`_.
.. image:: https://badge.fury.io/py/osmcha.svg
:target: http://badge.fury.io/py/osmcha
.. image:: https://coveralls.io/repos/willemarcel/osmcha/badge.svg
:target: https://coveralls.io/r/willemarcel/osmcha
.. image:: https://img.shields.io/badge/Contributor%20Covenant-v2.0%20adopted-ff69b4.svg
:target: code_of_conduct.md
Installation
============
.. code-block:: console
pip install osmcha
Usage
=====
Python Library
--------------
You can read a `replication changeset file <https://planet.openstreetmap.org/replication/changesets/>`_
directly from the web:
.. code-block:: python
from osmcha.changeset import ChangesetList
c = ChangesetList('https://planet.openstreetmap.org/replication/changesets/002/236/374.osm.gz')
or from your local filesystem.
.. code-block:: python
c = ChangesetList('tests/245.osm.gz')
``c.changesets`` will return a list containing data of all the changesets listed in the file.
You can filter the changesets passing a `GeoJSON` file with a polygon with your
interest area to `ChangesetList` as the second argument.
Finally, to analyse an especific changeset, do:
.. code-block:: python
from osmcha.changeset import Analyse
ch = Analyse(changeset_id)
ch.full_analysis()
Customizing Detection Rules
~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can customize the detection rules by defining your prefered values when
initializing the ``Analyze`` class. See below the default values.
.. code-block:: python
ch = Analyse(changeset_id, create_threshold=200, modify_threshold=200,
delete_threshold=30, percentage=0.7, top_threshold=1000,
suspect_words=[...], illegal_sources=[...], excluded_words=[...])
Command Line Interface
----------------------
The command line interface can be used to verify an especific changeset directly
from the terminal.
Usage: ``osmcha <changeset_id>``
Detection Rules
===============
``osmcha`` works by analysing how many map features the changeset created, modified
or deleted, and by verifying the presence of some suspect words in the ``comment``,
``source`` and ``imagery_used`` fields of the changeset. Furthermore, we also
consider if the software editor used allows to import data or to do mass edits.
We consider ``powerfull editors``: JOSM, Merkaartor, level0, QGIS and ArcGis.
In the ``Usage`` section, you can see how to customize some of these detection rules.
Possible Import
---------------
We tag a changeset as a ``possible import`` if the number of created elements is
greater than 70% of the sum of elements created, modified and deleted and if it
creates more than 1000 elements or 200 elements case it used one of the ``powerfull editors``.
Mass Modification
-----------------
We consider a changeset as a ``mass modification`` if the number of modified elements
is greater than 70% of the sum of elements created, modified and deleted and if it
modifies more than 200 elements.
Mass Deletion
-------------
All changesets that delete more than 1000 elements are considered a ``mass deletion``.
If the changeset deletes between 200 and 1000 elements and the number of deleted
elements is greater than 70% of the sum of elements created, modified and deleted
it's also tagged as a ``mass deletion``.
Suspect words
-------------
The suspect words are loaded from a `yaml file <osmcha/suspect_words.yaml>`_.
You can customize the words by setting another default file with a environment
variable:
.. code-block:: console
export SUSPECT_WORDS=<path_to_the_file>
or pass a list of words to the ``Analyse`` class, more information on the section
``Customizing Detection Rules``. We use a list of illegal sources to analyse the
``source`` and ``imagery_used`` fields and another more general list to examine
the comment field. We have also a list of excluded words to avoid false positives.
New mapper
-----------
Verify if the user has less than 5 edits or less than 5 mapping days.
User has multiple blocks
------------------------
Changesets created by users that has received more than one block will be
flagged.
OSM Server Configuration
------------------------
If you need to use OSMCha with another OSM server instance, you need to configure the OSM_SERVER_URL environment variable, without trailing slash. Example:
.. code-block:: console
export OSM_SERVER_URL='https://www.openhistoricalmap.org'
Tests
======
To run the tests on `osmcha`:
.. code-block:: console
git clone https://github.com/willemarcel/osmcha.git
cd osmcha
pip install -e .[test]
py.test -v
Publishing a new version
=========================
Update the version number in ``osmcha/__init__.py`` and execute
the following commands:
.. code-block:: console
python setup.py bdist_wheel
twine upload dist/osmcha-{version}...
Changelog
=========
Check `CHANGELOG <CHANGELOG.rst>`_ for the version history.
Related projects
================
* `osmcha-django <https://github.com/willemarcel/osmcha-django>`_ - backend and API
* `osmcha-frontend <https://github.com/mapbox/osmcha-frontend>`_ - frontend of the `OSMCha <https://osmcha.org>`_ application
* `osm-compare <https://github.com/mapbox/osm-compare>`_ - library that analyse OSM features to input it to OSMCha
License
=======
GPLv3
Raw data
{
"_id": null,
"home_page": "https://github.com/willemarcel/osmcha",
"name": "osmcha",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "openstreetmap,osm,QA,gis",
"author": "Wille Marcel",
"author_email": "wille.yyz@gmail.com",
"download_url": "",
"platform": null,
"description": "osmcha\n=======\n\nOSM Changeset Analyser, ``osmcha``, is a Python package to detect suspicious OSM changesets.\nIt was designed to be used with `osmcha-django <https://github.com/willemarcel/osmcha-django>`_,\nbut also can be used standalone or in other projects.\n\nYou can report issues or request new features in the the\n`osmcha-frontend repository <https://github.com/mapbox/osmcha-frontend>`_.\n\n.. image:: https://badge.fury.io/py/osmcha.svg\n :target: http://badge.fury.io/py/osmcha\n\n.. image:: https://coveralls.io/repos/willemarcel/osmcha/badge.svg\n :target: https://coveralls.io/r/willemarcel/osmcha\n\n.. image:: https://img.shields.io/badge/Contributor%20Covenant-v2.0%20adopted-ff69b4.svg\n :target: code_of_conduct.md\n\nInstallation\n============\n\n.. code-block:: console\n\n pip install osmcha\n\nUsage\n=====\n\nPython Library\n--------------\n\nYou can read a `replication changeset file <https://planet.openstreetmap.org/replication/changesets/>`_\ndirectly from the web:\n\n.. code-block:: python\n\n from osmcha.changeset import ChangesetList\n c = ChangesetList('https://planet.openstreetmap.org/replication/changesets/002/236/374.osm.gz')\n\nor from your local filesystem.\n\n.. code-block:: python\n\n c = ChangesetList('tests/245.osm.gz')\n\n\n``c.changesets`` will return a list containing data of all the changesets listed in the file.\n\nYou can filter the changesets passing a `GeoJSON` file with a polygon with your\ninterest area to `ChangesetList` as the second argument.\n\nFinally, to analyse an especific changeset, do:\n\n.. code-block:: python\n\n from osmcha.changeset import Analyse\n ch = Analyse(changeset_id)\n ch.full_analysis()\n\nCustomizing Detection Rules\n~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nYou can customize the detection rules by defining your prefered values when\ninitializing the ``Analyze`` class. See below the default values.\n\n.. code-block:: python\n\n ch = Analyse(changeset_id, create_threshold=200, modify_threshold=200,\n delete_threshold=30, percentage=0.7, top_threshold=1000,\n suspect_words=[...], illegal_sources=[...], excluded_words=[...])\n\nCommand Line Interface\n----------------------\n\nThe command line interface can be used to verify an especific changeset directly\nfrom the terminal.\n\nUsage: ``osmcha <changeset_id>``\n\nDetection Rules\n===============\n\n``osmcha`` works by analysing how many map features the changeset created, modified\nor deleted, and by verifying the presence of some suspect words in the ``comment``,\n``source`` and ``imagery_used`` fields of the changeset. Furthermore, we also\nconsider if the software editor used allows to import data or to do mass edits.\nWe consider ``powerfull editors``: JOSM, Merkaartor, level0, QGIS and ArcGis.\n\nIn the ``Usage`` section, you can see how to customize some of these detection rules.\n\nPossible Import\n---------------\n\nWe tag a changeset as a ``possible import`` if the number of created elements is\ngreater than 70% of the sum of elements created, modified and deleted and if it\ncreates more than 1000 elements or 200 elements case it used one of the ``powerfull editors``.\n\nMass Modification\n-----------------\n\nWe consider a changeset as a ``mass modification`` if the number of modified elements\nis greater than 70% of the sum of elements created, modified and deleted and if it\nmodifies more than 200 elements.\n\nMass Deletion\n-------------\n\nAll changesets that delete more than 1000 elements are considered a ``mass deletion``.\nIf the changeset deletes between 200 and 1000 elements and the number of deleted\nelements is greater than 70% of the sum of elements created, modified and deleted\nit's also tagged as a ``mass deletion``.\n\nSuspect words\n-------------\n\nThe suspect words are loaded from a `yaml file <osmcha/suspect_words.yaml>`_.\nYou can customize the words by setting another default file with a environment\nvariable:\n\n.. code-block:: console\n\n export SUSPECT_WORDS=<path_to_the_file>\n\nor pass a list of words to the ``Analyse`` class, more information on the section\n``Customizing Detection Rules``. We use a list of illegal sources to analyse the\n``source`` and ``imagery_used`` fields and another more general list to examine\nthe comment field. We have also a list of excluded words to avoid false positives.\n\n\nNew mapper\n-----------\n\nVerify if the user has less than 5 edits or less than 5 mapping days.\n\n\nUser has multiple blocks\n------------------------\n\nChangesets created by users that has received more than one block will be\nflagged.\n\nOSM Server Configuration\n------------------------\n\nIf you need to use OSMCha with another OSM server instance, you need to configure the OSM_SERVER_URL environment variable, without trailing slash. Example:\n\n.. code-block:: console\n\n export OSM_SERVER_URL='https://www.openhistoricalmap.org'\n\nTests\n======\n\nTo run the tests on `osmcha`:\n\n.. code-block:: console\n\n git clone https://github.com/willemarcel/osmcha.git\n cd osmcha\n pip install -e .[test]\n py.test -v\n\nPublishing a new version\n=========================\n\nUpdate the version number in ``osmcha/__init__.py`` and execute\nthe following commands:\n\n.. code-block:: console\n\n python setup.py bdist_wheel\n twine upload dist/osmcha-{version}...\n\nChangelog\n=========\n\nCheck `CHANGELOG <CHANGELOG.rst>`_ for the version history.\n\nRelated projects\n================\n\n* `osmcha-django <https://github.com/willemarcel/osmcha-django>`_ - backend and API\n* `osmcha-frontend <https://github.com/mapbox/osmcha-frontend>`_ - frontend of the `OSMCha <https://osmcha.org>`_ application\n* `osm-compare <https://github.com/mapbox/osm-compare>`_ - library that analyse OSM features to input it to OSMCha\n\nLicense\n=======\n\nGPLv3\n\n\n",
"bugtrack_url": null,
"license": "GPLv3+",
"summary": "Python package to detect suspicious OpenStreetMap changesets",
"version": "0.9.1",
"project_urls": {
"Homepage": "https://github.com/willemarcel/osmcha"
},
"split_keywords": [
"openstreetmap",
"osm",
"qa",
"gis"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "cc670fd916443478b3c6a612626e4e0205d0aef73b92816020168065c61cb938",
"md5": "3bdbb02062004e2b6009069d34d0297f",
"sha256": "424cf64b1d944b39c70d71cac36212ae2191baf09e434360deea3a1f5214a6f5"
},
"downloads": -1,
"filename": "osmcha-0.9.1-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "3bdbb02062004e2b6009069d34d0297f",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 22597,
"upload_time": "2024-02-23T21:07:29",
"upload_time_iso_8601": "2024-02-23T21:07:29.376850Z",
"url": "https://files.pythonhosted.org/packages/cc/67/0fd916443478b3c6a612626e4e0205d0aef73b92816020168065c61cb938/osmcha-0.9.1-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-23 21:07:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "willemarcel",
"github_project": "osmcha",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "osmcha"
}
JSON
