# DiffSync
DiffSync is a utility library that can be used to compare and synchronize different datasets.
For example, it can be used to compare a list of devices from 2 inventory systems and, if required, synchronize them in either direction.
# Primary Use Cases
DiffSync is at its most useful when you have multiple sources or sets of data to compare and/or synchronize, and especially if any of the following are true:
- If you need to repeatedly compare or synchronize the data sets as one or both change over time.
- If you need to account for not only the creation of new records, but also changes to and deletion of existing records as well.
- If various types of data in your data set naturally form a tree-like or parent-child relationship with other data.
- If the different data sets have some attributes in common and other attributes that are exclusive to one or the other.
# Overview of DiffSync
DiffSync acts as an intermediate translation layer between all of the data sets you are diffing and/or syncing. In practical terms, this means that to use DiffSync, you will define a set of data models as well as the “adapters” needed to translate between each base data source and the data model. In Python terms, the adapters will be subclasses of the `Adapter` class, and each data model class will be a subclass of the `DiffSyncModel` class.
Once you have used each adapter to load each data source into a collection of data model records, you can then ask DiffSync to “diff” the two data sets, and it will produce a structured representation of the difference between them. In Python, this is accomplished by calling the `diff_to()` or `diff_from()` method on one adapter and passing the other adapter as a parameter.
You can also ask DiffSync to “sync” one data set onto the other, and it will instruct your adapter as to the steps it needs to take to make sure that its data set accurately reflects the other. In Python, this is accomplished by calling the `sync_to()` or `sync_from()` method on one adapter and passing the other adapter as a parameter.
# Simple Example
```python
A = DiffSyncSystemA()
B = DiffSyncSystemB()
A.load()
B.load()
# Show the difference between both systems, that is, what would change if we applied changes from System B to System A
diff_a_b = A.diff_from(B)
print(diff_a_b.str())
# Update System A to align with the current status of system B
A.sync_from(B)
# Update System B to align with the current status of system A
A.sync_to(B)
```
> You may wish to peruse the `diffsync` [GitHub topic](https://github.com/topics/diffsync) for examples of projects using this library.
# Documentation
The documentation is available [on Read The Docs](https://diffsync.readthedocs.io/en/latest/index.html).
# Installation
### Option 1: Install from PyPI.
```
$ pip install diffsync
```
### Option 2: Install from a GitHub branch, such as main as shown below.
```
$ pip install git+https://github.com/networktocode/diffsync.git@main
```
# Contributing
Pull requests are welcomed and automatically built and tested against multiple versions of Python through GitHub Actions.
The project is following Network to Code software development guidelines and are leveraging the following:
- Black, Pylint, Bandit, flake8, and pydocstyle, mypy for Python linting, formatting and type hint checking.
- pytest, coverage, and unittest for unit tests.
You can ensure your contribution adheres to these checks by running `invoke tests` from the CLI.
The command `invoke build` builds a docker container with all the necessary dependencies (including the redis backend) locally to facilitate the execution of these tests.
# Questions
Please see the [documentation](https://diffsync.readthedocs.io/en/latest/index.html) for detailed documentation on how to use `diffsync`. For any additional questions or comments, feel free to swing by the [Network to Code slack channel](https://networktocode.slack.com/) (channel #networktocode). Sign up [here](http://slack.networktocode.com/)
Raw data
{
"_id": null,
"home_page": "https://diffsync.readthedocs.io",
"name": "diffsync",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8",
"maintainer_email": null,
"keywords": "source-of-truth, synchronization",
"author": "Network to Code, LLC",
"author_email": "info@networktocode.com",
"download_url": "https://files.pythonhosted.org/packages/b4/42/4aae9b5e73333803ca82a01652c4a10caeebdc719272527469727ddf1bb2/diffsync-2.0.1.tar.gz",
"platform": null,
"description": "# DiffSync\n\nDiffSync is a utility library that can be used to compare and synchronize different datasets.\n\nFor example, it can be used to compare a list of devices from 2 inventory systems and, if required, synchronize them in either direction.\n\n# Primary Use Cases\n\nDiffSync is at its most useful when you have multiple sources or sets of data to compare and/or synchronize, and especially if any of the following are true:\n\n- If you need to repeatedly compare or synchronize the data sets as one or both change over time.\n- If you need to account for not only the creation of new records, but also changes to and deletion of existing records as well.\n- If various types of data in your data set naturally form a tree-like or parent-child relationship with other data.\n- If the different data sets have some attributes in common and other attributes that are exclusive to one or the other.\n\n# Overview of DiffSync\n\nDiffSync acts as an intermediate translation layer between all of the data sets you are diffing and/or syncing. In practical terms, this means that to use DiffSync, you will define a set of data models as well as the \u201cadapters\u201d needed to translate between each base data source and the data model. In Python terms, the adapters will be subclasses of the `Adapter` class, and each data model class will be a subclass of the `DiffSyncModel` class.\n\n![DiffSync Components](https://raw.githubusercontent.com/networktocode/diffsync/develop/docs/images/diffsync_components.png \"DiffSync Components\")\n\n\nOnce you have used each adapter to load each data source into a collection of data model records, you can then ask DiffSync to \u201cdiff\u201d the two data sets, and it will produce a structured representation of the difference between them. In Python, this is accomplished by calling the `diff_to()` or `diff_from()` method on one adapter and passing the other adapter as a parameter.\n\n![DiffSync Diff Creation](https://raw.githubusercontent.com/networktocode/diffsync/develop/docs/images/diffsync_diff_creation.png \"DiffSync Diff Creation\")\n\nYou can also ask DiffSync to \u201csync\u201d one data set onto the other, and it will instruct your adapter as to the steps it needs to take to make sure that its data set accurately reflects the other. In Python, this is accomplished by calling the `sync_to()` or `sync_from()` method on one adapter and passing the other adapter as a parameter.\n\n![DiffSync Sync](https://raw.githubusercontent.com/networktocode/diffsync/develop/docs/images/diffsync_sync.png \"DiffSync Sync\")\n\n# Simple Example\n\n```python\nA = DiffSyncSystemA()\nB = DiffSyncSystemB()\n\nA.load()\nB.load()\n\n# Show the difference between both systems, that is, what would change if we applied changes from System B to System A\ndiff_a_b = A.diff_from(B)\nprint(diff_a_b.str())\n\n# Update System A to align with the current status of system B\nA.sync_from(B)\n\n# Update System B to align with the current status of system A\nA.sync_to(B)\n```\n\n> You may wish to peruse the `diffsync` [GitHub topic](https://github.com/topics/diffsync) for examples of projects using this library.\n\n# Documentation\n\nThe documentation is available [on Read The Docs](https://diffsync.readthedocs.io/en/latest/index.html).\n\n# Installation\n\n### Option 1: Install from PyPI.\n\n```\n$ pip install diffsync\n```\n\n### Option 2: Install from a GitHub branch, such as main as shown below.\n```\n$ pip install git+https://github.com/networktocode/diffsync.git@main\n```\n\n# Contributing\nPull requests are welcomed and automatically built and tested against multiple versions of Python through GitHub Actions.\n\nThe project is following Network to Code software development guidelines and are leveraging the following:\n\n- Black, Pylint, Bandit, flake8, and pydocstyle, mypy\u00a0for Python linting, formatting and type hint checking.\n- pytest, coverage, and unittest for unit tests.\n\nYou can ensure your contribution adheres to these checks by running `invoke tests` from the CLI.\nThe command `invoke build` builds a docker container with all the necessary dependencies (including the redis backend) locally to facilitate the execution of these tests.\n\n# Questions\nPlease see the [documentation](https://diffsync.readthedocs.io/en/latest/index.html) for detailed documentation on how to use `diffsync`. For any additional questions or comments, feel free to swing by the [Network to Code slack channel](https://networktocode.slack.com/) (channel #networktocode). Sign up [here](http://slack.networktocode.com/)\n\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Library to easily sync/diff/update 2 different data sources",
"version": "2.0.1",
"project_urls": {
"Documentation": "https://diffsync.readthedocs.io",
"Homepage": "https://diffsync.readthedocs.io",
"Repository": "https://github.com/networktocode/diffsync"
},
"split_keywords": [
"source-of-truth",
" synchronization"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2353b827778d2cd06db4c623ff931c10e53c550cfefc9b48ee197c16095e9dff",
"md5": "a946954cfa91043270bf830584bfa92a",
"sha256": "c375139d2d0c060106ef4f724044bd70ec03296f90255cd41831946e0544a585"
},
"downloads": -1,
"filename": "diffsync-2.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "a946954cfa91043270bf830584bfa92a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8",
"size": 39886,
"upload_time": "2024-10-22T05:43:09",
"upload_time_iso_8601": "2024-10-22T05:43:09.002840Z",
"url": "https://files.pythonhosted.org/packages/23/53/b827778d2cd06db4c623ff931c10e53c550cfefc9b48ee197c16095e9dff/diffsync-2.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "b4424aae9b5e73333803ca82a01652c4a10caeebdc719272527469727ddf1bb2",
"md5": "b6c9fea5b904de6208f24f5c8ac2375f",
"sha256": "7f6fe7705b0669f0249b18c97231b74bdfa2530117ad98f3b79a563c07ff7728"
},
"downloads": -1,
"filename": "diffsync-2.0.1.tar.gz",
"has_sig": false,
"md5_digest": "b6c9fea5b904de6208f24f5c8ac2375f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8",
"size": 31784,
"upload_time": "2024-10-22T05:43:10",
"upload_time_iso_8601": "2024-10-22T05:43:10.617187Z",
"url": "https://files.pythonhosted.org/packages/b4/42/4aae9b5e73333803ca82a01652c4a10caeebdc719272527469727ddf1bb2/diffsync-2.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-22 05:43:10",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "networktocode",
"github_project": "diffsync",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "diffsync"
}
JSON
