# data-file-merge
This python module/CLI allows you to merge multiple JSON files into a single destination JSON file using a set of rules defined in the [dfm config file](https://github.com/ServerlessSam/data-file-merge/wiki/Configuration)
Please refer to the [project wiki](https://github.com/ServerlessSam/data-file-merge/wiki) for complete documentation including [usage](https://github.com/ServerlessSam/data-file-merge/wiki/Usage) and [configuration file syntax](https://github.com/ServerlessSam/data-file-merge/wiki/Configuration).
# Hello World Example
You can install the CLI for your operating system from [GitHub here](https://github.com/ServerlessSam/data-file-merge/releases).
Using the following config file:
```json
{
"SourceFiles" : [
{
"SourceFileLocation" : {
"Path" : "foo/bar.json"
},
"SourceFileNode" : "$",
"DestinationFileNode" : "$"
},
{
"SourceFileLocation" : {
"Path" : "foo/baz.json"
},
"SourceFileNode" : "$",
"DestinationFileNode" : "$"
},
],
"DestinationFile" : {
"DestinationFileLocation" : {
"Path" : "hello/world.json"
}
}
}
```
...you are able to merge the content of `foo/bar.json` and `foo/baz.json` into `hello/world.json` (this can either be an existing or new file). The `$`s indicate that we are merging from the root node of both `foo/*.json` files to the root of `hello/world.json`.
This merge can be achieved with the CLI command `dfm merge --root-path <path to 'foo' directory> <path to this config file>`.
The specific rules followed by the merging logic can be found in [the wiki](https://github.com/ServerlessSam/data-file-merge/wiki/Merging-Logic).
More examples can be found in the [dfm example repo](https://github.com/ServerlessSam/dfm-examples).
## Considerations
* Currently only JSON files are supported but YAML files could be supported with ease in future.
* Doing the reverse operation will soon be supported (splitting a single sourcefile into multiple destination files).
* You should be aware of:
* [json-path's dollar-notation syntax](https://pypi.org/project/jsonpath-ng/)
* [path-lib's path syntax](https://docs.python.org/3/library/pathlib.html)
* [regex syntax](https://www.rexegg.com/regex-quickstart.html)
## Development
* install pre-commit hooks: ``poetry run pre-commit install``
* check and fix linting issues: ``make format``
## Contributing to the project
You are welcome to help us rid the world of monolithic data files! Reach out to [ServerlessSam](https://github.com/ServerlessSam) to either find some tickets to complete or bring up any ideas you have.
Raw data
{
"_id": null,
"home_page": "https://github.com/ServerlessSam/data-file-merge",
"name": "data-file-merge",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "configuration management merge split yaml json data files",
"author": "Samuel Lock",
"author_email": "serverlesssam@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/72/2b/e3a41e9a0b117b889dd79e8df09e2665d5bfd158a75a8622e8c71cb16186/data-file-merge-0.3.1.tar.gz",
"platform": null,
"description": "# data-file-merge\n\nThis python module/CLI allows you to merge multiple JSON files into a single destination JSON file using a set of rules defined in the [dfm config file](https://github.com/ServerlessSam/data-file-merge/wiki/Configuration)\n\nPlease refer to the [project wiki](https://github.com/ServerlessSam/data-file-merge/wiki) for complete documentation including [usage](https://github.com/ServerlessSam/data-file-merge/wiki/Usage) and [configuration file syntax](https://github.com/ServerlessSam/data-file-merge/wiki/Configuration).\n\n# Hello World Example\n\nYou can install the CLI for your operating system from [GitHub here](https://github.com/ServerlessSam/data-file-merge/releases).\n\nUsing the following config file:\n```json\n{\n \"SourceFiles\" : [\n {\n \"SourceFileLocation\" : {\n \"Path\" : \"foo/bar.json\"\n },\n \"SourceFileNode\" : \"$\",\n \"DestinationFileNode\" : \"$\"\n },\n {\n \"SourceFileLocation\" : {\n \"Path\" : \"foo/baz.json\"\n },\n \"SourceFileNode\" : \"$\",\n \"DestinationFileNode\" : \"$\"\n },\n ],\n \"DestinationFile\" : {\n \"DestinationFileLocation\" : {\n \"Path\" : \"hello/world.json\"\n }\n }\n}\n```\n...you are able to merge the content of `foo/bar.json` and `foo/baz.json` into `hello/world.json` (this can either be an existing or new file). The `$`s indicate that we are merging from the root node of both `foo/*.json` files to the root of `hello/world.json`.\nThis merge can be achieved with the CLI command `dfm merge --root-path <path to 'foo' directory> <path to this config file>`.\nThe specific rules followed by the merging logic can be found in [the wiki](https://github.com/ServerlessSam/data-file-merge/wiki/Merging-Logic).\n\nMore examples can be found in the [dfm example repo](https://github.com/ServerlessSam/dfm-examples).\n\n## Considerations\n\n* Currently only JSON files are supported but YAML files could be supported with ease in future.\n* Doing the reverse operation will soon be supported (splitting a single sourcefile into multiple destination files).\n* You should be aware of:\n * [json-path's dollar-notation syntax](https://pypi.org/project/jsonpath-ng/)\n * [path-lib's path syntax](https://docs.python.org/3/library/pathlib.html)\n * [regex syntax](https://www.rexegg.com/regex-quickstart.html)\n\n## Development\n\n* install pre-commit hooks: ``poetry run pre-commit install``\n* check and fix linting issues: ``make format``\n\n## Contributing to the project\n\nYou are welcome to help us rid the world of monolithic data files! Reach out to [ServerlessSam](https://github.com/ServerlessSam) to either find some tickets to complete or bring up any ideas you have.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Config-driven merging and splitting of JSON data files.",
"version": "0.3.1",
"split_keywords": [
"configuration",
"management",
"merge",
"split",
"yaml",
"json",
"data",
"files"
],
"urls": [
{
"comment_text": "",
"digests": {
"md5": "c9edd112a94fa7e9d11c2c9ab1c89e22",
"sha256": "93fbae88eb4377db7f6283b02795ba225e7fe5b21ea9026bf34c5c5177f01107"
},
"downloads": -1,
"filename": "data-file-merge-0.3.1.tar.gz",
"has_sig": false,
"md5_digest": "c9edd112a94fa7e9d11c2c9ab1c89e22",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 12272,
"upload_time": "2022-12-30T13:13:38",
"upload_time_iso_8601": "2022-12-30T13:13:38.158934Z",
"url": "https://files.pythonhosted.org/packages/72/2b/e3a41e9a0b117b889dd79e8df09e2665d5bfd158a75a8622e8c71cb16186/data-file-merge-0.3.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-12-30 13:13:38",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "ServerlessSam",
"github_project": "data-file-merge",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"circle": true,
"lcname": "data-file-merge"
}
JSON
