.. _readme:
Data.SyncMaster
===============
|Repo Status| |Docker image| |PyPI| |PyPI License| |PyPI Python Version| |Documentation|
|Build Status| |Coverage| |pre-commit.ci|
.. |Repo Status| image:: https://www.repostatus.org/badges/latest/wip.svg
:target: https://www.repostatus.org/#wip
.. |Docker image| image:: https://img.shields.io/docker/v/mtsrus/syncmaster-server?sort=semver&label=docker
:target: https://hub.docker.com/r/mtsrus/syncmaster-server
.. |PyPI| image:: https://img.shields.io/pypi/v/data-syncmaster
:target: https://pypi.org/project/data-syncmaster/
.. |PyPI License| image:: https://img.shields.io/pypi/l/data-syncmaster.svg
:target: https://github.com/MobileTeleSystems/syncmaster/blob/develop/LICENSE.txt
.. |PyPI Python Version| image:: https://img.shields.io/pypi/pyversions/data-syncmaster.svg
:target: https://badge.fury.io/py/data-syncmaster
.. |Documentation| image:: https://readthedocs.org/projects/syncmaster/badge/?version=stable
:target: https://syncmaster.readthedocs.io
.. |Build Status| image:: https://github.com/MobileTeleSystems/syncmaster/workflows/Run%20All%20Tests/badge.svg
:target: https://github.com/MobileTeleSystems/syncmaster/actions
.. |Coverage| image:: https://codecov.io/gh/MobileTeleSystems/syncmaster/graph/badge.svg?token=ky7UyUxolB
:target: https://codecov.io/gh/MobileTeleSystems/syncmaster
.. |pre-commit.ci| image:: https://results.pre-commit.ci/badge/github/MobileTeleSystems/syncmaster/develop.svg
:target: https://results.pre-commit.ci/latest/github/MobileTeleSystems/syncmaster/develop
What is Data.SyncMaster?
------------------------
Data.SyncMaster is as low-code ETL tool for transfering data between databases and file systems.
List of currently supported connections:
* Apache Hive
* Clickhouse
* Postgres
* Oracle
* MSSQL
* MySQL
* HDFS
* S3
* FTP
* FTPS
* SFTP
* Samba
* WebDAV
Based on `onETL <https://onetl.readthedocs.io/>`_ and `Apache Spark <https://spark.apache.org/>`_.
**Note**: service is under active development, and is not ready to use.
Goals
-----
* Make transfering data between databases and file systems as simple as possible
* Provide a lot of builtin connectors to transfer data in heterogeneous environment
* RBAC and multitenancy support
Non-goals
---------
* This is not a backup system
* Only batch, no streaming
.. documentation
Documentation
-------------
See https://syncmaster.readthedocs.io
Raw data
{
"_id": null,
"home_page": null,
"name": "data-syncmaster",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.11",
"maintainer_email": null,
"keywords": "Syncmaster, REST, API, Worker, Spark, Transfer, ETL",
"author": "DataOps.ETL",
"author_email": "onetools@mts.ru",
"download_url": "https://files.pythonhosted.org/packages/b7/18/110b185c6eaff6c3d5c79b0e65a2d58a95815b9e8cfc6f30dc1053ab82cc/data_syncmaster-0.2.4.tar.gz",
"platform": null,
"description": ".. _readme:\n\nData.SyncMaster\n===============\n\n|Repo Status| |Docker image| |PyPI| |PyPI License| |PyPI Python Version| |Documentation|\n|Build Status| |Coverage| |pre-commit.ci|\n\n.. |Repo Status| image:: https://www.repostatus.org/badges/latest/wip.svg\n :target: https://www.repostatus.org/#wip\n.. |Docker image| image:: https://img.shields.io/docker/v/mtsrus/syncmaster-server?sort=semver&label=docker\n :target: https://hub.docker.com/r/mtsrus/syncmaster-server\n.. |PyPI| image:: https://img.shields.io/pypi/v/data-syncmaster\n :target: https://pypi.org/project/data-syncmaster/\n.. |PyPI License| image:: https://img.shields.io/pypi/l/data-syncmaster.svg\n :target: https://github.com/MobileTeleSystems/syncmaster/blob/develop/LICENSE.txt\n.. |PyPI Python Version| image:: https://img.shields.io/pypi/pyversions/data-syncmaster.svg\n :target: https://badge.fury.io/py/data-syncmaster\n.. |Documentation| image:: https://readthedocs.org/projects/syncmaster/badge/?version=stable\n :target: https://syncmaster.readthedocs.io\n.. |Build Status| image:: https://github.com/MobileTeleSystems/syncmaster/workflows/Run%20All%20Tests/badge.svg\n :target: https://github.com/MobileTeleSystems/syncmaster/actions\n.. |Coverage| image:: https://codecov.io/gh/MobileTeleSystems/syncmaster/graph/badge.svg?token=ky7UyUxolB\n :target: https://codecov.io/gh/MobileTeleSystems/syncmaster\n.. |pre-commit.ci| image:: https://results.pre-commit.ci/badge/github/MobileTeleSystems/syncmaster/develop.svg\n :target: https://results.pre-commit.ci/latest/github/MobileTeleSystems/syncmaster/develop\n\n\nWhat is Data.SyncMaster?\n------------------------\n\nData.SyncMaster is as low-code ETL tool for transfering data between databases and file systems.\nList of currently supported connections:\n\n* Apache Hive\n* Clickhouse\n* Postgres\n* Oracle\n* MSSQL\n* MySQL\n* HDFS\n* S3\n* FTP\n* FTPS\n* SFTP\n* Samba\n* WebDAV\n\nBased on `onETL <https://onetl.readthedocs.io/>`_ and `Apache Spark <https://spark.apache.org/>`_.\n\n**Note**: service is under active development, and is not ready to use.\n\nGoals\n-----\n\n* Make transfering data between databases and file systems as simple as possible\n* Provide a lot of builtin connectors to transfer data in heterogeneous environment\n* RBAC and multitenancy support\n\nNon-goals\n---------\n\n* This is not a backup system\n* Only batch, no streaming\n\n.. documentation\n\nDocumentation\n-------------\n\nSee https://syncmaster.readthedocs.io\n\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Syncmaster REST API + Worker",
"version": "0.2.4",
"project_urls": {
"CI/CD": "https://github.com/MobileTeleSystems/syncmaster/actions",
"Documentation": "https://syncmaster.readthedocs.io",
"Homepage": "https://github.com/MobileTeleSystems/syncmaster",
"Source": "https://github.com/MobileTeleSystems/syncmaster",
"Tracker": "https://github.com/MobileTeleSystems/syncmaster/issues"
},
"split_keywords": [
"syncmaster",
" rest",
" api",
" worker",
" spark",
" transfer",
" etl"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "426efa4592b05a770c082ad65bde81748154d6d3f185bb5cf99ca1d80d2ddc3f",
"md5": "6fe5bfa0dc12223d1b8bd750bc215c42",
"sha256": "eb57c637845d954b28697672400b98dd761e6aa38fff2b282571320aa3272d55"
},
"downloads": -1,
"filename": "data_syncmaster-0.2.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6fe5bfa0dc12223d1b8bd750bc215c42",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.11",
"size": 175734,
"upload_time": "2025-07-09T14:35:27",
"upload_time_iso_8601": "2025-07-09T14:35:27.355334Z",
"url": "https://files.pythonhosted.org/packages/42/6e/fa4592b05a770c082ad65bde81748154d6d3f185bb5cf99ca1d80d2ddc3f/data_syncmaster-0.2.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "b718110b185c6eaff6c3d5c79b0e65a2d58a95815b9e8cfc6f30dc1053ab82cc",
"md5": "459271d9e23c5e5a677de5f85143d905",
"sha256": "789db17adc9d5f9a64075ba4d9cc3f34048ff5f833510b3c97b2fb10d7facbb3"
},
"downloads": -1,
"filename": "data_syncmaster-0.2.4.tar.gz",
"has_sig": false,
"md5_digest": "459271d9e23c5e5a677de5f85143d905",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.11",
"size": 80760,
"upload_time": "2025-07-09T14:35:29",
"upload_time_iso_8601": "2025-07-09T14:35:29.117081Z",
"url": "https://files.pythonhosted.org/packages/b7/18/110b185c6eaff6c3d5c79b0e65a2d58a95815b9e8cfc6f30dc1053ab82cc/data_syncmaster-0.2.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-09 14:35:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "MobileTeleSystems",
"github_project": "syncmaster",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "data-syncmaster"
}
JSON
