# Sync-stream
<p><img alt="Banner" src="https://repository-images.githubusercontent.com/370331958/6ca07907-3936-444e-8d0d-39c9a0867ef3"></p>
<p align="center">
<a href="https://github.com/cainmagi/sync-stream/releases/latest"><img alt="GitHub release (latest SemVer)" src="https://img.shields.io/github/v/release/cainmagi/sync-stream?logo=github&sort=semver&style=flat-square"></a>
<a href="https://github.com/cainmagi/sync-stream/releases"><img alt="GitHub all releases" src="https://img.shields.io/github/downloads/cainmagi/sync-stream/total?logo=github&style=flat-square"></a>
<a href="https://github.com/cainmagi/sync-stream/blob/main/LICENSE"><img alt="GitHub" src="https://img.shields.io/github/license/cainmagi/sync-stream?style=flat-square&logo=opensourceinitiative&logoColor=white"></a>
<a href="https://pypi.org/project/syncstream"><img alt="PyPI - Downloads" src="https://img.shields.io/pypi/dm/syncstream?style=flat-square&logo=pypi&logoColor=white&label=pypi"/></a>
</p>
<p align="center">
<a href="https://github.com/cainmagi/sync-stream/actions/workflows/python-package.yml"><img alt="GitHub Actions (Build)" src="https://img.shields.io/github/actions/workflow/status/cainmagi/sync-stream/python-package.yml?style=flat-square&logo=githubactions&logoColor=white&label=build"></a>
<a href="https://github.com/cainmagi/sync-stream/actions/workflows/python-publish.yml"><img alt="GitHub Actions (Release)" src="https://img.shields.io/github/actions/workflow/status/cainmagi/sync-stream/python-publish.yml?style=flat-square&logo=githubactions&logoColor=white&label=release"></a>
</p>
This project is designed for providing the synchoronization of the stdout / stderr among different threads, processes, devices or hosts. The package could be used for the following cases:
1. Use `syncstream.LineBuffer`: Multiple threads are created. The messages (stdout) from different threads are required to be collected.
2. Use `syncstream.LineProcBuffer` in the main process, and `syncstream.LineProcMirror` in the sub-processes: Multiple sub-processes are created on the same device. The stdout / stderr of each process is redirected to a `LineProcMirror`, and the results are collected by `LineProcBuffer`.
3. Use `syncstream.LineFileBuffer`: Multiple processes are created. These processes may be deployed on different devices (even with different platforms), but all devices could get accessed to the same shared disk. In this case, the message could be shared by locked files. Each process would hold an independent `LineFileBuffer` pointing to the same log files.
4. Use `syncstream.LineHostBuffer` on the server side, and `syncstream.LineHostMirror` on the client side: Multiple processes are deployed on different devices, and they could not share the same disk. In this case, the message are synchronized by the web service. Each process would hold a `LineHostMirror`, and the results would be collected by `LineHostBuffer`.
The basic package would not contain the `file` and `host` modules. To install the package, please use the following options:
```bash
pip install syncstream[option1,option2...]
```
| Option | Supports |
| :----: | :---------------------------------------------------------------------------------------------------------------------------- |
| `file` | Install dependencies for the `file` module. The module provides `syncstream.LineFileBuffer`. |
| `host` | Install dependencies for the `host` module. The module provides `syncstream.LineHostBuffer`, and `syncstream.LineHostMirror`. |
| `test` | Install dependencies for running tests. |
| `dev` | Install dependencies for developers. |
## Documentation
View the documentation here: [:blue_book: https://cainmagi.github.io/sync-stream/](https://cainmagi.github.io/sync-stream/)
## Update reports
See the versions in the [CHANGELOG :spiral_notepad:](./CHANGELOG.md) file.
Raw data
{
"_id": null,
"home_page": null,
"name": "syncstream",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "Yuchen Jin <cainmagi@gmail.com>",
"keywords": "python, python3, python-library, stdout, stdout-redirection, multiprocessing, synchronization, stream",
"author": null,
"author_email": "Yuchen Jin <cainmagi@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/6b/62/aca061cfe15de00348d9faaea387242cbcdee453c1b65f0ab48f5b0948d0/syncstream-1.2.2.tar.gz",
"platform": null,
"description": "# Sync-stream\n\n<p><img alt=\"Banner\" src=\"https://repository-images.githubusercontent.com/370331958/6ca07907-3936-444e-8d0d-39c9a0867ef3\"></p>\n\n<p align=\"center\">\n <a href=\"https://github.com/cainmagi/sync-stream/releases/latest\"><img alt=\"GitHub release (latest SemVer)\" src=\"https://img.shields.io/github/v/release/cainmagi/sync-stream?logo=github&sort=semver&style=flat-square\"></a>\n <a href=\"https://github.com/cainmagi/sync-stream/releases\"><img alt=\"GitHub all releases\" src=\"https://img.shields.io/github/downloads/cainmagi/sync-stream/total?logo=github&style=flat-square\"></a>\n <a href=\"https://github.com/cainmagi/sync-stream/blob/main/LICENSE\"><img alt=\"GitHub\" src=\"https://img.shields.io/github/license/cainmagi/sync-stream?style=flat-square&logo=opensourceinitiative&logoColor=white\"></a>\n <a href=\"https://pypi.org/project/syncstream\"><img alt=\"PyPI - Downloads\" src=\"https://img.shields.io/pypi/dm/syncstream?style=flat-square&logo=pypi&logoColor=white&label=pypi\"/></a>\n</p>\n<p align=\"center\">\n <a href=\"https://github.com/cainmagi/sync-stream/actions/workflows/python-package.yml\"><img alt=\"GitHub Actions (Build)\" src=\"https://img.shields.io/github/actions/workflow/status/cainmagi/sync-stream/python-package.yml?style=flat-square&logo=githubactions&logoColor=white&label=build\"></a>\n <a href=\"https://github.com/cainmagi/sync-stream/actions/workflows/python-publish.yml\"><img alt=\"GitHub Actions (Release)\" src=\"https://img.shields.io/github/actions/workflow/status/cainmagi/sync-stream/python-publish.yml?style=flat-square&logo=githubactions&logoColor=white&label=release\"></a>\n</p>\n\nThis project is designed for providing the synchoronization of the stdout / stderr among different threads, processes, devices or hosts. The package could be used for the following cases:\n\n1. Use `syncstream.LineBuffer`: Multiple threads are created. The messages (stdout) from different threads are required to be collected.\n2. Use `syncstream.LineProcBuffer` in the main process, and `syncstream.LineProcMirror` in the sub-processes: Multiple sub-processes are created on the same device. The stdout / stderr of each process is redirected to a `LineProcMirror`, and the results are collected by `LineProcBuffer`.\n3. Use `syncstream.LineFileBuffer`: Multiple processes are created. These processes may be deployed on different devices (even with different platforms), but all devices could get accessed to the same shared disk. In this case, the message could be shared by locked files. Each process would hold an independent `LineFileBuffer` pointing to the same log files.\n4. Use `syncstream.LineHostBuffer` on the server side, and `syncstream.LineHostMirror` on the client side: Multiple processes are deployed on different devices, and they could not share the same disk. In this case, the message are synchronized by the web service. Each process would hold a `LineHostMirror`, and the results would be collected by `LineHostBuffer`.\n\nThe basic package would not contain the `file` and `host` modules. To install the package, please use the following options:\n\n```bash\npip install syncstream[option1,option2...]\n```\n\n| Option | Supports |\n| :----: | :---------------------------------------------------------------------------------------------------------------------------- |\n| `file` | Install dependencies for the `file` module. The module provides `syncstream.LineFileBuffer`. |\n| `host` | Install dependencies for the `host` module. The module provides `syncstream.LineHostBuffer`, and `syncstream.LineHostMirror`. |\n| `test` | Install dependencies for running tests. |\n| `dev` | Install dependencies for developers. |\n\n## Documentation\n\nView the documentation here: [:blue_book: https://cainmagi.github.io/sync-stream/](https://cainmagi.github.io/sync-stream/)\n\n## Update reports\n\nSee the versions in the [CHANGELOG :spiral_notepad:](./CHANGELOG.md) file.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A python tool for synchronize the messages from different threads, processes or hosts.",
"version": "1.2.2",
"project_urls": {
"Changelog": "https://github.com/cainmagi/sync-stream/blob/main/Changelog.md",
"Documentation": "https://cainmagi.github.io/sync-stream/",
"Homepage": "https://github.com/cainmagi/sync-stream",
"Issues": "https://github.com/cainmagi/sync-stream/issues",
"Repository": "https://github.com/cainmagi/sync-stream.git"
},
"split_keywords": [
"python",
" python3",
" python-library",
" stdout",
" stdout-redirection",
" multiprocessing",
" synchronization",
" stream"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b6ab8efb7580b663b89c49d65b712a2b32d7b3fe3b32dbdad5631902424dbca9",
"md5": "6052335e750e9e44c58938d1071482b1",
"sha256": "a888a3f6cc588230fb1cea3f8f4c6492c7c49ca191dcb6a44de6b26eb6ff5516"
},
"downloads": -1,
"filename": "syncstream-1.2.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6052335e750e9e44c58938d1071482b1",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 33737,
"upload_time": "2025-01-08T06:09:02",
"upload_time_iso_8601": "2025-01-08T06:09:02.870432Z",
"url": "https://files.pythonhosted.org/packages/b6/ab/8efb7580b663b89c49d65b712a2b32d7b3fe3b32dbdad5631902424dbca9/syncstream-1.2.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "6b62aca061cfe15de00348d9faaea387242cbcdee453c1b65f0ab48f5b0948d0",
"md5": "9f457abc85e337677f0be6d0911380b4",
"sha256": "80b18e1ba6251c6cfc0779b334e63b73391b19d34520d051d45accdfa9da6d8f"
},
"downloads": -1,
"filename": "syncstream-1.2.2.tar.gz",
"has_sig": false,
"md5_digest": "9f457abc85e337677f0be6d0911380b4",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 50957,
"upload_time": "2025-01-08T06:09:05",
"upload_time_iso_8601": "2025-01-08T06:09:05.302566Z",
"url": "https://files.pythonhosted.org/packages/6b/62/aca061cfe15de00348d9faaea387242cbcdee453c1b65f0ab48f5b0948d0/syncstream-1.2.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-08 06:09:05",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cainmagi",
"github_project": "sync-stream",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "typing_extensions",
"specs": [
[
">=",
"4.0.0"
]
]
},
{
"name": "packaging",
"specs": []
}
],
"lcname": "syncstream"
}
JSON
