| Name | pyfsq JSON |
| Version |
0.1.5
JSON |
| download |
| home_page | None |
| Summary | Python-based console client and API for transferring data to FSQ server |
| upload_time | 2024-08-01 06:52:48 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.7 |
| license | None |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# pyfsq
Python-based FSQ API designed for transferring data to a FSQ server and
simple console client for demonstrating how to use the API.
The FSQ protocol explanation and server/client written in C is provided at [https://github.com/tstibor/fsq](https://github.com/tstibor/fsq)
## Installation
```pip3 install pyfsq```
## Requirements
### Python
* Python >= 3.7
## Console Tools
After installing the package, the console tools ```pyfsqc``` and ```pyfsqbench``` are available.
### FSQ Client
```bash
$ pyfsqc -f /lustre -a /lustre/fsqc -n polaris -p polaris -s localhost -v info src/fsqc.py
[2024-07-31 12:24:48,804 api.py:103 INFO] connect sucessfully to FSQ server localhost:7625
[2024-07-31 12:24:48,804 api.py:178 INFO] opened sucessfully on FSQ server file /lustre/fsqc/src/fsqc.py
[2024-07-31 12:24:48,847 api.py:230 INFO] send sucessfully 2471 bytes to file /lustre/fsqc/src/fsqc.py on FSQ server
[2024-07-31 12:24:48,848 api.py:278 INFO] closed sucessfully on FSQ server file /lustre/fsqc/src/fsqc.py
[2024-07-31 12:24:48,848 api.py:320 INFO] disconnect sucessfully from FSQ server localhost
```
### FSQ Benchmark Tool
```bash
$ pyfsqbench -t 2 -n polaris -p polaris -s localhost -v info
[2024-07-31 12:26:59,098 api.py:103 INFO] connect sucessfully to FSQ server localhost:7625
[2024-07-31 12:26:59,098 api.py:103 INFO] connect sucessfully to FSQ server localhost:7625
[2024-07-31 12:26:59,129 api.py:178 INFO] opened sucessfully on FSQ server file /lustre/fsqbench/PN18OzXV5mccV3Wl5Gm0VQz7vO6YheH1
[2024-07-31 12:26:59,129 api.py:178 INFO] opened sucessfully on FSQ server file /lustre/fsqbench/O83IhaSELK6uzPs8cvLW2rmkMiL3K4MQ
[2024-07-31 12:26:59,132 api.py:230 INFO] send sucessfully 16777216 bytes to file /lustre/fsqbench/PN18OzXV5mccV3Wl5Gm0VQz7vO6YheH1 on FSQ server
[2024-07-31 12:26:59,132 api.py:230 INFO] send sucessfully 16777216 bytes to file /lustre/fsqbench/O83IhaSELK6uzPs8cvLW2rmkMiL3K4MQ on FSQ server
[2024-07-31 12:26:59,133 api.py:278 INFO] closed sucessfully on FSQ server file /lustre/fsqbench/PN18OzXV5mccV3Wl5Gm0VQz7vO6YheH1
[2024-07-31 12:26:59,133 api.py:278 INFO] closed sucessfully on FSQ server file /lustre/fsqbench/O83IhaSELK6uzPs8cvLW2rmkMiL3K4MQ
...
...
[2024-07-31 12:26:59,180 api.py:278 INFO] closed sucessfully on FSQ server file /lustre/fsqbench/545Vutrtr8T9TQiv8GbMAE9ujBMltcrB
[2024-07-31 12:26:59,180 api.py:178 INFO] opened sucessfully on FSQ server file /lustre/fsqbench/RUnZKHB2zuAkbbqrFXjgrlCDsEzqajfj
[2024-07-31 12:26:59,183 api.py:230 INFO] send sucessfully 16777216 bytes to file /lustre/fsqbench/YuaSxx3ifs3yBWaki0DbLDhXHKNLudVP on FSQ server
[2024-07-31 12:26:59,183 api.py:278 INFO] closed sucessfully on FSQ server file /lustre/fsqbench/YuaSxx3ifs3yBWaki0DbLDhXHKNLudVP
[2024-07-31 12:26:59,183 api.py:320 INFO] disconnect sucessfully from FSQ server localhost
[2024-07-31 12:26:59,184 api.py:230 INFO] send sucessfully 16777216 bytes to file /lustre/fsqbench/RUnZKHB2zuAkbbqrFXjgrlCDsEzqajfj on FSQ server
[2024-07-31 12:26:59,184 api.py:278 INFO] closed sucessfully on FSQ server file /lustre/fsqbench/RUnZKHB2zuAkbbqrFXjgrlCDsEzqajfj
[2024-07-31 12:26:59,184 api.py:320 INFO] disconnect sucessfully from FSQ server localhost
Thread-1 (perform_task) transmitted 268435456 bytes in 0.0554 secs
Thread-2 (perform_task) transmitted 268435456 bytes in 0.0545 secs
total transmitted 536870912 bytes in 0.1099 secs (4.8848 Gbytes/secs)
```
## Example
### FSQ API
```
from fsq.api import Api as FsqApi
from fsq.struct import FsqStorageDest
fsq = FsqApi()
fsq.connect('polaris', 'polaris', 'localhost', 7625)
fsq.open(fs='/lustre', fpath='/lustre/physicsexp/pyfsq/helloworld.txt',
desc='some desc', dest=FsqStorageDest.FSQ_STORAGE_LUSTRE)
buffer = b'hello world!'
size = len(buffer)
fsq.write(buffer, size)
fsq.close()
fsq.disconnect()
```
### Contextmanager FSQ API
```
from fsq.contextmanager import Connect, Open
from fsq.struct import FsqStorageDest
with Connect('polaris', 'polaris', 'localhost', 7625) as fsq:
with Open(fsq, '/lustre', '/lustre/physicsexp/ctxmgr/helloworld.txt',
'some desc', FsqStorageDest.FSQ_STORAGE_LOCAL):
buffer = b'hello world!'
size = len(buffer)
fsq.write(buffer, size)
```
Raw data
{
"_id": null,
"home_page": null,
"name": "pyfsq",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": null,
"author": null,
"author_email": "Thomas Stibor <t.stibor@gsi.de>",
"download_url": "https://files.pythonhosted.org/packages/5e/13/8d8ffbe2773b5034a5a6fdeea8b15c8f5d9b394f3552e35ae5ce0c75b3b1/pyfsq-0.1.5.tar.gz",
"platform": null,
"description": "# pyfsq\nPython-based FSQ API designed for transferring data to a FSQ server and\nsimple console client for demonstrating how to use the API.\n\nThe FSQ protocol explanation and server/client written in C is provided at [https://github.com/tstibor/fsq](https://github.com/tstibor/fsq)\n\n## Installation\n```pip3 install pyfsq```\n\n## Requirements\n### Python\n* Python >= 3.7\n\n## Console Tools\nAfter installing the package, the console tools ```pyfsqc``` and ```pyfsqbench``` are available.\n### FSQ Client\n```bash\n$ pyfsqc -f /lustre -a /lustre/fsqc -n polaris -p polaris -s localhost -v info src/fsqc.py\n[2024-07-31 12:24:48,804 api.py:103 INFO] connect sucessfully to FSQ server localhost:7625\n[2024-07-31 12:24:48,804 api.py:178 INFO] opened sucessfully on FSQ server file /lustre/fsqc/src/fsqc.py\n[2024-07-31 12:24:48,847 api.py:230 INFO] send sucessfully 2471 bytes to file /lustre/fsqc/src/fsqc.py on FSQ server\n[2024-07-31 12:24:48,848 api.py:278 INFO] closed sucessfully on FSQ server file /lustre/fsqc/src/fsqc.py\n[2024-07-31 12:24:48,848 api.py:320 INFO] disconnect sucessfully from FSQ server localhost\n```\n### FSQ Benchmark Tool\n```bash\n$ pyfsqbench -t 2 -n polaris -p polaris -s localhost -v info\n[2024-07-31 12:26:59,098 api.py:103 INFO] connect sucessfully to FSQ server localhost:7625\n[2024-07-31 12:26:59,098 api.py:103 INFO] connect sucessfully to FSQ server localhost:7625\n[2024-07-31 12:26:59,129 api.py:178 INFO] opened sucessfully on FSQ server file /lustre/fsqbench/PN18OzXV5mccV3Wl5Gm0VQz7vO6YheH1\n[2024-07-31 12:26:59,129 api.py:178 INFO] opened sucessfully on FSQ server file /lustre/fsqbench/O83IhaSELK6uzPs8cvLW2rmkMiL3K4MQ\n[2024-07-31 12:26:59,132 api.py:230 INFO] send sucessfully 16777216 bytes to file /lustre/fsqbench/PN18OzXV5mccV3Wl5Gm0VQz7vO6YheH1 on FSQ server\n[2024-07-31 12:26:59,132 api.py:230 INFO] send sucessfully 16777216 bytes to file /lustre/fsqbench/O83IhaSELK6uzPs8cvLW2rmkMiL3K4MQ on FSQ server\n[2024-07-31 12:26:59,133 api.py:278 INFO] closed sucessfully on FSQ server file /lustre/fsqbench/PN18OzXV5mccV3Wl5Gm0VQz7vO6YheH1\n[2024-07-31 12:26:59,133 api.py:278 INFO] closed sucessfully on FSQ server file /lustre/fsqbench/O83IhaSELK6uzPs8cvLW2rmkMiL3K4MQ\n...\n...\n[2024-07-31 12:26:59,180 api.py:278 INFO] closed sucessfully on FSQ server file /lustre/fsqbench/545Vutrtr8T9TQiv8GbMAE9ujBMltcrB\n[2024-07-31 12:26:59,180 api.py:178 INFO] opened sucessfully on FSQ server file /lustre/fsqbench/RUnZKHB2zuAkbbqrFXjgrlCDsEzqajfj\n[2024-07-31 12:26:59,183 api.py:230 INFO] send sucessfully 16777216 bytes to file /lustre/fsqbench/YuaSxx3ifs3yBWaki0DbLDhXHKNLudVP on FSQ server\n[2024-07-31 12:26:59,183 api.py:278 INFO] closed sucessfully on FSQ server file /lustre/fsqbench/YuaSxx3ifs3yBWaki0DbLDhXHKNLudVP\n[2024-07-31 12:26:59,183 api.py:320 INFO] disconnect sucessfully from FSQ server localhost\n[2024-07-31 12:26:59,184 api.py:230 INFO] send sucessfully 16777216 bytes to file /lustre/fsqbench/RUnZKHB2zuAkbbqrFXjgrlCDsEzqajfj on FSQ server\n[2024-07-31 12:26:59,184 api.py:278 INFO] closed sucessfully on FSQ server file /lustre/fsqbench/RUnZKHB2zuAkbbqrFXjgrlCDsEzqajfj\n[2024-07-31 12:26:59,184 api.py:320 INFO] disconnect sucessfully from FSQ server localhost\nThread-1 (perform_task) transmitted 268435456 bytes in 0.0554 secs\nThread-2 (perform_task) transmitted 268435456 bytes in 0.0545 secs\ntotal transmitted 536870912 bytes in 0.1099 secs (4.8848 Gbytes/secs)\n```\n\n## Example\n### FSQ API\n```\nfrom fsq.api import Api as FsqApi\nfrom fsq.struct import FsqStorageDest\n\nfsq = FsqApi()\nfsq.connect('polaris', 'polaris', 'localhost', 7625)\nfsq.open(fs='/lustre', fpath='/lustre/physicsexp/pyfsq/helloworld.txt',\n desc='some desc', dest=FsqStorageDest.FSQ_STORAGE_LUSTRE)\nbuffer = b'hello world!'\nsize = len(buffer)\nfsq.write(buffer, size)\nfsq.close()\nfsq.disconnect()\n```\n\n### Contextmanager FSQ API\n```\nfrom fsq.contextmanager import Connect, Open\nfrom fsq.struct import FsqStorageDest\n\nwith Connect('polaris', 'polaris', 'localhost', 7625) as fsq:\n with Open(fsq, '/lustre', '/lustre/physicsexp/ctxmgr/helloworld.txt',\n 'some desc', FsqStorageDest.FSQ_STORAGE_LOCAL):\n buffer = b'hello world!'\n size = len(buffer)\n fsq.write(buffer, size)\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "Python-based console client and API for transferring data to FSQ server",
"version": "0.1.5",
"project_urls": {
"Homepage": "https://github.com/tstibor/pyfsq",
"Issues": "https://github.com/tstibor/pyfsq/issues"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3c58e0137c10ea9598ad0bccfed4f943149682c5c9762d1633184928d0d7e0d7",
"md5": "d1a15f1caa12ff4f2c1d03d5a9f86a33",
"sha256": "07c58fc55652cbcd593079dd1f49b8099dcf86eb3ff9a52f674a0c95626c9231"
},
"downloads": -1,
"filename": "pyfsq-0.1.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d1a15f1caa12ff4f2c1d03d5a9f86a33",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 21950,
"upload_time": "2024-08-01T06:52:46",
"upload_time_iso_8601": "2024-08-01T06:52:46.306544Z",
"url": "https://files.pythonhosted.org/packages/3c/58/e0137c10ea9598ad0bccfed4f943149682c5c9762d1633184928d0d7e0d7/pyfsq-0.1.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5e138d8ffbe2773b5034a5a6fdeea8b15c8f5d9b394f3552e35ae5ce0c75b3b1",
"md5": "ae68d1d6f5d07e028bbc6077006c92c7",
"sha256": "786ea8cf14f3e314b49f0ccde712996f42f7df329905da4cf8ab4db44b081314"
},
"downloads": -1,
"filename": "pyfsq-0.1.5.tar.gz",
"has_sig": false,
"md5_digest": "ae68d1d6f5d07e028bbc6077006c92c7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 22555,
"upload_time": "2024-08-01T06:52:48",
"upload_time_iso_8601": "2024-08-01T06:52:48.486364Z",
"url": "https://files.pythonhosted.org/packages/5e/13/8d8ffbe2773b5034a5a6fdeea8b15c8f5d9b394f3552e35ae5ce0c75b3b1/pyfsq-0.1.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-01 06:52:48",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "tstibor",
"github_project": "pyfsq",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "pyfsq"
}
