| Name | ctadata JSON |
| Version |
0.2.5
JSON |
| download |
| home_page | |
| Summary | |
| upload_time | 2023-05-05 16:43:18 |
| maintainer | |
| docs_url | None |
| author | Volodymyr Savchenko |
| requires_python | >=3.8,<4.0 |
| license | |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Client for CTA Data Download Service at Swiss CTA DC
## Disclaimer
This is a prototype development only for Swiss CTA DC. Please report bugs here, and for any details please inquire the CTAO CH DC management.
## Purpose
CTA data are/will be stored at CTAO-CH data center at CSCS.
These data should be accessible for selected external users.
In addition, these files should be available within interactive analysis platform at CSCS.
This client presents an API access to these services and data.
## Installation
```bash
$ pip install 'ctadata>=0.1.14'
```
## Usage from within CTA CSCS JupyterHub platform
From within CTA CSCS JupyterHub platform, selected authorized users are able to access the "data download service", like so:
```python
import ctadata
for url in ctadata.list_dir("cta/DL1/20241114/v0.1/"):
if 'datacheck' not in url and '.0100' in url and '11111' in url:
print("stored", ctadata.fetch_and_save_file(url)/1024/1024, "Mb")
print("found keys", h5py.File(url.split("/")[-1]).keys())
```
### Uploading files
To upload a file:
```python
ctadata.upload_file("latest", "filelists/latest-file-list")
```
The result is:
```
{'path': 'lst/users/volodymyr_savchenko_epfl_ch/filelists/latest-file-list',
'status': 'uploaded',
'total_written': 60098730}
```
Note that for every user, the file is uploaded to their own directory constructed from the user name. The path specified is relative to this directory. If you need to move the files to common directories, please as support. But you likely want to just share returned path to be used as so:
```python
ctadata.fetch_and_save_file("lst/users/volodymyr_savchenko_epfl_ch/filelists/latest-file-list")
```
You can also use command line interface to do this:
```bash
$ cta-data put latest-file-list latest-file-list-bla-bla
```
**Beware that all the files written are accessible to all CTAO members and all platform users.**
## From outside (possibly another jupyterhub)
You need to get yourself a jupyterhub token, it will be used to authenticate to the download service.
If you are in the session, navigate to the hub control panel this way:
Request a token:
The rest is similar to the previous case:
```python
import os
os.environ["CTADS_URL"] = "DATA-DISTRIBUTING-JUPYTERHUB/services/downloadservice/"
os.environ["JUPYTERHUB_API_TOKEN"] = "INSERT-YOUR-TOKEN-HERE"
for url in ctadata.list_dir("cta/DL1/20241114/v0.1"):
if 'datacheck' not in url and '.0100' in url and '11111' in url:
print("stored", ctadata.fetch_and_save_file(url)/1024/1024, "Mb")
print("found keys", h5py.File(url.split("/")[-1]).keys())
```
Raw data
{
"_id": null,
"home_page": "",
"name": "ctadata",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4.0",
"maintainer_email": "",
"keywords": "",
"author": "Volodymyr Savchenko",
"author_email": "contact@volodymyrsavchenko.com",
"download_url": "https://files.pythonhosted.org/packages/5e/bf/248bbf99652eaec2bda1daa5e5ef557a18113a072a08333eaaa26028a417/ctadata-0.2.5.tar.gz",
"platform": null,
"description": "# Client for CTA Data Download Service at Swiss CTA DC\n\n## Disclaimer\n\nThis is a prototype development only for Swiss CTA DC. Please report bugs here, and for any details please inquire the CTAO CH DC management.\n\n## Purpose\n\nCTA data are/will be stored at CTAO-CH data center at CSCS.\nThese data should be accessible for selected external users.\nIn addition, these files should be available within interactive analysis platform at CSCS.\nThis client presents an API access to these services and data.\n## Installation\n\n```bash\n$ pip install 'ctadata>=0.1.14'\n```\n\n## Usage from within CTA CSCS JupyterHub platform\n\nFrom within CTA CSCS JupyterHub platform, selected authorized users are able to access the \"data download service\", like so:\n\n```python\nimport ctadata\n\nfor url in ctadata.list_dir(\"cta/DL1/20241114/v0.1/\"):\n if 'datacheck' not in url and '.0100' in url and '11111' in url:\n print(\"stored\", ctadata.fetch_and_save_file(url)/1024/1024, \"Mb\")\n print(\"found keys\", h5py.File(url.split(\"/\")[-1]).keys())\n```\n\n### Uploading files\n\nTo upload a file:\n\n```python\nctadata.upload_file(\"latest\", \"filelists/latest-file-list\")\n```\n\nThe result is:\n\n```\n{'path': 'lst/users/volodymyr_savchenko_epfl_ch/filelists/latest-file-list',\n 'status': 'uploaded',\n 'total_written': 60098730}\n```\n\nNote that for every user, the file is uploaded to their own directory constructed from the user name. The path specified is relative to this directory. If you need to move the files to common directories, please as support. But you likely want to just share returned path to be used as so:\n\n```python\nctadata.fetch_and_save_file(\"lst/users/volodymyr_savchenko_epfl_ch/filelists/latest-file-list\")\n```\n\nYou can also use command line interface to do this:\n\n```bash\n$ cta-data put latest-file-list latest-file-list-bla-bla\n```\n\n**Beware that all the files written are accessible to all CTAO members and all platform users.**\n\n## From outside (possibly another jupyterhub)\n\nYou need to get yourself a jupyterhub token, it will be used to authenticate to the download service.\n\nIf you are in the session, navigate to the hub control panel this way:\n\n\n\nRequest a token:\n\n\n\nThe rest is similar to the previous case:\n\n```python\nimport os\nos.environ[\"CTADS_URL\"] = \"DATA-DISTRIBUTING-JUPYTERHUB/services/downloadservice/\"\nos.environ[\"JUPYTERHUB_API_TOKEN\"] = \"INSERT-YOUR-TOKEN-HERE\"\n\nfor url in ctadata.list_dir(\"cta/DL1/20241114/v0.1\"):\n if 'datacheck' not in url and '.0100' in url and '11111' in url:\n print(\"stored\", ctadata.fetch_and_save_file(url)/1024/1024, \"Mb\")\n print(\"found keys\", h5py.File(url.split(\"/\")[-1]).keys())\n```\n\n",
"bugtrack_url": null,
"license": "",
"summary": "",
"version": "0.2.5",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "4fbbcd385d451e1c38446c8400174756c5adb37f0a6eaa2d0146cd86a3f23bc6",
"md5": "de4a77171cde562127b0262f4a86526b",
"sha256": "4ced0a5e304ed18d52bb9c35f8d51d25625081065f3d9ad2d3be42ee877f93af"
},
"downloads": -1,
"filename": "ctadata-0.2.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "de4a77171cde562127b0262f4a86526b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4.0",
"size": 5513,
"upload_time": "2023-05-05T16:43:15",
"upload_time_iso_8601": "2023-05-05T16:43:15.243628Z",
"url": "https://files.pythonhosted.org/packages/4f/bb/cd385d451e1c38446c8400174756c5adb37f0a6eaa2d0146cd86a3f23bc6/ctadata-0.2.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5ebf248bbf99652eaec2bda1daa5e5ef557a18113a072a08333eaaa26028a417",
"md5": "1f5c8c561d3fb5cfadcfffa32386ff27",
"sha256": "1acbfab5ba4b97d32442aa7e016d9ddec69c890386a006a91ae74260ce6d1e0e"
},
"downloads": -1,
"filename": "ctadata-0.2.5.tar.gz",
"has_sig": false,
"md5_digest": "1f5c8c561d3fb5cfadcfffa32386ff27",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4.0",
"size": 4170,
"upload_time": "2023-05-05T16:43:18",
"upload_time_iso_8601": "2023-05-05T16:43:18.060360Z",
"url": "https://files.pythonhosted.org/packages/5e/bf/248bbf99652eaec2bda1daa5e5ef557a18113a072a08333eaaa26028a417/ctadata-0.2.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-05-05 16:43:18",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "ctadata"
}
