# HTCdaskGateway
* A Dask Gateway client extension for heterogeneous cluster mode combining the Kubernetes backend for pain-free scheduler networking, with COFFEA-powered HTCondor workers and/or OKD [coming soon].
* Latest [](https://badge.fury.io/py/htcdaskgateway)
is installed by default and deployed to the COFFEA-DASK notebook on EAF (https://analytics-hub.fnal.gov). A few lines will get you going!
* The current image for workers/schedulers is: coffeateam/coffea-dask-cc7-gateway:0.7.12-fastjet-3.3.4.0rc9-g8a990fa
## Basic usage @ Fermilab [EAF](https://analytics-hub.fnal.gov)
* Make sure the notebook launched supports this functionality (COFFEA-DASK notebook)
```
from htcdaskgateway import HTCGateway
gateway = HTCGateway()
cluster = gateway.new_cluster()
cluster
# Scale my cluster to 5 HTCondor workers
cluster.scale(5)
# Obtain a client for connecting to your cluster scheduler
# Your cluster should be ready to take requests
client = cluster.get_client()
client
# When computations are finished, shutdown the cluster
cluster.shutdown()
```
## Other functions worth checking out
* This is a multi-tenant environment, and you are authenticated via JupyterHub Oauth which means that you can create as many* clusters as you wish
* To list your clusters:
```
# Verify that the gateway is responding to requests by asking to list all its clusters
clusters = gateway.list_clusters()
clusters
```
* To connect to a specific cluster from the list:
```
cluster = gateway.connect(cluster_name)
cluster
cluster.shutdown()
```
* To gracefully close the cluster and remove HTCondor worker jobs associated to it:
```
cluster.shutdown()
```
* There are widgets implemented by Dask Gateway. Make sure to give them a try from your EAF COFFEA notebook, just execute the the `client` and `cluster` commands (after properly initializing them) in a cell like:
```
-------------
cluster = gateway.new_cluster()
cluster
< Widget will appear after this step>
-------------
client = cluster.get_client()
client
< Widget will apear after this step >
-------------
cluster
< Widget will appear after this step >
```
Raw data
{
"_id": null,
"home_page": "https://github.com/mapsacosta/htcdaskgateway",
"name": "htcdaskgateway",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": null,
"author": "Maria P. Acosta F./Fermilab EAF project",
"author_email": "macosta@fnal.gov",
"download_url": "https://files.pythonhosted.org/packages/0c/de/0cfa45f8e1056e761693dc3a6b22345d1a05f6c685b1183ba34826e13e5e/htcdaskgateway-0.1.24.tar.gz",
"platform": null,
"description": "# HTCdaskGateway\n\n* A Dask Gateway client extension for heterogeneous cluster mode combining the Kubernetes backend for pain-free scheduler networking, with COFFEA-powered HTCondor workers and/or OKD [coming soon].\n* Latest [](https://badge.fury.io/py/htcdaskgateway)\n is installed by default and deployed to the COFFEA-DASK notebook on EAF (https://analytics-hub.fnal.gov). A few lines will get you going!\n* The current image for workers/schedulers is: coffeateam/coffea-dask-cc7-gateway:0.7.12-fastjet-3.3.4.0rc9-g8a990fa\n\n## Basic usage @ Fermilab [EAF](https://analytics-hub.fnal.gov)\n* Make sure the notebook launched supports this functionality (COFFEA-DASK notebook)\n\n```\nfrom htcdaskgateway import HTCGateway\n\ngateway = HTCGateway()\ncluster = gateway.new_cluster()\ncluster\n\n# Scale my cluster to 5 HTCondor workers\ncluster.scale(5)\n\n# Obtain a client for connecting to your cluster scheduler\n# Your cluster should be ready to take requests\nclient = cluster.get_client()\nclient\n\n# When computations are finished, shutdown the cluster\ncluster.shutdown()\n```\n## Other functions worth checking out\n* This is a multi-tenant environment, and you are authenticated via JupyterHub Oauth which means that you can create as many* clusters as you wish\n* To list your clusters:\n```\n# Verify that the gateway is responding to requests by asking to list all its clusters\nclusters = gateway.list_clusters()\nclusters\n```\n* To connect to a specific cluster from the list:\n```\ncluster = gateway.connect(cluster_name)\ncluster\ncluster.shutdown()\n```\n* To gracefully close the cluster and remove HTCondor worker jobs associated to it:\n```\ncluster.shutdown()\n```\n* There are widgets implemented by Dask Gateway. Make sure to give them a try from your EAF COFFEA notebook, just execute the the `client` and `cluster` commands (after properly initializing them) in a cell like: \n```\n-------------\ncluster = gateway.new_cluster()\ncluster\n< Widget will appear after this step>\n-------------\nclient = cluster.get_client()\nclient\n< Widget will apear after this step >\n-------------\ncluster\n< Widget will appear after this step >\n```\n",
"bugtrack_url": null,
"license": "Apache",
"summary": "Launches a Dask Gateway cluster in K8s and joins HTCondor workers to it",
"version": "0.1.24",
"project_urls": {
"Homepage": "https://github.com/mapsacosta/htcdaskgateway"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6caf6432ffbb662eac75e5afe7f8c5dc7779677aad3a809b5a1719cb159008a9",
"md5": "5b7ac6d4625907cf22afc3820e9ef1de",
"sha256": "e07d41aaf3e8e984bc0eeff4e91ae8260eb16c1c1c568c411cfc50d37ab29948"
},
"downloads": -1,
"filename": "htcdaskgateway-0.1.24-py3-none-any.whl",
"has_sig": false,
"md5_digest": "5b7ac6d4625907cf22afc3820e9ef1de",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 11781,
"upload_time": "2025-03-10T19:42:24",
"upload_time_iso_8601": "2025-03-10T19:42:24.829469Z",
"url": "https://files.pythonhosted.org/packages/6c/af/6432ffbb662eac75e5afe7f8c5dc7779677aad3a809b5a1719cb159008a9/htcdaskgateway-0.1.24-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "0cde0cfa45f8e1056e761693dc3a6b22345d1a05f6c685b1183ba34826e13e5e",
"md5": "0674461a065be615cf7aee19964c758c",
"sha256": "3a92643870b41541ccaef859ffe44ea6941afc62c021ac363e45e1a1eacc31d6"
},
"downloads": -1,
"filename": "htcdaskgateway-0.1.24.tar.gz",
"has_sig": false,
"md5_digest": "0674461a065be615cf7aee19964c758c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 8165,
"upload_time": "2025-03-10T19:42:25",
"upload_time_iso_8601": "2025-03-10T19:42:25.946359Z",
"url": "https://files.pythonhosted.org/packages/0c/de/0cfa45f8e1056e761693dc3a6b22345d1a05f6c685b1183ba34826e13e5e/htcdaskgateway-0.1.24.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-03-10 19:42:25",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "mapsacosta",
"github_project": "htcdaskgateway",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "htcdaskgateway"
}
JSON
