# Simple GPU Queue
> Simple GPU Queue: For running lots of smaller experiments on your handful of GPUs
>
Functional, minimal.
## Quickstart
```
pip install simple_gpu_queue
simpleq_start 4 --memfrac-per-proc=0.45 # Start 2 processes per gpu, each taking 0.45 the total amount of GPU RAM available
```
Example python launch script
```python
from simple_gpu_queue.simple_gpu_queue import GPUQueueClient
queue = GPUQueueClient(git_commit_id="most_recent")
# Send commands as a list of (command, stdout_file) tuples. Stdoutfile can be "" for no saving of stdout logging.
commands = [
("python -c 'import time; import jax.numpy as jnp; a=jnp.ones(5); time.sleep(1); print(\"I SLEPT HAPPY 1\")'", "testlogs/v1.txt"),
("python -c 'import time; import jax.numpy as jnp; a=jnp.ones(5); time.sleep(2); print(\"I SLEPT HAPPY 2\")'", ""),
("python -c 'import time; import jax.numpy as jnp; a=jnp.ones(5); time.sleep(3); print(\"I SLEPT HAPPY 3\")'", "testlogs/v3.txt"),
]
if queue.git_repo.is_dirty():
user_input = input("Repo has uncommitted changes: continue? [Y/n] ").strip().lower()
if user_input == 'n':
raise ValueError("Aborting due to uncommitted changes in the repository.")
queue.send_command_list(commands)
```
## For Development
```
conda env create -f environment.yml
conda activate simple-gpu-queue
pip install -r requirements.txt
# pip install -r requirements-dev.txt
```
Check the server is running, navigate to `http://localhost:5034/docs`, test the routes
## Upload to pypi (reference)
```
rm -rf dist/
python setup.py sdist bdist_wheel
twine upload dist/*
```
Raw data
{
"_id": null,
"home_page": "https://github.com/bhoov/simple-gpu-queue",
"name": "simple-gpu-queue",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "Ben Hoover",
"author_email": "benhoover34@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/89/74/4d617eb8aec68b8b2988e80e6e39931b605ef54c9612c5f0fb909c2c9f02/simple_gpu_queue-0.1.1.tar.gz",
"platform": null,
"description": "# Simple GPU Queue\n> Simple GPU Queue: For running lots of smaller experiments on your handful of GPUs\n> \n\nFunctional, minimal.\n\n## Quickstart\n\n```\npip install simple_gpu_queue\nsimpleq_start 4 --memfrac-per-proc=0.45 # Start 2 processes per gpu, each taking 0.45 the total amount of GPU RAM available\n```\n\nExample python launch script\n\n```python\nfrom simple_gpu_queue.simple_gpu_queue import GPUQueueClient\nqueue = GPUQueueClient(git_commit_id=\"most_recent\")\n\n# Send commands as a list of (command, stdout_file) tuples. Stdoutfile can be \"\" for no saving of stdout logging.\ncommands = [\n (\"python -c 'import time; import jax.numpy as jnp; a=jnp.ones(5); time.sleep(1); print(\\\"I SLEPT HAPPY 1\\\")'\", \"testlogs/v1.txt\"),\n (\"python -c 'import time; import jax.numpy as jnp; a=jnp.ones(5); time.sleep(2); print(\\\"I SLEPT HAPPY 2\\\")'\", \"\"),\n (\"python -c 'import time; import jax.numpy as jnp; a=jnp.ones(5); time.sleep(3); print(\\\"I SLEPT HAPPY 3\\\")'\", \"testlogs/v3.txt\"),\n]\n\nif queue.git_repo.is_dirty():\n user_input = input(\"Repo has uncommitted changes: continue? [Y/n] \").strip().lower()\n if user_input == 'n':\n raise ValueError(\"Aborting due to uncommitted changes in the repository.\") \n\nqueue.send_command_list(commands)\n```\n\n\n## For Development\n```\nconda env create -f environment.yml\nconda activate simple-gpu-queue\npip install -r requirements.txt\n# pip install -r requirements-dev.txt\n```\n\nCheck the server is running, navigate to `http://localhost:5034/docs`, test the routes\n\n## Upload to pypi (reference)\n```\nrm -rf dist/\npython setup.py sdist bdist_wheel\ntwine upload dist/*\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "Simple GPU Queue: For running lots of smaller experiments on your handful of GPUs, designed for JAX",
"version": "0.1.1",
"project_urls": {
"Homepage": "https://github.com/bhoov/simple-gpu-queue"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b98b852d671900d1838bb3d4e931966666cb9df52c970c203dd9e8d91ed62b76",
"md5": "59ad2556a9c9cf1ce9279d2815a36e91",
"sha256": "008f865d9e83eb06c2bf90638c1944b07caa59350c115c10381f4a902f1d1e61"
},
"downloads": -1,
"filename": "simple_gpu_queue-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "59ad2556a9c9cf1ce9279d2815a36e91",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 9128,
"upload_time": "2024-08-07T20:43:10",
"upload_time_iso_8601": "2024-08-07T20:43:10.966218Z",
"url": "https://files.pythonhosted.org/packages/b9/8b/852d671900d1838bb3d4e931966666cb9df52c970c203dd9e8d91ed62b76/simple_gpu_queue-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "89744d617eb8aec68b8b2988e80e6e39931b605ef54c9612c5f0fb909c2c9f02",
"md5": "7f32ad46bf9b2b22dcc7a29d2fdf3463",
"sha256": "126589bf2df343253610ddef81bef6dbd2efa630c3014e8b059a30ee0d8e2cd7"
},
"downloads": -1,
"filename": "simple_gpu_queue-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "7f32ad46bf9b2b22dcc7a29d2fdf3463",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 8932,
"upload_time": "2024-08-07T20:43:12",
"upload_time_iso_8601": "2024-08-07T20:43:12.533848Z",
"url": "https://files.pythonhosted.org/packages/89/74/4d617eb8aec68b8b2988e80e6e39931b605ef54c9612c5f0fb909c2c9f02/simple_gpu_queue-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-07 20:43:12",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "bhoov",
"github_project": "simple-gpu-queue",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "simple-gpu-queue"
}
JSON
