| Name | hypo-run JSON |
| Version |
1.0.0
JSON |
| download |
| home_page | None |
| Summary | A CLI tool to run the complex experiments |
| upload_time | 2024-09-28 09:58:09 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.8 |
| license | Copyright (c) 2016 The Python Packaging Authority (PyPA) Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
cli tool
experiment
parallel controller
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Hypo is a CLI tool to run OS commands concurrently.
You are tired of running commands one by one manually when the previous command is finished. A lot of commands (maybe experiments) need to run in your project (maybe paper). You want to run them concurrently, without leaving the Free time.
Hypo is a tool to help you run commands concurrently. You can use Hypo to run them concurrently without wasting time. It run your complex commands in System-level. You can easily manipulate the commands in the way you like.
You can install it by `pip install hypo-run`.
```python
# In the file folder_a/index.py
from hypo import run, Run
@run()
def trial():
return [Run(command="echo this_is_a_very_complex_prompt_to_start_your_experiment_in_bash", name="indicate your task")]
@run(max_workers=10) # run 10 tasks concurrently
def trial():
return [Run(command="echo this_is_a_very_complex_prompt_to_start_your_experiment_in_bash", name="indicate your task")]
```
Then you can start your task parallel.
```bash
# hypo <dir_name> <function_name>
hypo folder_a index.trial # to start method trial. Create tasks, then run.
# if the file named `index.py`, then you can ignore the file name.
hypo folder_a trial # to start method trial. Create tasks, then run.
# if you are already in the folder_a
hypo trial # to start method trial. Create tasks, then run.
```
Or, directly call the function you need.
```python
trial()
# Then, `python folder_a/index.py`
```
After running all experiments, you can check the task summary in the output folder named `summary.json`.
```json
[
{
"name": "A very complex task",
"command": "echo this_is_a_very_complex_prompt_to_start_your_experiment_in_bash",
"cwd": "/data/Hypothesis/hypo",
"output": "/data/Hypothesis/hypo/a",
"datetime": "2024-06-27__18-35-21"
},
{
"name": "Git Version",
"command": "git rev-parse HEAD",
"cwd": "/data/Hypothesis/hypo",
"output": "/data/Hypothesis/hypo",
"datetime": "2024-06-27__18-35-21"
}
]
```
## CUDA CMD Friendly
You may have a lot of cuda tasks to do. Run them concurrently! Assume your GPU could have 2 task to run at the same time. `cuda_visible_devices` will be the environment variable `CUDA_VISIBLE_DEVICES` pass to processing.
```python
from hypo import run, Run, run_git_checkout
from itertools import product
@run(cuda_visible_devices={0, 1, 6, 7}, max_workers=8)
def compare():
cmd_templete = "python main.py --category {clz}"
clzs = [
"table",
"sofa",
"bench",
"watercraft",
# ... a really long list
]
tasks = []
for clz, method in product(clzs, ["my_method", "baseline", "sota"]): # the method you want to compare
task = [
run_git_checkout(method), # branch name. git checkout to the branch you want to run.
Run(
command=cmd_templete.format(clz=clz),
name=f"{method}-{clz}",
cwd="/path/to/your/project",
out="/summary.json/will/be/generated/here",
),
]
tasks.append(task)
return tasks
```
You do not need to worry about the `run_git_checkout`. Python will load all file in memory at the start. Your code will not go wrong.
## Extensions
You can use some pre-defined Run. for example, the `git version` using `run_git_status`.
```python
from hypo import runs, Run, run_git_status
@runs
def method():
return [
Run(name="a", cwd=".", output="./a", command="echo $cwd"),
run_git_status(),
]
```
If you want to run the command in a specific git version, you can use `run_git_checkout`.
```python
from hypo import run, run_git_checkout, Run
@run()
def test_run_git_checkout():
return [
[
run_git_checkout("a6bb0c3"), # commit name
Run(command="python main.py", output=".", cwd=".", name="run1"),
],
[
run_git_checkout("main"), # branch name
Run(command="python main.py", output=".", cwd=".", name="run2"),
],
]
```
Raw data
{
"_id": null,
"home_page": null,
"name": "hypo-run",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "ruimox <aruix@outlook.com>",
"keywords": "cli tool, experiment, Parallel controller",
"author": null,
"author_email": "ruimox <aruix@outlook.com>",
"download_url": "https://files.pythonhosted.org/packages/13/83/5c4e1b0fd32325e018487945f9cf35a3f3c5d4553cb39158e12fee7139c6/hypo_run-1.0.0.tar.gz",
"platform": null,
"description": "# Hypo is a CLI tool to run OS commands concurrently.\n\nYou are tired of running commands one by one manually when the previous command is finished. A lot of commands (maybe experiments) need to run in your project (maybe paper). You want to run them concurrently, without leaving the Free time. \n\nHypo is a tool to help you run commands concurrently. You can use Hypo to run them concurrently without wasting time. It run your complex commands in System-level. You can easily manipulate the commands in the way you like.\n\nYou can install it by `pip install hypo-run`.\n\n```python\n# In the file folder_a/index.py\nfrom hypo import run, Run\n\n@run()\ndef trial():\n return [Run(command=\"echo this_is_a_very_complex_prompt_to_start_your_experiment_in_bash\", name=\"indicate your task\")]\n\n@run(max_workers=10) # run 10 tasks concurrently\ndef trial():\n return [Run(command=\"echo this_is_a_very_complex_prompt_to_start_your_experiment_in_bash\", name=\"indicate your task\")]\n\n```\n\nThen you can start your task parallel.\n\n```bash\n# hypo <dir_name> <function_name>\n\nhypo folder_a index.trial # to start method trial. Create tasks, then run.\n\n# if the file named `index.py`, then you can ignore the file name.\nhypo folder_a trial # to start method trial. Create tasks, then run.\n\n# if you are already in the folder_a\nhypo trial # to start method trial. Create tasks, then run.\n\n```\n\nOr, directly call the function you need.\n\n```python\ntrial()\n# Then, `python folder_a/index.py`\n```\n\nAfter running all experiments, you can check the task summary in the output folder named `summary.json`.\n\n```json\n[\n {\n \"name\": \"A very complex task\",\n \"command\": \"echo this_is_a_very_complex_prompt_to_start_your_experiment_in_bash\",\n \"cwd\": \"/data/Hypothesis/hypo\",\n \"output\": \"/data/Hypothesis/hypo/a\",\n \"datetime\": \"2024-06-27__18-35-21\"\n },\n {\n \"name\": \"Git Version\",\n \"command\": \"git rev-parse HEAD\",\n \"cwd\": \"/data/Hypothesis/hypo\",\n \"output\": \"/data/Hypothesis/hypo\",\n \"datetime\": \"2024-06-27__18-35-21\"\n }\n]\n```\n\n## CUDA CMD Friendly\n\nYou may have a lot of cuda tasks to do. Run them concurrently! Assume your GPU could have 2 task to run at the same time. `cuda_visible_devices` will be the environment variable `CUDA_VISIBLE_DEVICES` pass to processing.\n\n```python\nfrom hypo import run, Run, run_git_checkout\nfrom itertools import product\n\n@run(cuda_visible_devices={0, 1, 6, 7}, max_workers=8) \ndef compare():\n cmd_templete = \"python main.py --category {clz}\"\n\n clzs = [\n \"table\",\n \"sofa\",\n \"bench\",\n \"watercraft\",\n # ... a really long list\n ]\n tasks = []\n for clz, method in product(clzs, [\"my_method\", \"baseline\", \"sota\"]): # the method you want to compare\n task = [\n run_git_checkout(method), # branch name. git checkout to the branch you want to run. \n Run(\n command=cmd_templete.format(clz=clz),\n name=f\"{method}-{clz}\",\n cwd=\"/path/to/your/project\",\n out=\"/summary.json/will/be/generated/here\",\n ),\n ]\n tasks.append(task)\n return tasks\n\n```\n\nYou do not need to worry about the `run_git_checkout`. Python will load all file in memory at the start. Your code will not go wrong.\n\n\n## Extensions\n\nYou can use some pre-defined Run. for example, the `git version` using `run_git_status`.\n\n```python\nfrom hypo import runs, Run, run_git_status\n\n@runs\ndef method():\n return [\n Run(name=\"a\", cwd=\".\", output=\"./a\", command=\"echo $cwd\"),\n run_git_status(),\n ]\n\n```\n\nIf you want to run the command in a specific git version, you can use `run_git_checkout`.\n\n```python\n\nfrom hypo import run, run_git_checkout, Run\n\n\n@run()\ndef test_run_git_checkout():\n return [\n [\n run_git_checkout(\"a6bb0c3\"), # commit name\n Run(command=\"python main.py\", output=\".\", cwd=\".\", name=\"run1\"),\n ],\n [\n run_git_checkout(\"main\"), # branch name\n Run(command=\"python main.py\", output=\".\", cwd=\".\", name=\"run2\"),\n ],\n ]\n\n```\n",
"bugtrack_url": null,
"license": "Copyright (c) 2016 The Python Packaging Authority (PyPA) Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ",
"summary": "A CLI tool to run the complex experiments",
"version": "1.0.0",
"project_urls": {
"Bug Reports": "https://github.com/aruiplex/hypo",
"Funding": "https://github.com/aruiplex/hypo",
"Homepage": "https://github.com/aruiplex/hypo",
"Say Thanks!": "https://github.com/aruiplex/hypo",
"Source": "https://github.com/aruiplex/hypo"
},
"split_keywords": [
"cli tool",
" experiment",
" parallel controller"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "dac5444340d90881190aa638125c4f03a38d1e48c5e2b9002734bc1f4de3c4ac",
"md5": "cf34a633f7e1470b9253b41100d54de7",
"sha256": "1167845c04358d4f7958fdf1356e19199938e120e953da79e5b7e75708d2739d"
},
"downloads": -1,
"filename": "hypo_run-1.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "cf34a633f7e1470b9253b41100d54de7",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 11568,
"upload_time": "2024-09-28T09:58:07",
"upload_time_iso_8601": "2024-09-28T09:58:07.592576Z",
"url": "https://files.pythonhosted.org/packages/da/c5/444340d90881190aa638125c4f03a38d1e48c5e2b9002734bc1f4de3c4ac/hypo_run-1.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "13835c4e1b0fd32325e018487945f9cf35a3f3c5d4553cb39158e12fee7139c6",
"md5": "621a9992242341727960f24890bc1fc2",
"sha256": "8e7bc5abd4cd6687f1a8eff4efd7322c2144580f08c2b3a140ab5ce8be11c27d"
},
"downloads": -1,
"filename": "hypo_run-1.0.0.tar.gz",
"has_sig": false,
"md5_digest": "621a9992242341727960f24890bc1fc2",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 13864,
"upload_time": "2024-09-28T09:58:09",
"upload_time_iso_8601": "2024-09-28T09:58:09.411424Z",
"url": "https://files.pythonhosted.org/packages/13/83/5c4e1b0fd32325e018487945f9cf35a3f3c5d4553cb39158e12fee7139c6/hypo_run-1.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-28 09:58:09",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "aruiplex",
"github_project": "hypo",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "hypo-run"
}
