# Taku
Taku is a library for easy manipulating pipelined-tasks. It manages intermediate caching, job depedencies, provide a easy interface to parallel jobs using multiple backends. Also, it supports direct instantiation of a torch `Dataset`, allowing for end-to-end data generation & training experience.
**Caveat**: Tasklib is only beta now, and for now only supports chained dependency (but one/multiple job in a task can be dependent on multiple/one jobs in another task).
## Example Task Class
```python
from pathlib import Path
from typing import Any, List, Optional
import taku
from taku.task import JobData, JobName, JobSpecs, TaskName
class ExampleTask(taku.Task):
def __init__(self, name: TaskName, hparams: Any, meta_override_dir: Path | None = None) -> None:
super().__init__(name, hparams, meta_override_dir)
def gather_jobs(self, upstream_job_names: List[JobName]) -> List[JobSpecs]:
return super().gather_jobs(upstream_job_names)
def run(self, job_name: JobName, job_data: JobData, upstream_job_names: List[str], upstream_job_data: List[JobData]) -> JobData:
return super().run(job_name, job_data, upstream_job_names, upstream_job_data)
def has_job_data(self, job_name: str) -> bool:
return super().has_job_data(job_name)
def save_job_data(self, job_name: str, job_data: JobData) -> None:
return super().save_job_data(job_name, job_data)
def load_job_data(self, job_name: str) -> JobData:
return super().load_job_data(job_name)
```
## Development
```bash
# Develop locally
python setup.py develop
# Update to PyPI
bash bump_and_upload.sh
```
Raw data
{
"_id": null,
"home_page": "https://taku.readthedocs.io/en/latest/",
"name": "taku",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "task,manager",
"author": "Jiahui Huang",
"author_email": "huangjh.work@outlook.com",
"download_url": "https://files.pythonhosted.org/packages/2e/20/36e650f98db102f0651cd6b6b5684b06fd87462c468cd6c727fa17dfd5ee/taku-0.7.0.tar.gz",
"platform": null,
"description": "# Taku\n\nTaku is a library for easy manipulating pipelined-tasks. It manages intermediate caching, job depedencies, provide a easy interface to parallel jobs using multiple backends. Also, it supports direct instantiation of a torch `Dataset`, allowing for end-to-end data generation & training experience.\n\n**Caveat**: Tasklib is only beta now, and for now only supports chained dependency (but one/multiple job in a task can be dependent on multiple/one jobs in another task).\n\n## Example Task Class\n\n```python\nfrom pathlib import Path\nfrom typing import Any, List, Optional\nimport taku\nfrom taku.task import JobData, JobName, JobSpecs, TaskName\n\n\nclass ExampleTask(taku.Task):\n def __init__(self, name: TaskName, hparams: Any, meta_override_dir: Path | None = None) -> None:\n super().__init__(name, hparams, meta_override_dir)\n\n def gather_jobs(self, upstream_job_names: List[JobName]) -> List[JobSpecs]:\n return super().gather_jobs(upstream_job_names)\n \n def run(self, job_name: JobName, job_data: JobData, upstream_job_names: List[str], upstream_job_data: List[JobData]) -> JobData:\n return super().run(job_name, job_data, upstream_job_names, upstream_job_data)\n \n def has_job_data(self, job_name: str) -> bool:\n return super().has_job_data(job_name)\n \n def save_job_data(self, job_name: str, job_data: JobData) -> None:\n return super().save_job_data(job_name, job_data)\n \n def load_job_data(self, job_name: str) -> JobData:\n return super().load_job_data(job_name)\n\n```\n\n## Development\n\n```bash\n# Develop locally\npython setup.py develop\n# Update to PyPI\nbash bump_and_upload.sh\n```\n",
"bugtrack_url": null,
"license": "",
"summary": "Taku: Task managing made easy",
"version": "0.7.0",
"project_urls": {
"Homepage": "https://taku.readthedocs.io/en/latest/"
},
"split_keywords": [
"task",
"manager"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "159ae2cf618be6eefbe276f420923fac6f319cc802962381679c0618a073b21b",
"md5": "42c648fa408f9698abc35205a2d0856d",
"sha256": "52ea4d16a6c03b46217dd1091bf26f80c58db4efbe6d47e23d6e0574172ec01d"
},
"downloads": -1,
"filename": "taku-0.7.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "42c648fa408f9698abc35205a2d0856d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 10928,
"upload_time": "2023-08-04T13:17:03",
"upload_time_iso_8601": "2023-08-04T13:17:03.116847Z",
"url": "https://files.pythonhosted.org/packages/15/9a/e2cf618be6eefbe276f420923fac6f319cc802962381679c0618a073b21b/taku-0.7.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "2e2036e650f98db102f0651cd6b6b5684b06fd87462c468cd6c727fa17dfd5ee",
"md5": "b446ac61d118ae10ade7430bdcdeb452",
"sha256": "bd17e3da884ffef7e056a10a1dba81e6069781eafb5d16894fad18ca4d19c6cc"
},
"downloads": -1,
"filename": "taku-0.7.0.tar.gz",
"has_sig": false,
"md5_digest": "b446ac61d118ae10ade7430bdcdeb452",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 11453,
"upload_time": "2023-08-04T13:17:05",
"upload_time_iso_8601": "2023-08-04T13:17:05.509255Z",
"url": "https://files.pythonhosted.org/packages/2e/20/36e650f98db102f0651cd6b6b5684b06fd87462c468cd6c727fa17dfd5ee/taku-0.7.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-08-04 13:17:05",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "taku"
}
JSON