| Name | doot JSON |
| Version |
0.7.2
JSON |
| download |
| home_page | None |
| Summary | An opinionated, TOML based task runner |
| upload_time | 2024-04-20 15:42:55 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.11 |
| license | ACAB License © 2022-12-09 John Grey To the maximum extent applicable by law, and any licenses of components of this work: ** Permissions: Except as prohibited below, any individual, group, organization, charity, business entity or university may use, modify, and distribute source code and software utilising this work. Users may extend this license, so long as these initial conditions remain in force. ** Obligations: Users of this work are obligated to freely provide public access to AI scripts written to be used by this work. ** Prohibitions: No Law Enforcement, Carceral Institutions, Immigration enforcement entities may use the work or products of the work. For any reason, be it for simulation, production of propaganda, or otherwise. No business entity where the ratio of pay (salaried, freelance, stocks, or other benefits) between the highest and lowest individual in the entity is greater than 50 : 1 may use the work for any reason. Business entities with boards/management comprising less than 1/2 POC may not use the work for any reason. No individual or entity may use this work for racist or bigoted purposes. Users must not remove this license from the work. ** Sanction Users recognise breach of the above terms may be sanctioned to the maximum extent applicable by law. ** Warranty No warranty, liability, or fitness for purpose is implied. |
| keywords |
toml
taskrunner
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
beautifulsoup4
bibtexparser
itemadapter
matplotlib
networkx
numpy
pandas
pdfrw
Pillow
pony
pylatexenc
pypandoc
PyYAML
regex
Requests
scikit-learn
scrapy
seaborn
tomlguard
tweepy
w3lib
wordcloud
xsdata
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# doot
Version : 0.7.2
Author : John Grey
Date : 2022-12-09
## Overview
This started out as an opinionated rewrite of the [doit](https://pydoit.org/contents.html) task runner.
For other, more mature alternatives, see [Luigi](https://github.com/spotify/luigi), and [SnakeMake](https://github.com/snakemake/snakemake)
and, of course, [GNU Make](https://www.gnu.org/software/make/).
Mainly, I found the process of authoring tasks and reusing them to be convoluted.
Doot is an attempt to simplify matters.
Settings and tasks are specified in toml files, which the doot program assists in writing.
More complicated logic is written in normal python, either as a library (eg: [Dootle](https://github.com/jgrey4296/dootle),
or as local task specific code.
To use doot, call `doot help`.
Examples and details can be found in the [Wiki](https://github.com/jgrey4296/doot/wiki)
### doot.toml
The `doot.toml` file provides a place for marking where doot considers the current working directory,
and lets you control general settings.
It is created from a default template by just running `doot`, if no `doot.toml` file exists.
If you don't want a separate file, everything can be added to `pyproject.toml` by prepending `tool.doot` to sections.
eg:
``` toml
# -*- mode:conf-toml; -*-
[settings.general]
notify = { say-on-exit = false }
loaders = { commands="default", task="default", parser="default"}
location_check = { make_missing = true, print_levels={action="WARN", execute="WARN" } }
[settings.tasks]
sources = [".tasks"] # Files or directories where task specs can be loaded from, expanded according to [[locations]] keys
code = [] # Directories where task specific code can be imported from, expanded according to [[locations]] keys
sleep = { task=0.2, subtask=1, batch=1 }
[logging]
stream = { level="WARN", allow=["doot"], format="{levelname:<8} : {message}", colour=true }
file = { level="DEBUG", allow=["doot"], format="{levelname:<8} : {message:<20} :|: (module:{module} line:{lineno} fn:{funcName})" }
printer = { level="INFO", colour=true}
[plugins]
# Allows for defining shorthands
command = { other-run = "doot.cmds.run_cmd:RunCmd", tasks = "doot.cmds.list_cmd:ListCmd" }
report-line = {}
reporter = {}
action = {}
task = {}
[commands]
run = { tracker = "default", runner = "default", reporter= "default", report-line = []}
[[locations]]
temp = ".temp"
desktop = "~/Desktop"
```
### Tasks
Tasks are specified in toml, by default in the `doot.toml` file, or toml files in a `.tasks` directory, but that can be changed in `doot.toml:settings.tasks.sources`.
The easiest way to write one is to use `doot stub`.
eg: `doot stub mygroup::mytask` produces:
``` toml
[[tasks.mygroup]] # mygroup is the group this task is part of
name = "mytask" # combined together, this means this specific task is `mygroup::mytask`
version = "0.1" # <str>
ctor = "task" # <type> the python class this task uses. See the plugins listed in 'doot plugins'
doc = ["Text to help understand what this task does"] # <list[str]>
required_for = [] # <list[DootTaskArtifact]> see below
depends_on = [] # <list[DootTaskArtifact]> see below
actions = [] # <list[Any]> See below
```
You can see what tasks are available by calling `doot list`.
You can get help on a specific task by calling `doot {task} --help` or `doot help {task}`.
You can run a task by calling `doot {task}` or `doot run {task}`.
eg: `doot mygroup::mytask`
### Actions
Tasks run a sequence of actions, specified in the following form. the argument to `do` can be an import specifier,
or an alias from the actions section of `doot plugins`:
``` toml
{ do="write!", args=[], aKwd="val" },
{ do="doot.actions.io:WriteAction", args=[], aKwd="val" },
```
You can get help on writing an action using `doot stub --Actions {action}`. eg: `doot stub --A write!` will print:
```
- Action write! : doot.actions.io:WriteAction
-- Declared kwargs for action: ['from_', 'to']
-- Toml Form:
{ do="write!", args=[] }
- For Custom Python Actions, implement the following in the .tasks director
def custom_action(spec:DootActionSpec, task_state:dict) -> None|bool|dict:...
```
When specifying values in toml you can use direct keys, or indirect keys.
For example, the action:
``` toml
{ do="log", msg="This is a test", level="INFO" }
```
will log that exact message, at that exact logging level.
Meanwhile the action:
``` toml
{ do="log", msg_="gen_msg", level="INFO" }
```
Will use the string stored in the task state's 'gen_msg' variable, assuming that variable has been set by a prior action, or the toml spec of the task.
This also allows you to specify the key to put information into:
``` toml
{ do="read!", from="{data}/names.txt", update_="names" }
```
This reads from the `names.txt` file, and adds it to the task state in the key `names`.
### Task Dependencies
Tasks can depend on other tasks, or artifacts like files.
These can be specified in the `required_for` and `depends_on` fields of a task spec.
To depend on a task, use its full name. eg: `examples::basic`.
To depend on a file, specify it with the prefix `file:>`. eg: `file:>./doot.toml`.
### String and Path expansion
To simplify using locations, both `doot.toml` and task toml files can have `[[locations]]` tables.
eg:
``` toml
[[locations]]
tasks = ".tasks"
home = "~" # will be made into an absolute path when a task uses the {home} key in a path.
data = "doot/data" # A relative path, will be made absolute, according to cwd.
other = "/somewhere/else" # Absolute paths can also be used.
subdata = "{data}/blah" # {data} will expand to doot/data at runtime
```
Expansion of arguments is relatively straight forward, and is basically python string expansion.
Actions will expand parameters they support.
eg:
``` toml
actions = [
# This will write from the task state['data'] into the path "doot/data/example.txt":
{do="write!", from_="data", to="{data}/example.txt" },
# This will print to stdout the task state['data']:
{do="log", msg="The data is: {data}" }
]
```
### Entry Points / Plugins
[Entry Points](https://packaging.python.org/en/latest/specifications/pyproject-toml/#entry-points) for doot to automatically recognise system installed packages are supported. Eg:
``` toml
# From dootle's pyproject.toml
[project.entry-points."doot.plugins.command"]
example = "dootle.cmds.example_cmd:ExampleCmd"
[project-entry-points."doot.plugins.action"]
an_action = "..."
```
Plugins are specified in the pyproject.toml table `[project.entry-points."doot.plugins.TYPE"]`,
where TYPE is one of the forms defined in variables found in `doot.constants`:
1) `FRONTEND_PLUGIN_TYPES` : "command", "reporter", "report-line",
2) `BACKEND_PLUGIN_TYPES` : "tracker", "runner", "command-loader", "task-loader", "parser", "action", "job", "database"
Currently available plugins are listed with the command `doot plugins`.
They can be filtered with a simple pattern (`pattern:str in plugin.fullname:str` essentially).
Raw data
{
"_id": null,
"home_page": null,
"name": "doot",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.11",
"maintainer_email": null,
"keywords": "toml, taskrunner",
"author": null,
"author_email": "jgrey <jgrey.n.plus.one@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/fb/c2/72669f371528f41b8d96f1ff114cf868e35b7a82da5c3791fd2de6a32308/doot-0.7.2.tar.gz",
"platform": null,
"description": "\n# doot\nVersion : 0.7.2 \nAuthor : John Grey \nDate : 2022-12-09 \n\n## Overview\nThis started out as an opinionated rewrite of the [doit](https://pydoit.org/contents.html) task runner.\nFor other, more mature alternatives, see [Luigi](https://github.com/spotify/luigi), and [SnakeMake](https://github.com/snakemake/snakemake)\nand, of course, [GNU Make](https://www.gnu.org/software/make/).\n\nMainly, I found the process of authoring tasks and reusing them to be convoluted.\nDoot is an attempt to simplify matters.\nSettings and tasks are specified in toml files, which the doot program assists in writing.\nMore complicated logic is written in normal python, either as a library (eg: [Dootle](https://github.com/jgrey4296/dootle),\nor as local task specific code.\n\nTo use doot, call `doot help`.\nExamples and details can be found in the [Wiki](https://github.com/jgrey4296/doot/wiki)\n\n### doot.toml\nThe `doot.toml` file provides a place for marking where doot considers the current working directory,\nand lets you control general settings.\nIt is created from a default template by just running `doot`, if no `doot.toml` file exists.\nIf you don't want a separate file, everything can be added to `pyproject.toml` by prepending `tool.doot` to sections.\n\neg:\n``` toml\n# -*- mode:conf-toml; -*-\n[settings.general]\nnotify = { say-on-exit = false }\nloaders = { commands=\"default\", task=\"default\", parser=\"default\"}\nlocation_check = { make_missing = true, print_levels={action=\"WARN\", execute=\"WARN\" } }\n\n[settings.tasks]\nsources = [\".tasks\"] # Files or directories where task specs can be loaded from, expanded according to [[locations]] keys\ncode = [] # Directories where task specific code can be imported from, expanded according to [[locations]] keys\nsleep = { task=0.2, subtask=1, batch=1 }\n\n[logging]\nstream = { level=\"WARN\", allow=[\"doot\"], format=\"{levelname:<8} : {message}\", colour=true }\nfile = { level=\"DEBUG\", allow=[\"doot\"], format=\"{levelname:<8} : {message:<20} :|: (module:{module} line:{lineno} fn:{funcName})\" }\nprinter = { level=\"INFO\", colour=true}\n\n[plugins]\n# Allows for defining shorthands\ncommand = { other-run = \"doot.cmds.run_cmd:RunCmd\", tasks = \"doot.cmds.list_cmd:ListCmd\" }\nreport-line = {}\nreporter = {}\naction = {}\ntask = {}\n\n[commands]\nrun = { tracker = \"default\", runner = \"default\", reporter= \"default\", report-line = []}\n\n[[locations]]\ntemp = \".temp\"\ndesktop = \"~/Desktop\"\n```\n\n\n### Tasks\nTasks are specified in toml, by default in the `doot.toml` file, or toml files in a `.tasks` directory, but that can be changed in `doot.toml:settings.tasks.sources`.\nThe easiest way to write one is to use `doot stub`.\neg: `doot stub mygroup::mytask` produces:\n\n``` toml\n[[tasks.mygroup]] # mygroup is the group this task is part of\nname = \"mytask\" # combined together, this means this specific task is `mygroup::mytask`\nversion = \"0.1\" # <str>\nctor = \"task\" # <type> the python class this task uses. See the plugins listed in 'doot plugins'\ndoc = [\"Text to help understand what this task does\"] # <list[str]>\nrequired_for = [] # <list[DootTaskArtifact]> see below\ndepends_on = [] # <list[DootTaskArtifact]> see below\nactions = [] # <list[Any]> See below\n```\n\nYou can see what tasks are available by calling `doot list`.\nYou can get help on a specific task by calling `doot {task} --help` or `doot help {task}`.\nYou can run a task by calling `doot {task}` or `doot run {task}`.\neg: `doot mygroup::mytask`\n\n### Actions\nTasks run a sequence of actions, specified in the following form. the argument to `do` can be an import specifier,\nor an alias from the actions section of `doot plugins`:\n\n``` toml\n{ do=\"write!\", args=[], aKwd=\"val\" },\n{ do=\"doot.actions.io:WriteAction\", args=[], aKwd=\"val\" },\n```\n\nYou can get help on writing an action using `doot stub --Actions {action}`. eg: `doot stub --A write!` will print:\n\n```\n- Action write! : doot.actions.io:WriteAction\n-- Declared kwargs for action: ['from_', 'to']\n\n-- Toml Form:\n{ do=\"write!\", args=[] }\n\n- For Custom Python Actions, implement the following in the .tasks director\ndef custom_action(spec:DootActionSpec, task_state:dict) -> None|bool|dict:...\n```\n\nWhen specifying values in toml you can use direct keys, or indirect keys.\nFor example, the action:\n``` toml\n{ do=\"log\", msg=\"This is a test\", level=\"INFO\" }\n```\nwill log that exact message, at that exact logging level.\nMeanwhile the action:\n\n``` toml\n{ do=\"log\", msg_=\"gen_msg\", level=\"INFO\" }\n```\nWill use the string stored in the task state's 'gen_msg' variable, assuming that variable has been set by a prior action, or the toml spec of the task.\nThis also allows you to specify the key to put information into:\n\n``` toml\n{ do=\"read!\", from=\"{data}/names.txt\", update_=\"names\" }\n```\nThis reads from the `names.txt` file, and adds it to the task state in the key `names`.\n\n### Task Dependencies\nTasks can depend on other tasks, or artifacts like files.\nThese can be specified in the `required_for` and `depends_on` fields of a task spec.\nTo depend on a task, use its full name. eg: `examples::basic`.\nTo depend on a file, specify it with the prefix `file:>`. eg: `file:>./doot.toml`.\n\n### String and Path expansion\nTo simplify using locations, both `doot.toml` and task toml files can have `[[locations]]` tables.\neg:\n``` toml\n[[locations]]\ntasks = \".tasks\"\nhome = \"~\" # will be made into an absolute path when a task uses the {home} key in a path.\ndata = \"doot/data\" # A relative path, will be made absolute, according to cwd.\nother = \"/somewhere/else\" # Absolute paths can also be used.\nsubdata = \"{data}/blah\" # {data} will expand to doot/data at runtime\n```\n\nExpansion of arguments is relatively straight forward, and is basically python string expansion.\nActions will expand parameters they support.\neg:\n``` toml\n\nactions = [\n# This will write from the task state['data'] into the path \"doot/data/example.txt\":\n{do=\"write!\", from_=\"data\", to=\"{data}/example.txt\" },\n# This will print to stdout the task state['data']:\n{do=\"log\", msg=\"The data is: {data}\" }\n]\n```\n\n### Entry Points / Plugins\n[Entry Points](https://packaging.python.org/en/latest/specifications/pyproject-toml/#entry-points) for doot to automatically recognise system installed packages are supported. Eg:\n\n``` toml\n# From dootle's pyproject.toml\n[project.entry-points.\"doot.plugins.command\"]\nexample = \"dootle.cmds.example_cmd:ExampleCmd\"\n\n[project-entry-points.\"doot.plugins.action\"]\nan_action = \"...\"\n```\n\nPlugins are specified in the pyproject.toml table `[project.entry-points.\"doot.plugins.TYPE\"]`,\nwhere TYPE is one of the forms defined in variables found in `doot.constants`:\n1) `FRONTEND_PLUGIN_TYPES` : \"command\", \"reporter\", \"report-line\",\n2) `BACKEND_PLUGIN_TYPES` : \"tracker\", \"runner\", \"command-loader\", \"task-loader\", \"parser\", \"action\", \"job\", \"database\"\n\nCurrently available plugins are listed with the command `doot plugins`.\nThey can be filtered with a simple pattern (`pattern:str in plugin.fullname:str` essentially).\n",
"bugtrack_url": null,
"license": "ACAB License \u00a9 2022-12-09 John Grey To the maximum extent applicable by law, and any licenses of components of this work: ** Permissions: Except as prohibited below, any individual, group, organization, charity, business entity or university may use, modify, and distribute source code and software utilising this work. Users may extend this license, so long as these initial conditions remain in force. ** Obligations: Users of this work are obligated to freely provide public access to AI scripts written to be used by this work. ** Prohibitions: No Law Enforcement, Carceral Institutions, Immigration enforcement entities may use the work or products of the work. For any reason, be it for simulation, production of propaganda, or otherwise. No business entity where the ratio of pay (salaried, freelance, stocks, or other benefits) between the highest and lowest individual in the entity is greater than 50 : 1 may use the work for any reason. Business entities with boards/management comprising less than 1/2 POC may not use the work for any reason. No individual or entity may use this work for racist or bigoted purposes. Users must not remove this license from the work. ** Sanction Users recognise breach of the above terms may be sanctioned to the maximum extent applicable by law. ** Warranty No warranty, liability, or fitness for purpose is implied. ",
"summary": "An opinionated, TOML based task runner",
"version": "0.7.2",
"project_urls": {
"changelog": "https://github.com//doot/blob/master/CHANGELOG.md",
"documentation": "https://github.com/jgrey4296/doot/wiki",
"homepage": "https://github.com/jgrey4296/doot",
"repository": "https://github.com/jgrey4296/doot"
},
"split_keywords": [
"toml",
" taskrunner"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e5d628fc09f6737793c747f12f7c7234cd3a0b3bb11988f87d26cd988f834b95",
"md5": "72c2c2e3adbb65a95772ff57bcf96359",
"sha256": "756ecb2704bdcf79343fab42266e02bd312b2feaab66efa36abdf23db2271d8d"
},
"downloads": -1,
"filename": "doot-0.7.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "72c2c2e3adbb65a95772ff57bcf96359",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11",
"size": 214603,
"upload_time": "2024-04-20T15:42:52",
"upload_time_iso_8601": "2024-04-20T15:42:52.977798Z",
"url": "https://files.pythonhosted.org/packages/e5/d6/28fc09f6737793c747f12f7c7234cd3a0b3bb11988f87d26cd988f834b95/doot-0.7.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "fbc272669f371528f41b8d96f1ff114cf868e35b7a82da5c3791fd2de6a32308",
"md5": "e672f70e2ac39a6cb71a4c808e03ad77",
"sha256": "059ac68cad67bb8262b8026b5fc3b5d9c2074f25ca60f51d92464396d124f940"
},
"downloads": -1,
"filename": "doot-0.7.2.tar.gz",
"has_sig": false,
"md5_digest": "e672f70e2ac39a6cb71a4c808e03ad77",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11",
"size": 131192,
"upload_time": "2024-04-20T15:42:55",
"upload_time_iso_8601": "2024-04-20T15:42:55.194957Z",
"url": "https://files.pythonhosted.org/packages/fb/c2/72669f371528f41b8d96f1ff114cf868e35b7a82da5c3791fd2de6a32308/doot-0.7.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-20 15:42:55",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "jgrey4296",
"github_project": "doot",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "beautifulsoup4",
"specs": []
},
{
"name": "bibtexparser",
"specs": []
},
{
"name": "itemadapter",
"specs": []
},
{
"name": "matplotlib",
"specs": []
},
{
"name": "networkx",
"specs": []
},
{
"name": "numpy",
"specs": []
},
{
"name": "pandas",
"specs": []
},
{
"name": "pdfrw",
"specs": []
},
{
"name": "Pillow",
"specs": []
},
{
"name": "pony",
"specs": []
},
{
"name": "pylatexenc",
"specs": []
},
{
"name": "pypandoc",
"specs": []
},
{
"name": "PyYAML",
"specs": []
},
{
"name": "regex",
"specs": []
},
{
"name": "Requests",
"specs": []
},
{
"name": "scikit-learn",
"specs": []
},
{
"name": "scrapy",
"specs": []
},
{
"name": "seaborn",
"specs": []
},
{
"name": "tomlguard",
"specs": []
},
{
"name": "tweepy",
"specs": []
},
{
"name": "w3lib",
"specs": []
},
{
"name": "wordcloud",
"specs": []
},
{
"name": "xsdata",
"specs": []
}
],
"lcname": "doot"
}
