# Qik
Qik (*quick*) is an extensible command runner for monorepos. Like [make](https://www.gnu.org/software/make/), but with hash-based caching, advanced dependencies like globs and module imports, and execution in managed virtual environments.
Qik's command caching and module parametrization can dramatically improve local development and CI/CD time. Qik's plugin ecosystem offers remote S3 command caching, Python import graph linting, and much more.
Although qik is currently tailored towards Python repos, any project can use it.
[Read the qik docs here](https://qik.build/en/stable/guide/) or [this blog post on why I built Qik](https://qik.build/en/stable/blog/2024/08/12/why-im-building-qik/).
## Installation
```bash
pip install qik
```
For local development, we recommend installing most optional dependencies with:
```bash
pip install "qik[dev]"
```
Qik is compatible with Python 3.10 - 3.13, Linux, OSX, and WSL. It requires [git](https://git-scm.com) for command caching.
## Getting Started
### Commands
Configure commands in `qik.toml`:
```toml
[commands]
format = "ruff format ."
lint = "ruff check . --fix"
type-check = "pyright"
```
Run `qik` to execute every command across available cores. `qik <cmd> ...` runs specific commands.
Specify `deps` to cache commands. Here we use `pip-compile` from [pip-tools](https://github.com/jazzband/pip-tools) to lock requirements:
```toml
[commands.lock]
exec = "pip-compile > requirements.txt"
deps = ["requirements.in"]
```
The requirements file from `qik lock` is cached locally until `requirements.in` changes.
Along with globs, commands can depend on constants, other commands, Python distributions, and more.
Here we update our previous command to depend on the `pip-tools` distribution, breaking the cache when the version changes:
```toml
[commands.lock]
exec = "pip-compile > requirements.txt"
deps = [
"requirements.in",
{type = "pydist", name = "pip-tools"}
]
```
Here we use command dependencies to ensure our linter runs after formatting:
```toml
[commands.format]
exec = "ruff format ."
deps = ["**.py"]
[commands.lint]
exec = "ruff check . --fix"
deps = ["**.py", {type = "command", name = "format"}]
```
The `qik.pygraph` plugin provides the `pygraph` dependency. Here we run [pyright](https://github.com/microsoft/pyright) over the `hello.world` module whenever its code or imported code changes:
```toml
[plugins]
pygraph = "qik.pygraph"
[commands.check-types]
exec = "pyright hello/world"
deps = [{type = "pygraph", pyimport = "hello.world"}]
```
### Caches
Cache results directly in your project with the `repo` cache:
```toml
[commands.lock]
exec = "pip-compile > requirements.txt"
deps = ["requirements.in"]
cache = "repo"
```
Us the `qik.s3` plugin to create a shared remote cache using [AWS S3](https://aws.amazon.com/s3/):
```toml
[plugins]
s3 = "qik.s3"
[caches.remote]
type = "s3"
bucket = "qik-cache"
[commands.lock]
exec = "pip-compile > requirements.txt"
deps = ["requirements.in"]
artifacts = ["requirements.txt"]
cache = "remote"
```
We specify `artifacts` here to ensure `requirements.txt` is cached and restored.
### Spaces
Qik *spaces* define isolated environments and metadata for commands.
Here we create a space for our linting command. The `qik.uv` plugin uses [uv](https://github.com/astral-sh/uv) to build virtualenvs:
```toml
[plugins]
uv = "qik.uv"
[spaces]
ruff = "ruff-requirements.txt"
[commands.format]
exec = "ruff format ."
deps = ["**.py"]
space = "ruff"
[commands.lint]
exec = "ruff check . --fix"
deps = ["**.py", {type = "command", name = "format"}]
space = "ruff"
```
Running `qik` will lock and install the virtualenv for the `ruff` space and execute commands inside it. Changes to `ruff-requirements.txt` will break the cache of downstream commands.
Along with managing virtual environments, spaces can:
- Declare a `dotenv` file to automatically set environment variables.
- Declare a `root`. Running `qik` under this folder only selects commands in the space.
Here's a more complete example:
```toml
[spaces.my-app]
root = "app"
dotenv = "app.env"
venv = "requirements.txt"
```
### Modules
Spaces can define *modules* for command parametrization. Here we parametrize `pyright` over six modules across two spaces:
```toml
[spaces.first]
modules = ["a", "b", "c"]
venv = "requirements-first.txt"
[spaces.second]
modules = ["d", "e", "f"]
venv = "requirements-second.txt"
[commands.check-types]
exec = "pyright {module.dir}"
deps = [{type = "pygraph", pyimport = "{module.pyimport}"}]
```
Using `{module...}` in a command definition will automatically parametrize it over ever module in every space.
Use `qik check-types -s first` to specify spaces or `qik check-types -m b -m c` to specific modules.
### Fences
Plugins like `qik.pygraph` can leverage the `fence` of a space to perform import linting and other useful tasks:
```toml
[plugins]
pygraph = "qik.pygraph"
[spaces.first]
modules = ["a", "b", "c"]
fence = true
```
Running `qik pygraph.check` will ensure these modules only import each other. Add additional internal imports or a virtual environment to extend the fence:
```toml
[plugins]
pygraph = "qik.pygraph"
[spaces.first]
modules = ["a", "b", "c"]
fence = ["other/module"]
venv = "requirements.txt"
```
Include another space in the fence:
```toml
[spaces.first]
modules = ["a", "b", "c"]
[spaces.second]
modules = ["d", "e", "f"]
fence = [{type = "space", name = "first"}]
```
Running `qik pygraph.check -s second` will ensure the `second` space can import its modules and the `first` space's modules.
### Context
Set context variables and use them in your configuration. Below we create a context variable for the bundle cache:
```toml
ctx = [{name = "bundle_cache", default = "local"}]
[command.build-bundle]
exec = "npm run build"
artifacts = ["js-build/*"]
cache = "{ctx.bundle_cache}"
```
Context variables can be supplied in the environment:
```bash
BUNDLE_CACHE=remote qik build-bundle
```
### Command Line Interface
The core `qik` CLI functionality is as follows:
- `qik` to run all commands.
- `qik <cmd_name> <cmd_name>` to select commands by name.
- `-m` to select by module.
- `-s` to select by space.
- `--watch` to reactively run selected commands.
- `--since` to select commands based on changes since a git reference.
- `-f` to run without the cache.
- `-n` to set the number of workers.
- `--ls` to list commands instead of running them.
See [the command runner section](https://qik.build/en/stable/commands/#runner) for other advanced options, such as selecting commands based on cache status.
The `qikx` utility is also available for running commands in spaces:
```bash
# Run in the default space
qikx my_command --args val
# Run in a specific space
qikx my_command@space_name --args val
```
## Next Steps
Read the following guide to become a qik expert:
- [Spaces](https://qik.build/en/stable/spaces/): How spaces work, their functionality, and how commands and plugins leverage them.
- [Commands](https://qik.build/en/stable/commands/): Configuring and running commands. Learn the dependencies, selectors, and runtime behavior.
- [Context](https://qik.build/en/stable/context/): Using environment-based runtime context.
- [Caching](https://qik.build/en/stable/caching/): How caching works and how to configure different cache types.
Learn more about plugins:
- [Intro](https://qik.build/en/stable/plugin_intro/): How to configure and create plugins.
- [Pygraph](https://qik.build/en/stable/plugin_pygraph/): Using the `qik.pygraph` plugin for import-based dependencies and import linting.
- [UV](https://qik.build/en/stable/plugin_uv/): How to leverage and configure the `qik.uv` plugin for virtualenvs, including constraint files, custom PyPI indices, and more.
- [S3](https://qik.build/en/stable/plugin_s3/): Configuring the `qik.s3` plugin for a remote S3 cache.
Read the cookbook for more examples and guides:
- [Spaces](https://qik.build/en/stable/cookbook_spaces/): More examples of leveraging spaces.
- [Commands](https://qik.build/en/stable/cookbook_commands/): Common command definition examples.
- [CLI Usage](https://qik.build/en/stable/cookbook_cli/): Command line interface snippets.
- [CI/CD](https://qik.build/en/stable/ookbook_cicd/): Patterns for optimizing CI/CD time.
Finish by checking out:
- [Roadmap](https://qik.build/en/stable/roadmap/) for all the exciting upcoming features.
- [Blog](https://qik.build/en/stable/blog/index/) for updates, how-tos, and other articles.
Questions or thoughts? Open a [discussion](https://github.com/Opus10/qik/discussions). Report bugs [here](https://github.com/Opus10/qik/issues).
## Disclaimer
Qik is currently in beta. Bumping the minor version (e.g. `0.1.0` to `0.2.0`) will indicate an API break until we release version `1.0.0`.
Be diligent when using qik in your CI/CD. We recommend including a [base dependency](https://qik.build/en/stable/commands/#base) in your commands to regularly break the cache. We also recommend [understanding how the import graph is built](https://qik.build/en/stable/plugin_pygraph/#building-graph) when using pygraph dependencies.
Raw data
{
"_id": null,
"home_page": "https://github.com/AmbitionEng/qik",
"name": "qik",
"maintainer": null,
"docs_url": null,
"requires_python": "<4,>=3.10.0",
"maintainer_email": null,
"keywords": null,
"author": "Wes Kendall",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/cd/80/3f5e976bfa736eb0d7e4a2bb91dfeab89f3afd9f1f17df10c08e3336658f/qik-0.2.2.tar.gz",
"platform": null,
"description": "\n# Qik\n\nQik (*quick*) is an extensible command runner for monorepos. Like [make](https://www.gnu.org/software/make/), but with hash-based caching, advanced dependencies like globs and module imports, and execution in managed virtual environments.\n\nQik's command caching and module parametrization can dramatically improve local development and CI/CD time. Qik's plugin ecosystem offers remote S3 command caching, Python import graph linting, and much more.\n\nAlthough qik is currently tailored towards Python repos, any project can use it.\n\n[Read the qik docs here](https://qik.build/en/stable/guide/) or [this blog post on why I built Qik](https://qik.build/en/stable/blog/2024/08/12/why-im-building-qik/).\n\n## Installation\n\n```bash\npip install qik\n```\n\nFor local development, we recommend installing most optional dependencies with:\n\n```bash\npip install \"qik[dev]\"\n```\n\nQik is compatible with Python 3.10 - 3.13, Linux, OSX, and WSL. It requires [git](https://git-scm.com) for command caching.\n\n## Getting Started\n\n### Commands\n\nConfigure commands in `qik.toml`:\n\n```toml\n[commands]\nformat = \"ruff format .\"\nlint = \"ruff check . --fix\"\ntype-check = \"pyright\"\n```\n\nRun `qik` to execute every command across available cores. `qik <cmd> ...` runs specific commands.\n\nSpecify `deps` to cache commands. Here we use `pip-compile` from [pip-tools](https://github.com/jazzband/pip-tools) to lock requirements:\n\n```toml\n[commands.lock]\nexec = \"pip-compile > requirements.txt\"\ndeps = [\"requirements.in\"]\n```\n\nThe requirements file from `qik lock` is cached locally until `requirements.in` changes.\n\nAlong with globs, commands can depend on constants, other commands, Python distributions, and more.\n\nHere we update our previous command to depend on the `pip-tools` distribution, breaking the cache when the version changes:\n\n```toml\n[commands.lock]\nexec = \"pip-compile > requirements.txt\"\ndeps = [\n \"requirements.in\",\n {type = \"pydist\", name = \"pip-tools\"}\n]\n```\n\nHere we use command dependencies to ensure our linter runs after formatting:\n\n```toml\n[commands.format]\nexec = \"ruff format .\"\ndeps = [\"**.py\"]\n\n[commands.lint]\nexec = \"ruff check . --fix\"\ndeps = [\"**.py\", {type = \"command\", name = \"format\"}]\n```\n\nThe `qik.pygraph` plugin provides the `pygraph` dependency. Here we run [pyright](https://github.com/microsoft/pyright) over the `hello.world` module whenever its code or imported code changes:\n\n```toml\n[plugins]\npygraph = \"qik.pygraph\"\n\n[commands.check-types]\nexec = \"pyright hello/world\"\ndeps = [{type = \"pygraph\", pyimport = \"hello.world\"}]\n```\n\n### Caches\n\nCache results directly in your project with the `repo` cache:\n\n```toml\n[commands.lock]\nexec = \"pip-compile > requirements.txt\"\ndeps = [\"requirements.in\"]\ncache = \"repo\"\n```\n\nUs the `qik.s3` plugin to create a shared remote cache using [AWS S3](https://aws.amazon.com/s3/):\n\n```toml\n[plugins]\ns3 = \"qik.s3\"\n\n[caches.remote]\ntype = \"s3\"\nbucket = \"qik-cache\"\n\n[commands.lock]\nexec = \"pip-compile > requirements.txt\"\ndeps = [\"requirements.in\"]\nartifacts = [\"requirements.txt\"]\ncache = \"remote\"\n```\n\nWe specify `artifacts` here to ensure `requirements.txt` is cached and restored.\n\n### Spaces\n\nQik *spaces* define isolated environments and metadata for commands.\n\nHere we create a space for our linting command. The `qik.uv` plugin uses [uv](https://github.com/astral-sh/uv) to build virtualenvs:\n\n```toml\n[plugins]\nuv = \"qik.uv\"\n\n[spaces]\nruff = \"ruff-requirements.txt\"\n\n[commands.format]\nexec = \"ruff format .\"\ndeps = [\"**.py\"]\nspace = \"ruff\"\n\n[commands.lint]\nexec = \"ruff check . --fix\"\ndeps = [\"**.py\", {type = \"command\", name = \"format\"}]\nspace = \"ruff\"\n```\n\nRunning `qik` will lock and install the virtualenv for the `ruff` space and execute commands inside it. Changes to `ruff-requirements.txt` will break the cache of downstream commands.\n\nAlong with managing virtual environments, spaces can:\n\n- Declare a `dotenv` file to automatically set environment variables.\n- Declare a `root`. Running `qik` under this folder only selects commands in the space.\n\nHere's a more complete example:\n\n```toml\n[spaces.my-app]\nroot = \"app\"\ndotenv = \"app.env\"\nvenv = \"requirements.txt\"\n```\n\n### Modules\n\nSpaces can define *modules* for command parametrization. Here we parametrize `pyright` over six modules across two spaces:\n\n```toml\n[spaces.first]\nmodules = [\"a\", \"b\", \"c\"]\nvenv = \"requirements-first.txt\"\n\n[spaces.second]\nmodules = [\"d\", \"e\", \"f\"]\nvenv = \"requirements-second.txt\"\n\n[commands.check-types]\nexec = \"pyright {module.dir}\"\ndeps = [{type = \"pygraph\", pyimport = \"{module.pyimport}\"}]\n```\n\nUsing `{module...}` in a command definition will automatically parametrize it over ever module in every space.\n\nUse `qik check-types -s first` to specify spaces or `qik check-types -m b -m c` to specific modules.\n\n### Fences\n\nPlugins like `qik.pygraph` can leverage the `fence` of a space to perform import linting and other useful tasks:\n\n```toml\n[plugins]\npygraph = \"qik.pygraph\"\n\n[spaces.first]\nmodules = [\"a\", \"b\", \"c\"]\nfence = true\n```\n\nRunning `qik pygraph.check` will ensure these modules only import each other. Add additional internal imports or a virtual environment to extend the fence:\n\n```toml\n[plugins]\npygraph = \"qik.pygraph\"\n\n[spaces.first]\nmodules = [\"a\", \"b\", \"c\"]\nfence = [\"other/module\"]\nvenv = \"requirements.txt\"\n```\n\nInclude another space in the fence:\n\n```toml\n[spaces.first]\nmodules = [\"a\", \"b\", \"c\"]\n\n[spaces.second]\nmodules = [\"d\", \"e\", \"f\"]\nfence = [{type = \"space\", name = \"first\"}]\n```\n\nRunning `qik pygraph.check -s second` will ensure the `second` space can import its modules and the `first` space's modules.\n\n### Context\n\nSet context variables and use them in your configuration. Below we create a context variable for the bundle cache:\n\n```toml\nctx = [{name = \"bundle_cache\", default = \"local\"}]\n\n[command.build-bundle]\nexec = \"npm run build\"\nartifacts = [\"js-build/*\"]\ncache = \"{ctx.bundle_cache}\"\n```\n\nContext variables can be supplied in the environment:\n\n```bash\nBUNDLE_CACHE=remote qik build-bundle\n```\n\n### Command Line Interface\n\nThe core `qik` CLI functionality is as follows:\n\n- `qik` to run all commands.\n- `qik <cmd_name> <cmd_name>` to select commands by name.\n- `-m` to select by module.\n- `-s` to select by space.\n- `--watch` to reactively run selected commands.\n- `--since` to select commands based on changes since a git reference.\n- `-f` to run without the cache.\n- `-n` to set the number of workers.\n- `--ls` to list commands instead of running them.\n\nSee [the command runner section](https://qik.build/en/stable/commands/#runner) for other advanced options, such as selecting commands based on cache status.\n\nThe `qikx` utility is also available for running commands in spaces:\n\n```bash\n# Run in the default space\nqikx my_command --args val\n\n# Run in a specific space\nqikx my_command@space_name --args val\n```\n\n## Next Steps\n\nRead the following guide to become a qik expert:\n\n- [Spaces](https://qik.build/en/stable/spaces/): How spaces work, their functionality, and how commands and plugins leverage them.\n- [Commands](https://qik.build/en/stable/commands/): Configuring and running commands. Learn the dependencies, selectors, and runtime behavior.\n- [Context](https://qik.build/en/stable/context/): Using environment-based runtime context.\n- [Caching](https://qik.build/en/stable/caching/): How caching works and how to configure different cache types.\n\nLearn more about plugins:\n\n- [Intro](https://qik.build/en/stable/plugin_intro/): How to configure and create plugins.\n- [Pygraph](https://qik.build/en/stable/plugin_pygraph/): Using the `qik.pygraph` plugin for import-based dependencies and import linting.\n- [UV](https://qik.build/en/stable/plugin_uv/): How to leverage and configure the `qik.uv` plugin for virtualenvs, including constraint files, custom PyPI indices, and more.\n- [S3](https://qik.build/en/stable/plugin_s3/): Configuring the `qik.s3` plugin for a remote S3 cache.\n\nRead the cookbook for more examples and guides:\n\n- [Spaces](https://qik.build/en/stable/cookbook_spaces/): More examples of leveraging spaces.\n- [Commands](https://qik.build/en/stable/cookbook_commands/): Common command definition examples.\n- [CLI Usage](https://qik.build/en/stable/cookbook_cli/): Command line interface snippets.\n- [CI/CD](https://qik.build/en/stable/ookbook_cicd/): Patterns for optimizing CI/CD time.\n\nFinish by checking out:\n\n- [Roadmap](https://qik.build/en/stable/roadmap/) for all the exciting upcoming features.\n- [Blog](https://qik.build/en/stable/blog/index/) for updates, how-tos, and other articles.\n\nQuestions or thoughts? Open a [discussion](https://github.com/Opus10/qik/discussions). Report bugs [here](https://github.com/Opus10/qik/issues).\n\n## Disclaimer\n\nQik is currently in beta. Bumping the minor version (e.g. `0.1.0` to `0.2.0`) will indicate an API break until we release version `1.0.0`.\n\nBe diligent when using qik in your CI/CD. We recommend including a [base dependency](https://qik.build/en/stable/commands/#base) in your commands to regularly break the cache. We also recommend [understanding how the import graph is built](https://qik.build/en/stable/plugin_pygraph/#building-graph) when using pygraph dependencies.\n",
"bugtrack_url": null,
"license": "BSD-3-Clause",
"summary": "Tame your monorepo. Make CI fast again.",
"version": "0.2.2",
"project_urls": {
"Documentation": "https://qik.readthedocs.io",
"Homepage": "https://github.com/AmbitionEng/qik",
"Repository": "https://github.com/AmbitionEng/qik"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6515bb7743ccf9ebbeaaab82f26aea7a88a2927b72fc05c8450da00ccf8fe942",
"md5": "c8452ac0039f711cb386cc0422a3425a",
"sha256": "d2f859a25b481df8bf56952d6c83acf21590ea46c0dc7eb8a30d51c032215984"
},
"downloads": -1,
"filename": "qik-0.2.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c8452ac0039f711cb386cc0422a3425a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4,>=3.10.0",
"size": 55981,
"upload_time": "2024-12-16T01:41:22",
"upload_time_iso_8601": "2024-12-16T01:41:22.149354Z",
"url": "https://files.pythonhosted.org/packages/65/15/bb7743ccf9ebbeaaab82f26aea7a88a2927b72fc05c8450da00ccf8fe942/qik-0.2.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "cd803f5e976bfa736eb0d7e4a2bb91dfeab89f3afd9f1f17df10c08e3336658f",
"md5": "d545ef90a7b21f1bd75395aaec3eea65",
"sha256": "1734bf55b990bcf4e5199e5cf855423f8daeac54a457b14e98707e8a372009de"
},
"downloads": -1,
"filename": "qik-0.2.2.tar.gz",
"has_sig": false,
"md5_digest": "d545ef90a7b21f1bd75395aaec3eea65",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4,>=3.10.0",
"size": 46293,
"upload_time": "2024-12-16T01:41:26",
"upload_time_iso_8601": "2024-12-16T01:41:26.068101Z",
"url": "https://files.pythonhosted.org/packages/cd/80/3f5e976bfa736eb0d7e4a2bb91dfeab89f3afd9f1f17df10c08e3336658f/qik-0.2.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-16 01:41:26",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "AmbitionEng",
"github_project": "qik",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"circle": true,
"tox": true,
"lcname": "qik"
}
JSON
