| Name | universal-test-runner JSON |
| Version |
0.6.0
JSON |
| download |
| home_page | |
| Summary | Universal, language-aware unit test runner. |
| upload_time | 2023-12-07 03:12:32 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.9 |
| license | |
| keywords |
testing
test-runner
pytest
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# universal-test-runner
The Universal Test Runner is a zero-configuration, language-aware way to run unit tests in any project. It installs a command, `t`, which will determine how to run your test suite (and then run it).
<p align="center">
<a href="https://github.com/xavdid/test-runner-demo/blob/main/_demo/demo-min.gif">
<img src="https://raw.githubusercontent.com/xavdid/test-runner-demo/main/_demo/demo-min.gif"/>
</a>
</p>
If you're working on a JS project, it runs `[your package manager here] test`. You've run `pytest` in this folder before? `pytest` it is. Rust project? `cargo test` coming right up. Is also clever about running all your `go` module tests (regardless of how they're organized). No matter the command, all args are passed directly into the test runner.
Currently [supports 7 languages](#supported-languages) (and their respective test frameworks). Please open an issue if I'm missing your favorite!
## Installation
The easiest way to install is by using [pipx](https://pypa.github.io/pipx/):
```bash
pipx install universal-test-runner
```
You can also use brew (which will build from source and take a little longer):
```bash
brew install xavdid/projects/universal-test-runner
```
## Usage
> You can also clone the [demo repo](https://github.com/xavdid/test-runner-demo) to play around with the test runner - it's got toy examples to show how tests are run in many languages!
Once installed, the command `t` will be available. Run it in a project folder's root and it'll do its best to run your unit tests:
```
% t
-> pytest
=============================== test session starts ================================
platform darwin -- Python 3.11.0, pytest-7.3.1, pluggy-1.0.0
rootdir: /Users/username/projects/test-runner
collected 78 items
tests/test_cli.py ... [ 3%]
tests/test_context.py ..................... [ 30%]
tests/test_matchers.py .................................................. [ 94%]
tests/test_runner.py .... [100%]
================================ 78 passed in 0.08s ================================
```
It passes all arguments and environment modifications down to the chosen test runner:
```
% t -k test_builder --verbose
-> pytest -k test_builder --verbose
=============================== test session starts ================================
platform darwin -- Python 3.11.0, pytest-7.3.1, pluggy-1.0.0
cachedir: .pytest_cache
rootdir: /Users/username/projects/test-runner
collected 78 items / 77 deselected / 1 selected
tests/test_context.py::test_builder PASSED [100%]
========================= 1 passed, 77 deselected in 0.03s =========================
```
It prints the command it's running as part of the output. To disable that behavior, set `UTR_DISABLE_ECHO` to any value in the environment.
If it can't guess the testing method, it will tell you so. Feel free to open an issue to request wider language support!
### Debugging
The package also ships a command to surface info about itself: `universal-test-runner`. It has a few key pieces of functionality:
- the `universal-test-runner --version` flag, which prints info about your installed package version
- the `universal-test-runner debug` command, which prints info about which matcher would run (and why):
```
% universal-test-runner debug
[universal-test-runner]: checking each handler for first match
[universal-test-runner]: Checking matcher 01/11: pytest
[universal-test-runner]: looking for: ".pytest_cache"
[universal-test-runner]: no match, continuing
[universal-test-runner]: Checking matcher 02/11: py
[universal-test-runner]: looking for: "tests.py"
[universal-test-runner]: no match, continuing
[universal-test-runner]: Checking matcher 03/11: go_multi
[universal-test-runner]: looking for: "go.mod" and no arguments
[universal-test-runner]: no match, continuing
[universal-test-runner]: Checking matcher 04/11: go_single
[universal-test-runner]: looking for: "go.mod" or a file named "..._test.go"
[universal-test-runner]: no match, continuing
...
[universal-test-runner]: no matching test handler. To add a new one, please file an issue: https://github.com/xavdid/universal-test-runner/issues
```
## Supported Languages
This list describes how each language behaves (but not the order in which languages are matched; use the [debugger](#debugging) for that).
- Python
- checks for `manage.py` (Django)
- else uses `pytest` if you've run `pytest` before. You'll need to run pytest manually on clean installs before `t` will work
- looks for a `tests.py` file if not
- Rust
- `cargo test`
- Go
- if there's a `X_test.go`, then runs a plain `go test`
- if you pass any args at all, runs `go test your-args-here`
- otherwise, runs `go test ./...`
- Elixir
- `mix test`
- Clojure
- `lein test`
- Javascript/Typescript
- if there's a `package.json` and it has a `test` script, runs `[package manager] test`, where `[package manager]` is:
- `npm` if there's a `package-lock.json`
- `yarn` if there's a `yarn.lock`
- `pnpm` if there's a `pnpm-lock.yaml`
- [justfile](https://github.com/casey/just)
- uses the JSON api to find a `test` command
- Makefile
- looks for a line that starts with `test:`
### Exercism
[Exercism](https://exercism.org/) is a platform for learning new programming languages. It has more than 65 tracks available. The Universal Test Runner supports nearly all of them out of the box using the [Exercism CLI](https://exercism.org/docs/using/solving-exercises/working-locally)'s `exercism test` command. Just like this tool, it knows how to run each track's tests and invokes the correct one automatically.
Rather than re-implement all of the test commands `exercism` can handle, the runner will invoke the Exercism CLI when run from an exercise directory. This requires version `3.2.0` of the Exercism CLI installed.
> fun fact: I [added the test command](https://github.com/exercism/cli/pull/1092) after it was suggested in the [forum thread](https://forum.exercism.org/t/introducing-the-universal-test-runner/6228) where I announced the Universal Test Runner
## Motivation
I work in a few languages at a time, so I've actually had a [version of this in my dotfiles](https://github.com/xavdid/dotfiles/blob/6bd5f56b1f9ad2dcef9f8b72413d30779b378aef/node/aliases.zsh#L45-L73) for a while. Also, as I've been doing [Exercism's #12in23 program](https://exercism.org/challenges/12in23), I'm _really_ switching languages. It's nice not to have to re-learn any muscle memory. Plus, increasingly complex `bash` was holding me back.
### Design Philosophy
1. The runner itself should need no configuration - it Just Works
2. It should pass all arguments through to the underlying test command
3. It should have wide language and test runner support; please open an issue if your use case isn't supported!
## FAQ
### `just` errors when passing CLI args
If you run with args (like `t -k matcher`) and see an error from `just` like:
```
error: Justfile does not contain recipes `-k` or `matcher`.
```
That means your `test` recipe doesn't accept any options. Make sure it has an `*options` arg that you pass through to your test command:
```makefile
test *options:
pytest {{options}}
```
## Development
This section is people making changes to this package.
When in a virtual environment, run the following:
```bash
pip install -e '.[test]'
```
This installs the package in `--edit` mode and makes its dependencies available. You can now run `t` to run tests and `universal-test-runner` to access help, version, and debugging info.
### Running Tests
In your virtual environment, a simple `pytest` should run the unit test suite. You can also run `pyright` for type checking.
### Releasing New Versions
> these notes are mostly for myself (or other contributors)
1. Run `just release` while your venv is active
2. paste the stored API key (If you're getting invalid password, verify that `~/.pypirc` is empty)
Raw data
{
"_id": null,
"home_page": "",
"name": "universal-test-runner",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "",
"keywords": "testing,test-runner,pytest",
"author": "",
"author_email": "David Brownman <beamneocube@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/c4/6a/2d12683a8bd1c17c990f3a2d9c94b2fe92418f8a6d5705a570a71acf6577/universal_test_runner-0.6.0.tar.gz",
"platform": null,
"description": "# universal-test-runner\n\nThe Universal Test Runner is a zero-configuration, language-aware way to run unit tests in any project. It installs a command, `t`, which will determine how to run your test suite (and then run it).\n\n<p align=\"center\">\n <a href=\"https://github.com/xavdid/test-runner-demo/blob/main/_demo/demo-min.gif\">\n <img src=\"https://raw.githubusercontent.com/xavdid/test-runner-demo/main/_demo/demo-min.gif\"/>\n </a>\n</p>\n\nIf you're working on a JS project, it runs `[your package manager here] test`. You've run `pytest` in this folder before? `pytest` it is. Rust project? `cargo test` coming right up. Is also clever about running all your `go` module tests (regardless of how they're organized). No matter the command, all args are passed directly into the test runner.\n\nCurrently [supports 7 languages](#supported-languages) (and their respective test frameworks). Please open an issue if I'm missing your favorite!\n\n## Installation\n\nThe easiest way to install is by using [pipx](https://pypa.github.io/pipx/):\n\n```bash\npipx install universal-test-runner\n```\n\nYou can also use brew (which will build from source and take a little longer):\n\n```bash\nbrew install xavdid/projects/universal-test-runner\n```\n\n## Usage\n\n> You can also clone the [demo repo](https://github.com/xavdid/test-runner-demo) to play around with the test runner - it's got toy examples to show how tests are run in many languages!\n\nOnce installed, the command `t` will be available. Run it in a project folder's root and it'll do its best to run your unit tests:\n\n```\n% t\n-> pytest\n=============================== test session starts ================================\nplatform darwin -- Python 3.11.0, pytest-7.3.1, pluggy-1.0.0\nrootdir: /Users/username/projects/test-runner\ncollected 78 items\n\ntests/test_cli.py ... [ 3%]\ntests/test_context.py ..................... [ 30%]\ntests/test_matchers.py .................................................. [ 94%]\ntests/test_runner.py .... [100%]\n\n================================ 78 passed in 0.08s ================================\n```\n\nIt passes all arguments and environment modifications down to the chosen test runner:\n\n```\n% t -k test_builder --verbose\n-> pytest -k test_builder --verbose\n=============================== test session starts ================================\nplatform darwin -- Python 3.11.0, pytest-7.3.1, pluggy-1.0.0\ncachedir: .pytest_cache\nrootdir: /Users/username/projects/test-runner\ncollected 78 items / 77 deselected / 1 selected\n\ntests/test_context.py::test_builder PASSED [100%]\n\n========================= 1 passed, 77 deselected in 0.03s =========================\n```\n\nIt prints the command it's running as part of the output. To disable that behavior, set `UTR_DISABLE_ECHO` to any value in the environment.\n\nIf it can't guess the testing method, it will tell you so. Feel free to open an issue to request wider language support!\n\n### Debugging\n\nThe package also ships a command to surface info about itself: `universal-test-runner`. It has a few key pieces of functionality:\n\n- the `universal-test-runner --version` flag, which prints info about your installed package version\n- the `universal-test-runner debug` command, which prints info about which matcher would run (and why):\n\n```\n% universal-test-runner debug\n[universal-test-runner]: checking each handler for first match\n[universal-test-runner]: Checking matcher 01/11: pytest\n[universal-test-runner]: looking for: \".pytest_cache\"\n[universal-test-runner]: no match, continuing\n[universal-test-runner]: Checking matcher 02/11: py\n[universal-test-runner]: looking for: \"tests.py\"\n[universal-test-runner]: no match, continuing\n[universal-test-runner]: Checking matcher 03/11: go_multi\n[universal-test-runner]: looking for: \"go.mod\" and no arguments\n[universal-test-runner]: no match, continuing\n[universal-test-runner]: Checking matcher 04/11: go_single\n[universal-test-runner]: looking for: \"go.mod\" or a file named \"..._test.go\"\n[universal-test-runner]: no match, continuing\n\n...\n\n[universal-test-runner]: no matching test handler. To add a new one, please file an issue: https://github.com/xavdid/universal-test-runner/issues\n```\n\n## Supported Languages\n\nThis list describes how each language behaves (but not the order in which languages are matched; use the [debugger](#debugging) for that).\n\n- Python\n - checks for `manage.py` (Django)\n - else uses `pytest` if you've run `pytest` before. You'll need to run pytest manually on clean installs before `t` will work\n - looks for a `tests.py` file if not\n- Rust\n - `cargo test`\n- Go\n - if there's a `X_test.go`, then runs a plain `go test`\n - if you pass any args at all, runs `go test your-args-here`\n - otherwise, runs `go test ./...`\n- Elixir\n - `mix test`\n- Clojure\n - `lein test`\n- Javascript/Typescript\n - if there's a `package.json` and it has a `test` script, runs `[package manager] test`, where `[package manager]` is:\n - `npm` if there's a `package-lock.json`\n - `yarn` if there's a `yarn.lock`\n - `pnpm` if there's a `pnpm-lock.yaml`\n- [justfile](https://github.com/casey/just)\n - uses the JSON api to find a `test` command\n- Makefile\n - looks for a line that starts with `test:`\n\n### Exercism\n\n[Exercism](https://exercism.org/) is a platform for learning new programming languages. It has more than 65 tracks available. The Universal Test Runner supports nearly all of them out of the box using the [Exercism CLI](https://exercism.org/docs/using/solving-exercises/working-locally)'s `exercism test` command. Just like this tool, it knows how to run each track's tests and invokes the correct one automatically.\n\nRather than re-implement all of the test commands `exercism` can handle, the runner will invoke the Exercism CLI when run from an exercise directory. This requires version `3.2.0` of the Exercism CLI installed.\n\n> fun fact: I [added the test command](https://github.com/exercism/cli/pull/1092) after it was suggested in the [forum thread](https://forum.exercism.org/t/introducing-the-universal-test-runner/6228) where I announced the Universal Test Runner\n\n## Motivation\n\nI work in a few languages at a time, so I've actually had a [version of this in my dotfiles](https://github.com/xavdid/dotfiles/blob/6bd5f56b1f9ad2dcef9f8b72413d30779b378aef/node/aliases.zsh#L45-L73) for a while. Also, as I've been doing [Exercism's #12in23 program](https://exercism.org/challenges/12in23), I'm _really_ switching languages. It's nice not to have to re-learn any muscle memory. Plus, increasingly complex `bash` was holding me back.\n\n### Design Philosophy\n\n1. The runner itself should need no configuration - it Just Works\n2. It should pass all arguments through to the underlying test command\n3. It should have wide language and test runner support; please open an issue if your use case isn't supported!\n\n## FAQ\n\n### `just` errors when passing CLI args\n\nIf you run with args (like `t -k matcher`) and see an error from `just` like:\n\n```\nerror: Justfile does not contain recipes `-k` or `matcher`.\n```\n\nThat means your `test` recipe doesn't accept any options. Make sure it has an `*options` arg that you pass through to your test command:\n\n```makefile\ntest *options:\n pytest {{options}}\n```\n\n## Development\n\nThis section is people making changes to this package.\n\nWhen in a virtual environment, run the following:\n\n```bash\npip install -e '.[test]'\n```\n\nThis installs the package in `--edit` mode and makes its dependencies available. You can now run `t` to run tests and `universal-test-runner` to access help, version, and debugging info.\n\n### Running Tests\n\nIn your virtual environment, a simple `pytest` should run the unit test suite. You can also run `pyright` for type checking.\n\n### Releasing New Versions\n\n> these notes are mostly for myself (or other contributors)\n\n1. Run `just release` while your venv is active\n2. paste the stored API key (If you're getting invalid password, verify that `~/.pypirc` is empty)\n\n",
"bugtrack_url": null,
"license": "",
"summary": "Universal, language-aware unit test runner.",
"version": "0.6.0",
"project_urls": {
"Author": "https://xavd.id",
"Bug Tracker": "https://github.com/xavdid/universal-test-runner/issues",
"Changelog": "https://github.com/xavdid/universal-test-runner/blob/main/CHANGELOG.md",
"Homepage": "https://github.com/xavdid/universal-test-runner"
},
"split_keywords": [
"testing",
"test-runner",
"pytest"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b3a9432a4782999f2beb40c7fe3151c4c4fdd8cc12206f95a8456415728627dc",
"md5": "c4df38fd9fb705e4d67366bb986e000e",
"sha256": "fa527411a1fb3ea28a51b5fd31a22c6994a46fd3ad856787899e1510c5af0d29"
},
"downloads": -1,
"filename": "universal_test_runner-0.6.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c4df38fd9fb705e4d67366bb986e000e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 10165,
"upload_time": "2023-12-07T03:12:31",
"upload_time_iso_8601": "2023-12-07T03:12:31.146454Z",
"url": "https://files.pythonhosted.org/packages/b3/a9/432a4782999f2beb40c7fe3151c4c4fdd8cc12206f95a8456415728627dc/universal_test_runner-0.6.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c46a2d12683a8bd1c17c990f3a2d9c94b2fe92418f8a6d5705a570a71acf6577",
"md5": "c8b0a6f5574b855d9469ae68eeb54e5e",
"sha256": "bba8e705f4bd44f60bbf4fe51994ad615fd7816d94a2335d155af9b73066ed0b"
},
"downloads": -1,
"filename": "universal_test_runner-0.6.0.tar.gz",
"has_sig": false,
"md5_digest": "c8b0a6f5574b855d9469ae68eeb54e5e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 8423,
"upload_time": "2023-12-07T03:12:32",
"upload_time_iso_8601": "2023-12-07T03:12:32.888741Z",
"url": "https://files.pythonhosted.org/packages/c4/6a/2d12683a8bd1c17c990f3a2d9c94b2fe92418f8a6d5705a570a71acf6577/universal_test_runner-0.6.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-12-07 03:12:32",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "xavdid",
"github_project": "universal-test-runner",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"tox": true,
"lcname": "universal-test-runner"
}
