# filecheck - A Python-native clone of LLVMs FileCheck tool
**Note:** This project is the successor of [mull-project/FileCheck.py](https://github.com/mull-project/FileCheck.py).
This tries to be as close a clone of LLVMs FileCheck as possible, without going crazy. It currently passes 1576 out of
1645 (95.8%) of LLVMs MLIR filecheck tests. We are tracking all 69 remaining test failures in GitHub issues.
There are some features that are left out for now (e.g.a
[pseudo-numeric variables](https://llvm.org/docs/CommandGuide/FileCheck.html#filecheck-pseudo-numeric-variables) and
parts of [numeric substitution](https://llvm.org/docs/CommandGuide/FileCheck.html#filecheck-numeric-substitution-blocks)
).
The codebase is fully type checked by `pyright`, and automatically formatted using `black`. We aim to have tests
covering everything from normal matching down to error messages.
Install by running `pip install filecheck`.
## Features:
Here's an overview of all FileCheck features and their implementation status.
- **Checks:**
- [X] `CHECK`
- [X] `CHECK-NEXT`
- [X] `CHECK-NOT` (Bug: [#10](https://github.com/AntonLydike/filecheck/issues/10))
- [X] `CHECK-LABEL` (Bug: [#8](https://github.com/AntonLydike/filecheck/issues/8))
- [X] `CHECK-EMPTY`
- [X] `CHECK-SAME`
- [X] `CHECK-DAG`
- [X] `CHECK-COUNT`
- **Flags:**
- [X] `--check-prefix`
- [X] `--check-prefixes`
- [X] `--comment-prefixes`
- [ ] `--allow-unused-prefixes`
- [X] `--input-file`
- [X] `--match-full-lines`
- [X] `--strict-whitespace` (Bug: [#6](https://github.com/AntonLydike/filecheck/issues/6))
- [ ] `--ignore-case`
- [ ] `--implicit-check-not` (Tracked: [#20](https://github.com/AntonLydike/filecheck/issues/20))
- [X] `--dump-input` (only `fail` and `never` supported)
- [ ] `--dump-input-context`
- [ ] `--dump-input-filter`
- [X] `--enable-var-scope`
- [X] `-D<VAR=VALUE>`
- [ ] `-D#<FMT>,<NUMVAR>=<NUMERIC EXPRESSION>`
- [X] `-version`
- [ ] `-v`
- [ ] `-vv`
- [ ] `--allow-deprecated-dag-overlap`
- [X] `--allow-empty`
- [ ] `--color` Colored output is supported and automatically detected. No support for the flag.
- **Base Features:**
- [X] Regex patterns
- [X] Captures and Capture Matches (Bug: [#11](https://github.com/AntonLydike/filecheck/issues/11))
- [X] Numeric Captures
- [ ] Numeric Substitutions (jesus christ, wtf man) (Tracked: [#21](https://github.com/AntonLydike/filecheck/issues/21))
- [X] Literal matching (`CHECK{LITERAL}`)
- [X] Weird regex features (`[:xdigits:]` and friends)
- [X] Correct(?) handling of matching check lines (Bug: [#22](https://github.com/AntonLydike/filecheck/issues/22))
- **Testing:**
- [X] Base cases
- [X] Negative tests
- [ ] Error messages (started)
- [ ] Lots of edge cases
- [ ] MLIR/xDSL integration tests
- **UX:**
- Good error messages: Error messages are on an okay level, not great, but not terrible either.
- [X] Parse errors
- [X] Matching errors
- [X] Print possible intended matches (could be better still)
- [X] Malformed regexes
- [ ] Wrong/unkown command line arguments
- [ ] Print variables and their origin in error messages
- **Infrastructure:**
- [X] Formatting: black
- [X] Pyright
- [X] `pre-commit`
- [X] CI for everything
We are open to PRs for bugfixes or any features listed here.
## Differences to LLVMs FileCheck:
We want to be as close as possible to the original FileCheck, and document our differences very clearly.
If you encounter a difference that is not documented here, feel free to file a bug report.
### Better Regexes:
We use pythons regex library, which is a flavour of a Perl compatible regular expression (PCRE), instead of FileChecks
POSIX regex flavour.
**Example:**
```
// LLVM filecheck:
// CHECK: %{{[[:alnum:]]+}}, %{{[[:digit:]]+}}
// our fileheck:
// CHECK: %{{[a-zA-Z0-9]+}}, %{{\d+}}
```
Some effort is made to translate character classes from POSIX to PCRE, although it might be wrong in edge cases.
### Relaxed Matching:
We relax some of the matching rules, like:
- Allow a file to start with `CHECK-NEXT`
### No Numerical Substitution:
This is used in 2 out of 1645 tests in our benchmark (upstream MLIR tests).
While our filecheck supports [numeric capture](https://llvm.org/docs/CommandGuide/FileCheck.html#filecheck-numeric-substitution-blocks)
(`[[#%.3x,VAR:]]` will capture a three-digit hex number), we don't support arithmetic expressions on these captured
values at the moment. We also don't support the "Pseudo Numeric Variable" `@LINE`.
### Special Feature Flags:
This version of filecheck implements some non-standard extensions to LLVMs filecheck. These are disabled by default but
can be enabled through the environment variable `FILECHECK_FEATURE_ENABLE=...`. Avialable extensions are documented here:
- `MLIR_REGEX_CLS`: Add additional special regex matchers to match MLIR/LLVM constructs:
- `\V` will match any SSA value name
### Reject Empty Captures:
We introduce a new flag called `reject-empty-vars` that throws an error when a capture expression captures an empty
string.
Raw data
{
"_id": null,
"home_page": "https://github.com/AntonLydike/filecheck",
"name": "filecheck",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.10",
"maintainer_email": null,
"keywords": "tests, filecheck, llvm",
"author": "Anton Lydike",
"author_email": "me@antonlydike.de",
"download_url": "https://files.pythonhosted.org/packages/93/d2/7e8bf9acf2ccb522fef4845d940b7db14782e6a36929da0ca5a4791bc2b6/filecheck-1.0.1.tar.gz",
"platform": null,
"description": "# filecheck - A Python-native clone of LLVMs FileCheck tool\n\n**Note:** This project is the successor of [mull-project/FileCheck.py](https://github.com/mull-project/FileCheck.py).\n\nThis tries to be as close a clone of LLVMs FileCheck as possible, without going crazy. It currently passes 1576 out of\n1645 (95.8%) of LLVMs MLIR filecheck tests. We are tracking all 69 remaining test failures in GitHub issues.\n\nThere are some features that are left out for now (e.g.a\n[pseudo-numeric variables](https://llvm.org/docs/CommandGuide/FileCheck.html#filecheck-pseudo-numeric-variables) and\nparts of [numeric substitution](https://llvm.org/docs/CommandGuide/FileCheck.html#filecheck-numeric-substitution-blocks)\n).\n\nThe codebase is fully type checked by `pyright`, and automatically formatted using `black`. We aim to have tests\ncovering everything from normal matching down to error messages.\n\nInstall by running `pip install filecheck`.\n\n## Features:\nHere's an overview of all FileCheck features and their implementation status.\n\n- **Checks:**\n - [X] `CHECK`\n - [X] `CHECK-NEXT`\n - [X] `CHECK-NOT` (Bug: [#10](https://github.com/AntonLydike/filecheck/issues/10))\n - [X] `CHECK-LABEL` (Bug: [#8](https://github.com/AntonLydike/filecheck/issues/8))\n - [X] `CHECK-EMPTY`\n - [X] `CHECK-SAME`\n - [X] `CHECK-DAG`\n - [X] `CHECK-COUNT`\n- **Flags:**\n - [X] `--check-prefix`\n - [X] `--check-prefixes`\n - [X] `--comment-prefixes`\n - [ ] `--allow-unused-prefixes`\n - [X] `--input-file`\n - [X] `--match-full-lines`\n - [X] `--strict-whitespace` (Bug: [#6](https://github.com/AntonLydike/filecheck/issues/6))\n - [ ] `--ignore-case`\n - [ ] `--implicit-check-not` (Tracked: [#20](https://github.com/AntonLydike/filecheck/issues/20))\n - [X] `--dump-input` (only `fail` and `never` supported)\n - [ ] `--dump-input-context`\n - [ ] `--dump-input-filter`\n - [X] `--enable-var-scope`\n - [X] `-D<VAR=VALUE>`\n - [ ] `-D#<FMT>,<NUMVAR>=<NUMERIC EXPRESSION>`\n - [X] `-version`\n - [ ] `-v`\n - [ ] `-vv`\n - [ ] `--allow-deprecated-dag-overlap`\n - [X] `--allow-empty`\n - [ ] `--color` Colored output is supported and automatically detected. No support for the flag.\n- **Base Features:**\n - [X] Regex patterns\n - [X] Captures and Capture Matches (Bug: [#11](https://github.com/AntonLydike/filecheck/issues/11))\n - [X] Numeric Captures\n - [ ] Numeric Substitutions (jesus christ, wtf man) (Tracked: [#21](https://github.com/AntonLydike/filecheck/issues/21))\n - [X] Literal matching (`CHECK{LITERAL}`)\n - [X] Weird regex features (`[:xdigits:]` and friends)\n - [X] Correct(?) handling of matching check lines (Bug: [#22](https://github.com/AntonLydike/filecheck/issues/22))\n- **Testing:**\n - [X] Base cases\n - [X] Negative tests\n - [ ] Error messages (started)\n - [ ] Lots of edge cases\n - [ ] MLIR/xDSL integration tests\n- **UX:**\n - Good error messages: Error messages are on an okay level, not great, but not terrible either.\n - [X] Parse errors\n - [X] Matching errors\n - [X] Print possible intended matches (could be better still)\n - [X] Malformed regexes\n - [ ] Wrong/unkown command line arguments\n - [ ] Print variables and their origin in error messages\n- **Infrastructure:**\n - [X] Formatting: black\n - [X] Pyright\n - [X] `pre-commit`\n - [X] CI for everything\n\nWe are open to PRs for bugfixes or any features listed here.\n\n## Differences to LLVMs FileCheck:\nWe want to be as close as possible to the original FileCheck, and document our differences very clearly.\n\nIf you encounter a difference that is not documented here, feel free to file a bug report.\n\n### Better Regexes:\nWe use pythons regex library, which is a flavour of a Perl compatible regular expression (PCRE), instead of FileChecks\nPOSIX regex flavour.\n\n**Example:**\n```\n// LLVM filecheck:\n// CHECK: %{{[[:alnum:]]+}}, %{{[[:digit:]]+}}\n\n// our fileheck:\n// CHECK: %{{[a-zA-Z0-9]+}}, %{{\\d+}}\n```\n\nSome effort is made to translate character classes from POSIX to PCRE, although it might be wrong in edge cases.\n\n### Relaxed Matching:\n\nWe relax some of the matching rules, like:\n\n- Allow a file to start with `CHECK-NEXT`\n\n\n### No Numerical Substitution:\n\nThis is used in 2 out of 1645 tests in our benchmark (upstream MLIR tests).\n\nWhile our filecheck supports [numeric capture](https://llvm.org/docs/CommandGuide/FileCheck.html#filecheck-numeric-substitution-blocks)\n(`[[#%.3x,VAR:]]` will capture a three-digit hex number), we don't support arithmetic expressions on these captured\nvalues at the moment. We also don't support the \"Pseudo Numeric Variable\" `@LINE`.\n\n### Special Feature Flags:\n\nThis version of filecheck implements some non-standard extensions to LLVMs filecheck. These are disabled by default but\ncan be enabled through the environment variable `FILECHECK_FEATURE_ENABLE=...`. Avialable extensions are documented here:\n\n- `MLIR_REGEX_CLS`: Add additional special regex matchers to match MLIR/LLVM constructs:\n - `\\V` will match any SSA value name\n\n### Reject Empty Captures:\n\nWe introduce a new flag called `reject-empty-vars` that throws an error when a capture expression captures an empty\nstring.\n\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "A Python-native clone of LLVMs FileCheck tool",
"version": "1.0.1",
"project_urls": {
"Homepage": "https://github.com/AntonLydike/filecheck",
"Repository": "https://github.com/AntonLydike/filecheck"
},
"split_keywords": [
"tests",
" filecheck",
" llvm"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c729827b9f240e03c2cc6a2fd534ac980a12b3c7e8d8aa71c2f1039a5f91e932",
"md5": "4376a22275d3f409071979235e3e9c62",
"sha256": "2d1a0e8784b723a4b04a655cff3af09dd159a31f4e39d477186f5547553124ab"
},
"downloads": -1,
"filename": "filecheck-1.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4376a22275d3f409071979235e3e9c62",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.10",
"size": 23702,
"upload_time": "2024-09-07T13:46:21",
"upload_time_iso_8601": "2024-09-07T13:46:21.442669Z",
"url": "https://files.pythonhosted.org/packages/c7/29/827b9f240e03c2cc6a2fd534ac980a12b3c7e8d8aa71c2f1039a5f91e932/filecheck-1.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "93d27e8bf9acf2ccb522fef4845d940b7db14782e6a36929da0ca5a4791bc2b6",
"md5": "cdd41d1bc31c5e97e37ac11441366cfe",
"sha256": "bbc3c49c190bd3af2445426a193ff0b54e1fad5a81ea7c2116c4dc36f36614f2"
},
"downloads": -1,
"filename": "filecheck-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "cdd41d1bc31c5e97e37ac11441366cfe",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.10",
"size": 20480,
"upload_time": "2024-09-07T13:46:22",
"upload_time_iso_8601": "2024-09-07T13:46:22.689100Z",
"url": "https://files.pythonhosted.org/packages/93/d2/7e8bf9acf2ccb522fef4845d940b7db14782e6a36929da0ca5a4791bc2b6/filecheck-1.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-07 13:46:22",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "AntonLydike",
"github_project": "filecheck",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "filecheck"
}
JSON
