# phmutest 0.0.4
## Detect broken Python examples in Markdown
- Command line program checks Python syntax highlighted examples.
- Equivalent Python library for calling from test suite. | [Here](#call-from-python)
- Python tools to get fenced code block contents from Markdown. | [Here](docs/api.md)
Treats each Markdown file as a single long example, of many FCBs which continues
across multiple Markdown [fenced code blocks][3] (FCBs or blocks).
- Checks either Python code examples plus output **or** ">>>" REPL examples
| [doctest][5].
- Reports pass/failed/error/skip status and line number for each block.
- An example can continue **across** files.
- Runs files in user specified order.
- TOML configuration available.
### Your Python setup and cleanup
Specify a Python function which is called first before checking examples.
Change context, acquire resources, create objects, register cleanup functions.
Pass objects as global variables to the examples.
Cleans up even when fail-fast.
| [Suite initialization and cleanup](#suite-initialization-and-cleanup)
### Some Markdown edits required
No edits required for REPL examples.
Remove or use `expected-output`
as the info string on
expected output FCBs.
### Extendable
Designated and stable **patch points** for Python standard library
**unittest.mock.patch()** patches. | [Here](#patch-points)
### Advanced features
These features require adding tool specific HTML comment **directives**
to the Markdown. Because directives are HTML comments they are not visible in
rendered Markdown. View directives on GitHub
by pressing the `Code` button in the banner at the top of the file.
| [Advanced feature details](docs/advanced.md).
- Assign test group names to blocks. Command line options select or
deselect test groups by name.
- Skip blocks or skip checking printed output.
- Label any fenced code block for later retrieval.
- Accepts [phmdoctest][17] directives except share-names and clear-names.
- Specify blocks as setup and teardown code for the file or setup across files.
## main branch status
[](https://github.com/tmarktaylor/phmutest/blob/main/LICENSE)
[](https://pypi.python.org/pypi/phmutest)
[](https://pypi.python.org/pypi/phmutest)
[](https://github.com/psf/black)
[](https://github.com/tmarktaylor/phmutest/actions/workflows/ci.yml)
[](https://ci.appveyor.com/project/tmarktaylor/phmutest)
[](https://phmutest.readthedocs.io/en/latest/?badge=latest)
[](https://codecov.io/gh/tmarktaylor/phmutest?branch=main)
[Docs RTD](https://phmutest.readthedocs.io/en/latest/) |
[Docs GitHub](https://github.com/tmarktaylor/phmutest/blob/main/README.md) |
[Repos](https://github.com/tmarktaylor/phmutest) |
[pytest][13] |
[Codecov](https://codecov.io/gh/tmarktaylor/phmutest?branch=main) |
[License](https://github.com/tmarktaylor/phmutest/blob/main/LICENSE)
[Installation](#installation) |
[Usage](#usage) |
[FILE](#file) |
[REPL mode](#repl-mode) |
[Suite initialization and cleanup](#suite-initialization-and-cleanup) |
[Extend an example across files](#extend-an-example-across-files) |
[Skip blocks from the command line](#skip-blocks-from-the-command-line) |
[--summary](#summary-option) |
[TOML configuration](#toml-configuration) |
[Run as a Python module](#run-as-a-python-module) |
[Call from Python](#call-from-python) |
[Patch points](#patch-points) |
[Hints](#hints) |
[Related projects](#related-projects) |
[Differences between phmutest and phmdoctest](#differences-between-phmutest-and-phmdoctest)
[Sections](docs/demos.md#sections) |
[Demos](docs/demos.md#demos) |
[Changes](docs/recent_changes.md) |
[Contributions](CONTRIBUTING.md)
## Markdown code/output demo
The are no phmutest directives in this file.
The example starts by creating the object m.
```python
from hashlib import sha256
m = sha256()
```
The example continues here.
```python
m.update(b"hello World")
print(m.hexdigest()[0:5])
```
Expected output here is checked. This fenced code block does not
have an info string.
```
db406
```
The example continues here. It will continue for the entire file. This is
the last Python fenced code/output code block (FCB) in the file.
```python
m.update(b"more bytes")
print(m.hexdigest()[0:5])
```
Note the expected output below is different.
This FCB has the info string `expected-output` to avoid
Markdown linting tool nag.
phmutest treats a block with this info string as expected
output if it is the first FCB after a Python code FCB.
```expected-output
4c6ea
```
### phmutest command line
```shell
phmutest README.md --log
```
### phmutest output
Here is output from the command line.
The output produced by Python standard library unittest module
is not shown here. This is printed after the unittest `OK` line.
The lower case "o" after the line number indicates a
subsequent FCB containing expected printed output got checked.
```txt
log:
args.files: 'README.md'
args.log: 'True'
location|label result
--------------- ------
README.md:99... pass
README.md:106 o pass
README.md:121 o pass
--------------- ------
```
## Markdown REPL demo
- Run phmutest with --replmode to test Python interactive session FCBs.
- The REPL FCB's start with ">>>".
- No Markdown edits needed. Tests examples the way they were written.
```python
>>> a = "Greetings Planet!"
>>> a
'Greetings Planet!'
>>> b = 12
>>> b
12
```
Example borrowed from Python Standard Library fractions documentation.
```py
>>> from fractions import Fraction
>>> Fraction(16, -10)
Fraction(-8, 5)
>>> Fraction(123)
Fraction(123, 1)
>>> Fraction()
Fraction(0, 1)
>>> Fraction('3/7')
Fraction(3, 7)
```
Here we show names assigned in prior FCBs are still visible.
```py
>>> b
12
```
```py
>>> Fraction('3/7')
Fraction(3, 7)
```
### phmutest --replmode command line
```shell
phmutest README.md --replmode --log
```
### phmutest --replmode output
```txt
log:
args.files: 'README.md'
args.replmode: 'True'
args.log: 'True'
location|label result
-------------- ------
README.md:171. pass
README.md:182. pass
README.md:196. pass
README.md:201. pass
-------------- ------
```
See [list of demos](docs/demos.md)
See [How it works](docs/howitworks.md)
## Installation
```shell
python -m pip install phmutest
```
- No dependencies since Python 3.11. Depends on tomli before Python 3.11.
- Pure Python. No binaries.
- It is advisable to install in a virtual environment.
## Usage
`phmutest --help`
```txt
usage: phmutest [-h] [--version] [--skip [TEXT ...]] [--fixture DOTTED_PATH.FUNCTION]
[--share-across-files [FILE ...]] [--setup-across-files [FILE ...]]
[--select [GROUP ...] | --deselect [GROUP ...]]
[--config TOMLFILE] [--replmode]
[-g OUTFILE] [--progress]
[--sharing [FILE ...]] [--log] [--summary]
[--report]
[FILE ...]
Detect broken Python examples in Markdown. Accepts relevant unittest options.
positional arguments:
FILE Markdown input file.
options:
-h, --help show this help message and exit
--version show program's version number and exit
--skip [TEXT ...] Any block that contains the substring TEXT is not tested.
--fixture DOTTED_PATH.FUNCTION
Function run before testing.
--share-across-files [FILE ...]
Shares names from Markdown file to later positional files.
--setup-across-files [FILE ...]
Apply Markdown file setup blocks to files.
--select [GROUP ...] Select all blocks with phmutest-group GROUP directive for testing.
--deselect [GROUP ...]
Exclude all blocks with phmutest-group GROUP directive from testing.
--config TOMLFILE .toml configuration file.
--replmode Test Python interactive sessions.
-g OUTFILE, --generate OUTFILE
Write generated Python or docstring to output file or stdout.
--progress Print block by block test progress. File by file in --replmode.
--sharing [FILE ...] For these files print name sharing. . means all files.
--log Print log items when done.
--summary Print test count and skipped tests.
--report Print fenced code block configuration, deselected blocks.
```
- The **-f** option indicates fail fast.
## FILE
The Markdown files are processed in the same order they are present as positional
arguments on the command line.
Shell wildcards can be used. Be aware that the shell expansion and operating system
will determine the order.
## REPL mode
When --replmode is specified Python interactive sessions are tested and
Python code and expected output blocks are not tested. REPL mode tests are
implemented using [doctest][5].
The option --setup-across-files and the setup and teardown directives
have no effect in REPL mode.
--progress has file by file granularity.
## Suite initialization and cleanup
For background refer to definitions at the top of [unittest][18].
Use --fixture to specify a Python initialization function that runs before the tests.
It works with or without --replmode, but there are differences.
In both modes, the fixture function may create objects (globs) that are visible
as globals to the FCBs under test.
In the event of test errors orderly cleanup/release of resources is assured.
For Python code blocks the fixture may register cleanup functions by
calling **unittest.addModuleCleanup()**.
In REPL mode the fixture function optionally returns a cleanup function.
- The fixture can acquire and release resources or change context.
- The fixture can make entries to the log displayed by --log.
Specify the --fixture function as
a relative **dotted path** where `/` is replaced with `.`.
For example, the function **my_init()** in the file **tests/myfixture.py**
would be specified:
--fixture **tests.myfixture.my_init**
The function is passed keyword only arguments and **optionally**
returns a Fixture instance.
The keyword arguments and return type are described by
[src/phmutest/fixture.py](docs/fixture_py.md).
The fixture file should be in the project directory tree. Fixture demos:
- [fixture change workdir](docs/fix/code/chdir.md)
- [fixture set globals](docs/fix/code/globdemo.md)
- [fixture cleanup REPL Mode](docs/fix/repl/drink.md)
When calling phmutest from Python, mock.patch() patching
would typically be implemented as enclosing with statements.
When invoking phmutest from a shell,
the --fixture function can be used to install patches.
See example near the end of tests/test_patching.py and
in tests/test_subprocess.py. The example shows how to patch to
apply doctest optionflags in --replmode.
Do patching cleanup when not in --replmode by calling
`unittest.addModuleCleanup(stack.pop_all().close)`.
### Dotted path details
The fixture function must be at the top level of a .py file.
- The dotted_path has components separated by ".".
- The last component is the function name.
- The next to last component is the python file name without the .py suffix.
- The preceding components identify parent folders. Folders should be
relative to the current working directory which is typically the
project root.
## Extend an example across files
Names assigned by all the blocks in a file can be shared, as global variables,
to files specified later in the command line.
Add a markdown file path to the --share-across-files command line option.
The 'shared' file(s) must also be specified as a FILE positional command line argument.
- [share demo](docs/share/share_demo.md) |
[how it works](docs/codemode.md#share-across-files)
- [--replmode share demo](docs/repl/replshare_demo.md) |
[how it works](docs/sessionmode.md#share-across-files)
## Skip blocks from the command line
The skip `--skip TEXT` command line option
prevents testing of any Python code or REPL block that contains the substring TEXT.
The block is logged as skip with `--skip TEXT` as the reason.
## summary option
The example [here](docs/share/share_demo.md) shows --summary output.
## TOML configuration
Command line options can be augmented with values from a `[tool.phmutest]` section in
a .toml configuration file. It can be in a new file or added to an existing
.toml file like pyproject.toml.
The configuration file is specified by the `--config FILE` command line option.
Zero or more of these TOML keys may be present in the `[tool.phmutest]` section.
| TOML key | Usage option | TOML value - double quoted strings
| :------------------| :-----------------: | :---------:
| include-globs | positional arg FILE | list of filename glob to select files
| exclude-globs | positional arg FILE | list of filename glob to deselect files
| share-across-files | --share-across-files | list of path
| setup-across-files | --setup-across-files | list of path
| fixture | --fixture | dotted path
| select | --select | list of group directive name
| deselect | --deselect | list of group directive name
Only one of select and deselect can have strings.
- globs are described by Python standard library **pathlib.Path.glob()**.
- Any FILEs on the command line extend the files selected by include-globs and
exclude-globs.
- Command line options supercede the keys in the config file.
- See the example **tests/toml/project.toml**.
## Run as a Python module
To run phmutest as a Python module:
```bash
python -m phmutest README.md --log
```
## Call from Python
Call **phmutest.main.command()** with a string that looks like a
command line less the phmutest, like this:
`"md/project.md --replmode"`
- A `phmutest.summary.PhmResult` instance is returned.
- When calling from Python there is no shell wildcard expansion.
- The --fixture function can be in the same Python file. Caveat: The Python file is
imported again to a new module object. The Python file's module level code will
be run a second time.
- Call from pytest to get overall result as Junit XML | [Suggestion](docs/junit.md)
[Example](docs/callfrompython.md) | [Limitation](docs/callfrompython.md#limitation)
## Patch points
Feel free to **unittest.mock.patch()** at these places in the code and not worry about
breakage in future versions. Look for examples in tests/test_patching.py.
### List of patch points
| patched function | purpose
| :--------------------------------: | :----------:
| phmutest.direct.directive_finders() | Add directive aliases
| phmutest.fenced.python_matcher() | Add detect Python from FCB info string
| phmutest.select.OUTPUT_INFO_STRINGS | Change detect expected output FCB info string
| phmutest.session.modify_docstring() | Inspect/modify REPL text before testing
| phmutest.reader.post() | Inspect/modify DocNode detected in Markdown
## Hints
- Since phmutest generates code, the input files should be from a trusted
source.
- The phmutest Markdown parser finds fenced code blocks enclosed by
html `<details>` and `</details>` tags.
The tags may require a preceding and trailing blank line
to render correctly. See example at the bottom tests/md/readerfcb.md.
- Markdown indented code blocks ([Spec][4] section 4.4) are ignored.
- A malformed HTML comment ending is bad. Make sure
it ends with both dashes like `-->`.
- A misspelled directive will be missing from the --report output.
- If the generated test file has a compile error phmutest will raise an
ImportError when importing it.
- Blocks skipped with --skip and the phmutest-skip directive
are not rendered. This is useful to avoid above import error.
- In repl mode **no** skipped blocks are rendered.
- Try redirecting `--generate -` standard output into PYPI Pygments to
colorize the generated test file.
- pytest will run a generated test file (--generate TESTFILE). pytest won't
run functions added by unittest.addModuleCleanup().
## Related projects
- phmdoctest
- rundoc
- byexample
- sphinx.ext.doctest
- sybil
- doxec
- egtest
- pytest-phmdoctest
- pytest-codeblocks
## Differences between phmutest and phmdoctest
- phmutest treats each Markdown file as a single long example. phmdoctest
tests each FCB in isolation. Adding a share-names directive is necessary to
extend an example across FCBs within a file.
- Only phmutest can extend an example across files.
- phmutest uses Python standard library unittest and doctest as test runners.
phmdoctest writes a pytest testfile for each Markdown file
which requires a separate step to run. The testfiles then need to be discarded.
- phmdoctest offers two pytest fixtures that can be used in a pytest test case
to generate and run a testfile in one step.
- phmutest generates tests for multiple Markdown files in one step
and runs them internally so there are no leftover test files.
- The --fixture test suite initialization and cleanup is only available on phmutest.
phmdoctest offers some initialization behaviour using an FCB with a setup
directive and its --setup-doctest option and it only works with sessions.
See phmdoctest documentation "Execution Context"
section for an explanation.
- phmutest does not support inline annotations.
[1]: https://github.com/tmarktaylor/phmutest/blob/master/README.md?plain=1
[3]: https://github.github.com/gfm/#fenced-code-blocks
[4]: https://spec.commonmark.org
[11]: https://github.github.com/gfm/#info-string
[10]: https://phmutest.readthedocs.io/en/latest/docs/api.html
[5]: https://docs.python.org/3/library/doctest.html
[6]: https://pypi.python.org/project/coverage
[13]: https://ci.appveyor.com/project/tmarktaylor/phmutest
[15]: https://docs.pytest.org/en/stable
[16]: https://tmarktaylor.github.io/pytest-phmdoctest
[17]: https://pypi.python.org/pypi/phmdoctest
[18]: https://docs.python.org/3/library/unittest.html
MIT License
Copyright (c) 2024 Mark Taylor
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Raw data
{
"_id": null,
"home_page": "https://phmutest.readthedocs.io/en/latest/",
"name": "phmutest",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "documentation, markdown, testing",
"author": "Mark Taylor",
"author_email": "mark66547ta2@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/6b/8b/f29fbc6189d1c873fbf52ae48f2aad6cd47600e5800c11c97fd734fe3103/phmutest-0.0.4.tar.gz",
"platform": null,
"description": "# phmutest 0.0.4\n\n## Detect broken Python examples in Markdown\n\n- Command line program checks Python syntax highlighted examples.\n- Equivalent Python library for calling from test suite. | [Here](#call-from-python)\n- Python tools to get fenced code block contents from Markdown. | [Here](docs/api.md)\n\nTreats each Markdown file as a single long example, of many FCBs which continues\nacross multiple Markdown [fenced code blocks][3] (FCBs or blocks).\n\n- Checks either Python code examples plus output **or** \">>>\" REPL examples\n | [doctest][5].\n- Reports pass/failed/error/skip status and line number for each block.\n- An example can continue **across** files.\n- Runs files in user specified order.\n- TOML configuration available.\n\n### Your Python setup and cleanup\n\nSpecify a Python function which is called first before checking examples.\nChange context, acquire resources, create objects, register cleanup functions.\nPass objects as global variables to the examples.\nCleans up even when fail-fast.\n| [Suite initialization and cleanup](#suite-initialization-and-cleanup)\n\n### Some Markdown edits required\n\nNo edits required for REPL examples.\nRemove or use `expected-output`\nas the info string on\nexpected output FCBs.\n\n### Extendable\n\nDesignated and stable **patch points** for Python standard library\n**unittest.mock.patch()** patches. | [Here](#patch-points)\n\n### Advanced features\n\nThese features require adding tool specific HTML comment **directives**\nto the Markdown. Because directives are HTML comments they are not visible in\nrendered Markdown. View directives on GitHub\nby pressing the `Code` button in the banner at the top of the file.\n| [Advanced feature details](docs/advanced.md).\n\n- Assign test group names to blocks. Command line options select or\n deselect test groups by name.\n- Skip blocks or skip checking printed output.\n- Label any fenced code block for later retrieval.\n- Accepts [phmdoctest][17] directives except share-names and clear-names.\n- Specify blocks as setup and teardown code for the file or setup across files.\n\n## main branch status\n\n[](https://github.com/tmarktaylor/phmutest/blob/main/LICENSE)\n[](https://pypi.python.org/pypi/phmutest)\n[](https://pypi.python.org/pypi/phmutest)\n[](https://github.com/psf/black)\n\n[](https://github.com/tmarktaylor/phmutest/actions/workflows/ci.yml)\n[](https://ci.appveyor.com/project/tmarktaylor/phmutest)\n[](https://phmutest.readthedocs.io/en/latest/?badge=latest)\n[](https://codecov.io/gh/tmarktaylor/phmutest?branch=main)\n\n[Docs RTD](https://phmutest.readthedocs.io/en/latest/) |\n[Docs GitHub](https://github.com/tmarktaylor/phmutest/blob/main/README.md) |\n[Repos](https://github.com/tmarktaylor/phmutest) |\n[pytest][13] |\n[Codecov](https://codecov.io/gh/tmarktaylor/phmutest?branch=main) |\n[License](https://github.com/tmarktaylor/phmutest/blob/main/LICENSE)\n\n[Installation](#installation) |\n[Usage](#usage) |\n[FILE](#file) |\n[REPL mode](#repl-mode) |\n[Suite initialization and cleanup](#suite-initialization-and-cleanup) |\n[Extend an example across files](#extend-an-example-across-files) |\n[Skip blocks from the command line](#skip-blocks-from-the-command-line) |\n[--summary](#summary-option) |\n[TOML configuration](#toml-configuration) |\n[Run as a Python module](#run-as-a-python-module) |\n[Call from Python](#call-from-python) |\n[Patch points](#patch-points) |\n[Hints](#hints) |\n[Related projects](#related-projects) |\n[Differences between phmutest and phmdoctest](#differences-between-phmutest-and-phmdoctest)\n\n[Sections](docs/demos.md#sections) |\n[Demos](docs/demos.md#demos) |\n[Changes](docs/recent_changes.md) |\n[Contributions](CONTRIBUTING.md)\n\n## Markdown code/output demo\n\nThe are no phmutest directives in this file.\nThe example starts by creating the object m.\n\n```python\nfrom hashlib import sha256\nm = sha256()\n```\n\nThe example continues here.\n\n```python\nm.update(b\"hello World\")\nprint(m.hexdigest()[0:5])\n```\n\nExpected output here is checked. This fenced code block does not\nhave an info string.\n\n```\ndb406\n```\n\nThe example continues here. It will continue for the entire file. This is\nthe last Python fenced code/output code block (FCB) in the file.\n\n```python\nm.update(b\"more bytes\")\nprint(m.hexdigest()[0:5])\n```\n\nNote the expected output below is different.\n\nThis FCB has the info string `expected-output` to avoid\nMarkdown linting tool nag.\nphmutest treats a block with this info string as expected\noutput if it is the first FCB after a Python code FCB.\n\n```expected-output\n4c6ea\n```\n\n### phmutest command line\n\n```shell\nphmutest README.md --log\n```\n\n### phmutest output\n\nHere is output from the command line.\nThe output produced by Python standard library unittest module\nis not shown here. This is printed after the unittest `OK` line.\n\nThe lower case \"o\" after the line number indicates a\nsubsequent FCB containing expected printed output got checked.\n\n```txt\nlog:\nargs.files: 'README.md'\nargs.log: 'True'\n\nlocation|label result\n--------------- ------\nREADME.md:99... pass\nREADME.md:106 o pass\nREADME.md:121 o pass\n--------------- ------\n```\n\n## Markdown REPL demo\n\n- Run phmutest with --replmode to test Python interactive session FCBs.\n- The REPL FCB's start with \">>>\".\n- No Markdown edits needed. Tests examples the way they were written.\n\n```python\n>>> a = \"Greetings Planet!\"\n>>> a\n'Greetings Planet!'\n>>> b = 12\n>>> b\n12\n```\n\nExample borrowed from Python Standard Library fractions documentation.\n\n```py\n>>> from fractions import Fraction\n>>> Fraction(16, -10)\nFraction(-8, 5)\n>>> Fraction(123)\nFraction(123, 1)\n>>> Fraction()\nFraction(0, 1)\n>>> Fraction('3/7')\nFraction(3, 7)\n```\n\nHere we show names assigned in prior FCBs are still visible.\n\n```py\n>>> b\n12\n```\n\n```py\n>>> Fraction('3/7')\nFraction(3, 7)\n```\n\n### phmutest --replmode command line\n\n```shell\nphmutest README.md --replmode --log\n```\n\n### phmutest --replmode output\n\n```txt\nlog:\nargs.files: 'README.md'\nargs.replmode: 'True'\nargs.log: 'True'\n\nlocation|label result\n-------------- ------\nREADME.md:171. pass\nREADME.md:182. pass\nREADME.md:196. pass\nREADME.md:201. pass\n-------------- ------\n```\n\nSee [list of demos](docs/demos.md)\nSee [How it works](docs/howitworks.md)\n\n## Installation\n\n```shell\npython -m pip install phmutest\n```\n\n- No dependencies since Python 3.11. Depends on tomli before Python 3.11.\n- Pure Python. No binaries.\n- It is advisable to install in a virtual environment.\n\n## Usage\n\n`phmutest --help`\n\n```txt\nusage: phmutest [-h] [--version] [--skip [TEXT ...]] [--fixture DOTTED_PATH.FUNCTION]\n [--share-across-files [FILE ...]] [--setup-across-files [FILE ...]]\n [--select [GROUP ...] | --deselect [GROUP ...]]\n [--config TOMLFILE] [--replmode]\n [-g OUTFILE] [--progress]\n [--sharing [FILE ...]] [--log] [--summary]\n [--report]\n [FILE ...]\n\nDetect broken Python examples in Markdown. Accepts relevant unittest options.\n\npositional arguments:\n FILE Markdown input file.\n\noptions:\n -h, --help show this help message and exit\n --version show program's version number and exit\n --skip [TEXT ...] Any block that contains the substring TEXT is not tested.\n --fixture DOTTED_PATH.FUNCTION\n Function run before testing.\n --share-across-files [FILE ...]\n Shares names from Markdown file to later positional files.\n --setup-across-files [FILE ...]\n Apply Markdown file setup blocks to files.\n --select [GROUP ...] Select all blocks with phmutest-group GROUP directive for testing.\n --deselect [GROUP ...]\n Exclude all blocks with phmutest-group GROUP directive from testing.\n --config TOMLFILE .toml configuration file.\n --replmode Test Python interactive sessions.\n -g OUTFILE, --generate OUTFILE\n Write generated Python or docstring to output file or stdout.\n --progress Print block by block test progress. File by file in --replmode.\n --sharing [FILE ...] For these files print name sharing. . means all files.\n --log Print log items when done.\n --summary Print test count and skipped tests.\n --report Print fenced code block configuration, deselected blocks.\n```\n\n- The **-f** option indicates fail fast.\n\n## FILE\n\nThe Markdown files are processed in the same order they are present as positional\narguments on the command line.\nShell wildcards can be used. Be aware that the shell expansion and operating system\nwill determine the order.\n\n## REPL mode\n\nWhen --replmode is specified Python interactive sessions are tested and\nPython code and expected output blocks are not tested. REPL mode tests are\nimplemented using [doctest][5].\nThe option --setup-across-files and the setup and teardown directives\nhave no effect in REPL mode.\n--progress has file by file granularity.\n\n## Suite initialization and cleanup\n\nFor background refer to definitions at the top of [unittest][18].\nUse --fixture to specify a Python initialization function that runs before the tests.\nIt works with or without --replmode, but there are differences.\nIn both modes, the fixture function may create objects (globs) that are visible\nas globals to the FCBs under test.\nIn the event of test errors orderly cleanup/release of resources is assured.\nFor Python code blocks the fixture may register cleanup functions by\ncalling **unittest.addModuleCleanup()**.\nIn REPL mode the fixture function optionally returns a cleanup function.\n\n- The fixture can acquire and release resources or change context.\n- The fixture can make entries to the log displayed by --log.\n\nSpecify the --fixture function as\na relative **dotted path** where `/` is replaced with `.`.\nFor example, the function **my_init()** in the file **tests/myfixture.py**\nwould be specified:\n\n--fixture **tests.myfixture.my_init**\n\nThe function is passed keyword only arguments and **optionally**\nreturns a Fixture instance.\nThe keyword arguments and return type are described by\n[src/phmutest/fixture.py](docs/fixture_py.md).\nThe fixture file should be in the project directory tree. Fixture demos:\n\n- [fixture change workdir](docs/fix/code/chdir.md)\n- [fixture set globals](docs/fix/code/globdemo.md)\n- [fixture cleanup REPL Mode](docs/fix/repl/drink.md)\n\nWhen calling phmutest from Python, mock.patch() patching\nwould typically be implemented as enclosing with statements.\n\nWhen invoking phmutest from a shell,\nthe --fixture function can be used to install patches.\nSee example near the end of tests/test_patching.py and\nin tests/test_subprocess.py. The example shows how to patch to\napply doctest optionflags in --replmode.\nDo patching cleanup when not in --replmode by calling\n`unittest.addModuleCleanup(stack.pop_all().close)`.\n\n### Dotted path details\n\nThe fixture function must be at the top level of a .py file.\n\n- The dotted_path has components separated by \".\".\n- The last component is the function name.\n- The next to last component is the python file name without the .py suffix.\n- The preceding components identify parent folders. Folders should be\n relative to the current working directory which is typically the\n project root.\n\n## Extend an example across files\n\nNames assigned by all the blocks in a file can be shared, as global variables,\nto files specified later in the command line.\nAdd a markdown file path to the --share-across-files command line option.\nThe 'shared' file(s) must also be specified as a FILE positional command line argument.\n\n- [share demo](docs/share/share_demo.md) |\n [how it works](docs/codemode.md#share-across-files)\n- [--replmode share demo](docs/repl/replshare_demo.md) |\n [how it works](docs/sessionmode.md#share-across-files)\n\n## Skip blocks from the command line\n\nThe skip `--skip TEXT` command line option\nprevents testing of any Python code or REPL block that contains the substring TEXT.\nThe block is logged as skip with `--skip TEXT` as the reason.\n\n## summary option\n\nThe example [here](docs/share/share_demo.md) shows --summary output.\n\n## TOML configuration\n\nCommand line options can be augmented with values from a `[tool.phmutest]` section in\na .toml configuration file. It can be in a new file or added to an existing\n.toml file like pyproject.toml.\nThe configuration file is specified by the `--config FILE` command line option.\n\nZero or more of these TOML keys may be present in the `[tool.phmutest]` section.\n\n| TOML key | Usage option | TOML value - double quoted strings\n| :------------------| :-----------------: | :---------:\n| include-globs | positional arg FILE | list of filename glob to select files\n| exclude-globs | positional arg FILE | list of filename glob to deselect files\n| share-across-files | --share-across-files | list of path\n| setup-across-files | --setup-across-files | list of path\n| fixture | --fixture | dotted path\n| select | --select | list of group directive name\n| deselect | --deselect | list of group directive name\n\nOnly one of select and deselect can have strings.\n\n- globs are described by Python standard library **pathlib.Path.glob()**.\n- Any FILEs on the command line extend the files selected by include-globs and\n exclude-globs.\n- Command line options supercede the keys in the config file.\n- See the example **tests/toml/project.toml**.\n\n## Run as a Python module\n\nTo run phmutest as a Python module:\n\n```bash\npython -m phmutest README.md --log\n```\n\n## Call from Python\n\nCall **phmutest.main.command()** with a string that looks like a\ncommand line less the phmutest, like this:\n`\"md/project.md --replmode\"`\n\n- A `phmutest.summary.PhmResult` instance is returned.\n- When calling from Python there is no shell wildcard expansion.\n- The --fixture function can be in the same Python file. Caveat: The Python file is\n imported again to a new module object. The Python file's module level code will\n be run a second time.\n- Call from pytest to get overall result as Junit XML | [Suggestion](docs/junit.md)\n\n[Example](docs/callfrompython.md) | [Limitation](docs/callfrompython.md#limitation)\n\n## Patch points\n\nFeel free to **unittest.mock.patch()** at these places in the code and not worry about\nbreakage in future versions. Look for examples in tests/test_patching.py.\n\n### List of patch points\n\n| patched function | purpose\n| :--------------------------------: | :----------:\n| phmutest.direct.directive_finders() | Add directive aliases\n| phmutest.fenced.python_matcher() | Add detect Python from FCB info string\n| phmutest.select.OUTPUT_INFO_STRINGS | Change detect expected output FCB info string\n| phmutest.session.modify_docstring() | Inspect/modify REPL text before testing\n| phmutest.reader.post() | Inspect/modify DocNode detected in Markdown\n\n## Hints\n\n- Since phmutest generates code, the input files should be from a trusted\n source.\n- The phmutest Markdown parser finds fenced code blocks enclosed by\n html `<details>` and `</details>` tags.\n The tags may require a preceding and trailing blank line\n to render correctly. See example at the bottom tests/md/readerfcb.md.\n- Markdown indented code blocks ([Spec][4] section 4.4) are ignored.\n- A malformed HTML comment ending is bad. Make sure\n it ends with both dashes like `-->`.\n- A misspelled directive will be missing from the --report output.\n- If the generated test file has a compile error phmutest will raise an\n ImportError when importing it.\n- Blocks skipped with --skip and the phmutest-skip directive\n are not rendered. This is useful to avoid above import error.\n- In repl mode **no** skipped blocks are rendered.\n- Try redirecting `--generate -` standard output into PYPI Pygments to\n colorize the generated test file.\n- pytest will run a generated test file (--generate TESTFILE). pytest won't\n run functions added by unittest.addModuleCleanup().\n\n## Related projects\n\n- phmdoctest\n- rundoc\n- byexample\n- sphinx.ext.doctest\n- sybil\n- doxec\n- egtest\n- pytest-phmdoctest\n- pytest-codeblocks\n\n## Differences between phmutest and phmdoctest\n\n- phmutest treats each Markdown file as a single long example. phmdoctest\n tests each FCB in isolation. Adding a share-names directive is necessary to\n extend an example across FCBs within a file.\n- Only phmutest can extend an example across files.\n- phmutest uses Python standard library unittest and doctest as test runners.\n phmdoctest writes a pytest testfile for each Markdown file\n which requires a separate step to run. The testfiles then need to be discarded.\n- phmdoctest offers two pytest fixtures that can be used in a pytest test case\n to generate and run a testfile in one step.\n- phmutest generates tests for multiple Markdown files in one step\n and runs them internally so there are no leftover test files.\n- The --fixture test suite initialization and cleanup is only available on phmutest.\n phmdoctest offers some initialization behaviour using an FCB with a setup\n directive and its --setup-doctest option and it only works with sessions.\n See phmdoctest documentation \"Execution Context\"\n section for an explanation.\n- phmutest does not support inline annotations.\n\n[1]: https://github.com/tmarktaylor/phmutest/blob/master/README.md?plain=1\n[3]: https://github.github.com/gfm/#fenced-code-blocks\n[4]: https://spec.commonmark.org\n[11]: https://github.github.com/gfm/#info-string\n[10]: https://phmutest.readthedocs.io/en/latest/docs/api.html\n[5]: https://docs.python.org/3/library/doctest.html\n[6]: https://pypi.python.org/project/coverage\n[13]: https://ci.appveyor.com/project/tmarktaylor/phmutest\n[15]: https://docs.pytest.org/en/stable\n[16]: https://tmarktaylor.github.io/pytest-phmdoctest\n[17]: https://pypi.python.org/pypi/phmdoctest\n[18]: https://docs.python.org/3/library/unittest.html\nMIT License\n\nCopyright (c) 2024 Mark Taylor\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Detect broken Python examples in Markdown.",
"version": "0.0.4",
"project_urls": {
"Bug Reports": "https://github.com/tmarktaylor/phmutest/issues",
"Homepage": "https://phmutest.readthedocs.io/en/latest/",
"Source": "https://github.com/tmarktaylor/phmutest/"
},
"split_keywords": [
"documentation",
" markdown",
" testing"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "da8b2bdb111d69e88f97a26abe7b340d4a55b703a81ca00bed2fb6c797594468",
"md5": "89426c3fe94adc709fd6e1b92379ee69",
"sha256": "c75c7cafe22eb04c67d23c11a222adf2e876583851a5ae2f23699f43a47c3127"
},
"downloads": -1,
"filename": "phmutest-0.0.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "89426c3fe94adc709fd6e1b92379ee69",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 44171,
"upload_time": "2024-09-06T16:03:44",
"upload_time_iso_8601": "2024-09-06T16:03:44.415529Z",
"url": "https://files.pythonhosted.org/packages/da/8b/2bdb111d69e88f97a26abe7b340d4a55b703a81ca00bed2fb6c797594468/phmutest-0.0.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "6b8bf29fbc6189d1c873fbf52ae48f2aad6cd47600e5800c11c97fd734fe3103",
"md5": "77250e31f33136c1041eb53c2d5cccc4",
"sha256": "17880e17c319d5bc7ee00324656649ab85e8d25e77f674e859a9d92a17951af0"
},
"downloads": -1,
"filename": "phmutest-0.0.4.tar.gz",
"has_sig": false,
"md5_digest": "77250e31f33136c1041eb53c2d5cccc4",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 110373,
"upload_time": "2024-09-06T16:03:45",
"upload_time_iso_8601": "2024-09-06T16:03:45.878670Z",
"url": "https://files.pythonhosted.org/packages/6b/8b/f29fbc6189d1c873fbf52ae48f2aad6cd47600e5800c11c97fd734fe3103/phmutest-0.0.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-06 16:03:45",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "tmarktaylor",
"github_project": "phmutest",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"appveyor": true,
"requirements": [
{
"name": "tomli",
"specs": [
[
">=",
"1.0.0"
]
]
}
],
"tox": true,
"lcname": "phmutest"
}
JSON
