## checkpy
A Python tool for running tests on Python source files. Intended to be
used by students whom are taking courses from [Programming Lab at the UvA](https://www.proglab.nl).
Check out <https://github.com/spcourse/tests> for examples of tests.
### Installation
pip install checkpy
To download tests, run checkPy with ``-d`` as follows:
checkpy -d YOUR_GITHUB_TESTS_URL
For instance:
checkpy -d spcourse/tests
Here spcourse/tests points to <https://github.com/spcourse/tests>. You can also use the full url. This tests repository contains a test for `hello.py`. Here is how to run it:
$ echo 'print("hello world")' > hello.py
$ checkpy hello
Testing: hello.py
:( prints "Hello, world!"
assert 'hello world\n' == 'Hello, world!\n'
- hello world
? ^
+ Hello, world!
? ^ + +
:) prints exactly 1 line of output
### Usage
usage: checkpy [-h] [-module MODULE] [-download GITHUBLINK] [-register LOCALLINK] [-update] [-list] [-clean] [--dev]
[--silent] [--json] [--gh-auth GH_AUTH]
[files ...]
checkPy: a python testing framework for education. You are running Python version 3.10.6 and checkpy version 2.0.0.
positional arguments:
files names of files to be tested
options:
-h, --help show this help message and exit
-module MODULE provide a module name or path to run all tests from the module, or target a module for a
specific test
-download GITHUBLINK download tests from a Github repository and exit
-register LOCALLINK register a local folder that contains tests and exit
-update update all downloaded tests and exit
-list list all download locations and exit
-clean remove all tests from the tests folder and exit
--dev get extra information to support the development of tests
--silent do not print test results to stdout
--json return output as json, implies silent
--gh-auth GH_AUTH username:personal_access_token for authentication with GitHub. Only used to increase GitHub api's rate limit.
To test a single file call:
checkpy YOUR_FILE_NAME
### An example
Tests in checkpy are functions with assertions. For instance:
```Py
from checkpy import *
@test()
def printsHello():
"""prints Hello, world!"""
assert outputOf() == "Hello, world!\n"
```
checkpy's `test` decorator marks the function below as a test. The docstring is a short description of the test for the student. This test does just one thing, assert that the output of the student's code matches the expected output exactly. checkpy leverages pytest's assertion rewriting to autmatically create assertion messages. For instance, a student might see the following when running this test:
$ checkpy hello
Testing: hello.py
:( prints Hello, world!
assert 'hello world\n' == 'Hello, world!\n'
- hello world
? ^
+ Hello, world!
? ^ + +
### Writing tests
Tests are discovered by filename. If you want to test a file called ``hello.py``, the corresponding test must be named ``helloTest.py``. These tests must be placed in a folder called `tests`. For instance: `tests/helloTest.py`. Tests are distributed via GitHub repositories, but for development purposes tests can also be registered locally via the `-r` flag. For instance:
mkdir tests
touch tests/helloTest.py
checkpy -r tests/helloTest.py
Once registered, checkpy will start looking in that directory for tests. Now we need a test. A test minimally consists of the following:
```Py
from checkpy import *
@test()
def printsHello():
"""prints Hello, world!"""
assert outputOf() == "Hello, world!\n"
```
A function marked as a test through checkpy's test decorator. The docstring is a short, generally one-line, description of the test for the student. Then at least one assert.
> Quick tip, use only binary expressions in assertions and keep them relatively simple for students to understand. If a binary expression is not possible, or you do not want to spoil the output, raise your own assertionError instead: ```raise AssertionError("Your program did not output the answer to the ultimate question of life, the universe, and everything")```.
While developing, you can run checkpy with the `--dev` flag to get verbose error messages and full tracebacks. So here we might do:
$ checkpy --dev hello
Testing: hello.py
:( prints "Hello, world!"
assert 'hello world\n' == 'Hello, world!\n'
- hello world
? ^
+ Hello, world!
? ^ + +
:) prints exactly 1 line of output
Check out <https://github.com/spcourse/tests> for many examples of checkpy tests.
### Short examples
#### Dependencies between tests
```Py
@test()
def exactHello():
"""prints \"Hello, world!\""""
assert outputOf() == "Hello, world!\n"
@failed(exactHello)
def oneLine():
"""prints exactly 1 line of output"""
assert outputOf().count("\n") == 1
@passed(exactHello)
def allGood():
"""Good job, everything is correct! You are ready to hand in."""
```
#### Test functions
```Py
@test()
def testSquare():
"""square(2) returns 4"""
assert getFunction("square")(4) == 4
```
#### Give hints
```Py
@test()
def testSquare():
"""square(2) returns 4"""
assert getFunction("square")(4) == 4, "did you remember to round your output?"
```
#### Handle randomness with pytest's `approx`
```Py
@test()
def testThrowDice():
"""throw() returns 7 on average"""
throw = getFunction("throw")
avg = sum(throw() for i in range(1000)) / 1000
assert avg == approx(7, abs=0.5)
```
#### Ban language constructs
```Py
import ast
@test()
def testSquare():
"""square(2) returns 4"""
assert ast.While not in static.AbstractSyntaxTree()
assert getFunction("square")(4) == 4
```
#### Check types
```Py
@test()
def testFibonacci():
"""fibonacci(10) returns the first ten fibonacci numbers"""
fibonacci = getFunction("fibonacci")
assert fibonacci(10) == Type(list[int])
assert fibonacci(10) == [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
```
#### Configure which files should be present
> Calling `only`, `include`, `exclude`, `require` or `download` has checkpy create a temporary directory to which the specified files are copied/downloaded. The tests then run in that directory.
```Py
only("sentiment.py")
download("pos_words.txt", "https://github.com/spcourse/text/raw/main/en/sentiment/pos_words.txt")
download("neg_words.txt", "https://github.com/spcourse/text/raw/main/en/sentiment/neg_words.txt")
@test()
def testPositiveSentiment():
"""recognises a positive sentence"""
...
```
#### Change the timeout
```Py
@test(timeout=60)
def exactHello():
"""prints \"Hello, world!\""""
assert outputOf() == "Hello, world!\n"
```
#### Short declarative tests
> This is a new style of tests for simple repetative use cases. Be sure to check out <https://github.com/spcourse/tests> for many more examples. For example [sentimentTest.py](https://github.com/spcourse/tests/blob/676cf5f0d2b0fbc82c7580a76b4359af273b0ca7/tests/text/sentimentTest.py)
```Py
correctForPos = test()(declarative
.function("sentiment_of_text")
.params("text")
.returnType(int)
.call("Coronet has the best lines of all day cruisers.")
.returns(1)
.description("recognises a positive sentence")
)
```
### Distributing tests
checkpy downloads tests directly from Github repos. The requirement is that a folder called ``tests`` exists within the repo that contains only tests and folders (which checkpy treats as modules). There must also be at least one release in the Github repo. checkpy will automatically target the latest release. To download tests call checkpy with the optional ``-d`` argument and pass your github repo url. checkpy will automatically keep tests up to date by checking for any new releases on GitHub.
### Testing checkpy
python3 run_tests.py
Raw data
{
"_id": null,
"home_page": "https://github.com/Jelleas/CheckPy",
"name": "checkPy",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "new unexperienced programmers automatic testing minor programming",
"author": "Jelleas",
"author_email": "jelle.van.assema@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/4f/01/1bf8c19fe7bad707e327559b54152ab80cc9d2196b62a4029e117257fa4c/checkPy-2.0.12.tar.gz",
"platform": null,
"description": "## checkpy\n\nA Python tool for running tests on Python source files. Intended to be\nused by students whom are taking courses from [Programming Lab at the UvA](https://www.proglab.nl).\n\nCheck out <https://github.com/spcourse/tests> for examples of tests.\n\n### Installation\n\n pip install checkpy\n\nTo download tests, run checkPy with ``-d`` as follows:\n\n checkpy -d YOUR_GITHUB_TESTS_URL\n\nFor instance:\n\n checkpy -d spcourse/tests\n\nHere spcourse/tests points to <https://github.com/spcourse/tests>. You can also use the full url. This tests repository contains a test for `hello.py`. Here is how to run it:\n\n $ echo 'print(\"hello world\")' > hello.py\n $ checkpy hello\n Testing: hello.py\n :( prints \"Hello, world!\"\n assert 'hello world\\n' == 'Hello, world!\\n'\n - hello world\n ? ^\n + Hello, world!\n ? ^ + +\n :) prints exactly 1 line of output\n\n### Usage\n\n usage: checkpy [-h] [-module MODULE] [-download GITHUBLINK] [-register LOCALLINK] [-update] [-list] [-clean] [--dev]\n [--silent] [--json] [--gh-auth GH_AUTH]\n [files ...]\n\n checkPy: a python testing framework for education. You are running Python version 3.10.6 and checkpy version 2.0.0.\n\n positional arguments:\n files names of files to be tested\n\n options:\n -h, --help show this help message and exit\n -module MODULE provide a module name or path to run all tests from the module, or target a module for a\n specific test\n -download GITHUBLINK download tests from a Github repository and exit\n -register LOCALLINK register a local folder that contains tests and exit\n -update update all downloaded tests and exit\n -list list all download locations and exit\n -clean remove all tests from the tests folder and exit\n --dev get extra information to support the development of tests\n --silent do not print test results to stdout\n --json return output as json, implies silent\n --gh-auth GH_AUTH username:personal_access_token for authentication with GitHub. Only used to increase GitHub api's rate limit.\n\nTo test a single file call:\n\n checkpy YOUR_FILE_NAME\n\n### An example\n\nTests in checkpy are functions with assertions. For instance:\n\n```Py\nfrom checkpy import *\n\n@test()\ndef printsHello():\n \"\"\"prints Hello, world!\"\"\"\n assert outputOf() == \"Hello, world!\\n\"\n```\n\ncheckpy's `test` decorator marks the function below as a test. The docstring is a short description of the test for the student. This test does just one thing, assert that the output of the student's code matches the expected output exactly. checkpy leverages pytest's assertion rewriting to autmatically create assertion messages. For instance, a student might see the following when running this test:\n\n $ checkpy hello\n Testing: hello.py\n :( prints Hello, world!\n assert 'hello world\\n' == 'Hello, world!\\n'\n - hello world\n ? ^\n + Hello, world!\n ? ^ + +\n\n### Writing tests\n\nTests are discovered by filename. If you want to test a file called ``hello.py``, the corresponding test must be named ``helloTest.py``. These tests must be placed in a folder called `tests`. For instance: `tests/helloTest.py`. Tests are distributed via GitHub repositories, but for development purposes tests can also be registered locally via the `-r` flag. For instance:\n\n mkdir tests\n touch tests/helloTest.py\n checkpy -r tests/helloTest.py\n\nOnce registered, checkpy will start looking in that directory for tests. Now we need a test. A test minimally consists of the following:\n\n```Py\nfrom checkpy import *\n\n@test()\ndef printsHello():\n \"\"\"prints Hello, world!\"\"\"\n assert outputOf() == \"Hello, world!\\n\"\n```\n\nA function marked as a test through checkpy's test decorator. The docstring is a short, generally one-line, description of the test for the student. Then at least one assert.\n\n> Quick tip, use only binary expressions in assertions and keep them relatively simple for students to understand. If a binary expression is not possible, or you do not want to spoil the output, raise your own assertionError instead: ```raise AssertionError(\"Your program did not output the answer to the ultimate question of life, the universe, and everything\")```.\n\nWhile developing, you can run checkpy with the `--dev` flag to get verbose error messages and full tracebacks. So here we might do:\n\n $ checkpy --dev hello \n Testing: hello.py\n :( prints \"Hello, world!\"\n assert 'hello world\\n' == 'Hello, world!\\n'\n - hello world\n ? ^\n + Hello, world!\n ? ^ + +\n :) prints exactly 1 line of output\n\nCheck out <https://github.com/spcourse/tests> for many examples of checkpy tests.\n\n### Short examples\n\n#### Dependencies between tests\n\n```Py\n@test()\ndef exactHello():\n \"\"\"prints \\\"Hello, world!\\\"\"\"\"\n assert outputOf() == \"Hello, world!\\n\"\n\n@failed(exactHello)\ndef oneLine():\n \"\"\"prints exactly 1 line of output\"\"\"\n assert outputOf().count(\"\\n\") == 1\n\n@passed(exactHello)\ndef allGood():\n \"\"\"Good job, everything is correct! You are ready to hand in.\"\"\"\n```\n\n#### Test functions\n\n```Py\n@test()\ndef testSquare():\n \"\"\"square(2) returns 4\"\"\"\n assert getFunction(\"square\")(4) == 4\n```\n\n#### Give hints\n\n```Py\n@test()\ndef testSquare():\n \"\"\"square(2) returns 4\"\"\"\n assert getFunction(\"square\")(4) == 4, \"did you remember to round your output?\"\n```\n\n#### Handle randomness with pytest's `approx`\n\n```Py\n@test()\ndef testThrowDice():\n \"\"\"throw() returns 7 on average\"\"\"\n throw = getFunction(\"throw\")\n avg = sum(throw() for i in range(1000)) / 1000 \n assert avg == approx(7, abs=0.5)\n```\n\n#### Ban language constructs\n\n```Py\nimport ast\n\n@test()\ndef testSquare():\n \"\"\"square(2) returns 4\"\"\"\n assert ast.While not in static.AbstractSyntaxTree()\n assert getFunction(\"square\")(4) == 4\n```\n\n#### Check types\n\n```Py\n@test()\ndef testFibonacci():\n \"\"\"fibonacci(10) returns the first ten fibonacci numbers\"\"\"\n fibonacci = getFunction(\"fibonacci\")\n assert fibonacci(10) == Type(list[int])\n assert fibonacci(10) == [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]\n```\n\n#### Configure which files should be present\n\n> Calling `only`, `include`, `exclude`, `require` or `download` has checkpy create a temporary directory to which the specified files are copied/downloaded. The tests then run in that directory.\n\n```Py\nonly(\"sentiment.py\")\ndownload(\"pos_words.txt\", \"https://github.com/spcourse/text/raw/main/en/sentiment/pos_words.txt\")\ndownload(\"neg_words.txt\", \"https://github.com/spcourse/text/raw/main/en/sentiment/neg_words.txt\")\n\n@test()\ndef testPositiveSentiment():\n \"\"\"recognises a positive sentence\"\"\"\n ...\n```\n\n#### Change the timeout\n\n```Py\n@test(timeout=60)\ndef exactHello():\n \"\"\"prints \\\"Hello, world!\\\"\"\"\"\n assert outputOf() == \"Hello, world!\\n\"\n```\n\n#### Short declarative tests\n\n> This is a new style of tests for simple repetative use cases. Be sure to check out <https://github.com/spcourse/tests> for many more examples. For example [sentimentTest.py](https://github.com/spcourse/tests/blob/676cf5f0d2b0fbc82c7580a76b4359af273b0ca7/tests/text/sentimentTest.py)\n\n```Py\ncorrectForPos = test()(declarative\n .function(\"sentiment_of_text\")\n .params(\"text\")\n .returnType(int)\n .call(\"Coronet has the best lines of all day cruisers.\")\n .returns(1)\n .description(\"recognises a positive sentence\")\n)\n```\n\n### Distributing tests\n\ncheckpy downloads tests directly from Github repos. The requirement is that a folder called ``tests`` exists within the repo that contains only tests and folders (which checkpy treats as modules). There must also be at least one release in the Github repo. checkpy will automatically target the latest release. To download tests call checkpy with the optional ``-d`` argument and pass your github repo url. checkpy will automatically keep tests up to date by checking for any new releases on GitHub.\n\n\n### Testing checkpy\n\n python3 run_tests.py\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A simple python testing framework for educational purposes",
"version": "2.0.12",
"project_urls": {
"Homepage": "https://github.com/Jelleas/CheckPy"
},
"split_keywords": [
"new",
"unexperienced",
"programmers",
"automatic",
"testing",
"minor",
"programming"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "00621a2e0cbf6eaca82f74e5a1d292b3d862d0410518deb75bea639188041e8b",
"md5": "ef94f8360d4d0815a52b10b154d3507d",
"sha256": "2c6c70265d2939d7270f6b871f4f248384445aad11e8fd954077b51857817f39"
},
"downloads": -1,
"filename": "checkPy-2.0.12-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ef94f8360d4d0815a52b10b154d3507d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 42411,
"upload_time": "2024-03-21T13:42:12",
"upload_time_iso_8601": "2024-03-21T13:42:12.558905Z",
"url": "https://files.pythonhosted.org/packages/00/62/1a2e0cbf6eaca82f74e5a1d292b3d862d0410518deb75bea639188041e8b/checkPy-2.0.12-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4f011bf8c19fe7bad707e327559b54152ab80cc9d2196b62a4029e117257fa4c",
"md5": "a7cf29fd61dc5b5144093895292fcb3b",
"sha256": "35cf8cbc25bbf159242eb55587f2e93f57cbbac09d0e9bcc33b22e831f2e541f"
},
"downloads": -1,
"filename": "checkPy-2.0.12.tar.gz",
"has_sig": false,
"md5_digest": "a7cf29fd61dc5b5144093895292fcb3b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 35193,
"upload_time": "2024-03-21T13:42:14",
"upload_time_iso_8601": "2024-03-21T13:42:14.600530Z",
"url": "https://files.pythonhosted.org/packages/4f/01/1bf8c19fe7bad707e327559b54152ab80cc9d2196b62a4029e117257fa4c/checkPy-2.0.12.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-21 13:42:14",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Jelleas",
"github_project": "CheckPy",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "checkpy"
}
JSON
