# Python Interface for Autograder
The Python interface for the autograding server.
## Quick Links
- [Autograder Server](https://github.com/eriq-augustine/autograder-server)
- [Autograder Python Interface](https://github.com/eriq-augustine/autograder-py)
- [Autograder Sample Course](https://github.com/eriq-augustine/cse-cracks-course)
## The CLI
This project contains several tools for interacting with an autograding server and
working with autograder assignments via the `autograder.cli` package.
All tools will show their usage if given the `--help` options.
You can get a list of the package for each set of tools by invoking `autograder.cli` directly:
```sh
python3 -m autograder.cli
```
There are many available tools and instead of discussing each one here,
this document will highlight the tools each type of user (student, TA, course developer) will generally use.
### Configuration
Before discussing specific tools, you should know some general information about
configuring and sending options to each tool.
To know who you are and what you are working on the autograder needs a few configuration options:
- `server` -- The autograding server to connect to.
- `course` -- The ID for the course you are enrolled in.
- `assignment` -- The current assignment you are working on (does not always apply)..
- `user` -- Your username (which is also your email).
- `pass` -- You password (probably sent to you by a TA or the autograding server in an email).
All these options can be set on the command line when invoking on of these tools, e.g.,:
```sh
python3 -m autograder.cli.submission.submit --user sammy@ucsc.edu --pass pass123 my_file.py
```
However, it will generally be more convenient to hold these common options in a more reusable location.
There are several other places that config options can be specified,
with each later location overriding any earlier options.
Here are the places options can be specified in the order that they are checked:
1. `./config.json` -- If a `config.json` exists in the current directory, it is loaded.
2. `<platform-specific user config location>/autograder.json` -- A directory which is considered the "proper" place to store user-related config for the platform you are using (according to [platformdirs](https://github.com/platformdirs/platformdirs)). Use `--help` to see the exact place in your specific case. This is a great place to store login credentials.
3. Files specified by `--config` -- These files are loaded in the order they appear on the command-line.
4. Bare Options -- Options specified directly like `--user` or `--pass`. These will override all previous options.
A base config file (`config.json`) is often distributed with assignments that contains most the settings you need.
You can modify this config to include your settings and use that for setting all your configuration options.
Using the default config file (`config.json`):
```sh
# `./config.json` will be looked for and loaded if it exists.
python3 -m autograder.cli.submission.submit my_file.py
```
Using a custom config file (`my_config.json`):
```sh
# `./my_config.json` will be used.
python3 -m autograder.cli.submission.submit --config my_config.json my_file.py
```
You can also use multiple config files (latter files will override settings from previous ones).
This is useful if you want to use the config files provided with assignments, but keep your user credentials in a more secure location:
```sh
# Use the default config file (config.json), but then override any settings in there with another config file:
python3 -m autograder.cli.submission.submit --config config.json --config ~/.secrets/autograder.json my_file.py
```
For brevity, all future commands in this document will assume that all standard config options are in the default
config files (and thus will not need to be specified).
### Commands for Students
Students will mainly be concerned with submitting assignments and checking on the status of their submission.
Therefore, the `autograder.cli.submission` package will be their primary resource.
This package contains tools for making, managing, and querying submissions.
#### Submitting an Assignment
Submitting an assignment to an autograder is done using the `autograder.cli.submission.submit` command.
This command takes the standard config options as well as an optional message to attach to the submission (like a commit message)
as well as all files to be included in the submission.
```sh
python3 -m autograder.cli.submission.submit --message "This is my submit message!" my_file.py
```
As many files as you need can be submitted (directories cannot be submitted):
```sh
python3 -m autograder.cli.submission.submit my_first_file.py my_second_file.java some_dir/*
```
The autograder will attempt to grade your assignment and will return some message about the result of grading.
For example, a successful grading may look like:
```
The autograder successfully graded your assignment.
Autograder transcript for assignment: HO0.
Grading started at 2023-09-26 08:35 and ended at 2023-09-26 08:35.
Task 1.A (my_function): 40 / 40
Task 2.A (test_my_function_value): 30 / 30
Task 2.B (TestMyFunction): 30 / 30
Style: 0 / 0
Style is clean!
Total: 100 / 100
```
On any successful grading (even if you got a zero), your result has been saved by the autograder and is in the system.
On a submission failure, the autograder will tell you and you will not receive any grade for your submission.
A failure may look like:
```
The autograder failed to grade your assignment.
Message from the autograder: Request could not be authenticated. Ensure that your username, password, and course are properly set.
```
#### Checking Your Last Submission
You can ask the autograder to show you the grade report for your last submission using the
`autograder.cli.submission.peek` command.
```sh
python3 -m autograder.cli.submission.peek
```
The output may look like:
```
Found a past submission for this assignment.
Autograder transcript for assignment: HO0.
Grading started at 2023-09-26 08:35 and ended at 2023-09-26 08:35.
Task 1.A (my_function): 40 / 40
Task 2.A (test_my_function_value): 30 / 30
Task 2.B (TestMyFunction): 30 / 30
Style: 0 / 0
Style is clean!
Total: 100 / 100
```
If you have made no past (successful) submissions, then your output may look like:
```
No past submission found for this assignment.
```
#### Getting a History of All Past Submissions
You can use the `autograder.cli.submission.history` command to get a summary of all your past submissions for an assignment.
```sh
python3 -m autograder.cli.submission.history
```
The output may look like:
```
Found 2 submissions.
Submission ID: 1695682455, Score: 24 / 100, Time: 2023-09-25 17:54.
Submission ID: 1695735313, Score: 100 / 100, Time: 2023-09-26 08:35, Message: 'I did it!'
```
If you have made no past (successful) submissions, then your output may look like:
```
No past submission found for this assignment.
```
### Commands for TAs and Instructors
For those that are managing a course and students,
most commands will be useful to you.
So you should have a look through all commands via:
```sh
python3 -m autograder.cli
```
Below is a list of commands you may want to look into.
The help prompt of each command (accessible using the `--help` option)
will give a more in-depth description of the command and available options.
- `autograder.lms.sync-users` -- Get information about users from your LMS (e.g. Canvas).
- `autograder.lms.upload-scores` -- Upload scores for any LMS assignment straight to your LMS. Very useful for avoiding a clunky LMS interface.
- `autograder.submission.fetch-scores` -- Get all the most recent scores for an assignment.
- `autograder.submission.fetch-submission` -- Get a student's submission (code) and grading output.
- `autograder.submission.fetch-submissions` -- Get all the most recent submissions (code and grading output) for an assignment.
- `autograder.user.*` -- Several different tools for managing users (adding, removing, changing passwords, etc).
### Commands for Course Builders
Users who are building courses should generally be aware of all the available tools,
but most of your time will probably be spent in the
`autograder.cli.testing` and `autograder.cli.grading` packages.
`autograder.cli.testing` is for running tests and checks (usually locally) on assignments.
`autograder.cli.grading` lets you grade assignments locally (without using an autograding server).
Because the autograding server runs this package inside a Docker container to do grading,
it can be much faster and more convenient to build assignments fully locally before using an autograding server.
Raw data
{
"_id": null,
"home_page": "https://github.com/eriq-augustine/autograder-py",
"name": "autograder-py",
"maintainer": "Eriq Augustine",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "eaugusti@ucsc.edu",
"keywords": "grading",
"author": null,
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/a5/af/39fe42aae61622ef28ed070f437528ff00cbc0ce11e3936026d8889c1732/autograder-py-0.4.2.tar.gz",
"platform": null,
"description": "# Python Interface for Autograder\n\nThe Python interface for the autograding server.\n\n## Quick Links\n\n - [Autograder Server](https://github.com/eriq-augustine/autograder-server)\n - [Autograder Python Interface](https://github.com/eriq-augustine/autograder-py)\n - [Autograder Sample Course](https://github.com/eriq-augustine/cse-cracks-course)\n\n## The CLI\n\nThis project contains several tools for interacting with an autograding server and\nworking with autograder assignments via the `autograder.cli` package.\nAll tools will show their usage if given the `--help` options.\n\nYou can get a list of the package for each set of tools by invoking `autograder.cli` directly:\n```sh\npython3 -m autograder.cli\n```\n\nThere are many available tools and instead of discussing each one here,\nthis document will highlight the tools each type of user (student, TA, course developer) will generally use.\n\n### Configuration\n\nBefore discussing specific tools, you should know some general information about\nconfiguring and sending options to each tool.\n\nTo know who you are and what you are working on the autograder needs a few configuration options:\n - `server` -- The autograding server to connect to.\n - `course` -- The ID for the course you are enrolled in.\n - `assignment` -- The current assignment you are working on (does not always apply)..\n - `user` -- Your username (which is also your email).\n - `pass` -- You password (probably sent to you by a TA or the autograding server in an email).\n\nAll these options can be set on the command line when invoking on of these tools, e.g.,:\n```sh\npython3 -m autograder.cli.submission.submit --user sammy@ucsc.edu --pass pass123 my_file.py\n```\nHowever, it will generally be more convenient to hold these common options in a more reusable location.\n\nThere are several other places that config options can be specified,\nwith each later location overriding any earlier options.\nHere are the places options can be specified in the order that they are checked:\n 1. `./config.json` -- If a `config.json` exists in the current directory, it is loaded.\n 2. `<platform-specific user config location>/autograder.json` -- A directory which is considered the \"proper\" place to store user-related config for the platform you are using (according to [platformdirs](https://github.com/platformdirs/platformdirs)). Use `--help` to see the exact place in your specific case. This is a great place to store login credentials.\n 3. Files specified by `--config` -- These files are loaded in the order they appear on the command-line.\n 4. Bare Options -- Options specified directly like `--user` or `--pass`. These will override all previous options.\n\nA base config file (`config.json`) is often distributed with assignments that contains most the settings you need.\nYou can modify this config to include your settings and use that for setting all your configuration options.\n\nUsing the default config file (`config.json`):\n```sh\n# `./config.json` will be looked for and loaded if it exists.\npython3 -m autograder.cli.submission.submit my_file.py\n```\n\nUsing a custom config file (`my_config.json`):\n```sh\n# `./my_config.json` will be used.\npython3 -m autograder.cli.submission.submit --config my_config.json my_file.py\n```\n\nYou can also use multiple config files (latter files will override settings from previous ones).\nThis is useful if you want to use the config files provided with assignments, but keep your user credentials in a more secure location:\n```sh\n# Use the default config file (config.json), but then override any settings in there with another config file:\npython3 -m autograder.cli.submission.submit --config config.json --config ~/.secrets/autograder.json my_file.py\n```\n\nFor brevity, all future commands in this document will assume that all standard config options are in the default\nconfig files (and thus will not need to be specified).\n\n### Commands for Students\n\nStudents will mainly be concerned with submitting assignments and checking on the status of their submission.\nTherefore, the `autograder.cli.submission` package will be their primary resource.\nThis package contains tools for making, managing, and querying submissions.\n\n#### Submitting an Assignment\n\nSubmitting an assignment to an autograder is done using the `autograder.cli.submission.submit` command.\nThis command takes the standard config options as well as an optional message to attach to the submission (like a commit message)\nas well as all files to be included in the submission.\n\n```sh\npython3 -m autograder.cli.submission.submit --message \"This is my submit message!\" my_file.py\n```\n\nAs many files as you need can be submitted (directories cannot be submitted):\n```sh\npython3 -m autograder.cli.submission.submit my_first_file.py my_second_file.java some_dir/*\n```\n\nThe autograder will attempt to grade your assignment and will return some message about the result of grading.\nFor example, a successful grading may look like:\n```\nThe autograder successfully graded your assignment.\nAutograder transcript for assignment: HO0.\nGrading started at 2023-09-26 08:35 and ended at 2023-09-26 08:35.\nTask 1.A (my_function): 40 / 40\nTask 2.A (test_my_function_value): 30 / 30\nTask 2.B (TestMyFunction): 30 / 30\nStyle: 0 / 0\n Style is clean!\n\nTotal: 100 / 100\n```\n\nOn any successful grading (even if you got a zero), your result has been saved by the autograder and is in the system.\nOn a submission failure, the autograder will tell you and you will not receive any grade for your submission.\nA failure may look like:\n```\nThe autograder failed to grade your assignment.\nMessage from the autograder: Request could not be authenticated. Ensure that your username, password, and course are properly set.\n```\n\n#### Checking Your Last Submission\n\nYou can ask the autograder to show you the grade report for your last submission using the\n`autograder.cli.submission.peek` command.\n\n```sh\npython3 -m autograder.cli.submission.peek\n```\n\nThe output may look like:\n```\nFound a past submission for this assignment.\nAutograder transcript for assignment: HO0.\nGrading started at 2023-09-26 08:35 and ended at 2023-09-26 08:35.\nTask 1.A (my_function): 40 / 40\nTask 2.A (test_my_function_value): 30 / 30\nTask 2.B (TestMyFunction): 30 / 30\nStyle: 0 / 0\n Style is clean!\n\nTotal: 100 / 100\n```\n\nIf you have made no past (successful) submissions, then your output may look like:\n```\nNo past submission found for this assignment.\n```\n\n#### Getting a History of All Past Submissions\n\nYou can use the `autograder.cli.submission.history` command to get a summary of all your past submissions for an assignment.\n\n```sh\npython3 -m autograder.cli.submission.history\n```\n\nThe output may look like:\n```\nFound 2 submissions.\n Submission ID: 1695682455, Score: 24 / 100, Time: 2023-09-25 17:54.\n Submission ID: 1695735313, Score: 100 / 100, Time: 2023-09-26 08:35, Message: 'I did it!'\n```\n\nIf you have made no past (successful) submissions, then your output may look like:\n```\nNo past submission found for this assignment.\n```\n\n### Commands for TAs and Instructors\n\nFor those that are managing a course and students,\nmost commands will be useful to you.\nSo you should have a look through all commands via:\n```sh\npython3 -m autograder.cli\n```\n\nBelow is a list of commands you may want to look into.\nThe help prompt of each command (accessible using the `--help` option)\nwill give a more in-depth description of the command and available options.\n\n - `autograder.lms.sync-users` -- Get information about users from your LMS (e.g. Canvas).\n - `autograder.lms.upload-scores` -- Upload scores for any LMS assignment straight to your LMS. Very useful for avoiding a clunky LMS interface.\n - `autograder.submission.fetch-scores` -- Get all the most recent scores for an assignment.\n - `autograder.submission.fetch-submission` -- Get a student's submission (code) and grading output.\n - `autograder.submission.fetch-submissions` -- Get all the most recent submissions (code and grading output) for an assignment.\n - `autograder.user.*` -- Several different tools for managing users (adding, removing, changing passwords, etc).\n\n### Commands for Course Builders\n\nUsers who are building courses should generally be aware of all the available tools,\nbut most of your time will probably be spent in the\n`autograder.cli.testing` and `autograder.cli.grading` packages.\n`autograder.cli.testing` is for running tests and checks (usually locally) on assignments.\n`autograder.cli.grading` lets you grade assignments locally (without using an autograding server).\nBecause the autograding server runs this package inside a Docker container to do grading,\nit can be much faster and more convenient to build assignments fully locally before using an autograding server.\n",
"bugtrack_url": null,
"license": null,
"summary": "The Python interface for the autograding server.",
"version": "0.4.2",
"project_urls": {
"Homepage": "https://github.com/eriq-augustine/autograder-py"
},
"split_keywords": [
"grading"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d7205ea03eef16d72eaa0aa0ac6363073a73c46ce711fc72d48e3a300717be59",
"md5": "f75679c0662241841a08a5e7587ccda4",
"sha256": "21800cd1bcf785d7c927ba60e352fd4b9c4bea1ea26882c6272899ed8ea07706"
},
"downloads": -1,
"filename": "autograder_py-0.4.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f75679c0662241841a08a5e7587ccda4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 68230,
"upload_time": "2024-03-30T21:13:19",
"upload_time_iso_8601": "2024-03-30T21:13:19.616579Z",
"url": "https://files.pythonhosted.org/packages/d7/20/5ea03eef16d72eaa0aa0ac6363073a73c46ce711fc72d48e3a300717be59/autograder_py-0.4.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a5af39fe42aae61622ef28ed070f437528ff00cbc0ce11e3936026d8889c1732",
"md5": "4b52d99014f2901d2b5e010ce17162c6",
"sha256": "6609a066cab1e5dcf9c33509f60e0e2217cb6549cc6935f9110b5cc36c445e6c"
},
"downloads": -1,
"filename": "autograder-py-0.4.2.tar.gz",
"has_sig": false,
"md5_digest": "4b52d99014f2901d2b5e010ce17162c6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 43670,
"upload_time": "2024-03-30T21:13:21",
"upload_time_iso_8601": "2024-03-30T21:13:21.272248Z",
"url": "https://files.pythonhosted.org/packages/a5/af/39fe42aae61622ef28ed070f437528ff00cbc0ce11e3936026d8889c1732/autograder-py-0.4.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-30 21:13:21",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "eriq-augustine",
"github_project": "autograder-py",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "argon2-cffi",
"specs": [
[
">=",
"23.1.0"
]
]
},
{
"name": "flake8",
"specs": [
[
">=",
"6.0.0"
]
]
},
{
"name": "GitPython",
"specs": [
[
">=",
"3.1.31"
]
]
},
{
"name": "platformdirs",
"specs": [
[
">=",
"3.10.0"
]
]
},
{
"name": "requests",
"specs": [
[
">=",
"2.31.0"
]
]
},
{
"name": "build",
"specs": []
},
{
"name": "twine",
"specs": []
},
{
"name": "packaging",
"specs": [
[
">=",
"21.3"
]
]
},
{
"name": "vermin",
"specs": []
}
],
"lcname": "autograder-py"
}
JSON
