| Name | pht-train-container-library JSON |
| Version |
2.0.5
JSON |
| download |
| home_page | |
| Summary | Python library for handling containerized PHT trains |
| upload_time | 2023-04-17 18:07:34 |
| maintainer | |
| docs_url | None |
| author | Michael Graf |
| requires_python | >=3.8,<4.0 |
| license | |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
[](https://train-container-library.readthedocs.io/en/latest/?badge=latest)
[](https://github.com/PHT-Medic/train-container-library/actions/workflows/codeql-analysis.yml)
[](https://github.com/PHT-EU/train-container-library/actions/workflows/main.yml)
[](https://codecov.io/gh/PHT-Medic/train-container-library)
[](https://github.com/psf/black)
# 🚆 Train Container Library
Python library for validating and interacting with pht-train images/containers.
## Installation
```shell
pip install pht-train-container-library
```
## Setup development environment
Make sure you have [poetry](https://python-poetry.org/docs/#installation) and [pre-commit](https://pre-commit.com/#install) installed.
Install the dependencies and pre-commit hooks:
```shell
poetry install --with dev
```
```shell
poetry run pre-commit install
```
### Run tests
```shell
poetry run pytest
```
### Linting and formatting
These commands are also run as pre-commit hooks.
Linting with ruff:
```shell
poetry run ruff . --fix
```
Formatting with black:
```shell
poetry run black .
```
## Security Protocol
The pht security protocol adapted from `docs/Secure_PHT_latest__official.pdf` performs two main tasks:
1. Before executing a train-image on the local machine, unless the station is the first station on the route, the
previous results need to be decrypted and the content of the image needs to be validated based on the configuration
of the individual train -> `pre-run`.
2. After executing the train the updated results need to be encrypted and the train configuration needs to be updated to
reflect the current state ->`post-run`.
### Train image structure
To ensure the protocol is working correctly train docker images are required to keep the following structure:
- `/opt/train_config.json`: Stores the configuration file of the train.
- `/opt/pht_train/`: Stores all the files containing code or other things required for the train algorithm to run. The
contents of this directory can never change and is validated by the `pre-run` step.
- `/opt/pht_results/`: Stores the results of the train. Which will be decrypted in the `pre-run` step and encrypted in
the `post-run` step.
No files in the image outside the `/opt/pht_results/` directory should change during the execution of the algorithm.
### Usage - Python Script
To use the protocol in your own python application, after installing the library
with `pip install pht-train-container-library` an instance of the protocol can be to validate docker images as follows:
```python
from train_lib.security.protocol import SecurityProtocol
from train_lib.docker_util.docker_ops import extract_train_config
image_name = '<image-repo>:<image-tag>'
station_id = '<station-id>'
# Get the train configuration from the image
config = extract_train_config(image_name)
# Initialize the protocol with the extracted config and station_id
protocol = SecurityProtocol(station_id=station_id, config=config)
# execute one of the protocol steps
protocol.pre_run_protocol(image_name, private_key_path='<path-to-private-key>')
# protocol.post_run_protocol(image_name, private_key_path='<path-to-private-key>')
```
### Usage - Container
A containerized version of the protocol is also available it can be used with the following command:
```shell
docker run -e STATION_ID=<station_id> -e PRIVATE_KEY_PATH=/opt/private_key.pem -v /var/run/docker.sock:/var/run/docker.sock -v <path_to_your_key>:/opt/private_key.pem ghcr.io/pht-medic/protocol <pre-run/post-run> <image-repo>:<image-tag>
```
`STATION_ID` and `PRIVATE_KEY_PATH` are required to be set in the environment variables. As well as passing the docker
socket `/var/run/docker.sock` to the container as a volume to enable docker-in-docker functionality.
### Pre-run protocol
The pre-run protocol consists of the following steps
1. The hash of the immutable files (train definition) is verified making sure that the executable files did not change
during the the train definition.
2. The digital signature is verified ensuring the correctness of the results at each stop of the train.
3. The symmetric key is decrypted using the provided station private key
4. The mutable files in `/opt/pht_results` are decrypted using the symmetric key obtained in the previous step
5. The decrypted files are hashed and the hash is compared to the one stored in the train configuration file.
Once these steps have been completed the image is ready to be executed.
### Post-run protocol
1. Calculate the hash of the newly generated results
2. Sign the hash of the results using the provided `PRIVATE_KEY_PATH`
3. Update the the train signature using the session id that is randomly generated at each execution step
4. Encrypt the resulting files using a newly generated symmetric key
5. Encrypt the generated symmetric key with the public keys of the train participants
6. Update the train configuration file
With the completion of these steps the train is ready to be pushed into the registry for further processing
## Tests
Run the tests to validate the security protocol is working as intended. From this projects root directory run
`pytest train_lib`
Raw data
{
"_id": null,
"home_page": "",
"name": "pht-train-container-library",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4.0",
"maintainer_email": "",
"keywords": "",
"author": "Michael Graf",
"author_email": "michael.graf@uni-tuebingen.com",
"download_url": "https://files.pythonhosted.org/packages/75/01/0b2b51a9d785aeb99ed39bab72a647f9e0509e56c6e5269612a3c218ffa4/pht_train_container_library-2.0.5.tar.gz",
"platform": null,
"description": "[](https://train-container-library.readthedocs.io/en/latest/?badge=latest)\n[](https://github.com/PHT-Medic/train-container-library/actions/workflows/codeql-analysis.yml)\n[](https://github.com/PHT-EU/train-container-library/actions/workflows/main.yml)\n[](https://codecov.io/gh/PHT-Medic/train-container-library)\n\n\n[](https://github.com/psf/black)\n\n# 🚆 Train Container Library\n\nPython library for validating and interacting with pht-train images/containers.\n\n## Installation\n\n```shell\npip install pht-train-container-library\n```\n\n\n## Setup development environment\nMake sure you have [poetry](https://python-poetry.org/docs/#installation) and [pre-commit](https://pre-commit.com/#install) installed.\n\nInstall the dependencies and pre-commit hooks:\n```shell\npoetry install --with dev\n```\n\n```shell\npoetry run pre-commit install\n```\n\n### Run tests\n\n```shell\npoetry run pytest\n```\n\n### Linting and formatting\n\nThese commands are also run as pre-commit hooks.\n\nLinting with ruff:\n```shell\npoetry run ruff . --fix\n```\n\nFormatting with black:\n```shell\npoetry run black .\n```\n\n## Security Protocol\n\nThe pht security protocol adapted from `docs/Secure_PHT_latest__official.pdf` performs two main tasks:\n\n1. Before executing a train-image on the local machine, unless the station is the first station on the route, the\n previous results need to be decrypted and the content of the image needs to be validated based on the configuration\n of the individual train -> `pre-run`.\n2. After executing the train the updated results need to be encrypted and the train configuration needs to be updated to\n reflect the current state ->`post-run`.\n\n### Train image structure\n\nTo ensure the protocol is working correctly train docker images are required to keep the following structure:\n\n- `/opt/train_config.json`: Stores the configuration file of the train.\n- `/opt/pht_train/`: Stores all the files containing code or other things required for the train algorithm to run. The\n contents of this directory can never change and is validated by the `pre-run` step.\n- `/opt/pht_results/`: Stores the results of the train. Which will be decrypted in the `pre-run` step and encrypted in\n the `post-run` step.\n\nNo files in the image outside the `/opt/pht_results/` directory should change during the execution of the algorithm.\n\n### Usage - Python Script\n\nTo use the protocol in your own python application, after installing the library\nwith `pip install pht-train-container-library` an instance of the protocol can be to validate docker images as follows:\n\n```python\nfrom train_lib.security.protocol import SecurityProtocol\nfrom train_lib.docker_util.docker_ops import extract_train_config\n\nimage_name = '<image-repo>:<image-tag>'\nstation_id = '<station-id>'\n\n# Get the train configuration from the image\nconfig = extract_train_config(image_name)\n# Initialize the protocol with the extracted config and station_id\nprotocol = SecurityProtocol(station_id=station_id, config=config)\n\n# execute one of the protocol steps\nprotocol.pre_run_protocol(image_name, private_key_path='<path-to-private-key>')\n# protocol.post_run_protocol(image_name, private_key_path='<path-to-private-key>')\n```\n\n### Usage - Container\n\nA containerized version of the protocol is also available it can be used with the following command:\n\n```shell\ndocker run -e STATION_ID=<station_id> -e PRIVATE_KEY_PATH=/opt/private_key.pem -v /var/run/docker.sock:/var/run/docker.sock -v <path_to_your_key>:/opt/private_key.pem ghcr.io/pht-medic/protocol <pre-run/post-run> <image-repo>:<image-tag>\n```\n\n`STATION_ID` and `PRIVATE_KEY_PATH` are required to be set in the environment variables. As well as passing the docker\nsocket `/var/run/docker.sock` to the container as a volume to enable docker-in-docker functionality.\n\n### Pre-run protocol\n\nThe pre-run protocol consists of the following steps\n\n1. The hash of the immutable files (train definition) is verified making sure that the executable files did not change\n during the the train definition.\n2. The digital signature is verified ensuring the correctness of the results at each stop of the train.\n3. The symmetric key is decrypted using the provided station private key\n4. The mutable files in `/opt/pht_results` are decrypted using the symmetric key obtained in the previous step\n5. The decrypted files are hashed and the hash is compared to the one stored in the train configuration file.\n\nOnce these steps have been completed the image is ready to be executed.\n\n### Post-run protocol\n\n1. Calculate the hash of the newly generated results\n2. Sign the hash of the results using the provided `PRIVATE_KEY_PATH`\n3. Update the the train signature using the session id that is randomly generated at each execution step\n4. Encrypt the resulting files using a newly generated symmetric key\n5. Encrypt the generated symmetric key with the public keys of the train participants\n6. Update the train configuration file\n\nWith the completion of these steps the train is ready to be pushed into the registry for further processing\n\n## Tests\n\nRun the tests to validate the security protocol is working as intended. From this projects root directory run\n`pytest train_lib`\n\n\n\n\n\n\n\n",
"bugtrack_url": null,
"license": "",
"summary": "Python library for handling containerized PHT trains",
"version": "2.0.5",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e38d306c6e7cfc50d6eaeec3b77eb92a72481bc3697e5ce05f0b881a410d170f",
"md5": "758b52a90bddaeb386aff6849c2a56d9",
"sha256": "726c42aaef132dcd34727da14fa409a6c141ac4c6c8517c640ebbbf4748e478f"
},
"downloads": -1,
"filename": "pht_train_container_library-2.0.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "758b52a90bddaeb386aff6849c2a56d9",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4.0",
"size": 45243,
"upload_time": "2023-04-17T18:07:32",
"upload_time_iso_8601": "2023-04-17T18:07:32.161617Z",
"url": "https://files.pythonhosted.org/packages/e3/8d/306c6e7cfc50d6eaeec3b77eb92a72481bc3697e5ce05f0b881a410d170f/pht_train_container_library-2.0.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "75010b2b51a9d785aeb99ed39bab72a647f9e0509e56c6e5269612a3c218ffa4",
"md5": "af978a8229d6def5a88c8fd9d7145a1d",
"sha256": "9dbb1c877c947caeb23a53953e6804dae3a40c87d0555041b27205615afd986b"
},
"downloads": -1,
"filename": "pht_train_container_library-2.0.5.tar.gz",
"has_sig": false,
"md5_digest": "af978a8229d6def5a88c8fd9d7145a1d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4.0",
"size": 38360,
"upload_time": "2023-04-17T18:07:34",
"upload_time_iso_8601": "2023-04-17T18:07:34.317290Z",
"url": "https://files.pythonhosted.org/packages/75/01/0b2b51a9d785aeb99ed39bab72a647f9e0509e56c6e5269612a3c218ffa4/pht_train_container_library-2.0.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-04-17 18:07:34",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "pht-train-container-library"
}
