# TAF (The Archive Framework)
TAF is a framework that aims to provide archival authentication
and ensure that Git repositories can be securely cloned/updated. TAF's implementation
strongly relies on [The Update Framework (TUF)](https://theupdateframework.github.io),
which helps developers maintain the security of a software update system. It provides a
flexible framework and specification that developers can integrate into any software update
system. TAF integrates Git with TUF:
- TUF targets were modified to authenticate Git commits instead of individual files.
This reduces the metadata size and simplifies authentication.
- The TUF metadata repository storage utilizes Git. That means TUF metadata files are
stored in a Git repository, which is referred to as an authentication repository.
When a TAF authentication repository is cloned, all target repositories are also cloned, and
TUF validation is performed against every commit since the repository's inception. When a TAF
repository is updated, data is fetched from upstream and each commit is authenticated. A TAF
clone/update differs from a standard Git clone/fetch in that remote commits aren't added to the
local Git repositories until they've been fully authenticated locally. TAF can be used to secure
any git repository, regardless of its content.
## Threats
A git repository can be compromised in several ways:
- An attacker might hack a user's account on a code hosting platform, like GitHub or GitLab.
- An attacker might compromise the hosting platform itself.
- An attacker might gain access to a developer's personal computer.
Such an attacker could then:
- Upload a new GPG key to GitHub.
- Push new commits to any repository.
- Add another authorized user with write access.
- Unprotect the master branch of any repository and force-push to it.
TAF's primary objective is not to prevent the attacks listed above but rather to detect when
an attack has occurred and halt an update if necessary. Thus, TAF should be used instead of
directly calling `git pull` and `git clone`.
## Further reading
1. [UELMA whitepaper](whitepapers/UELMA-Open-Law-White-Paper.pdf)
1. [TAF implementation and integration with TUF](docs/updater/specification.md)
## Installation Steps
From _PyPI_
```bash
pip install taf
```
From source:
```bash
pip install -e .
```
Install extra dependencies when using _Yubikey_:
```bash
pip install taf[yubikey]
```
Add bash completion:
1. copy `taf-complete.sh` to user's directory
1. add `source ./taf-complete.sh` to `~/.bash_profile` or `~/.bashrc`
1. source `~/.bash_profile`
## Development Setup
We are using [pre-commit](https://pre-commit.com/) to run _black_ code formatter, _flake8_ and _bandit_ code quality checks,
as well as _Mypy_ static type checker.
```bash
pip install -e .[dev]
pip install -e .[test]
pre-commit install # registers git pre-commit hook
pre-commit run --all-files # runs code formatting and quality checks for all files
```
NOTE: For _Windows_ users: Open [settings.json](.vscode/settings.json) and replace paths.
## Running Tests
To run tests with mocked Yubikey:
```bash
pytest
```
To run tests with real Yubikey:
1. Insert **test** Yubikey
2. Run `taf setup_test_key`
WARNING: This command will import targets private key to signature slot of your Yubikey, as well as new self-signed x509 certificate!
3. Run `REAL_YK=True pytest` or `set REAL_YK=True pytest` depending on platform.
## Platform-specific Wheels
1. Open https://dev.azure.com/openlawlibrary/TAF/_build
2. Click on latest build
3. Open _Summary_ tab
4. Under _Build artifacts published_, click on \*wheels to download zip
More info in [devops document](./docs/devops.md).
## Building Wheels on Ubuntu 16.04 and 18.04
**Binary wheels exists only for macOS, windows-32bit and windows-64bit platforms for python 3.10!**
- Install dependencies
```bash
sudo add-apt-repository ppa:jonathonf/python-3.10
sudo apt-get update
sudo apt-get install python3.10
sudo apt-get install python3.10-venv
sudo apt-get install python3.10-dev
sudo apt-get install swig
sudo apt-get install libpcsclite-dev
sudo apt-get install libssl-dev
sudo apt-get install libykpers-1-dev
```
- Create virtual environment
```bash
python3.10 -m venv env
pip install --upgrade pip
pip install wheel
pip install taf
```
- Test CLI
```bash
taf
```
## Acknowledgements
This project was made possible in part by the Institute of Museum and Library Services [(LG-246285-OLS-20)](https://www.imls.gov/grants/awarded/lg-246285-ols-20)
Raw data
{
"_id": null,
"home_page": "https://github.com/openlawlibrary/taf/tree/master",
"name": "taf",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "update updater secure authentication archival",
"author": "Open Law Library",
"author_email": "info@openlawlib.org",
"download_url": "https://files.pythonhosted.org/packages/df/2d/7fb994dc64fe8936c91cf7dc557cf6bf383ba3333a727a15c5a5624cfe24/taf-0.29.1.tar.gz",
"platform": null,
"description": "# TAF (The Archive Framework)\r\n\r\nTAF is a framework that aims to provide archival authentication\r\nand ensure that Git repositories can be securely cloned/updated. TAF's implementation\r\nstrongly relies on [The Update Framework (TUF)](https://theupdateframework.github.io),\r\nwhich helps developers maintain the security of a software update system. It provides a\r\nflexible framework and specification that developers can integrate into any software update\r\nsystem. TAF integrates Git with TUF:\r\n\r\n- TUF targets were modified to authenticate Git commits instead of individual files.\r\n This reduces the metadata size and simplifies authentication.\r\n- The TUF metadata repository storage utilizes Git. That means TUF metadata files are\r\n stored in a Git repository, which is referred to as an authentication repository.\r\n\r\nWhen a TAF authentication repository is cloned, all target repositories are also cloned, and\r\nTUF validation is performed against every commit since the repository's inception. When a TAF\r\nrepository is updated, data is fetched from upstream and each commit is authenticated. A TAF\r\nclone/update differs from a standard Git clone/fetch in that remote commits aren't added to the\r\nlocal Git repositories until they've been fully authenticated locally. TAF can be used to secure\r\nany git repository, regardless of its content.\r\n\r\n## Threats\r\n\r\nA git repository can be compromised in several ways:\r\n\r\n- An attacker might hack a user's account on a code hosting platform, like GitHub or GitLab.\r\n- An attacker might compromise the hosting platform itself.\r\n- An attacker might gain access to a developer's personal computer.\r\n\r\nSuch an attacker could then:\r\n\r\n- Upload a new GPG key to GitHub.\r\n- Push new commits to any repository.\r\n- Add another authorized user with write access.\r\n- Unprotect the master branch of any repository and force-push to it.\r\n\r\nTAF's primary objective is not to prevent the attacks listed above but rather to detect when\r\nan attack has occurred and halt an update if necessary. Thus, TAF should be used instead of\r\ndirectly calling `git pull` and `git clone`.\r\n\r\n\r\n## Further reading\r\n\r\n1. [UELMA whitepaper](whitepapers/UELMA-Open-Law-White-Paper.pdf)\r\n1. [TAF implementation and integration with TUF](docs/updater/specification.md)\r\n\r\n## Installation Steps\r\n\r\nFrom _PyPI_\r\n\r\n```bash\r\npip install taf\r\n```\r\n\r\nFrom source:\r\n\r\n```bash\r\npip install -e .\r\n```\r\n\r\nInstall extra dependencies when using _Yubikey_:\r\n\r\n```bash\r\npip install taf[yubikey]\r\n```\r\n\r\nAdd bash completion:\r\n\r\n1. copy `taf-complete.sh` to user's directory\r\n1. add `source ./taf-complete.sh` to `~/.bash_profile` or `~/.bashrc`\r\n1. source `~/.bash_profile`\r\n\r\n## Development Setup\r\n\r\nWe are using [pre-commit](https://pre-commit.com/) to run _black_ code formatter, _flake8_ and _bandit_ code quality checks,\r\nas well as _Mypy_ static type checker.\r\n\r\n```bash\r\npip install -e .[dev]\r\npip install -e .[test]\r\n\r\npre-commit install # registers git pre-commit hook\r\n\r\npre-commit run --all-files # runs code formatting and quality checks for all files\r\n```\r\n\r\nNOTE: For _Windows_ users: Open [settings.json](.vscode/settings.json) and replace paths.\r\n\r\n## Running Tests\r\n\r\nTo run tests with mocked Yubikey:\r\n\r\n```bash\r\npytest\r\n```\r\n\r\nTo run tests with real Yubikey:\r\n\r\n1. Insert **test** Yubikey\r\n2. Run `taf setup_test_key`\r\n WARNING: This command will import targets private key to signature slot of your Yubikey, as well as new self-signed x509 certificate!\r\n3. Run `REAL_YK=True pytest` or `set REAL_YK=True pytest` depending on platform.\r\n\r\n## Platform-specific Wheels\r\n\r\n1. Open https://dev.azure.com/openlawlibrary/TAF/_build\r\n2. Click on latest build\r\n3. Open _Summary_ tab\r\n4. Under _Build artifacts published_, click on \\*wheels to download zip\r\n\r\nMore info in [devops document](./docs/devops.md).\r\n\r\n## Building Wheels on Ubuntu 16.04 and 18.04\r\n\r\n**Binary wheels exists only for macOS, windows-32bit and windows-64bit platforms for python 3.10!**\r\n\r\n- Install dependencies\r\n\r\n```bash\r\nsudo add-apt-repository ppa:jonathonf/python-3.10\r\nsudo apt-get update\r\nsudo apt-get install python3.10\r\nsudo apt-get install python3.10-venv\r\nsudo apt-get install python3.10-dev\r\nsudo apt-get install swig\r\nsudo apt-get install libpcsclite-dev\r\nsudo apt-get install libssl-dev\r\nsudo apt-get install libykpers-1-dev\r\n```\r\n\r\n- Create virtual environment\r\n\r\n```bash\r\npython3.10 -m venv env\r\npip install --upgrade pip\r\npip install wheel\r\npip install taf\r\n```\r\n\r\n- Test CLI\r\n\r\n```bash\r\ntaf\r\n```\r\n\r\n\r\n## Acknowledgements\r\n\r\nThis project was made possible in part by the Institute of Museum and Library Services [(LG-246285-OLS-20)](https://www.imls.gov/grants/awarded/lg-246285-ols-20)\r\n",
"bugtrack_url": null,
"license": "",
"summary": "Implementation of archival authentication",
"version": "0.29.1",
"project_urls": {
"Homepage": "https://github.com/openlawlibrary/taf/tree/master"
},
"split_keywords": [
"update",
"updater",
"secure",
"authentication",
"archival"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "01f49dbf15ebb8248d3373bd7f20736dff7b3966bc3da57ff458db382befb104",
"md5": "72db7a03be49d91ad7fb371e8f3f9590",
"sha256": "9a5df940a3a229196c14a0d4f22766251af33d695dd2ac3cb231849928dbfc3a"
},
"downloads": -1,
"filename": "taf-0.29.1-cp310-cp310-macosx_11_0_x86_64.whl",
"has_sig": false,
"md5_digest": "72db7a03be49d91ad7fb371e8f3f9590",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": null,
"size": 533526,
"upload_time": "2024-02-07T17:06:07",
"upload_time_iso_8601": "2024-02-07T17:06:07.477296Z",
"url": "https://files.pythonhosted.org/packages/01/f4/9dbf15ebb8248d3373bd7f20736dff7b3966bc3da57ff458db382befb104/taf-0.29.1-cp310-cp310-macosx_11_0_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9f9b76a4f52187c594914de29cb5a7a920d7e4d15d295637d1598f2793cce341",
"md5": "8a633a3ecde6751ef01f148eafb0bdb2",
"sha256": "975f5b9d3fde50d2d15f23cb550197160857f9b2fefee6082d2d38dfb60298af"
},
"downloads": -1,
"filename": "taf-0.29.1-cp310-cp310-win32.whl",
"has_sig": false,
"md5_digest": "8a633a3ecde6751ef01f148eafb0bdb2",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": null,
"size": 731682,
"upload_time": "2024-02-07T16:52:18",
"upload_time_iso_8601": "2024-02-07T16:52:18.382374Z",
"url": "https://files.pythonhosted.org/packages/9f/9b/76a4f52187c594914de29cb5a7a920d7e4d15d295637d1598f2793cce341/taf-0.29.1-cp310-cp310-win32.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5270f85956eb4db910a15d31740e84cba8efe0606a4ad4c6fb36b2ed27b00ee2",
"md5": "191f4ebeceb8f25fd4dd56d51567ab13",
"sha256": "8e101f298884f5708ad21f338bf7dd81c8587493f5f2efa503e881b90bb292a0"
},
"downloads": -1,
"filename": "taf-0.29.1-cp310-cp310-win_amd64.whl",
"has_sig": false,
"md5_digest": "191f4ebeceb8f25fd4dd56d51567ab13",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": null,
"size": 791681,
"upload_time": "2024-02-07T16:56:02",
"upload_time_iso_8601": "2024-02-07T16:56:02.436462Z",
"url": "https://files.pythonhosted.org/packages/52/70/f85956eb4db910a15d31740e84cba8efe0606a4ad4c6fb36b2ed27b00ee2/taf-0.29.1-cp310-cp310-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "67f5a39790a261ec9f486a4d2f966867cc0e1ff26aed6493e31914cb47f3ec1a",
"md5": "5d2fdd0bbdbb89eea78acea30b66dd39",
"sha256": "f647cbfd7c225ea062fcd6f3de35689210d2835f71db3dabd4ee36369105cf6e"
},
"downloads": -1,
"filename": "taf-0.29.1-cp37-cp37m-macosx_11_0_x86_64.whl",
"has_sig": false,
"md5_digest": "5d2fdd0bbdbb89eea78acea30b66dd39",
"packagetype": "bdist_wheel",
"python_version": "cp37",
"requires_python": null,
"size": 533521,
"upload_time": "2024-02-07T17:06:18",
"upload_time_iso_8601": "2024-02-07T17:06:18.692939Z",
"url": "https://files.pythonhosted.org/packages/67/f5/a39790a261ec9f486a4d2f966867cc0e1ff26aed6493e31914cb47f3ec1a/taf-0.29.1-cp37-cp37m-macosx_11_0_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "442f719e5875a418ea5d44be4088a7c5522999f635acd9646661fb9434c8a593",
"md5": "1d1498536a79d92364d287752dd529b6",
"sha256": "cc33e698f93facd82fe4dc31f28e3d3e0548ba82a8ee395ed860806c9963a723"
},
"downloads": -1,
"filename": "taf-0.29.1-cp37-cp37m-win32.whl",
"has_sig": false,
"md5_digest": "1d1498536a79d92364d287752dd529b6",
"packagetype": "bdist_wheel",
"python_version": "cp37",
"requires_python": null,
"size": 731676,
"upload_time": "2024-02-07T16:54:16",
"upload_time_iso_8601": "2024-02-07T16:54:16.054568Z",
"url": "https://files.pythonhosted.org/packages/44/2f/719e5875a418ea5d44be4088a7c5522999f635acd9646661fb9434c8a593/taf-0.29.1-cp37-cp37m-win32.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "895ecd1a9e9520db7a37f0c68f92cbf222f90d43b594ac169bab006c7cc11934",
"md5": "7aa727abfe0fb02f90a8db661d72b7ff",
"sha256": "32b5cc3e52b23c6c62d68f6cdfde4adef3d0894f59761818bc6b6676390ed8a7"
},
"downloads": -1,
"filename": "taf-0.29.1-cp37-cp37m-win_amd64.whl",
"has_sig": false,
"md5_digest": "7aa727abfe0fb02f90a8db661d72b7ff",
"packagetype": "bdist_wheel",
"python_version": "cp37",
"requires_python": null,
"size": 791679,
"upload_time": "2024-02-07T16:56:24",
"upload_time_iso_8601": "2024-02-07T16:56:24.361635Z",
"url": "https://files.pythonhosted.org/packages/89/5e/cd1a9e9520db7a37f0c68f92cbf222f90d43b594ac169bab006c7cc11934/taf-0.29.1-cp37-cp37m-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "358a0c59ab1c35e46c569a2dcfb330d8b8386f1b9a5dcf675c88e74fc277b95e",
"md5": "7a962feb78e87d577af8f05aa8f8f725",
"sha256": "e41c2a9bc42bcbf076bc81a02e8bad4f5e48035e4478a4eaafa85775f1cc8190"
},
"downloads": -1,
"filename": "taf-0.29.1-cp38-cp38-macosx_11_0_x86_64.whl",
"has_sig": false,
"md5_digest": "7a962feb78e87d577af8f05aa8f8f725",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": null,
"size": 533524,
"upload_time": "2024-02-07T17:09:10",
"upload_time_iso_8601": "2024-02-07T17:09:10.180535Z",
"url": "https://files.pythonhosted.org/packages/35/8a/0c59ab1c35e46c569a2dcfb330d8b8386f1b9a5dcf675c88e74fc277b95e/taf-0.29.1-cp38-cp38-macosx_11_0_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5c449f975b565cfa3a01976b3d430adb14f043eb4135e1ce43ebc3d5aec03d9b",
"md5": "ef484d1f960dcdd4bd24ef0bbd6aaa51",
"sha256": "07799faf1fae53d9b64f16e6549c4b0e9d88010833951f8c143a42833fc20cce"
},
"downloads": -1,
"filename": "taf-0.29.1-cp38-cp38-win32.whl",
"has_sig": false,
"md5_digest": "ef484d1f960dcdd4bd24ef0bbd6aaa51",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": null,
"size": 731673,
"upload_time": "2024-02-07T16:54:23",
"upload_time_iso_8601": "2024-02-07T16:54:23.793029Z",
"url": "https://files.pythonhosted.org/packages/5c/44/9f975b565cfa3a01976b3d430adb14f043eb4135e1ce43ebc3d5aec03d9b/taf-0.29.1-cp38-cp38-win32.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9ce54324cce76967d12229ffa4789759dc5415a938e6ccfe3ee423b25d8bac51",
"md5": "c5109a706b206874247c34097fe827b5",
"sha256": "831306cce69003d07a260136b70e1930261e2bbee48ec26311a49fd59285e9d2"
},
"downloads": -1,
"filename": "taf-0.29.1-cp38-cp38-win_amd64.whl",
"has_sig": false,
"md5_digest": "c5109a706b206874247c34097fe827b5",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": null,
"size": 791676,
"upload_time": "2024-02-07T16:57:11",
"upload_time_iso_8601": "2024-02-07T16:57:11.165240Z",
"url": "https://files.pythonhosted.org/packages/9c/e5/4324cce76967d12229ffa4789759dc5415a938e6ccfe3ee423b25d8bac51/taf-0.29.1-cp38-cp38-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "6a8616985cff79505287cc389b3b6eac195389944a5d91f4b4a28233967388a5",
"md5": "002d044e39053f5ceda59f18754f4e0e",
"sha256": "5e781db86f2599eed2da390cb9350ea14ac0da5613b543aa329cb9b99cc0e270"
},
"downloads": -1,
"filename": "taf-0.29.1-cp39-cp39-win32.whl",
"has_sig": false,
"md5_digest": "002d044e39053f5ceda59f18754f4e0e",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": null,
"size": 731676,
"upload_time": "2024-02-07T16:55:04",
"upload_time_iso_8601": "2024-02-07T16:55:04.253884Z",
"url": "https://files.pythonhosted.org/packages/6a/86/16985cff79505287cc389b3b6eac195389944a5d91f4b4a28233967388a5/taf-0.29.1-cp39-cp39-win32.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d8499cc55ce892187efc2731103a6b592ec376d89777669c78e1526bffb8e420",
"md5": "a8e007a7c126d044c9d5fe2e8e3aabda",
"sha256": "149f39f9fb036b874eb32a4dff75bb0f06baebfeab408aba937d2353d12c558f"
},
"downloads": -1,
"filename": "taf-0.29.1-cp39-cp39-win_amd64.whl",
"has_sig": false,
"md5_digest": "a8e007a7c126d044c9d5fe2e8e3aabda",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": null,
"size": 791679,
"upload_time": "2024-02-07T16:57:30",
"upload_time_iso_8601": "2024-02-07T16:57:30.154588Z",
"url": "https://files.pythonhosted.org/packages/d8/49/9cc55ce892187efc2731103a6b592ec376d89777669c78e1526bffb8e420/taf-0.29.1-cp39-cp39-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "df2d7fb994dc64fe8936c91cf7dc557cf6bf383ba3333a727a15c5a5624cfe24",
"md5": "9f57ff528364979fb7a54e32cfafef1b",
"sha256": "5eb9af03e97faaa2bdcf53a4033474110caaab4432d37cd1ddc8c8f17154ad6d"
},
"downloads": -1,
"filename": "taf-0.29.1.tar.gz",
"has_sig": false,
"md5_digest": "9f57ff528364979fb7a54e32cfafef1b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 651463,
"upload_time": "2024-02-07T16:52:23",
"upload_time_iso_8601": "2024-02-07T16:52:23.554081Z",
"url": "https://files.pythonhosted.org/packages/df/2d/7fb994dc64fe8936c91cf7dc557cf6bf383ba3333a727a15c5a5624cfe24/taf-0.29.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-07 16:52:23",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "openlawlibrary",
"github_project": "taf",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "taf"
}
JSON
