# template-python
Python service template for reuse.
## Installation
<details>
<summary>Install Python 3.12 if it is not available in your package manager</summary>
These instructions are for Ubuntu 22.04. If you're on a different distribution,
or - God forbid! - Windows, you should adjust these accordingly.
Also, these instructions are about using Poetry with Pyenv-managed (non-system) Python.
### Step 1: Update and Install Dependencies
Before we install pyenv, we need to update our package lists for upgrades and new package installations. We also need to install dependencies for pyenv.
Open your terminal and type:
```bash
sudo apt-get update
sudo apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev \
libreadline-dev libsqlite3-dev wget curl llvm libncursesw5-dev xz-utils \
tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev
```
### Step 2: Install Pyenv
We will clone pyenv from the official GitHub repository and add it to our system path.
```bash
git clone https://github.com/pyenv/pyenv.git ~/.pyenv
echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bashrc
echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(pyenv init -)"' >> ~/.bashrc
exec "$SHELL"
```
### Step 3: Install Python 3.12
Now that pyenv is installed, we can install different Python versions. To install Python 3.12, use the following command:
```bash
pyenv install 3.12
```
### Step 4: Connect Poetry to it
Do this in the template dir. Pycharm will automatically connect to it later
```bash
poetry env use ~/.pyenv/versions/3.12.1/bin/python
```
(change the version number accordingly to what is installed)
Finally, verify that Poetry indeed is connected to the proper version:
```bash
poetry enf info
```
</details>
1. If you don't have `Poetry` installed run:
```bash
pip install poetry
```
2. Install dependencies:
```bash
poetry config virtualenvs.in-project true
poetry install --no-root --with dev,test
```
3. Install `pre-commit` hooks:
```bash
poetry run pre-commit install
```
4. Launch the project:
```bash
poetry run uvicorn app.main:app [--reload]
```
or do it in two steps:
```bash
poetry shell
uvicorn app.main:app
```
5. Running tests:
```bash
poetry run pytest
```
You can test the application for multiple versions of Python. To do this, you need to install the required Python versions on your operating system, specify these versions in the tox.ini file, and then run the tests:
```bash
poetry run tox
```
## Package
To generate and publish a package on pypi.org, execute the following commands:
```bash
poetry config pypi-token.pypi <pypi_token>
poetry build
poetry publish
```
pypi_token - API token for authentication on PyPI. https://pypi.org/help/#apitoken
## Docker
Build a docker image and run a container:
```bash
docker build . -t <image_name>:<image_tag>
docker run <image_name>:<image_tag>
```
Upload the Docker image to the repository:
```bash
docker login -u <username>
docker push <image_name>:<image_tag>
```
https://docs.docker.com/
## Helm chart
Authenticate your Helm client in the container registry:
```bash
helm registry login <repo_url> -u <username>
```
Create a Helm chart:
```bash
helm package charts/<chart_name>
```
Push the Helm chart to container registry:
```bash
helm push <helm_chart_package> <repo_url>
```
Deploy the Helm chart:
```bash
helm repo add <repo_name> <repo_url>
helm repo update <repo_name>
helm upgrade --install <release_name> <repo_name>/<chart_name>
```
https://helm.sh/ru/docs/
## OpenaAPI schema
To manually generate the OpenAPI schema, execute the command:
```bash
poetry run python ./tools/extract_openapi.py app.main:app --app-dir . --out openapi.yaml --app_version <version>
```
## Release
To create a release, add a tag in GIT with the format a.a.a, where 'a' is an integer.
The release version for branches, pull requests, and other tags will be generated based on the last tag of the form a.a.a.
## GitHub Actions
GitHub Actions triggers testing, builds, and application publishing for each release.
https://docs.github.com/en/actions
You can set up automatic testing in GitHub Actions for different versions of Python. To do this, you need to specify the set of versions in the .github/workflows/test_and_build.yaml file. For example:
```yaml
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12"]
```
The process of building and publishing differs for web services and libraries.
### Web service
The default build and publish process is configured for a web application (.github\workflows\build_web.yaml).
During this process, a Docker image is built, a Helm chart is created, an openapi.yaml is generated, and the web service is deployed to a Kubernetes cluster.
**Initial setup**
Create the branch gh-pages and use it as a GitHub page https://pages.github.com/.
Set up secrets at `https://github.com/<workspace>/<project>/settings/secrets/actions`:
1. DOCKER_IMAGE_NAME - The name of the Docker image for uploading to the repository.
2. DOCKER_USERNAME - The username for the Docker repository on https://hub.docker.com/.
3. DOCKER_PASSWORD - The password for the Docker repository.
4. AWS_ACCESS_KEY_ID - AWS Access Key ID. https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html
5. AWS_SECRET_ACCESS_KEY - AWS Secret Access Key
6. AWS_REGION - AWS region. https://aws.amazon.com/about-aws/global-infrastructure/regions_az/
7. EKS_CLUSTER_ROLE_ARN - The IAM role's ARN in AWS, providing permissions for managing an Amazon EKS Kubernetes cluster.
8. EKS_CLUSTER_NAME - Amazon EKS Kubernetes cluster name.
9. EKS_CLUSTER_NAMESPACE - Amazon EKS Kubernetes cluster namespace.
10. HELM_REPO_URL - `https://<workspace>.github.io/<project>/`
**After execution**
The OpenAPI schema will be available at `https://github.com/<workspace>/<project>/releases/`.
The index.yaml file containing the list of Helm charts will be available at `https://<workspace>.github.io/<project>/charts-repo/index.yaml`. You can this URL on https://artifacthub.io/.
### Library
To change the build process for the library, you need to replace the nested workflow `./.github/workflows/build_web.yaml` to `./.github/workflows/build_lib.yaml` in `.github/workflows/test_and_build.yaml`:
```yaml
build:
needs: [test]
secrets: inherit
uses: ./.github/workflows/build_lib.yaml
```
After that, during the build process, the package will be built and published on pypi.org.
Uploading the package to pypi.org only occurs when a.a.a release is created.
**Initial setup**
Set up a secret token for PyPI at `https://github.com/<workspace>/<project>/settings/secrets/actions`.
https://pypi.org/help/#apitoken
**After execution**
A package will be available at `https://github.com/<workspace>/<project>/releases/` and pypi.org.
## Act
You can run your GitHub Actions locally using https://github.com/nektos/act.
Usage example:
```bash
act push -j deploy --secret-file my.secrets
```
## Classy-FastAPI
Classy-FastAPI allows you to easily do dependency injection of
object instances that should persist between FastAPI routes invocations,
e.g. database connections.
More on that (with examples) at [Classy-FastAPI GitLab page](https://gitlab.com/companionlabs-opensource/classy-fastapi).
# Collaboration guidelines
HIRO uses and requires from its partners [GitFlow with Forks](https://hirodevops.notion.site/GitFlow-with-Forks-3b737784e4fc40eaa007f04aed49bb2e?pvs=4)
Raw data
{
"_id": null,
"home_page": "https://github.com/Shockedshodan/template-python-test",
"name": "template-python-test",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10,<4.0",
"maintainer_email": "",
"keywords": "",
"author": "HIRO-MicroDataCenters",
"author_email": "all-hiro@hiro-microdatacenters.nl",
"download_url": "https://files.pythonhosted.org/packages/d3/f5/6c62703c960c3526927a95e381dfc5a74f5daad0a318d0251b22b8639835/template_python_test-0.0.0.tar.gz",
"platform": null,
"description": "# template-python\nPython service template for reuse.\n\n## Installation\n<details>\n<summary>Install Python 3.12 if it is not available in your package manager</summary>\n\nThese instructions are for Ubuntu 22.04. If you're on a different distribution,\nor - God forbid! - Windows, you should adjust these accordingly.\n\nAlso, these instructions are about using Poetry with Pyenv-managed (non-system) Python.\n \n### Step 1: Update and Install Dependencies\nBefore we install pyenv, we need to update our package lists for upgrades and new package installations. We also need to install dependencies for pyenv. \n\nOpen your terminal and type:\n```bash\nsudo apt-get update\nsudo apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev \\\nlibreadline-dev libsqlite3-dev wget curl llvm libncursesw5-dev xz-utils \\\ntk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev\n```\n\n### Step 2: Install Pyenv\nWe will clone pyenv from the official GitHub repository and add it to our system path.\n```bash\ngit clone https://github.com/pyenv/pyenv.git ~/.pyenv\necho 'export PYENV_ROOT=\"$HOME/.pyenv\"' >> ~/.bashrc\necho 'export PATH=\"$PYENV_ROOT/bin:$PATH\"' >> ~/.bashrc\necho 'eval \"$(pyenv init -)\"' >> ~/.bashrc\nexec \"$SHELL\"\n```\n\n### Step 3: Install Python 3.12\nNow that pyenv is installed, we can install different Python versions. To install Python 3.12, use the following command:\n```bash\npyenv install 3.12\n```\n\n### Step 4: Connect Poetry to it\nDo this in the template dir. Pycharm will automatically connect to it later\n```bash\npoetry env use ~/.pyenv/versions/3.12.1/bin/python\n```\n(change the version number accordingly to what is installed)\n\nFinally, verify that Poetry indeed is connected to the proper version:\n```bash\npoetry enf info\n```\n</details> \n\n1. If you don't have `Poetry` installed run:\n```bash\npip install poetry\n```\n\n2. Install dependencies:\n```bash\npoetry config virtualenvs.in-project true\npoetry install --no-root --with dev,test\n```\n\n3. Install `pre-commit` hooks:\n```bash\npoetry run pre-commit install\n```\n\n4. Launch the project:\n```bash\npoetry run uvicorn app.main:app [--reload]\n```\nor do it in two steps:\n```bash\npoetry shell\nuvicorn app.main:app\n```\n\n5. Running tests:\n```bash\npoetry run pytest\n```\n\nYou can test the application for multiple versions of Python. To do this, you need to install the required Python versions on your operating system, specify these versions in the tox.ini file, and then run the tests:\n```bash\npoetry run tox\n```\n\n## Package\nTo generate and publish a package on pypi.org, execute the following commands:\n```bash\npoetry config pypi-token.pypi <pypi_token>\npoetry build\npoetry publish\n```\n\npypi_token - API token for authentication on PyPI. https://pypi.org/help/#apitoken\n\n## Docker\nBuild a docker image and run a container:\n```bash\ndocker build . -t <image_name>:<image_tag>\ndocker run <image_name>:<image_tag>\n```\n\nUpload the Docker image to the repository:\n```bash\ndocker login -u <username>\ndocker push <image_name>:<image_tag>\n```\n\nhttps://docs.docker.com/\n\n## Helm chart\nAuthenticate your Helm client in the container registry:\n```bash\nhelm registry login <repo_url> -u <username>\n```\n\nCreate a Helm chart:\n```bash\nhelm package charts/<chart_name>\n```\n\nPush the Helm chart to container registry:\n```bash\nhelm push <helm_chart_package> <repo_url>\n```\n\nDeploy the Helm chart:\n```bash\nhelm repo add <repo_name> <repo_url>\nhelm repo update <repo_name>\nhelm upgrade --install <release_name> <repo_name>/<chart_name>\n```\n\nhttps://helm.sh/ru/docs/\n\n## OpenaAPI schema\nTo manually generate the OpenAPI schema, execute the command:\n```bash\npoetry run python ./tools/extract_openapi.py app.main:app --app-dir . --out openapi.yaml --app_version <version>\n```\n\n## Release\nTo create a release, add a tag in GIT with the format a.a.a, where 'a' is an integer.\nThe release version for branches, pull requests, and other tags will be generated based on the last tag of the form a.a.a.\n\n## GitHub Actions\nGitHub Actions triggers testing, builds, and application publishing for each release. \nhttps://docs.github.com/en/actions \n\nYou can set up automatic testing in GitHub Actions for different versions of Python. To do this, you need to specify the set of versions in the .github/workflows/test_and_build.yaml file. For example:\n```yaml\nstrategy:\n matrix:\n python-version: [\"3.10\", \"3.11\", \"3.12\"]\n```\n\nThe process of building and publishing differs for web services and libraries.\n\n### Web service\nThe default build and publish process is configured for a web application (.github\\workflows\\build_web.yaml).\nDuring this process, a Docker image is built, a Helm chart is created, an openapi.yaml is generated, and the web service is deployed to a Kubernetes cluster.\n\n**Initial setup** \nCreate the branch gh-pages and use it as a GitHub page https://pages.github.com/. \nSet up secrets at `https://github.com/<workspace>/<project>/settings/secrets/actions`:\n1. DOCKER_IMAGE_NAME - The name of the Docker image for uploading to the repository.\n2. DOCKER_USERNAME - The username for the Docker repository on https://hub.docker.com/.\n3. DOCKER_PASSWORD - The password for the Docker repository.\n4. AWS_ACCESS_KEY_ID - AWS Access Key ID. https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html\n5. AWS_SECRET_ACCESS_KEY - AWS Secret Access Key\n6. AWS_REGION - AWS region. https://aws.amazon.com/about-aws/global-infrastructure/regions_az/\n7. EKS_CLUSTER_ROLE_ARN - The IAM role's ARN in AWS, providing permissions for managing an Amazon EKS Kubernetes cluster.\n8. EKS_CLUSTER_NAME - Amazon EKS Kubernetes cluster name.\n9. EKS_CLUSTER_NAMESPACE - Amazon EKS Kubernetes cluster namespace.\n10. HELM_REPO_URL - `https://<workspace>.github.io/<project>/`\n\n**After execution** \nThe OpenAPI schema will be available at `https://github.com/<workspace>/<project>/releases/`. \nThe index.yaml file containing the list of Helm charts will be available at `https://<workspace>.github.io/<project>/charts-repo/index.yaml`. You can this URL on https://artifacthub.io/.\n\n### Library\nTo change the build process for the library, you need to replace the nested workflow `./.github/workflows/build_web.yaml` to `./.github/workflows/build_lib.yaml` in `.github/workflows/test_and_build.yaml`:\n```yaml\nbuild:\n needs: [test]\n secrets: inherit\n uses: ./.github/workflows/build_lib.yaml\n```\n\nAfter that, during the build process, the package will be built and published on pypi.org. \nUploading the package to pypi.org only occurs when a.a.a release is created.\n\n**Initial setup** \nSet up a secret token for PyPI at `https://github.com/<workspace>/<project>/settings/secrets/actions`.\nhttps://pypi.org/help/#apitoken\n\n**After execution** \nA package will be available at `https://github.com/<workspace>/<project>/releases/` and pypi.org. \n\n## Act\nYou can run your GitHub Actions locally using https://github.com/nektos/act. \n\nUsage example:\n```bash\nact push -j deploy --secret-file my.secrets\n```\n\n## Classy-FastAPI\nClassy-FastAPI allows you to easily do dependency injection of \nobject instances that should persist between FastAPI routes invocations,\ne.g. database connections.\nMore on that (with examples) at [Classy-FastAPI GitLab page](https://gitlab.com/companionlabs-opensource/classy-fastapi).\n\n# Collaboration guidelines\nHIRO uses and requires from its partners [GitFlow with Forks](https://hirodevops.notion.site/GitFlow-with-Forks-3b737784e4fc40eaa007f04aed49bb2e?pvs=4)\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Web service template in Python for reuse.",
"version": "0.0.0",
"project_urls": {
"Homepage": "https://github.com/Shockedshodan/template-python-test",
"Repository": "https://github.com/Shockedshodan/template-python-test"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "808f8ae84ce2aefffee4e7db74f7ce08bb65536b49ef924f15e3bbb4b0cd140e",
"md5": "22ccc9949f22cce3624b8d0c13feb7df",
"sha256": "0e88045f816204a83413375efe7e2ec20cf151d631f4f3533d5507982e09baeb"
},
"downloads": -1,
"filename": "template_python_test-0.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "22ccc9949f22cce3624b8d0c13feb7df",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10,<4.0",
"size": 7357,
"upload_time": "2024-02-28T16:54:17",
"upload_time_iso_8601": "2024-02-28T16:54:17.372764Z",
"url": "https://files.pythonhosted.org/packages/80/8f/8ae84ce2aefffee4e7db74f7ce08bb65536b49ef924f15e3bbb4b0cd140e/template_python_test-0.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d3f56c62703c960c3526927a95e381dfc5a74f5daad0a318d0251b22b8639835",
"md5": "979bff43c1b91d687f67668bcb38084e",
"sha256": "edf6292afc3ced8291f038f0b2cdbbb4ee161250f897d34ba4ff9b0823b0efc0"
},
"downloads": -1,
"filename": "template_python_test-0.0.0.tar.gz",
"has_sig": false,
"md5_digest": "979bff43c1b91d687f67668bcb38084e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10,<4.0",
"size": 10845,
"upload_time": "2024-02-28T16:54:19",
"upload_time_iso_8601": "2024-02-28T16:54:19.483483Z",
"url": "https://files.pythonhosted.org/packages/d3/f5/6c62703c960c3526927a95e381dfc5a74f5daad0a318d0251b22b8639835/template_python_test-0.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-28 16:54:19",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Shockedshodan",
"github_project": "template-python-test",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "template-python-test"
}
JSON
