| Name | toolforge-weld JSON |
| Version |
1.6.6
JSON |
| download |
| home_page | None |
| Summary | Shared Python code for Toolforge infrastructure components |
| upload_time | 2025-02-07 01:31:52 |
| maintainer | None |
| docs_url | None |
| author | Taavi Väänänen |
| requires_python | >=3.9 |
| license | AGPL-3.0-or-later |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# python3-toolforge-weld
Shared Python code for Toolforge infrastructure components.
## Local development environment (guideline)
### Testing with tox on debian
Clone the repo (instructions here https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-weld).
Install tox (this is the only debian-specific part):
```
~:$ apt install tox
```
Move to the directory where you cloned the repo, and run tox:
```
/path/to/repo/toolforge-weld:$ tox
```
That will run the tests and create a virtualenv that you can use to manually debug anything you need, to enter it:
```
/path/to/repo/toolforge-weld:$ source .tox/py3-tests/bin/activate
```
## Building the debian packages
The process will be:
* Create new branch
* Bump the version
* Build and deploy the package
* Upload the package to the toolforge repositories
* Merge PR and Create a release
Let's get started!
### Create new branch
To get started, create a new branch from main:
```
~:$ git checkout -b <new-branch-name>
```
### Bump the version
#### Update the changelog and setup.py
1. To do so, you can run the script:
```
~:$ utils/bump_version.sh
```
That will:
* create an entry in `debian/changelog` from the git log since the last tag
* bump the version in `setup.py` too
* create the required commit and tag.
2. Once this is done, you should push the new commit and tag for review:
```
git push -u origin <new-branch-name>
git push origin <new_version>
```
You can find out the value of the newly created tag by looking at your terminal output or running `git tags --list`
#### Create a patch and get it reviewed
On gitlab you should create a patch based on the above PR, then Review the `changelog` and the `setup.py` changes to make sure it's what you want (it uses your name, email, etc.), and ask
for reviews.
### Build and deploy the package
#### Build and deploy with cookbook (Recommended)
> **NOTE**: Currently the PR created above needs to be open before you can use this cookbook. If you choose to use the more complicated alternatives below, keeping the PR open is not mandatory.
1. Building and deploying the package has been greatly simplified using the cookbook. To do this simply run:
```
~:$ cookbook wmcs.toolforge.component.deploy --cluster-name toolsbeta --component toolforge-weld --git-branch bump_version
```
The above builds the package, uploads it to the toolsbeta package repository, and installs it on all the toolsbeta bastions. To do the same for tools use `--cluster-name tools`.
See https://wikitech.wikimedia.org/wiki/Spicerack/Cookbooks for details on cookbooks.
#### Build and deploy with containers
> **NOTE**: This will not allow you to sign your package, so if you need that try using the manual process.
1. You can build the package with:
```
path/to/repo/toolforge-weld:$ utils/build_deb.sh
```
The first time it might take a bit more time as it will build the core image to build packages, downloading many
dependencies. The next run it will not need to download all those dependencies, so it will be way faster.
> **NOTE**: If it failed when installing packages, try passing `--no-cache` to force rebuilding the cached layers.
2. After building, you'll need to upload the built package to toolforge package repository. See [`Uploading to the toolforge repository`](#uploading-to-the-toolforge-repository) for more details.
3. Once you are done uploading, you also need to login to the various bastions on both tools and toolsbeta to manually install the package there.
For example to install on toolsbeta bastion 6:
```
~:$ ssh toolsbeta-bastion-6.toolsbeta.eqiad1.wikimedia.cloud
~:$ sudo apt-get update && sudo apt-get install python3-toolforge-weld
```
It is important to check how many bastions we have for both tools and toolsbeta and do this for all. You can ask the toolforge team if you don't have this information.
#### Build and deploy with wmcs-package-build script
1. Another alternative is using the wmcs-package-build.py script that you can find in
the operations/puppet repo at modules/toolforge/files
```
$ ./wmcs-package-build.py --git-repo https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-weld -a buster-toolsbeta -a bullseye-toolsbeta --git-branch main --build-dist=bullseye --backports --toolforge-repo=tools
```
The script will SSH into a build server, build the package there, and publish it
to two repos: `buster-toolsbeta` and `bullseye-tooslbeta`.
The additional params `--backports, --toolforge-repo=tools
--build-dist=bullseye` are necessary because the build requires Poetry and other
build tools not available in the buster repos.
2. If that command is successful, you should then copy the package from the
"toolsbeta" to the "tools" distribution. See [`Uploading to the toolforge repository`](#uploading-to-the-toolforge-repository) for more details.
3. Once you are done uploading, you also need to login to the various bastions on both tools and toolsbeta to manually install the package there.
For example to install on toolsbeta bastion 6:
```
~:$ ssh toolsbeta-bastion-6.toolsbeta.eqiad1.wikimedia.cloud
~:$ sudo apt-get update && sudo apt-get install python3-toolforge-weld
```
It is important to check how many bastions we have for both tools and toolsbeta and do this for all. You can ask the toolforge team if you don't have this information.
Additional documentation on the wmcs-package-build script is available at
https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Packaging#wmcs-package-build
### Uploading to the toolforge repository
If you built the package using any of the manual methods, you can uploade it following:
https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Packaging#Uploading_a_package
### Merge PR and Create a release
Depending on the deployment method you chose, the PR might still be open. If that's the case remember to merge the PR and create a new Gitlab release.
Raw data
{
"_id": null,
"home_page": null,
"name": "toolforge-weld",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": null,
"author": "Taavi V\u00e4\u00e4n\u00e4nen",
"author_email": "hi@taavi.wtf",
"download_url": "https://files.pythonhosted.org/packages/af/f2/d395cd757863b051abc3b07cb47d1faf012de8e4b1af0a91199a1f7d7747/toolforge_weld-1.6.6.tar.gz",
"platform": null,
"description": "# python3-toolforge-weld\n\nShared Python code for Toolforge infrastructure components.\n\n## Local development environment (guideline)\n\n### Testing with tox on debian\n\nClone the repo (instructions here https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-weld).\n\nInstall tox (this is the only debian-specific part):\n```\n~:$ apt install tox\n```\n\nMove to the directory where you cloned the repo, and run tox:\n```\n/path/to/repo/toolforge-weld:$ tox\n```\n\nThat will run the tests and create a virtualenv that you can use to manually debug anything you need, to enter it:\n```\n/path/to/repo/toolforge-weld:$ source .tox/py3-tests/bin/activate\n```\n\n## Building the debian packages\n\nThe process will be:\n* Create new branch\n* Bump the version\n* Build and deploy the package\n* Upload the package to the toolforge repositories\n* Merge PR and Create a release\n\n\n\nLet's get started!\n\n### Create new branch\nTo get started, create a new branch from main:\n```\n~:$ git checkout -b <new-branch-name>\n```\n\n### Bump the version\n#### Update the changelog and setup.py\n1. To do so, you can run the script:\n ```\n ~:$ utils/bump_version.sh\n ```\n\n That will:\n\n * create an entry in `debian/changelog` from the git log since the last tag\n * bump the version in `setup.py` too\n * create the required commit and tag.\n\n2. Once this is done, you should push the new commit and tag for review:\n ```\n git push -u origin <new-branch-name>\n git push origin <new_version>\n ```\n You can find out the value of the newly created tag by looking at your terminal output or running `git tags --list`\n\n#### Create a patch and get it reviewed\n\nOn gitlab you should create a patch based on the above PR, then Review the `changelog` and the `setup.py` changes to make sure it's what you want (it uses your name, email, etc.), and ask\nfor reviews.\n\n### Build and deploy the package\n#### Build and deploy with cookbook (Recommended)\n\n> **NOTE**: Currently the PR created above needs to be open before you can use this cookbook. If you choose to use the more complicated alternatives below, keeping the PR open is not mandatory.\n\n1. Building and deploying the package has been greatly simplified using the cookbook. To do this simply run:\n ```\n ~:$ cookbook wmcs.toolforge.component.deploy --cluster-name toolsbeta --component toolforge-weld --git-branch bump_version\n ```\n The above builds the package, uploads it to the toolsbeta package repository, and installs it on all the toolsbeta bastions. To do the same for tools use `--cluster-name tools`.\n\n See https://wikitech.wikimedia.org/wiki/Spicerack/Cookbooks for details on cookbooks.\n\n#### Build and deploy with containers\n> **NOTE**: This will not allow you to sign your package, so if you need that try using the manual process.\n\n1. You can build the package with:\n ```\n path/to/repo/toolforge-weld:$ utils/build_deb.sh\n ```\n The first time it might take a bit more time as it will build the core image to build packages, downloading many\n dependencies. The next run it will not need to download all those dependencies, so it will be way faster.\n\n > **NOTE**: If it failed when installing packages, try passing `--no-cache` to force rebuilding the cached layers.\n\n2. After building, you'll need to upload the built package to toolforge package repository. See [`Uploading to the toolforge repository`](#uploading-to-the-toolforge-repository) for more details.\n\n3. Once you are done uploading, you also need to login to the various bastions on both tools and toolsbeta to manually install the package there.\n For example to install on toolsbeta bastion 6:\n ```\n ~:$ ssh toolsbeta-bastion-6.toolsbeta.eqiad1.wikimedia.cloud\n ~:$ sudo apt-get update && sudo apt-get install python3-toolforge-weld\n ```\n It is important to check how many bastions we have for both tools and toolsbeta and do this for all. You can ask the toolforge team if you don't have this information.\n\n\n\n\n#### Build and deploy with wmcs-package-build script\n1. Another alternative is using the wmcs-package-build.py script that you can find in\nthe operations/puppet repo at modules/toolforge/files\n\n ```\n $ ./wmcs-package-build.py --git-repo https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-weld -a buster-toolsbeta -a bullseye-toolsbeta --git-branch main --build-dist=bullseye --backports --toolforge-repo=tools\n ```\n\n The script will SSH into a build server, build the package there, and publish it\n to two repos: `buster-toolsbeta` and `bullseye-tooslbeta`.\n\n The additional params `--backports, --toolforge-repo=tools\n --build-dist=bullseye` are necessary because the build requires Poetry and other\n build tools not available in the buster repos.\n\n2. If that command is successful, you should then copy the package from the\n\"toolsbeta\" to the \"tools\" distribution. See [`Uploading to the toolforge repository`](#uploading-to-the-toolforge-repository) for more details.\n\n3. Once you are done uploading, you also need to login to the various bastions on both tools and toolsbeta to manually install the package there.\n For example to install on toolsbeta bastion 6:\n ```\n ~:$ ssh toolsbeta-bastion-6.toolsbeta.eqiad1.wikimedia.cloud\n ~:$ sudo apt-get update && sudo apt-get install python3-toolforge-weld\n ```\n It is important to check how many bastions we have for both tools and toolsbeta and do this for all. You can ask the toolforge team if you don't have this information.\n\nAdditional documentation on the wmcs-package-build script is available at\nhttps://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Packaging#wmcs-package-build\n\n\n### Uploading to the toolforge repository\n\nIf you built the package using any of the manual methods, you can uploade it following:\nhttps://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Packaging#Uploading_a_package\n\n### Merge PR and Create a release\nDepending on the deployment method you chose, the PR might still be open. If that's the case remember to merge the PR and create a new Gitlab release.\n",
"bugtrack_url": null,
"license": "AGPL-3.0-or-later",
"summary": "Shared Python code for Toolforge infrastructure components",
"version": "1.6.6",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f8e325e50c2f6c703f9fa663f7cd66249be489b388e63691fc168d2359b603e6",
"md5": "1b864082d13f389607f3e09e86266c4e",
"sha256": "d2a1327178304ccaaf7bf0ecb4bf6590636b0c269fe3eb9f11396e097b586538"
},
"downloads": -1,
"filename": "toolforge_weld-1.6.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1b864082d13f389607f3e09e86266c4e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 28786,
"upload_time": "2025-02-07T01:31:50",
"upload_time_iso_8601": "2025-02-07T01:31:50.304285Z",
"url": "https://files.pythonhosted.org/packages/f8/e3/25e50c2f6c703f9fa663f7cd66249be489b388e63691fc168d2359b603e6/toolforge_weld-1.6.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "aff2d395cd757863b051abc3b07cb47d1faf012de8e4b1af0a91199a1f7d7747",
"md5": "305d37dccbf54a01b54098d67e812db7",
"sha256": "22f3ce199656c5bde28bce888b5934b88927955ab56f98ce66a5781557b8b161"
},
"downloads": -1,
"filename": "toolforge_weld-1.6.6.tar.gz",
"has_sig": false,
"md5_digest": "305d37dccbf54a01b54098d67e812db7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 31006,
"upload_time": "2025-02-07T01:31:52",
"upload_time_iso_8601": "2025-02-07T01:31:52.122551Z",
"url": "https://files.pythonhosted.org/packages/af/f2/d395cd757863b051abc3b07cb47d1faf012de8e4b1af0a91199a1f7d7747/toolforge_weld-1.6.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-07 01:31:52",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "toolforge-weld"
}
