# Docker Shaper
This is a spin-off package for the `docker-shaper` tool, which can be used to monitor Docker stuff
like containers, images and volumes and automatically enforce certain cleanup-rules.
## Installation
```sh
[<PYTHON> -m] pip[3] install [--user] [--upgrade] docker-shaper
```
## Usage
```
docker-shaper serve`
```
=> Navigate to e.g. http://my-build-node:5432/
## Development & Contribution
### Prerequisites
* Python 3.8.10+ (e.g. via `pyenv`)
* `poetry` and `pre-commit`
`python3 -m pip install --upgrade --user poetry pre-commit`
```sh
git clone https://github.com/Checkmk/checkmk-dev-tools
cd checkmk-dev-tools/additional_packages/docker_shaper
pre-commit install
# if you need a specific version of Python inside your dev environment
poetry env use ~/.pyenv/versions/3.8.10/bin/python3
poetry install
```
### Workflow
* (once and only for publishing to PyPi) Get token on PyPi.org
* (maybe) setup distinct repository setup `poetry config repositories.checkmk https://upload.pypi.org/legacy/`
* (once and only for publishing to PyPi) `poetry config pypi-token.checkmk pypi-<LONG-STRING>`
(will write to `~/.config/pypoetry/auth.toml`)
* modify and check commits via `pre-commit`
* after work is done locally:
- adapt version in `pyproject.toml`
- build and check a package
```sh
poetry build && \
twine check dist/* &&
python3 -m pip uninstall -y checkmk_dev_tools && \
python3 -m pip install --user dist/checkmk_dev_tools-$(grep -E "^version.?=" pyproject.toml | cut -d '"' -f 2)-py3-none-any.whl
```
- check installed package
- go through review process
- publish the new package `poetry publish --build --repository checkmk`
- commit new version && push
## Todo
- [x] Fix: stuck auto-reload: was: postpone=True + monitoring log
- [x] Fix: crawl-images should fix parents not having them listed
- [-] Fix: stuck crawl images (too many async requests, does not happen in production)
- [ ] Fix: `'677aff0727' could not be removed: DockerError(404, 'No such image: 677aff0727:latest')`
- [ ] Fix: `tried to remove container 4f5fb0848c unknown to us`
- [ ] Fix: Crashes (see runlog)
- [ ] Image update message only if needed
- [ ] New TUI
- make ongoing progress visible
- [ ] review log levels (too verbose)
- [ ] answer https://stackoverflow.com/questions/32723111
- [x] installable via `pip install`
- [x] Quart interface (instead of `flask`)
- [x] auto-apply changes to source and configuration files
- [x] outsourced config file
- [x] bring in features of former `dgcd`
- [x] bring in features of former `dockermon`
- [x] untag certain tags
- [x] container cleanup
- [x] Fix `none` image lookup
- [x] Exceptions to messages
- [x] Clip
- [x] Increase/decrease logging via web / signal
- [x] Link: cleanup (images/containers) now
- [x] Add volumes list (with recent owners)
- [x] Containers: Store CPU / Memory usage over time
- [x] Containers: store history
- [x] Persist messages
- [ ] Remove old message/container logs
- [ ] Show different color for unmatched images
- [ ] Warn about use of unpinned / upstream images
- [ ] Handle 'build cache objects' (found on system prune)
- [ ] Bring in volume monitoring: which volumes have been created and used by which containers?
- [ ] Containers: show total CPU usage
- [ ] Containers: list volumes
- [ ] Images: list parents / children
- [ ] Volumes: list usage
- [ ] Instructions to readme
- [ ] List unmatched / overmatched tags
- [ ] Links to `delete` / `remove`
- [ ] Links to jobs
- [ ] Link: inspect
- [ ] Graph: cpu / containers (idle/up)
- [ ] Authenticate (at least if we can modify behavior, like stopping/removing images/containers)
## Knowledge
(just misc links to articles that helped me out)
* [How to delete docker images from Nexus Repository 3](https://support.sonatype.com/hc/en-us/articles/360009696054-How-to-delete-docker-images-from-Nexus-Repository-3)
* [Showing Text Box On Hover (In Table)](https://stackoverflow.com/questions/52562345/showing-text-box-on-hover-in-table)
* [Beautiful Interactive Tables for your Flask Templates](https://blog.miguelgrinberg.com/post/beautiful-interactive-tables-for-your-flask-templates)
* https://github.com/torfsen/python-systemd-tutorial
* https://www.digitalocean.com/community/tutorials/how-to-use-templates-in-a-flask-application
* https://stackoverflow.com/questions/49957034/live-updating-dynamic-variable-on-html-with-flask
* https://pgjones.gitlab.io/quart/how_to_guides/templating.html
### Logging
* https://pgjones.gitlab.io/hypercorn/how_to_guides/logging.html#how-to-log
Raw data
{
"_id": null,
"home_page": "https://github.com/Checkmk/checkmk-dev-tools",
"name": "docker-shaper",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0.0,>=3.10.4",
"maintainer_email": null,
"keywords": null,
"author": "Frans F\u00fcrst",
"author_email": "frans.fuerst@checkmk.com",
"download_url": "https://files.pythonhosted.org/packages/c5/ab/719e48a5942af38c8c18364ad2b62d6817b2f6f38d89259fbf28ae6e5a05/docker_shaper-2.0.1.tar.gz",
"platform": null,
"description": "# Docker Shaper\n\nThis is a spin-off package for the `docker-shaper` tool, which can be used to monitor Docker stuff\nlike containers, images and volumes and automatically enforce certain cleanup-rules.\n\n\n## Installation\n\n```sh\n[<PYTHON> -m] pip[3] install [--user] [--upgrade] docker-shaper\n```\n\n\n## Usage\n\n```\ndocker-shaper serve`\n```\n\n=> Navigate to e.g. http://my-build-node:5432/\n\n\n## Development & Contribution\n\n### Prerequisites\n\n* Python 3.8.10+ (e.g. via `pyenv`)\n* `poetry` and `pre-commit`\n `python3 -m pip install --upgrade --user poetry pre-commit`\n\n```sh\ngit clone https://github.com/Checkmk/checkmk-dev-tools\ncd checkmk-dev-tools/additional_packages/docker_shaper\npre-commit install\n# if you need a specific version of Python inside your dev environment\npoetry env use ~/.pyenv/versions/3.8.10/bin/python3\npoetry install\n```\n\n\n### Workflow\n\n* (once and only for publishing to PyPi) Get token on PyPi.org\n* (maybe) setup distinct repository setup `poetry config repositories.checkmk https://upload.pypi.org/legacy/`\n* (once and only for publishing to PyPi) `poetry config pypi-token.checkmk pypi-<LONG-STRING>`\n (will write to `~/.config/pypoetry/auth.toml`)\n* modify and check commits via `pre-commit`\n* after work is done locally:\n - adapt version in `pyproject.toml`\n - build and check a package\n```sh\npoetry build && \\\ntwine check dist/* &&\npython3 -m pip uninstall -y checkmk_dev_tools && \\\npython3 -m pip install --user dist/checkmk_dev_tools-$(grep -E \"^version.?=\" pyproject.toml | cut -d '\"' -f 2)-py3-none-any.whl\n```\n - check installed package\n - go through review process\n - publish the new package `poetry publish --build --repository checkmk`\n - commit new version && push\n\n\n## Todo\n\n- [x] Fix: stuck auto-reload: was: postpone=True + monitoring log\n- [x] Fix: crawl-images should fix parents not having them listed\n- [-] Fix: stuck crawl images (too many async requests, does not happen in production)\n- [ ] Fix: `'677aff0727' could not be removed: DockerError(404, 'No such image: 677aff0727:latest')`\n- [ ] Fix: `tried to remove container 4f5fb0848c unknown to us`\n- [ ] Fix: Crashes (see runlog)\n- [ ] Image update message only if needed\n- [ ] New TUI\n - make ongoing progress visible\n- [ ] review log levels (too verbose)\n- [ ] answer https://stackoverflow.com/questions/32723111\n\n- [x] installable via `pip install`\n- [x] Quart interface (instead of `flask`)\n- [x] auto-apply changes to source and configuration files\n- [x] outsourced config file\n- [x] bring in features of former `dgcd`\n- [x] bring in features of former `dockermon`\n- [x] untag certain tags\n- [x] container cleanup\n- [x] Fix `none` image lookup\n- [x] Exceptions to messages\n- [x] Clip\n- [x] Increase/decrease logging via web / signal\n- [x] Link: cleanup (images/containers) now\n- [x] Add volumes list (with recent owners)\n- [x] Containers: Store CPU / Memory usage over time\n- [x] Containers: store history\n- [x] Persist messages\n- [ ] Remove old message/container logs\n- [ ] Show different color for unmatched images\n- [ ] Warn about use of unpinned / upstream images\n- [ ] Handle 'build cache objects' (found on system prune)\n- [ ] Bring in volume monitoring: which volumes have been created and used by which containers?\n- [ ] Containers: show total CPU usage\n- [ ] Containers: list volumes\n- [ ] Images: list parents / children\n- [ ] Volumes: list usage\n- [ ] Instructions to readme\n- [ ] List unmatched / overmatched tags\n- [ ] Links to `delete` / `remove`\n- [ ] Links to jobs\n- [ ] Link: inspect\n- [ ] Graph: cpu / containers (idle/up)\n- [ ] Authenticate (at least if we can modify behavior, like stopping/removing images/containers)\n\n\n## Knowledge\n\n(just misc links to articles that helped me out)\n* [How to delete docker images from Nexus Repository 3](https://support.sonatype.com/hc/en-us/articles/360009696054-How-to-delete-docker-images-from-Nexus-Repository-3)\n* [Showing Text Box On Hover (In Table)](https://stackoverflow.com/questions/52562345/showing-text-box-on-hover-in-table)\n* [Beautiful Interactive Tables for your Flask Templates](https://blog.miguelgrinberg.com/post/beautiful-interactive-tables-for-your-flask-templates)\n* https://github.com/torfsen/python-systemd-tutorial\n* https://www.digitalocean.com/community/tutorials/how-to-use-templates-in-a-flask-application\n* https://stackoverflow.com/questions/49957034/live-updating-dynamic-variable-on-html-with-flask\n* https://pgjones.gitlab.io/quart/how_to_guides/templating.html\n\n\n### Logging\n\n* https://pgjones.gitlab.io/hypercorn/how_to_guides/logging.html#how-to-log\n\n",
"bugtrack_url": null,
"license": null,
"summary": "Keeps Docker resources in shape based on rules and usage",
"version": "2.0.1",
"project_urls": {
"Homepage": "https://github.com/Checkmk/checkmk-dev-tools",
"Repository": "https://github.com/Checkmk/checkmk-dev-tools"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c67864e37d99dc420c78224323561a8f0472399f6bd064e4f5bc35b404be732a",
"md5": "3a38c612f3a0dfc0810f7847cbc6cc8d",
"sha256": "95e36b30d38dc474bf7118eb84e91ec704b2e92ec5dc0c340b55d1f7232cb7a6"
},
"downloads": -1,
"filename": "docker_shaper-2.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "3a38c612f3a0dfc0810f7847cbc6cc8d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0.0,>=3.10.4",
"size": 39580,
"upload_time": "2024-05-02T14:08:03",
"upload_time_iso_8601": "2024-05-02T14:08:03.976890Z",
"url": "https://files.pythonhosted.org/packages/c6/78/64e37d99dc420c78224323561a8f0472399f6bd064e4f5bc35b404be732a/docker_shaper-2.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c5ab719e48a5942af38c8c18364ad2b62d6817b2f6f38d89259fbf28ae6e5a05",
"md5": "6dfffa25ed8b9a8aad27d65938585eef",
"sha256": "dadb5d8d10a02d8e990c97e91dbd416bc7a24e7a04f25ae9398d9dd0085e78b3"
},
"downloads": -1,
"filename": "docker_shaper-2.0.1.tar.gz",
"has_sig": false,
"md5_digest": "6dfffa25ed8b9a8aad27d65938585eef",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0.0,>=3.10.4",
"size": 37756,
"upload_time": "2024-05-02T14:08:06",
"upload_time_iso_8601": "2024-05-02T14:08:06.627497Z",
"url": "https://files.pythonhosted.org/packages/c5/ab/719e48a5942af38c8c18364ad2b62d6817b2f6f38d89259fbf28ae6e5a05/docker_shaper-2.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-02 14:08:06",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Checkmk",
"github_project": "checkmk-dev-tools",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "docker-shaper"
}
JSON
