dindbox
=======
Lightweight, docker-capable virtual testing environment.
DindBox creates virtual testing environments, each comprising two docker containers. These function like lightweight
virtual machines, allowing for the creation and operation of docker containers within them without affecting
the host system's main docker daemon. This "docker-in-docker" principle, used by CI platforms like GitLab CI or GitHub
Actions, enables docker capability within a containerized runner. DindBox offers this setup for local testing and
prototyping, and can also be used in a CI context with a dedicated runner.
Installation
-----------
Dindbox only runs on Linux-based systems. It requires a Python version >= 3.10, a running docker server, and an
installation of the Docker CLI.
Dindbox is mostly intended to be used as a command-line tool. For this use case, it is recommended to install it via
pipx in order to make it available globally but keep its dependencies within a separated environment:
```sh
python3 -m pip install --user pipx
python3 -m pipx ensurepath
pipx install dindbox
```
If you however do plan to use dindbox as a python library (see below), simply run `pip install dindbox` within the
virtual environment of your project.
Usage
----
All commands and options are documented in detail in the command-line help. Run `dindbox -h` for a list of available
commands, and `dindbox COMMAND -h` for help on each command.
The most basic use case is to directly start an interactive session:
`dindbox my_box run`
Where `my_box` is the name of your dindbox which you can re-use in further commands to interact with it. The `run`
command without further parameters will give you an Alpine Linux based environment, with a fully functional docker
client & daemon - encapsulated within docker containers themselves (docker-in-docker). The container containing the
docker daemon is referred to as *service container*. The container with which you're interacting, which contains the
docker client is referred to as *build container*.
If you are on an ubuntu-based system, `run` will also open a status screen in a separate console window. You may close
this status screen at any time, it is only for monitoring purposes. Also, you can reopen it at any time running
`dindbox my_box status` from a separate shell window.
If you'd rather use an ubuntu image, do something like the following to change the default build container image:
`dindbox my_box run -i ubuntu:focal`
Note that in this case you will need to manually install the docker **client**, since it is not part of the ubuntu
image. The docker daemon however, will already be available, since it runs inside the service container.
Note that you may also replace the service container's image (using `-I`). Normally, it should always be a `dind`
tag of the official docker image, but you can use this to test with different versions of docker, for example.
Nevertheless, other use cases are possible, too, like running a SQL server is in the service image, and the client in
the build image.
When you're done, you should always clean up boxes which you do not need anymore. In order to remove all docker entities
associated with your dindbox, run:
`dindbox my_box remove`
Or, if you know it is a throw-away box, directly append `--rm` to the `run` command for automatic cleanup after exiting.
For further commands, check out the command-line help. Note that `run` is a shortcut for `create` + `start` + `attach`.
Only in special cases it makes sense to use these commands individually, for the most part you should be fine with
`run`.
Config file
----------
All command-line parameters can also be set through a config file, so you do not have to re-type them every time. Note
that config file settings only affect CLI usage. They have **no** effect at all when using dindbox as a Python library.
### Config file locations
Dindbox will look for a config file in three locations:
* First, it will traverse the directory structure upwards, starting with the current directory and looking for a
`pyproject.toml` file, as is usually present within a poetry environment. Any sections within that file of the format
`[tool.dindbox.*]` will be considered (see below for exact format).
* Next, any file in the *current directory* of name `dindbox.conf` will be parsed.
* Lastly, a config file specified with the `--config-file` or `-c` command-line argument will be parsed.
Config files will be parsed in the order specified above. Duplicate options provided in a later config file will
override values from previous config files. Additionally, a command-line parameter will always override a value set in
any of the config files.
### Config file format
Config files need to be in the TOML format. Arguments to each `dindbox` command are specified in a section named
`[dindbox.COMMAND]`, where `COMMAND` is the name of the command, e.g. `run`. The section name is case-sensitive.
Additionally, global arguments (the ones provided before a command and listed when running `dindbox --help`) can be
specified in a section named `[dindbox.global]`.
Argument names must be provided in their long form, without the leading `--`. Here's an example:
```toml title="dindbox.conf"
[dindbox.global]
log-level = "DEBUG"
[dindbox.run]
no-status-screen = true
rm = true
[dindbox.remove]
force = true
```
**Note:** When setting options in poetry's `pyproject.toml`, the section name needs to be prefixed with `tool.`:
```toml
[tool.dindbox.run]
rm = true
```
Usage as a Python Library
------------------------
Althogh `dindbox` is primarily intended to be used as a command-line tool, it can also be used as a Python library and
you can create your own custom scripts and applications based on it. The main entry point is the `DindBox` class. Here's
a minimal example of how to use it:
```python
from dindbox import DindBox
box = DindBox("my_box", build_image="ubuntu:focal")
box.run()
```
For further usage details, rely on the docstrings and type annotations of the `DindBox` class and its methods.
Use Cases
--------
Although there are probably more, DindBox was created with a particular set of use cases in mind. Most of these are
aimed at local testing on your development machine, but there are some rare cases, where it may even make sense to run
DindBox on your CI server. (In that case however, you're gonna need a dedicated runner, since DindBox itself should not
be run within a container.)
* **Test installation and execution of dockerized apps in a safe environment and on different Linux distributions**
You've created an application which is nicely bundled into microservices. Maybe there's also a nice little install
script to set it all up on a machine. But how do you test it? Hopefully not by running an untested script with root
permissions directly on your development machine. You'll either need a VM, or a tedious manually cobbled-together
docker setup - or you just type `dindbox run` and give it a go! And by switching out the image for the build container,
you can even easily test if your script runs fine on different Linux distros (and different releases of these).
* **Test a dockerized application against different versions of docker**
Switching the docker version locally means uninstalling and re-installing for each version switch. With DindBox, you
can just swap out the images and re-run your application (or re-build your images) based on a different release of
docker.
* **Reproduce and debug errors which occur in continuous integration (CI) jobs**
CI pipelines can be difficult to debug. And sometimes, the problem is not in the pipeline code alone, but also has to
do with the fact that the job is executed within a containerized runner. This can be particularly tricky, when
running docker commands as part of the pipeline, e.g. to test a self-built image. DindBox allows to easily reproduce
the containerized setup locally, with one simple command-line.
* **Build docker images locally within a well-defined environment**
In CI, your docker images are built in a containerized environment with a fixed docker version etc. DindBox makes it
easy to reproduce the exact same conditions locally.
* **Anything else you would otherwise use a virtual machine for**, because you need it to be isolated, and it requires
docker.
Note: the service running in the second container does not *have* to be dockerd. Since the image is freely configurable,
you could also switch it our for a database image like MySQL, and test your DB-dependent, non-dockerized application
in a DindBox. That's totally possible, but it is not what I had in mind when I created it.
How is it different from docker-compose?
---------------------------------------
Good question. For many use cases, docker-compose may be a good solution. But specifically for the testing use cases
listed above, DindBox offers some advantages, and also some specific features, which are not easily reproduced with
docker-compose:
* Ease of use: no need to create a config file, just type a short command-line with a few options, and you're up and
running. DindBox allows for some helpful configuration options, but the defaults will work fine, usually.
* Easy management of multiple boxes: each box has a specific name. By referencing that name you can easily manage,
re-use, and clean up your various test environments. All docker entities (containers, networks, volumes) that make
up a single box, will be set up or removed automatically, with a one-line command.
* Nice status screen: allows you to monitor what is going on inside a box, even if you're not interacting with it
directly.
* Bind volumes: In addition to attaching docker volumes to your containers, or bind-mounting existing host directories,
DindBox also offers to configure so-called "bind volumes". These are basically host directories which behave like
docker volumes: they are created on-demand in a temporary folder, mounted to both DindBox containers, and cleaned up
on removal of the DindBox. They can be used for specific cases, which are not supported by standard docker volumes.
One example for this is running a user-space file system within on of your docker-ind-docker containers, and have it
access the volume.
Development
----------
DindBox uses [poetry](https://python-poetry.org) for dependency management, isolation and packaging.
* If you do not have it already, install poetry on your machine as described in
[its documentation](https://python-poetry.org/docs/#installation). Make sure to install in an isolated environment
(the official installer does that), do not simply `pip install` it.
* Optional: enable poetry bash completion, as described in the docs
* Clone this repository and enter the directory
* Run `poetry install`
Raw data
{
"_id": null,
"home_page": "https://gitlab.com/orbelico/dindbox",
"name": "dindbox",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10,<4.0",
"maintainer_email": "",
"keywords": "docker,docker-in-docker,dind,gitlab CI,gitlab runner,github actions",
"author": "Orbelico",
"author_email": "orbelico@posteo.de",
"download_url": "https://files.pythonhosted.org/packages/b5/30/e233ba0971d3ff04380d47fb5ade1a60cb28d5817c678e5522b82198eefb/dindbox-1.0.0.tar.gz",
"platform": null,
"description": "dindbox\n=======\n\nLightweight, docker-capable virtual testing environment.\n\nDindBox creates virtual testing environments, each comprising two docker containers. These function like lightweight\nvirtual machines, allowing for the creation and operation of docker containers within them without affecting\nthe host system's main docker daemon. This \"docker-in-docker\" principle, used by CI platforms like GitLab CI or GitHub\nActions, enables docker capability within a containerized runner. DindBox offers this setup for local testing and\nprototyping, and can also be used in a CI context with a dedicated runner.\n\nInstallation\n-----------\n\nDindbox only runs on Linux-based systems. It requires a Python version >= 3.10, a running docker server, and an\ninstallation of the Docker CLI.\n\nDindbox is mostly intended to be used as a command-line tool. For this use case, it is recommended to install it via\npipx in order to make it available globally but keep its dependencies within a separated environment:\n\n```sh\npython3 -m pip install --user pipx\npython3 -m pipx ensurepath\npipx install dindbox\n```\n\nIf you however do plan to use dindbox as a python library (see below), simply run `pip install dindbox` within the\nvirtual environment of your project.\n\nUsage\n----\n\nAll commands and options are documented in detail in the command-line help. Run `dindbox -h` for a list of available\ncommands, and `dindbox COMMAND -h` for help on each command.\n\nThe most basic use case is to directly start an interactive session: \n`dindbox my_box run` \nWhere `my_box` is the name of your dindbox which you can re-use in further commands to interact with it. The `run`\ncommand without further parameters will give you an Alpine Linux based environment, with a fully functional docker\nclient & daemon - encapsulated within docker containers themselves (docker-in-docker). The container containing the\ndocker daemon is referred to as *service container*. The container with which you're interacting, which contains the\ndocker client is referred to as *build container*.\n\nIf you are on an ubuntu-based system, `run` will also open a status screen in a separate console window. You may close\nthis status screen at any time, it is only for monitoring purposes. Also, you can reopen it at any time running \n`dindbox my_box status` from a separate shell window.\n\nIf you'd rather use an ubuntu image, do something like the following to change the default build container image: \n`dindbox my_box run -i ubuntu:focal` \nNote that in this case you will need to manually install the docker **client**, since it is not part of the ubuntu\nimage. The docker daemon however, will already be available, since it runs inside the service container.\n\nNote that you may also replace the service container's image (using `-I`). Normally, it should always be a `dind`\ntag of the official docker image, but you can use this to test with different versions of docker, for example.\nNevertheless, other use cases are possible, too, like running a SQL server is in the service image, and the client in\nthe build image.\n\nWhen you're done, you should always clean up boxes which you do not need anymore. In order to remove all docker entities\nassociated with your dindbox, run: \n`dindbox my_box remove` \nOr, if you know it is a throw-away box, directly append `--rm` to the `run` command for automatic cleanup after exiting.\n\nFor further commands, check out the command-line help. Note that `run` is a shortcut for `create` + `start` + `attach`.\nOnly in special cases it makes sense to use these commands individually, for the most part you should be fine with\n`run`.\n\nConfig file\n----------\n\nAll command-line parameters can also be set through a config file, so you do not have to re-type them every time. Note\nthat config file settings only affect CLI usage. They have **no** effect at all when using dindbox as a Python library.\n\n### Config file locations\n\nDindbox will look for a config file in three locations:\n\n* First, it will traverse the directory structure upwards, starting with the current directory and looking for a\n `pyproject.toml` file, as is usually present within a poetry environment. Any sections within that file of the format\n `[tool.dindbox.*]` will be considered (see below for exact format).\n* Next, any file in the *current directory* of name `dindbox.conf` will be parsed.\n* Lastly, a config file specified with the `--config-file` or `-c` command-line argument will be parsed.\n\nConfig files will be parsed in the order specified above. Duplicate options provided in a later config file will\noverride values from previous config files. Additionally, a command-line parameter will always override a value set in\nany of the config files.\n\n### Config file format\n\nConfig files need to be in the TOML format. Arguments to each `dindbox` command are specified in a section named\n`[dindbox.COMMAND]`, where `COMMAND` is the name of the command, e.g. `run`. The section name is case-sensitive.\nAdditionally, global arguments (the ones provided before a command and listed when running `dindbox --help`) can be \nspecified in a section named `[dindbox.global]`.\n\nArgument names must be provided in their long form, without the leading `--`. Here's an example:\n\n```toml title=\"dindbox.conf\"\n[dindbox.global]\nlog-level = \"DEBUG\"\n\n[dindbox.run]\nno-status-screen = true\nrm = true\n\n[dindbox.remove]\nforce = true\n```\n\n**Note:** When setting options in poetry's `pyproject.toml`, the section name needs to be prefixed with `tool.`:\n\n```toml\n[tool.dindbox.run]\nrm = true\n```\n\nUsage as a Python Library\n------------------------\n\nAlthogh `dindbox` is primarily intended to be used as a command-line tool, it can also be used as a Python library and\nyou can create your own custom scripts and applications based on it. The main entry point is the `DindBox` class. Here's\na minimal example of how to use it:\n\n```python\nfrom dindbox import DindBox\n\nbox = DindBox(\"my_box\", build_image=\"ubuntu:focal\")\nbox.run()\n```\n\nFor further usage details, rely on the docstrings and type annotations of the `DindBox` class and its methods.\n\nUse Cases\n--------\n\nAlthough there are probably more, DindBox was created with a particular set of use cases in mind. Most of these are\naimed at local testing on your development machine, but there are some rare cases, where it may even make sense to run\nDindBox on your CI server. (In that case however, you're gonna need a dedicated runner, since DindBox itself should not\nbe run within a container.)\n\n* **Test installation and execution of dockerized apps in a safe environment and on different Linux distributions** \n You've created an application which is nicely bundled into microservices. Maybe there's also a nice little install\n script to set it all up on a machine. But how do you test it? Hopefully not by running an untested script with root\n permissions directly on your development machine. You'll either need a VM, or a tedious manually cobbled-together\n docker setup - or you just type `dindbox run` and give it a go! And by switching out the image for the build container,\n you can even easily test if your script runs fine on different Linux distros (and different releases of these).\n\n* **Test a dockerized application against different versions of docker** \n Switching the docker version locally means uninstalling and re-installing for each version switch. With DindBox, you\n can just swap out the images and re-run your application (or re-build your images) based on a different release of\n docker.\n\n* **Reproduce and debug errors which occur in continuous integration (CI) jobs** \n CI pipelines can be difficult to debug. And sometimes, the problem is not in the pipeline code alone, but also has to\n do with the fact that the job is executed within a containerized runner. This can be particularly tricky, when\n running docker commands as part of the pipeline, e.g. to test a self-built image. DindBox allows to easily reproduce\n the containerized setup locally, with one simple command-line.\n\n* **Build docker images locally within a well-defined environment** \n In CI, your docker images are built in a containerized environment with a fixed docker version etc. DindBox makes it\n easy to reproduce the exact same conditions locally.\n\n* **Anything else you would otherwise use a virtual machine for**, because you need it to be isolated, and it requires\n docker.\n\nNote: the service running in the second container does not *have* to be dockerd. Since the image is freely configurable,\nyou could also switch it our for a database image like MySQL, and test your DB-dependent, non-dockerized application\nin a DindBox. That's totally possible, but it is not what I had in mind when I created it.\n\nHow is it different from docker-compose?\n---------------------------------------\n\nGood question. For many use cases, docker-compose may be a good solution. But specifically for the testing use cases\nlisted above, DindBox offers some advantages, and also some specific features, which are not easily reproduced with\ndocker-compose:\n\n* Ease of use: no need to create a config file, just type a short command-line with a few options, and you're up and\n running. DindBox allows for some helpful configuration options, but the defaults will work fine, usually.\n* Easy management of multiple boxes: each box has a specific name. By referencing that name you can easily manage,\n re-use, and clean up your various test environments. All docker entities (containers, networks, volumes) that make\n up a single box, will be set up or removed automatically, with a one-line command.\n* Nice status screen: allows you to monitor what is going on inside a box, even if you're not interacting with it\n directly.\n* Bind volumes: In addition to attaching docker volumes to your containers, or bind-mounting existing host directories,\n DindBox also offers to configure so-called \"bind volumes\". These are basically host directories which behave like\n docker volumes: they are created on-demand in a temporary folder, mounted to both DindBox containers, and cleaned up\n on removal of the DindBox. They can be used for specific cases, which are not supported by standard docker volumes.\n One example for this is running a user-space file system within on of your docker-ind-docker containers, and have it\n access the volume.\n\nDevelopment\n----------\n\nDindBox uses [poetry](https://python-poetry.org) for dependency management, isolation and packaging.\n\n* If you do not have it already, install poetry on your machine as described in\n [its documentation](https://python-poetry.org/docs/#installation). Make sure to install in an isolated environment\n (the official installer does that), do not simply `pip install` it.\n* Optional: enable poetry bash completion, as described in the docs\n* Clone this repository and enter the directory\n* Run `poetry install`\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Lightweight, docker-capable virtual testing environment based on docker-in-docker (dind)",
"version": "1.0.0",
"project_urls": {
"Homepage": "https://gitlab.com/orbelico/dindbox",
"Repository": "https://gitlab.com/orbelico/dindbox"
},
"split_keywords": [
"docker",
"docker-in-docker",
"dind",
"gitlab ci",
"gitlab runner",
"github actions"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "65becf60f48c4d27615a5bb995e74e3ec84b75abf48d57f89a9d6a1fac2236d6",
"md5": "b1b12881526fdf00ca97a7e4603b7001",
"sha256": "a3cb90d5942928bd9f7c5a981d26854bc9c5d5985ca1a5eb50d11e03dead86cf"
},
"downloads": -1,
"filename": "dindbox-1.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b1b12881526fdf00ca97a7e4603b7001",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10,<4.0",
"size": 37730,
"upload_time": "2024-01-17T23:12:19",
"upload_time_iso_8601": "2024-01-17T23:12:19.855020Z",
"url": "https://files.pythonhosted.org/packages/65/be/cf60f48c4d27615a5bb995e74e3ec84b75abf48d57f89a9d6a1fac2236d6/dindbox-1.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "b530e233ba0971d3ff04380d47fb5ade1a60cb28d5817c678e5522b82198eefb",
"md5": "0678fae8a8bed7d884942d23b41394fe",
"sha256": "4db4248bdea74b9b5a19424016152794a5c1b413c1c60b72142baff3c310bea6"
},
"downloads": -1,
"filename": "dindbox-1.0.0.tar.gz",
"has_sig": false,
"md5_digest": "0678fae8a8bed7d884942d23b41394fe",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10,<4.0",
"size": 38342,
"upload_time": "2024-01-17T23:12:21",
"upload_time_iso_8601": "2024-01-17T23:12:21.872605Z",
"url": "https://files.pythonhosted.org/packages/b5/30/e233ba0971d3ff04380d47fb5ade1a60cb28d5817c678e5522b82198eefb/dindbox-1.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-17 23:12:21",
"github": false,
"gitlab": true,
"bitbucket": false,
"codeberg": false,
"gitlab_user": "orbelico",
"gitlab_project": "dindbox",
"lcname": "dindbox"
}
JSON