dazel
=====
Run Google's bazel inside a docker container via a seamless proxy.
bazel is awesome at creating fast and reproducible builds on your own
development environment. The problem is that it works in an imperfect
and non-portable environment.
Enter dazel.
dazel allows you to create your build environment as a Docker image,
either via a Dockerfile or a prebuilt repository. The tool itself is a
simple python script that sends the command line arguments directly to
bazel inside the container, and maps all of the necessary volumes to
make it seamless to you. It uses the 'docker exec' command to achieve
this, and maps the current directory and the bazel-WORKDIR link
directory so that the results appear on the host as if you ran the
command locally on the host.
It is run the same way you would bazel:
.. code:: bash
dazel build //my/cool/package/...
dazel run //my/cool/package:target
This was a simple build and run. The command line arguments were sent
as-is into the docker container, and the output was run in the same
manner inside the container.
Running the command for the first time will start the container on it's
own, and it will automatically detect if there is need to rebuild or
restart the container (if the Dockerfile is newer than the conatiner).
You can configure anything you need through the ".dazelrc" file in the
same directory. Take a look at the configuration section for information
on how to write one.
Installation
------------
Dependencies
~~~~~~~~~~~~
.. code:: bash
apt-get install python python-pip
apt-get install docker-ce
Install dazel
~~~~~~~~~~~~~
.. code:: bash
pip install dazel
That's all there is to it. Even bazel is not required!
Configuration
-------------
You can configure dazel in two ways (or combine): \* A .dazelrc file in
the current directory. \* Environment variables with the configuration
parameters mentioned below.
Note that specific environment variables supercede the values in the
.dazelrc file.
The possible parameters to set are (with their defaults):
.. code:: python
# The name of the docker container to run.
DAZEL_INSTANCE_NAME="dazel"
# The name of the dazel image to build or pull.
DAZEL_IMAGE_NAME="dazel"
# The command to run when running the image (the continuous command that will
# hold the container active while we connect to it).
DAZEL_RUN_COMMAND="/bin/bash"
# The command to run to invoke docker (can be changed to `nvidia-docker` for GPUs).
DAZEL_DOCKER_COMMAND="docker"
# The path to the Dockerfile to use to build the dazel image.
DAZEL_DOCKERFILE="Dockerfile.dazel" # in DAZEL_DIRECTORY
# The repository to pull the dazel image from.
DAZEL_REPOSITORY="dazel"
# The directory to build the dazel image in.
DAZEL_DIRECTORY=$PWD
# The command to run inside the container.
# NOTE: You should add flags to the .bazelrc file instead of here, since it is
# also shared in the volume and it is a much cleaner way.
DAZEL_COMMAND="/usr/bin/bazel"
# Add any additional volumes you want to share between the host and the docker
# container, in the normal "hostdir:dockerdir" format.
# This can be a python iterable, or a comma-separated string.
DAZEL_VOLUMES=[]
# Add any ports you want to publish from the dazel container to the host, in the
# normal "interface:dockerport:hostport" (e.g. "0.0.0.0:80:80").
# This can be useful if you use the "dazel run //my/cool/webserver/target"
# command for example, and need to publish port 80.
DAZEL_PORTS=[]
# Add any environment variables you want to set in the dazel container
# They will be set via -e in the docker run command
# This can be a python iterable, or a comma-separated string.
DAZEL_ENV_VARS=[]
# The GPUs to expose to the Docker container, "all" for every installed GPU
DAZEL_GPUS=""
# The name of the network on which to load all run dependencies and dazel container.
# If you are using a docker-compose.yml file to load the environment, this must
# be the network name to which all of the necessary dependencies are connected.
DAZEL_NETWORK="dazel"
# Add any additional images that you want to run as dependencies and hook up to
# the same docker network as the main container.
# The format is the standard "repository/image:tag", but you can optionally add
# the name of the container to create with "repository/image:tag::container".
# This is useful if you want to add "postgres" or "rabbitmq" for instance, and
# have them run as part of your test environment in a seamless reproducible way.
# This can be a python iterable, or a comma-separated string.
# Note: alternatively, you can use a docker-compose.yml file for dependencies.
DAZEL_RUN_DEPS=[]
# Add a docker-compose.yml file here to use it to load any services you want to
# launch as part of the environment for running bazel.
# This can be a much more complex environment than what is possible using run
# dependencies.
# Note: you can control both the project name and which services to run with the
# variables below.
DAZEL_DOCKER_COMPOSE_FILE=""
# The command to run to invoke docker-compose (can be changed to
# `nvidia-docker-compose` for GPUs).
DAZEL_DOCKER_COMPOSE_COMMAND="docker-compose"
# If using a docker-compose.yml file, this will set the COMPOSE_PROJECT_NAME
# environment variable and thus the project name.
DAZEL_DOCKER_COMPOSE_PROJECT_NAME="dazel"
# If using a docker-compose.yml file, you can specify the services to run in the
# file (and empty string means all services - as in running: docker-compose up).
# This can be a python iterable, or a comma-separated string.
DAZEL_DOCKER_COMPOSE_SERVICES=""
# Whether or not to run in privileged mode (fixes bazel sandboxing issues on some
# systems). Note that this can be a python boolean equivalent, so if setting
# this from the environment, simply set it to an empty string.
DAZEL_DOCKER_RUN_PRIVILEGED=False
# Path to custom .bazelrc file to use when running the bazel commands.
DAZEL_BAZEL_RC_FILE=""
# Use the :delegated flag of the --volume option of docker for the bind-mounting of
# the bazel cache directory. This vastly improves throughput on MacOSX.
# More information on the :delegated flag: https://docs.docker.com/docker-for-mac/osxfs-caching/.
# NOTE: This will fail on Docker versions < 17.04.
DAZEL_DELEGATED_VOLUME=True
# The user, in the same format as the --user option docker run and docker exec takes,
# to use when starting the container and executing commands inside of the container
DAZEL_USER=""
Raw data
{
"_id": null,
"home_page": "https://github.com/nadirizr/dazel",
"name": "dazel",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "docker,bazel",
"author": "nadir.izr",
"author_email": "nadir@armis.com",
"download_url": "https://files.pythonhosted.org/packages/b3/f0/402920316af5c7a02037910c202291897fc6f15e56fc80d500d3b2ad74bb/dazel-0.0.42.tar.gz",
"platform": null,
"description": "dazel\n=====\n\nRun Google's bazel inside a docker container via a seamless proxy.\n\nbazel is awesome at creating fast and reproducible builds on your own\ndevelopment environment. The problem is that it works in an imperfect\nand non-portable environment.\n\nEnter dazel.\n\ndazel allows you to create your build environment as a Docker image,\neither via a Dockerfile or a prebuilt repository. The tool itself is a\nsimple python script that sends the command line arguments directly to\nbazel inside the container, and maps all of the necessary volumes to\nmake it seamless to you. It uses the 'docker exec' command to achieve\nthis, and maps the current directory and the bazel-WORKDIR link\ndirectory so that the results appear on the host as if you ran the\ncommand locally on the host.\n\nIt is run the same way you would bazel:\n\n.. code:: bash\n\n dazel build //my/cool/package/...\n dazel run //my/cool/package:target\n\nThis was a simple build and run. The command line arguments were sent\nas-is into the docker container, and the output was run in the same\nmanner inside the container.\n\nRunning the command for the first time will start the container on it's\nown, and it will automatically detect if there is need to rebuild or\nrestart the container (if the Dockerfile is newer than the conatiner).\nYou can configure anything you need through the \".dazelrc\" file in the\nsame directory. Take a look at the configuration section for information\non how to write one.\n\nInstallation\n------------\n\nDependencies\n~~~~~~~~~~~~\n\n.. code:: bash\n\n apt-get install python python-pip\n apt-get install docker-ce\n\nInstall dazel\n~~~~~~~~~~~~~\n\n.. code:: bash\n\n pip install dazel\n\nThat's all there is to it. Even bazel is not required!\n\nConfiguration\n-------------\n\nYou can configure dazel in two ways (or combine): \\* A .dazelrc file in\nthe current directory. \\* Environment variables with the configuration\nparameters mentioned below.\n\nNote that specific environment variables supercede the values in the\n.dazelrc file.\n\nThe possible parameters to set are (with their defaults):\n\n.. code:: python\n\n # The name of the docker container to run.\n DAZEL_INSTANCE_NAME=\"dazel\"\n\n # The name of the dazel image to build or pull.\n DAZEL_IMAGE_NAME=\"dazel\"\n\n # The command to run when running the image (the continuous command that will\n # hold the container active while we connect to it).\n DAZEL_RUN_COMMAND=\"/bin/bash\"\n\n # The command to run to invoke docker (can be changed to `nvidia-docker` for GPUs).\n DAZEL_DOCKER_COMMAND=\"docker\"\n\n # The path to the Dockerfile to use to build the dazel image.\n DAZEL_DOCKERFILE=\"Dockerfile.dazel\" # in DAZEL_DIRECTORY\n\n # The repository to pull the dazel image from.\n DAZEL_REPOSITORY=\"dazel\"\n\n # The directory to build the dazel image in.\n DAZEL_DIRECTORY=$PWD\n\n # The command to run inside the container.\n # NOTE: You should add flags to the .bazelrc file instead of here, since it is\n # also shared in the volume and it is a much cleaner way.\n DAZEL_COMMAND=\"/usr/bin/bazel\"\n\n # Add any additional volumes you want to share between the host and the docker\n # container, in the normal \"hostdir:dockerdir\" format.\n # This can be a python iterable, or a comma-separated string.\n DAZEL_VOLUMES=[]\n\n # Add any ports you want to publish from the dazel container to the host, in the\n # normal \"interface:dockerport:hostport\" (e.g. \"0.0.0.0:80:80\").\n # This can be useful if you use the \"dazel run //my/cool/webserver/target\"\n # command for example, and need to publish port 80.\n DAZEL_PORTS=[]\n\n # Add any environment variables you want to set in the dazel container\n # They will be set via -e in the docker run command\n # This can be a python iterable, or a comma-separated string.\n DAZEL_ENV_VARS=[]\n\n # The GPUs to expose to the Docker container, \"all\" for every installed GPU\n DAZEL_GPUS=\"\"\n\n # The name of the network on which to load all run dependencies and dazel container.\n # If you are using a docker-compose.yml file to load the environment, this must\n # be the network name to which all of the necessary dependencies are connected.\n DAZEL_NETWORK=\"dazel\"\n\n # Add any additional images that you want to run as dependencies and hook up to\n # the same docker network as the main container.\n # The format is the standard \"repository/image:tag\", but you can optionally add\n # the name of the container to create with \"repository/image:tag::container\".\n # This is useful if you want to add \"postgres\" or \"rabbitmq\" for instance, and\n # have them run as part of your test environment in a seamless reproducible way.\n # This can be a python iterable, or a comma-separated string.\n # Note: alternatively, you can use a docker-compose.yml file for dependencies.\n DAZEL_RUN_DEPS=[]\n\n # Add a docker-compose.yml file here to use it to load any services you want to\n # launch as part of the environment for running bazel.\n # This can be a much more complex environment than what is possible using run\n # dependencies.\n # Note: you can control both the project name and which services to run with the\n # variables below.\n DAZEL_DOCKER_COMPOSE_FILE=\"\"\n\n # The command to run to invoke docker-compose (can be changed to\n # `nvidia-docker-compose` for GPUs).\n DAZEL_DOCKER_COMPOSE_COMMAND=\"docker-compose\"\n\n # If using a docker-compose.yml file, this will set the COMPOSE_PROJECT_NAME\n # environment variable and thus the project name.\n DAZEL_DOCKER_COMPOSE_PROJECT_NAME=\"dazel\"\n\n # If using a docker-compose.yml file, you can specify the services to run in the\n # file (and empty string means all services - as in running: docker-compose up).\n # This can be a python iterable, or a comma-separated string.\n DAZEL_DOCKER_COMPOSE_SERVICES=\"\"\n\n # Whether or not to run in privileged mode (fixes bazel sandboxing issues on some\n # systems). Note that this can be a python boolean equivalent, so if setting\n # this from the environment, simply set it to an empty string.\n DAZEL_DOCKER_RUN_PRIVILEGED=False\n\n # Path to custom .bazelrc file to use when running the bazel commands.\n DAZEL_BAZEL_RC_FILE=\"\"\n\n # Use the :delegated flag of the --volume option of docker for the bind-mounting of\n # the bazel cache directory. This vastly improves throughput on MacOSX.\n # More information on the :delegated flag: https://docs.docker.com/docker-for-mac/osxfs-caching/.\n # NOTE: This will fail on Docker versions < 17.04.\n DAZEL_DELEGATED_VOLUME=True\n\n # The user, in the same format as the --user option docker run and docker exec takes,\n # to use when starting the container and executing commands inside of the container\n DAZEL_USER=\"\"\n\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Run bazel in Docker, in a reproducible and portable container.",
"version": "0.0.42",
"project_urls": {
"Homepage": "https://github.com/nadirizr/dazel"
},
"split_keywords": [
"docker",
"bazel"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "637ad394f114cf9dc87ff17642a191ecad7cad3c82b9a068c55dc5fc31b47847",
"md5": "c8e110d6ffba814f04b2b5e83ba42e8d",
"sha256": "b877015edfc29a11543f0ce6dc71072f357772b562ddeee646182cc0eb57296b"
},
"downloads": -1,
"filename": "dazel-0.0.42-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "c8e110d6ffba814f04b2b5e83ba42e8d",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 11449,
"upload_time": "2023-11-16T15:44:09",
"upload_time_iso_8601": "2023-11-16T15:44:09.154762Z",
"url": "https://files.pythonhosted.org/packages/63/7a/d394f114cf9dc87ff17642a191ecad7cad3c82b9a068c55dc5fc31b47847/dazel-0.0.42-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "b3f0402920316af5c7a02037910c202291897fc6f15e56fc80d500d3b2ad74bb",
"md5": "9aee310314f7c183554db0b97b8a21c8",
"sha256": "244efe192ec3a6e1680b62cadddbd8be3b4e77347114c1f381945fbeb1810ed4"
},
"downloads": -1,
"filename": "dazel-0.0.42.tar.gz",
"has_sig": false,
"md5_digest": "9aee310314f7c183554db0b97b8a21c8",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 11929,
"upload_time": "2023-11-16T15:44:10",
"upload_time_iso_8601": "2023-11-16T15:44:10.487536Z",
"url": "https://files.pythonhosted.org/packages/b3/f0/402920316af5c7a02037910c202291897fc6f15e56fc80d500d3b2ad74bb/dazel-0.0.42.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-16 15:44:10",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "nadirizr",
"github_project": "dazel",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "dazel"
}
JSON
