|Build Status| |Coverage Status| |PyPI version|
Python module and command-line tool for storing and retrieving data in a
Docker registry.
- Store arbitrary data (blob-store)
- Content addressable
- Set up named aliases to blobs
- Supports Docker registry schemas v1 and v2
- Works on Python 3.6+
Please note that ``dxf`` does *not* generate Docker container
configuration, so you won’t be able to ``docker pull`` data you store
using ``dxf``. See `this
issue <https://github.com/davedoesdev/dxf/issues/3>`__ for more details.
Command-line example:
.. code:: shell
dxf push-blob fred/datalogger logger.dat @may15-readings
dxf pull-blob fred/datalogger @may15-readings
which is the same as:
.. code:: shell
dxf set-alias fred/datalogger may15-readings $(dxf push-blob fred/datalogger logger.dat)
dxf pull-blob fred/datalogger $(dxf get-alias fred/datalogger may15-readings)
Module example:
.. code:: python
from dxf import DXF
def auth(dxf, response):
dxf.authenticate('fred', 'somepassword', response=response)
dxf = DXF('registry-1.docker.io', 'fred/datalogger', auth)
dgst = dxf.push_blob('logger.dat')
dxf.set_alias('may15-readings', dgst)
assert dxf.get_alias('may15-readings') == [dgst]
for chunk in dxf.pull_blob(dgst):
sys.stdout.write(chunk)
Usage
-----
The module API is described
`here <http://rawgit.davedoesdev.com/davedoesdev/dxf/master/docs/_build/html/index.html>`__.
The ``dxf`` command-line tool uses the following environment variables:
- ``DXF_HOST`` - Host where Docker registry is running.
- ``DXF_INSECURE`` - Set this to ``1`` if you want to connect to the
registry using ``http`` rather than ``https`` (which is the default).
- ``DXF_USERNAME`` - Name of user to authenticate as.
- ``DXF_PASSWORD`` - User’s password.
- ``DXF_AUTHORIZATION`` - HTTP ``Authorization`` header value.
- ``DXF_AUTH_HOST`` - If set, always perform token authentication to
this host, overriding the value returned by the registry.
- ``DXF_PROGRESS`` - If this is set to ``1``, a progress bar is
displayed (on standard error) during ``push-blob`` and ``pull-blob``.
If this is set to ``0``, a progress bar is not displayed. If this is
set to any other value, a progress bar is only displayed if standard
error is a terminal.
- ``DXF_BLOB_INFO`` - Set this to ``1`` if you want ``pull-blob`` to
prepend each blob with its digest and size (printed in plain text,
separated by a space and followed by a newline).
- ``DXF_CHUNK_SIZE`` - Number of bytes ``pull-blob`` should download at
a time. Defaults to 8192.
- ``DXF_SKIPTLSVERIFY`` - Set this to ``1`` to skip TLS certificate
verification.
- ``DXF_TLSVERIFY`` - Optional path to custom CA bundle to use for TLS
verification.
- ``DXF_PLATFORM`` - Optional platform (e.g. ``linux/amd64``) to use
for multi-arch manifests. If a multi-arch manifest is encountered and
this is not set then a dict containing entries for each platform will
be displayed.
You can use the following options with ``dxf``. Supply the name of the
repository you wish to work with in each case as the second argument.
- ``dxf push-blob <repo> <file> [@alias]``
Upload a file to the registry and optionally give it a name
(alias). The blob’s hash is printed to standard output.
..
The hash or the alias can be used to fetch the blob later using
``pull-blob``.
- ``dxf pull-blob <repo> <hash>|<@alias>...``
Download blobs from the registry to standard output. For each blob
you can specify its hash, prefixed by ``sha256:`` (remember the
registry is content-addressable) or an alias you’ve given it
(using ``push-blob`` or ``set-alias``).
- ``dxf blob-size <repo> <hash>|<@alias>...``
Print the size of blobs in the registry. If you specify an alias,
the sum of all the blobs it points to will be printed.
- ``dxf mount-blob <repo> <from-repo> <hash> [@alias]``
Cross mount a blob from another repository and optionally give it
an alias. Specify the blob by its hash, prefixed by ``sha256:``.
..
This is useful to avoid having to upload a blob to your repository
if you know it already exists in the registry.
- ``dxf del-blob <repo> <hash>|<@alias>...``
Delete blobs from the registry. If you specify an alias the blobs
it points to will be deleted, not the alias itself. Use
``del-alias`` for that.
- ``dxf set-alias <repo> <alias> <hash>|<file>...``
Give a name (alias) to a set of blobs. For each blob you can
either specify its hash (as printed by ``push-blob`` or
``get-alias``) or, if you have the blob’s contents on disk, its
filename (including a path separator to distinguish it from a
hash).
- ``dxf get-alias <repo> <alias>...``
For each alias you specify, print the hashes of all the blobs it
points to.
- ``dxf del-alias <repo> <alias>...``
Delete each specified alias. The blobs they point to won’t be
deleted (use ``del-blob`` for that), but their hashes will be
printed.
- ``dxf get-digest <repo> <alias>...``
For each alias you specify, print the hash of its configuration
blob. For an alias created using ``dxf``, this is the hash of the
first blob it points to. For a Docker image tag, this is the same
as ``docker inspect alias --format='{{.Id}}'``.
- ``dxf get-manifest <repo> <alias>...``
For each alias you specify, print its manifest obtained from the
registry.
- ``dxf list-aliases <repo>``
Print all the aliases defined in the repository.
- ``dxf list-repos``
Print the names of all the repositories in the registry. Not all
versions of the registry support this.
Certificates
------------
If your registry uses SSL with a self-issued certificate, you’ll need to
supply ``dxf`` with a set of trusted certificate authorities.
Set the ``REQUESTS_CA_BUNDLE`` environment variable to the path of a PEM
file containing the trusted certificate authority certificates.
Both the module and command-line tool support ``REQUESTS_CA_BUNDLE``.
Alternatively, you can set the ``DXF_TLSVERIFY`` environment variable
for the command-line tool or pass the ``tlsverify`` option to the
module.
Authentication tokens
---------------------
``dxf`` automatically obtains Docker registry authentication tokens
using your ``DXF_USERNAME`` and ``DXF_PASSWORD``, or
``DXF_AUTHORIZATION``, environment variables as necessary.
However, if you wish to override this then you can use the following
command:
- ``dxf auth <repo> <action>...``
Authenticate to the registry using ``DXF_USERNAME`` and
``DXF_PASSWORD``, or ``DXF_AUTHORIZATION``, and print the
resulting token.
..
``action`` can be ``pull``, ``push`` or ``*``.
If you assign the token to the ``DXF_TOKEN`` environment variable, for
example:
``DXF_TOKEN=$(dxf auth fred/datalogger pull)``
then subsequent ``dxf`` commands will use the token without needing
``DXF_USERNAME`` and ``DXF_PASSWORD``, or ``DXF_AUTHORIZATION``, to be
set.
Note however that the token expires after a few minutes, after which
``dxf`` will exit with ``EACCES``.
Docker Cloud authentication
---------------------------
You can use the
```dockercloud`` <https://github.com/docker/python-dockercloud>`__
library to read authentication information from your Docker
configuration file and pass it to ``dxf``:
.. code:: python
auth = 'Basic ' + dockercloud.api.auth.load_from_file()
dxf_obj = dxf.DXF('index.docker.io', repo='myorganization/myimage')
dxf_obj.authenticate(authorization=auth, actions=['pull'])
dxf_obj.list_aliases()
Thanks to `cyrilleverrier <https://github.com/cyrilleverrier>`__ for
this tip.
Installation
------------
.. code:: shell
pip install python-dxf
Licence
-------
`MIT <https://raw.github.com/davedoesdev/dxf/master/LICENCE>`__
Other projects that use DXF
---------------------------
Docker-charon
~~~~~~~~~~~~~
https://github.com/gabrieldemarmiesse/docker-charon
This package allows you to transfer Docker images from one registry to
another. The second one being disconnected from the internet.
Unlike ``docker save`` and ``docker load``, it creates the payload
directly from the registry (it’s faster) and is able to compute diffs to
only take the layers needed, hence reducing the size.
Tests
-----
.. code:: shell
make test
Lint
----
.. code:: shell
make lint
Code Coverage
-------------
.. code:: shell
make coverage
`coverage.py <http://nedbatchelder.com/code/coverage/>`__ results are
available
`here <http://rawgit.davedoesdev.com/davedoesdev/dxf/master/htmlcov/index.html>`__.
Coveralls page is `here <https://coveralls.io/r/davedoesdev/dxf>`__.
.. |Build Status| image:: https://github.com/davedoesdev/dxf/workflows/ci/badge.svg
:target: https://github.com/davedoesdev/dxf/actions
.. |Coverage Status| image:: https://coveralls.io/repos/davedoesdev/dxf/badge.png?branch=master
:target: https://coveralls.io/r/davedoesdev/dxf?branch=master
.. |PyPI version| image:: https://badge.fury.io/py/python-dxf.png
:target: http://badge.fury.io/py/python-dxf
Raw data
{
"_id": null,
"home_page": "https://github.com/davedoesdev/dxf",
"name": "python-dxf",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "docker registry",
"author": "David Halls",
"author_email": "dave@davedoesdev.com",
"download_url": "https://files.pythonhosted.org/packages/4f/af/f45c943768ce944d742b2e0473c4aa22f6ff229cba5ced172887b917681b/python-dxf-11.4.0.tar.gz",
"platform": null,
"description": "|Build Status| |Coverage Status| |PyPI version|\n\nPython module and command-line tool for storing and retrieving data in a\nDocker registry.\n\n- Store arbitrary data (blob-store)\n- Content addressable\n- Set up named aliases to blobs\n- Supports Docker registry schemas v1 and v2\n- Works on Python 3.6+\n\nPlease note that ``dxf`` does *not* generate Docker container\nconfiguration, so you won\u2019t be able to ``docker pull`` data you store\nusing ``dxf``. See `this\nissue <https://github.com/davedoesdev/dxf/issues/3>`__ for more details.\n\nCommand-line example:\n\n.. code:: shell\n\n dxf push-blob fred/datalogger logger.dat @may15-readings\n dxf pull-blob fred/datalogger @may15-readings\n\nwhich is the same as:\n\n.. code:: shell\n\n dxf set-alias fred/datalogger may15-readings $(dxf push-blob fred/datalogger logger.dat)\n dxf pull-blob fred/datalogger $(dxf get-alias fred/datalogger may15-readings)\n\nModule example:\n\n.. code:: python\n\n from dxf import DXF\n\n def auth(dxf, response):\n dxf.authenticate('fred', 'somepassword', response=response)\n\n dxf = DXF('registry-1.docker.io', 'fred/datalogger', auth)\n\n dgst = dxf.push_blob('logger.dat')\n dxf.set_alias('may15-readings', dgst)\n\n assert dxf.get_alias('may15-readings') == [dgst]\n\n for chunk in dxf.pull_blob(dgst):\n sys.stdout.write(chunk)\n\nUsage\n-----\n\nThe module API is described\n`here <http://rawgit.davedoesdev.com/davedoesdev/dxf/master/docs/_build/html/index.html>`__.\n\nThe ``dxf`` command-line tool uses the following environment variables:\n\n- ``DXF_HOST`` - Host where Docker registry is running.\n- ``DXF_INSECURE`` - Set this to ``1`` if you want to connect to the\n registry using ``http`` rather than ``https`` (which is the default).\n- ``DXF_USERNAME`` - Name of user to authenticate as.\n- ``DXF_PASSWORD`` - User\u2019s password.\n- ``DXF_AUTHORIZATION`` - HTTP ``Authorization`` header value.\n- ``DXF_AUTH_HOST`` - If set, always perform token authentication to\n this host, overriding the value returned by the registry.\n- ``DXF_PROGRESS`` - If this is set to ``1``, a progress bar is\n displayed (on standard error) during ``push-blob`` and ``pull-blob``.\n If this is set to ``0``, a progress bar is not displayed. If this is\n set to any other value, a progress bar is only displayed if standard\n error is a terminal.\n- ``DXF_BLOB_INFO`` - Set this to ``1`` if you want ``pull-blob`` to\n prepend each blob with its digest and size (printed in plain text,\n separated by a space and followed by a newline).\n- ``DXF_CHUNK_SIZE`` - Number of bytes ``pull-blob`` should download at\n a time. Defaults to 8192.\n- ``DXF_SKIPTLSVERIFY`` - Set this to ``1`` to skip TLS certificate\n verification.\n- ``DXF_TLSVERIFY`` - Optional path to custom CA bundle to use for TLS\n verification.\n- ``DXF_PLATFORM`` - Optional platform (e.g.\u00a0``linux/amd64``) to use\n for multi-arch manifests. If a multi-arch manifest is encountered and\n this is not set then a dict containing entries for each platform will\n be displayed.\n\nYou can use the following options with ``dxf``. Supply the name of the\nrepository you wish to work with in each case as the second argument.\n\n- ``dxf push-blob <repo> <file> [@alias]``\n\n Upload a file to the registry and optionally give it a name\n (alias). The blob\u2019s hash is printed to standard output.\n\n ..\n\n The hash or the alias can be used to fetch the blob later using\n ``pull-blob``.\n\n- ``dxf pull-blob <repo> <hash>|<@alias>...``\n\n Download blobs from the registry to standard output. For each blob\n you can specify its hash, prefixed by ``sha256:`` (remember the\n registry is content-addressable) or an alias you\u2019ve given it\n (using ``push-blob`` or ``set-alias``).\n\n- ``dxf blob-size <repo> <hash>|<@alias>...``\n\n Print the size of blobs in the registry. If you specify an alias,\n the sum of all the blobs it points to will be printed.\n\n- ``dxf mount-blob <repo> <from-repo> <hash> [@alias]``\n\n Cross mount a blob from another repository and optionally give it\n an alias. Specify the blob by its hash, prefixed by ``sha256:``.\n\n ..\n\n This is useful to avoid having to upload a blob to your repository\n if you know it already exists in the registry.\n\n- ``dxf del-blob <repo> <hash>|<@alias>...``\n\n Delete blobs from the registry. If you specify an alias the blobs\n it points to will be deleted, not the alias itself. Use\n ``del-alias`` for that.\n\n- ``dxf set-alias <repo> <alias> <hash>|<file>...``\n\n Give a name (alias) to a set of blobs. For each blob you can\n either specify its hash (as printed by ``push-blob`` or\n ``get-alias``) or, if you have the blob\u2019s contents on disk, its\n filename (including a path separator to distinguish it from a\n hash).\n\n- ``dxf get-alias <repo> <alias>...``\n\n For each alias you specify, print the hashes of all the blobs it\n points to.\n\n- ``dxf del-alias <repo> <alias>...``\n\n Delete each specified alias. The blobs they point to won\u2019t be\n deleted (use ``del-blob`` for that), but their hashes will be\n printed.\n\n- ``dxf get-digest <repo> <alias>...``\n\n For each alias you specify, print the hash of its configuration\n blob. For an alias created using ``dxf``, this is the hash of the\n first blob it points to. For a Docker image tag, this is the same\n as ``docker inspect alias --format='{{.Id}}'``.\n\n- ``dxf get-manifest <repo> <alias>...``\n\n For each alias you specify, print its manifest obtained from the\n registry.\n\n- ``dxf list-aliases <repo>``\n\n Print all the aliases defined in the repository.\n\n- ``dxf list-repos``\n\n Print the names of all the repositories in the registry. Not all\n versions of the registry support this.\n\nCertificates\n------------\n\nIf your registry uses SSL with a self-issued certificate, you\u2019ll need to\nsupply ``dxf`` with a set of trusted certificate authorities.\n\nSet the ``REQUESTS_CA_BUNDLE`` environment variable to the path of a PEM\nfile containing the trusted certificate authority certificates.\n\nBoth the module and command-line tool support ``REQUESTS_CA_BUNDLE``.\n\nAlternatively, you can set the ``DXF_TLSVERIFY`` environment variable\nfor the command-line tool or pass the ``tlsverify`` option to the\nmodule.\n\nAuthentication tokens\n---------------------\n\n``dxf`` automatically obtains Docker registry authentication tokens\nusing your ``DXF_USERNAME`` and ``DXF_PASSWORD``, or\n``DXF_AUTHORIZATION``, environment variables as necessary.\n\nHowever, if you wish to override this then you can use the following\ncommand:\n\n- ``dxf auth <repo> <action>...``\n\n Authenticate to the registry using ``DXF_USERNAME`` and\n ``DXF_PASSWORD``, or ``DXF_AUTHORIZATION``, and print the\n resulting token.\n\n ..\n\n ``action`` can be ``pull``, ``push`` or ``*``.\n\nIf you assign the token to the ``DXF_TOKEN`` environment variable, for\nexample:\n\n``DXF_TOKEN=$(dxf auth fred/datalogger pull)``\n\nthen subsequent ``dxf`` commands will use the token without needing\n``DXF_USERNAME`` and ``DXF_PASSWORD``, or ``DXF_AUTHORIZATION``, to be\nset.\n\nNote however that the token expires after a few minutes, after which\n``dxf`` will exit with ``EACCES``.\n\nDocker Cloud authentication\n---------------------------\n\nYou can use the\n```dockercloud`` <https://github.com/docker/python-dockercloud>`__\nlibrary to read authentication information from your Docker\nconfiguration file and pass it to ``dxf``:\n\n.. code:: python\n\n auth = 'Basic ' + dockercloud.api.auth.load_from_file()\n dxf_obj = dxf.DXF('index.docker.io', repo='myorganization/myimage')\n dxf_obj.authenticate(authorization=auth, actions=['pull'])\n dxf_obj.list_aliases()\n\nThanks to `cyrilleverrier <https://github.com/cyrilleverrier>`__ for\nthis tip.\n\nInstallation\n------------\n\n.. code:: shell\n\n pip install python-dxf\n\nLicence\n-------\n\n`MIT <https://raw.github.com/davedoesdev/dxf/master/LICENCE>`__\n\nOther projects that use DXF\n---------------------------\n\nDocker-charon\n~~~~~~~~~~~~~\n\nhttps://github.com/gabrieldemarmiesse/docker-charon\n\nThis package allows you to transfer Docker images from one registry to\nanother. The second one being disconnected from the internet.\n\nUnlike ``docker save`` and ``docker load``, it creates the payload\ndirectly from the registry (it\u2019s faster) and is able to compute diffs to\nonly take the layers needed, hence reducing the size.\n\nTests\n-----\n\n.. code:: shell\n\n make test\n\nLint\n----\n\n.. code:: shell\n\n make lint\n\nCode Coverage\n-------------\n\n.. code:: shell\n\n make coverage\n\n`coverage.py <http://nedbatchelder.com/code/coverage/>`__ results are\navailable\n`here <http://rawgit.davedoesdev.com/davedoesdev/dxf/master/htmlcov/index.html>`__.\n\nCoveralls page is `here <https://coveralls.io/r/davedoesdev/dxf>`__.\n\n.. |Build Status| image:: https://github.com/davedoesdev/dxf/workflows/ci/badge.svg\n :target: https://github.com/davedoesdev/dxf/actions\n.. |Coverage Status| image:: https://coveralls.io/repos/davedoesdev/dxf/badge.png?branch=master\n :target: https://coveralls.io/r/davedoesdev/dxf?branch=master\n.. |PyPI version| image:: https://badge.fury.io/py/python-dxf.png\n :target: http://badge.fury.io/py/python-dxf\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Package for accessing a Docker v2 registry",
"version": "11.4.0",
"project_urls": {
"Homepage": "https://github.com/davedoesdev/dxf"
},
"split_keywords": [
"docker",
"registry"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "4faff45c943768ce944d742b2e0473c4aa22f6ff229cba5ced172887b917681b",
"md5": "27ccf5d1726b9ab3e25b7b35fb0fc456",
"sha256": "3f3a2b96ad602232559f6f7467b03e2ec63d5a332cbc018f118d187c4ad4c443"
},
"downloads": -1,
"filename": "python-dxf-11.4.0.tar.gz",
"has_sig": false,
"md5_digest": "27ccf5d1726b9ab3e25b7b35fb0fc456",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 27593,
"upload_time": "2023-11-29T22:07:15",
"upload_time_iso_8601": "2023-11-29T22:07:15.250454Z",
"url": "https://files.pythonhosted.org/packages/4f/af/f45c943768ce944d742b2e0473c4aa22f6ff229cba5ced172887b917681b/python-dxf-11.4.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-29 22:07:15",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "davedoesdev",
"github_project": "dxf",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"requirements": [],
"lcname": "python-dxf"
}
JSON
