| Name | curldl JSON |
| Version |
1.0.1
JSON |
| download |
| home_page | |
| Summary | Safely and reliably download files with PycURL |
| upload_time | 2023-08-19 07:19:43 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.8 |
| license | LGPL-3.0-or-later |
| keywords |
curl
pycurl
download
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Introduction
The __curldl__ Python module safely and reliably downloads files with [PycURL](http://pycurl.io/), which in turn is a wrapper for [libcurl](https://curl.se/libcurl/) file transfer library. The purpose of __curldl__ is providing a straightforward API for downloading files with the following features:
* Multi-protocol support: protocol support is delegated to [curl](https://curl.se/) in as protocol-neutral way as possible. This means that there is no reliance on HTTP-specific header and statuses, for example. If a feature like download resuming and _if-modified-since_ condition is supported by the underlying protocol, it can be used by _curldl_.
* If a partial download is abandoned, most chances are that it may be resumed later (supported for HTTP(S), FTP(S) and FILE protocols). A `.part` extension is added to the partial download file, and it is renamed to the target file name once the download completes.
* If the downloaded file exists, it is not downloaded again unless the timestamp of file on server is newer (supported for HTTP(S), FTP(S), RTSP and FILE protocols). Note that file download may be skipped before timestamp is considered due to file size matching the expected file size (see below).
* Downloads are configured relative to a base directory, and relative download path is verified not to escape the base directory directly, via symlinks, or otherwise.
* Downloaded file size and/or cryptographic digest(s) can be verified upon download completion. This verification, together with the relative path safety above, allows for easy implementation of mirroring scripts — e.g., when relative path, file size and digest are located in a downloaded XML file.
* Speed: since native _libcurl_ writes directly to the output stream file descriptor, there are no transfers of large chunks of data inside Python interpreter.
# Usage
Most examples below use the _curldl_ wrapper script instead of Python code. Of course, in all cases it is easy to write a few lines of code with identical functionality — see the first example. Also, note that inline documentation is available for all functions.
## Simple Download
The following code snippet downloads a file and verifies its size and SHA-1 digest. A progress bar is shown on _stderr_ while download is in progress.
```python
import curldl, os
dl = curldl.Curldl(basedir="downloads", progress=True)
dl.get("https://kernel.org/pub/linux/kernel/Historic/linux-0.01.tar.gz",
"linux-0.01.tar.gz", size=73091,
digests={"sha1": "566b6fb6365e25f47b972efa1506932b87d3ca7d"})
assert os.path.exists("downloads/linux-0.01.tar.gz")
```
If verification fails, the partial download is removed; otherwise it is renamed to the target file after being timestamped with _last-modified_ timestamp received from the server.
A similar result is achieved on command-line by running the CLI wrapper script, which is useful for quickly testing _curldl_ functionality:
```shell
curldl -b downloads -s 73091 -a sha1 -d 566b6fb6365e25f47b972efa1506932b87d3ca7d \
-p -l debug https://kernel.org/pub/linux/kernel/Historic/linux-0.01.tar.gz
```
The corresponding (redacted) log output:
```text
Saving download(s) to: linux-0.01.tar.gz
Creating directory: downloads
Downloading https://kernel.org/pub/linux/kernel/Historic/linux-0.01.tar.gz
to downloads/linux-0.01.tar.gz.part
Finished downloading downloads/linux-0.01.tar.gz.part 0 -> 73,091 B
(HTTPS 200: OK) [0:00:01]
Timestamping downloads/linux-0.01.tar.gz.part with 1993-10-30 00:00:00+00:00
Successfully verified file size of downloads/linux-0.01.tar.gz.part
Successfully verified SHA1 of downloads/linux-0.01.tar.gz.part
Partial download of downloads/linux-0.01.tar.gz passed verification
(73091 / {'sha1': '566b6fb6365e25f47b972efa1506932b87d3ca7d'})
Moving downloads/linux-0.01.tar.gz.part to downloads/linux-0.01.tar.gz
```
Note that renaming of `downloads/linux-0.01.tar.gz.part` to `downloads/linux-0.01.tar.gz` is the very last action of `Curldl.get()` method. If the target filename exists, the download succeeded and passed verification, if requested.
## Repeated Download
Running the same command again doesn't actually result in a server request since file size matches (digest is not checked since it would be time-prohibitive when mirroring large repositories):
```text
Saving download(s) to: linux-0.01.tar.gz
Skipping update of downloads/linux-0.01.tar.gz since it has
the expected size 73,091 B
```
We can also request the same file without providing an expected size:
```python
import curldl
dl = curldl.Curldl(basedir="downloads", progress=True)
dl.get("ftp://ftp.hosteurope.de/mirror/ftp.kernel.org/pub/linux/kernel/Historic/linux-0.01.tar.gz",
"linux-0.01.tar.gz")
```
In this case, the download is skipped due to _If-Modified-Since_ check:
```text
Will update downloads/linux-0.01.tar.gz.part
if modified since 1993-10-30 00:00:00+00:00
Discarding downloads/linux-0.01.tar.gz.part because
it is not more recent
```
Note that FTP protocol was used this time — _curldl_ is entirely protocol-agnostic when using the underlying _libcurl_ functionality.
## Resuming Download
If a download is interrupted, it will be resumed on the next attempt (which may also be a retry according to the configured retry policy). Here is what happens when _Ctrl-C_ is used to send a SIGINT signal to the Python process. This example also demonstrates how to construct a filename from a URL (CLI interface does the same when `--output` switch is omitted).
```python
import curldl, os, urllib.parse as uparse
dl = curldl.Curldl(basedir="downloads", progress=True)
url = "https://releases.ubuntu.com/22.04.2/ubuntu-22.04.2-live-server-amd64.iso"
filename = os.path.basename(uparse.unquote(uparse.urlparse(url).path))
dl.get(url, filename)
```
The corresponding (redacted) log output:
```text
Downloading https://releases.ubuntu.com/22.04.2/ubuntu-22.04.2-live-server-amd64.iso
to downloads/ubuntu-22.04.2-live-server-amd64.iso.part
ubuntu-22.04.2-live-server-amd64.iso: 13%|██▋ | 244M/1.84G
[00:06<00:38, 44.7MB/s] ^C KeyboardInterrupt:
Download interrupted downloads/ubuntu-22.04.2-live-server-amd64.iso.part 0 -> 259,981,312 B
(42: Callback aborted / HTTPS 200: OK) [0:00:07]
```
Attempting the same download again resumes the download:
```text
Resuming download of https://releases.ubuntu.com/22.04.2/ubuntu-22.04.2-live-server-amd64.iso
to downloads/ubuntu-22.04.2-live-server-amd64.iso.part at 259,981,312 B
Finished downloading downloads/ubuntu-22.04.2-live-server-amd64.iso.part
259,981,312 -> 1,975,971,840 B (HTTPS 206: Partial Content) [0:01:24]
```
Note, however, that we didn't provide a size or digest for verification. Since the downloaded file is timestamped only once download completes, how does _curldl_ know that the file wasn't changed on the server in the meantime? The answer is that _curldl_ simply avoids removing large partial downloads in such cases — see documentation for _always_keep_part_bytes_ constructor parameter of _Curldl_.
## Enabling Additional Protocols
By default, _curldl_ enables the following protocols:
- HTTP(S)
- FTP(S)
- SFTP
In order to enable a different set of protocols, use the `allowed_protocols_bitmask` constructor argument. For instance, the code below downloads a _file://_ URI:
```python
import curldl, pycurl, pathlib
protocols = pycurl.PROTO_FILE | pycurl.PROTO_HTTPS
dl = curldl.Curldl(basedir="downloads", allowed_protocols_bitmask=protocols)
file_uri = pathlib.Path(__file__).absolute().as_uri()
dl.get(file_uri, "current_source.py")
```
To enable all protocols, use `allowed_protocols_bitmask=pycurl.PROTO_ALL`. Note, however, that there might be security repercussions.
## Escaping Base Directory
Attempts to escape base directory are prevented, e.g.:
```python
import curldl, os
dl = curldl.Curldl(basedir=os.curdir)
dl.get("http://example.com/", os.path.join(os.pardir, "file.txt"))
```
The above results in:
```text
ValueError: Relative path ../file.txt escapes base path /home/user/curldl
```
_curldl_ performs extensive checks to prevent escaping the base download directory — see _FileSystem_ class implementation and unit tests for details.
# Installation
The only requirement for _curldl_ is Python 3.8+. Install the package as follows:
```shell
pip install curldl
```
If you encounter a build failure during installation of _pycurl_ dependency, the following should help:
* On Linux, install one of:
* _pycurl_ package from distribution repo — e.g., on Ubuntu run `sudo apt install python3-pycurl`
* _libcurl_ development files with `sudo apt install build-essential libcurl4-openssl-dev`
* On Windows, install an unofficial _pycurl_ build since official builds are not available at the moment — e.g., from [Christoph Gohlke](https://www.lfd.uci.edu/~gohlke/pythonlibs/#pycurl)'s packages, or use _Conda_ (see below).
* On Windows and macOS, use _Conda_ or _Miniconda_ with [conda-forge](https://conda-forge.org/) channel. For instance, see runtime dependencies in the following [test environment](https://github.com/noexec/curldl/blob/develop/misc/conda/test-environment.yml).
Overall, _curldl_ is expected to not have any issues in any environment with Python 3.8+ (CPython or PyPy) — see the [Testing](#testing) section below.
# Testing
A simplified configuration matrix covered by [CI/CD test + build pipeline](https://github.com/noexec/curldl/actions/workflows/ci.yml) at the time of writing this document is presented below:
| Platform | CPython 3.8 | CPython 3.9, PyPy 3.8+3.10 | CPython 3.10 | CPython 3.11 |
|-------------|-----------------------|----------------------------|----------------|--------------|
| Ubuntu-x64 | venv, conda, platform | venv | venv, platform | venv, conda |
| Windows-x64 | venv, conda | | | venv, conda |
| Windows-x86 | venv | | | venv |
| macOS-x64 | conda | | | conda |
In the table:
* _venv_ — virtual environment with all package dependencies and [editable package install](https://pip.pypa.io/en/stable/topics/local-project-installs/); on Ubuntu includes tests with minimal versions of package dependencies;
* _conda_ — _Miniconda_ with package dependencies installed from _mini-forge_ channel, and _curldl_ as editable package install;
* _platform_ — as many dependencies as possible satisfied via Ubuntu package repository, and _curldl_ as _wheel_ install.
The CI/CD pipeline succeeds only if _curldl_ package successfully builds and passes all the [pytest](https://pytest.org/) test cases with 100% [code coverage](https://coverage.readthedocs.io/), as well as [Pylint](https://pylint.readthedocs.io/), [Mypy](https://mypy-lang.org/) and [Bandit](https://bandit.readthedocs.io/) static code analysis. Code style checks are also a part of the pipeline. Note that the testing code is also covered by these restrictions.
# Development
## Python Environment
The following commands install and activate a _venv_ environment in Linux, using the available Python 3 interpreter:
```shell
./venv.sh install-venv
. venv/bin/activate
```
`venv.sh` is a convenience _venv_ wrapper that also enables some additional Python checks; you can use it to run Python code, or just activate the _venv_ environment instead as shown above.
## Running Tests
Use _pytest_ in order to run all test cases:
```shell
./venv.sh pytest
```
In addition to the actual tests, pytest executes _Pylint_, _Mypy_, code coverage and code formatting plugins (_black_ and _isort_). Thus, make sure that all new code is covered by tests.
Testing with _Conda_ is possible as well — see the [CI/CD pipeline execution](https://github.com/noexec/curldl/actions) for details.
## Code Formatting
Reformat the code with _black_ and _isort_ by running the following scripts:
```shell
misc/scripts/run-black.sh
misc/scripts/run-isort.sh
```
This is only necessary if the tests fail due to code formatting.
## Changelog Entries
Upon authoring a set of code or documentation changes, prepare a changelog fragment using [towncrier](https://towncrier.readthedocs.io/) as follows:
```shell
towncrier create -c "Extend package usage documentation" 54.doc.md
```
The command above creates a changelog fragment file in `docs/changelog.d`. See the output of `towncrier create --help` for supported fragment types. Note also that `54` above is the number of GitHub [pull request](https://github.com/noexec/curldl/pull/54), which is used to format a pull request link when generating the combined changelog. So creation of new fragment needs to be done _after_ opening a pull request.
When releasing a new version, the changelog file can be updated as follows:
```shell
towncrier build --version 1.x.y [--draft]
```
Add _--draft_ option for a dry run first, because otherwise fragment files will be removed, and changelog file extended with the new entries.
# Changelog
See the [Changelog](https://github.com/noexec/curldl/blob/develop/docs/CHANGELOG.md) file for a summary of changes in each release.
# License
This project is released under the [GNU LGPL License Version 3](https://github.com/noexec/curldl/blob/develop/LICENSE.md) or any later version.
[](https://pypi.org/project/curldl/)
[](https://pypi.org/project/curldl/)
[](https://github.com/noexec/curldl/actions/workflows/ci.yml)
[](https://codecov.io/gh/noexec/curldl)
[](https://github.com/PyCQA/bandit)
[](https://curldl.readthedocs.io/)
[](https://github.com/psf/black)
[](https://pycqa.github.io/isort/)
[](https://github.com/noexec/curldl/blob/develop/LICENSE.md)
Raw data
{
"_id": null,
"home_page": "",
"name": "curldl",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "cURL,PycURL,download",
"author": "",
"author_email": "Michael Orlov <orlovm@noexec.org>",
"download_url": "https://files.pythonhosted.org/packages/87/ef/a5c7f54538ac45f4b5ad64f2e10bce747147b6e264925b43b74ff7b2ef1e/curldl-1.0.1.tar.gz",
"platform": null,
"description": "# Introduction\n\nThe __curldl__ Python module safely and reliably downloads files with [PycURL](http://pycurl.io/), which in turn is a wrapper for [libcurl](https://curl.se/libcurl/) file transfer library. The purpose of __curldl__ is providing a straightforward API for downloading files with the following features:\n\n* Multi-protocol support: protocol support is delegated to [curl](https://curl.se/) in as protocol-neutral way as possible. This means that there is no reliance on HTTP-specific header and statuses, for example. If a feature like download resuming and _if-modified-since_ condition is supported by the underlying protocol, it can be used by _curldl_.\n* If a partial download is abandoned, most chances are that it may be resumed later (supported for HTTP(S), FTP(S) and FILE protocols). A `.part` extension is added to the partial download file, and it is renamed to the target file name once the download completes.\n* If the downloaded file exists, it is not downloaded again unless the timestamp of file on server is newer (supported for HTTP(S), FTP(S), RTSP and FILE protocols). Note that file download may be skipped before timestamp is considered due to file size matching the expected file size (see below).\n* Downloads are configured relative to a base directory, and relative download path is verified not to escape the base directory directly, via symlinks, or otherwise.\n* Downloaded file size and/or cryptographic digest(s) can be verified upon download completion. This verification, together with the relative path safety above, allows for easy implementation of mirroring scripts \u2014 e.g., when relative path, file size and digest are located in a downloaded XML file.\n* Speed: since native _libcurl_ writes directly to the output stream file descriptor, there are no transfers of large chunks of data inside Python interpreter.\n\n\n# Usage\n\nMost examples below use the _curldl_ wrapper script instead of Python code. Of course, in all cases it is easy to write a few lines of code with identical functionality \u2014 see the first example. Also, note that inline documentation is available for all functions.\n\n\n## Simple Download\n\nThe following code snippet downloads a file and verifies its size and SHA-1 digest. A progress bar is shown on _stderr_ while download is in progress.\n\n```python\nimport curldl, os\ndl = curldl.Curldl(basedir=\"downloads\", progress=True)\ndl.get(\"https://kernel.org/pub/linux/kernel/Historic/linux-0.01.tar.gz\",\n \"linux-0.01.tar.gz\", size=73091,\n digests={\"sha1\": \"566b6fb6365e25f47b972efa1506932b87d3ca7d\"})\nassert os.path.exists(\"downloads/linux-0.01.tar.gz\")\n```\n\nIf verification fails, the partial download is removed; otherwise it is renamed to the target file after being timestamped with _last-modified_ timestamp received from the server.\n\nA similar result is achieved on command-line by running the CLI wrapper script, which is useful for quickly testing _curldl_ functionality:\n\n```shell\ncurldl -b downloads -s 73091 -a sha1 -d 566b6fb6365e25f47b972efa1506932b87d3ca7d \\\n -p -l debug https://kernel.org/pub/linux/kernel/Historic/linux-0.01.tar.gz\n```\n\nThe corresponding (redacted) log output:\n\n```text\nSaving download(s) to: linux-0.01.tar.gz\nCreating directory: downloads\nDownloading https://kernel.org/pub/linux/kernel/Historic/linux-0.01.tar.gz\n to downloads/linux-0.01.tar.gz.part\nFinished downloading downloads/linux-0.01.tar.gz.part 0 -> 73,091 B\n (HTTPS 200: OK) [0:00:01]\nTimestamping downloads/linux-0.01.tar.gz.part with 1993-10-30 00:00:00+00:00\nSuccessfully verified file size of downloads/linux-0.01.tar.gz.part\nSuccessfully verified SHA1 of downloads/linux-0.01.tar.gz.part\nPartial download of downloads/linux-0.01.tar.gz passed verification\n (73091 / {'sha1': '566b6fb6365e25f47b972efa1506932b87d3ca7d'})\nMoving downloads/linux-0.01.tar.gz.part to downloads/linux-0.01.tar.gz\n```\n\nNote that renaming of `downloads/linux-0.01.tar.gz.part` to `downloads/linux-0.01.tar.gz` is the very last action of `Curldl.get()` method. If the target filename exists, the download succeeded and passed verification, if requested.\n\n\n## Repeated Download\n\nRunning the same command again doesn't actually result in a server request since file size matches (digest is not checked since it would be time-prohibitive when mirroring large repositories):\n\n```text\nSaving download(s) to: linux-0.01.tar.gz\nSkipping update of downloads/linux-0.01.tar.gz since it has\n the expected size 73,091 B\n```\n\nWe can also request the same file without providing an expected size:\n\n```python\nimport curldl\ndl = curldl.Curldl(basedir=\"downloads\", progress=True)\ndl.get(\"ftp://ftp.hosteurope.de/mirror/ftp.kernel.org/pub/linux/kernel/Historic/linux-0.01.tar.gz\",\n \"linux-0.01.tar.gz\")\n```\n\nIn this case, the download is skipped due to _If-Modified-Since_ check:\n\n```text\nWill update downloads/linux-0.01.tar.gz.part\n if modified since 1993-10-30 00:00:00+00:00\nDiscarding downloads/linux-0.01.tar.gz.part because\n it is not more recent\n```\n\nNote that FTP protocol was used this time \u2014 _curldl_ is entirely protocol-agnostic when using the underlying _libcurl_ functionality.\n\n\n## Resuming Download\n\nIf a download is interrupted, it will be resumed on the next attempt (which may also be a retry according to the configured retry policy). Here is what happens when _Ctrl-C_ is used to send a SIGINT signal to the Python process. This example also demonstrates how to construct a filename from a URL (CLI interface does the same when `--output` switch is omitted).\n\n```python\nimport curldl, os, urllib.parse as uparse\ndl = curldl.Curldl(basedir=\"downloads\", progress=True)\nurl = \"https://releases.ubuntu.com/22.04.2/ubuntu-22.04.2-live-server-amd64.iso\"\nfilename = os.path.basename(uparse.unquote(uparse.urlparse(url).path))\ndl.get(url, filename)\n```\n\nThe corresponding (redacted) log output:\n\n```text\nDownloading https://releases.ubuntu.com/22.04.2/ubuntu-22.04.2-live-server-amd64.iso\n to downloads/ubuntu-22.04.2-live-server-amd64.iso.part\nubuntu-22.04.2-live-server-amd64.iso: 13%|\u2588\u2588\u258b | 244M/1.84G\n [00:06<00:38, 44.7MB/s] ^C KeyboardInterrupt:\nDownload interrupted downloads/ubuntu-22.04.2-live-server-amd64.iso.part 0 -> 259,981,312 B\n (42: Callback aborted / HTTPS 200: OK) [0:00:07]\n```\n\nAttempting the same download again resumes the download:\n\n```text\nResuming download of https://releases.ubuntu.com/22.04.2/ubuntu-22.04.2-live-server-amd64.iso\n to downloads/ubuntu-22.04.2-live-server-amd64.iso.part at 259,981,312 B\nFinished downloading downloads/ubuntu-22.04.2-live-server-amd64.iso.part\n 259,981,312 -> 1,975,971,840 B (HTTPS 206: Partial Content) [0:01:24]\n```\n\nNote, however, that we didn't provide a size or digest for verification. Since the downloaded file is timestamped only once download completes, how does _curldl_ know that the file wasn't changed on the server in the meantime? The answer is that _curldl_ simply avoids removing large partial downloads in such cases \u2014 see documentation for _always_keep_part_bytes_ constructor parameter of _Curldl_.\n\n\n## Enabling Additional Protocols\n\nBy default, _curldl_ enables the following protocols:\n\n- HTTP(S)\n- FTP(S)\n- SFTP\n\nIn order to enable a different set of protocols, use the `allowed_protocols_bitmask` constructor argument. For instance, the code below downloads a _file://_ URI:\n\n```python\nimport curldl, pycurl, pathlib\nprotocols = pycurl.PROTO_FILE | pycurl.PROTO_HTTPS\ndl = curldl.Curldl(basedir=\"downloads\", allowed_protocols_bitmask=protocols)\nfile_uri = pathlib.Path(__file__).absolute().as_uri()\ndl.get(file_uri, \"current_source.py\")\n```\n\nTo enable all protocols, use `allowed_protocols_bitmask=pycurl.PROTO_ALL`. Note, however, that there might be security repercussions.\n\n\n## Escaping Base Directory\n\nAttempts to escape base directory are prevented, e.g.:\n\n```python\nimport curldl, os\ndl = curldl.Curldl(basedir=os.curdir)\ndl.get(\"http://example.com/\", os.path.join(os.pardir, \"file.txt\"))\n```\n\nThe above results in:\n\n```text\nValueError: Relative path ../file.txt escapes base path /home/user/curldl\n```\n\n_curldl_ performs extensive checks to prevent escaping the base download directory \u2014 see _FileSystem_ class implementation and unit tests for details.\n\n\n# Installation\n\nThe only requirement for _curldl_ is Python 3.8+. Install the package as follows:\n```shell\npip install curldl\n```\n\nIf you encounter a build failure during installation of _pycurl_ dependency, the following should help:\n* On Linux, install one of:\n * _pycurl_ package from distribution repo \u2014 e.g., on Ubuntu run `sudo apt install python3-pycurl`\n * _libcurl_ development files with `sudo apt install build-essential libcurl4-openssl-dev`\n* On Windows, install an unofficial _pycurl_ build since official builds are not available at the moment \u2014 e.g., from [Christoph Gohlke](https://www.lfd.uci.edu/~gohlke/pythonlibs/#pycurl)'s packages, or use _Conda_ (see below).\n* On Windows and macOS, use _Conda_ or _Miniconda_ with [conda-forge](https://conda-forge.org/) channel. For instance, see runtime dependencies in the following [test environment](https://github.com/noexec/curldl/blob/develop/misc/conda/test-environment.yml).\n\nOverall, _curldl_ is expected to not have any issues in any environment with Python 3.8+ (CPython or PyPy) \u2014 see the [Testing](#testing) section below.\n\n\n# Testing\n\nA simplified configuration matrix covered by [CI/CD test + build pipeline](https://github.com/noexec/curldl/actions/workflows/ci.yml) at the time of writing this document is presented below:\n\n| Platform | CPython 3.8 | CPython 3.9, PyPy 3.8+3.10 | CPython 3.10 | CPython 3.11 |\n|-------------|-----------------------|----------------------------|----------------|--------------|\n| Ubuntu-x64 | venv, conda, platform | venv | venv, platform | venv, conda |\n| Windows-x64 | venv, conda | | | venv, conda |\n| Windows-x86 | venv | | | venv |\n| macOS-x64 | conda | | | conda |\n\nIn the table:\n* _venv_ \u2014 virtual environment with all package dependencies and [editable package install](https://pip.pypa.io/en/stable/topics/local-project-installs/); on Ubuntu includes tests with minimal versions of package dependencies;\n* _conda_ \u2014 _Miniconda_ with package dependencies installed from _mini-forge_ channel, and _curldl_ as editable package install;\n* _platform_ \u2014 as many dependencies as possible satisfied via Ubuntu package repository, and _curldl_ as _wheel_ install.\n\nThe CI/CD pipeline succeeds only if _curldl_ package successfully builds and passes all the [pytest](https://pytest.org/) test cases with 100% [code coverage](https://coverage.readthedocs.io/), as well as [Pylint](https://pylint.readthedocs.io/), [Mypy](https://mypy-lang.org/) and [Bandit](https://bandit.readthedocs.io/) static code analysis. Code style checks are also a part of the pipeline. Note that the testing code is also covered by these restrictions.\n\n\n# Development\n\n## Python Environment\n\nThe following commands install and activate a _venv_ environment in Linux, using the available Python 3 interpreter:\n```shell\n./venv.sh install-venv\n. venv/bin/activate\n```\n\n`venv.sh` is a convenience _venv_ wrapper that also enables some additional Python checks; you can use it to run Python code, or just activate the _venv_ environment instead as shown above.\n\n\n## Running Tests\n\nUse _pytest_ in order to run all test cases:\n```shell\n./venv.sh pytest\n```\n\nIn addition to the actual tests, pytest executes _Pylint_, _Mypy_, code coverage and code formatting plugins (_black_ and _isort_). Thus, make sure that all new code is covered by tests.\n\nTesting with _Conda_ is possible as well \u2014 see the [CI/CD pipeline execution](https://github.com/noexec/curldl/actions) for details.\n\n\n## Code Formatting\n\nReformat the code with _black_ and _isort_ by running the following scripts:\n```shell\nmisc/scripts/run-black.sh\nmisc/scripts/run-isort.sh\n```\n\nThis is only necessary if the tests fail due to code formatting.\n\n\n## Changelog Entries\n\nUpon authoring a set of code or documentation changes, prepare a changelog fragment using [towncrier](https://towncrier.readthedocs.io/) as follows:\n```shell\ntowncrier create -c \"Extend package usage documentation\" 54.doc.md\n```\n\nThe command above creates a changelog fragment file in `docs/changelog.d`. See the output of `towncrier create --help` for supported fragment types. Note also that `54` above is the number of GitHub [pull request](https://github.com/noexec/curldl/pull/54), which is used to format a pull request link when generating the combined changelog. So creation of new fragment needs to be done _after_ opening a pull request.\n\nWhen releasing a new version, the changelog file can be updated as follows:\n```shell\ntowncrier build --version 1.x.y [--draft]\n```\n\nAdd _--draft_ option for a dry run first, because otherwise fragment files will be removed, and changelog file extended with the new entries.\n\n\n# Changelog\n\nSee the [Changelog](https://github.com/noexec/curldl/blob/develop/docs/CHANGELOG.md) file for a summary of changes in each release.\n\n\n# License\n\nThis project is released under the [GNU LGPL License Version 3](https://github.com/noexec/curldl/blob/develop/LICENSE.md) or any later version.\n\n\n[](https://pypi.org/project/curldl/)\n[](https://pypi.org/project/curldl/)\n[](https://github.com/noexec/curldl/actions/workflows/ci.yml)\n[](https://codecov.io/gh/noexec/curldl)\n[](https://github.com/PyCQA/bandit)\n[](https://curldl.readthedocs.io/)\n[](https://github.com/psf/black)\n[](https://pycqa.github.io/isort/)\n[](https://github.com/noexec/curldl/blob/develop/LICENSE.md)\n",
"bugtrack_url": null,
"license": "LGPL-3.0-or-later",
"summary": "Safely and reliably download files with PycURL",
"version": "1.0.1",
"project_urls": {
"Bug Tracker": "https://github.com/noexec/curldl/issues",
"Changelog": "https://github.com/noexec/curldl/blob/develop/docs/CHANGELOG.md",
"Documentation": "https://curldl.readthedocs.io/",
"Homepage": "https://github.com/noexec/curldl"
},
"split_keywords": [
"curl",
"pycurl",
"download"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f939a88f895a856c479105ee0d915caf7456a607427bfc96f808b7c16ceb2147",
"md5": "ec79eed5d7745194cd86d038a0a69fbe",
"sha256": "13d17ab5991582c77c3ce2add9ce0adbbcb3782f6fc0cb9c8e9a09f881f07d94"
},
"downloads": -1,
"filename": "curldl-1.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ec79eed5d7745194cd86d038a0a69fbe",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 24109,
"upload_time": "2023-08-19T07:19:42",
"upload_time_iso_8601": "2023-08-19T07:19:42.320090Z",
"url": "https://files.pythonhosted.org/packages/f9/39/a88f895a856c479105ee0d915caf7456a607427bfc96f808b7c16ceb2147/curldl-1.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "87efa5c7f54538ac45f4b5ad64f2e10bce747147b6e264925b43b74ff7b2ef1e",
"md5": "b1fe2acd5a753fef98e9c4505cb0a171",
"sha256": "b8c17f16cf9ce44e8ceb19e9bb01778cc8a7fd1b6011c2f752f630697a95f852"
},
"downloads": -1,
"filename": "curldl-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "b1fe2acd5a753fef98e9c4505cb0a171",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 47650,
"upload_time": "2023-08-19T07:19:43",
"upload_time_iso_8601": "2023-08-19T07:19:43.971463Z",
"url": "https://files.pythonhosted.org/packages/87/ef/a5c7f54538ac45f4b5ad64f2e10bce747147b6e264925b43b74ff7b2ef1e/curldl-1.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-08-19 07:19:43",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "noexec",
"github_project": "curldl",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "curldl"
}
