| Name | afetchx JSON |
| Version |
1.1.1
JSON |
| download |
| home_page | None |
| Summary | None |
| upload_time | 2024-11-03 16:55:38 |
| maintainer | None |
| docs_url | None |
| author | DJ Stomp |
| requires_python | <4.0,>=3.10 |
| license | None |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# afetchx
Asynchronously fetch image URLs using httpx.
`afetchx` parses command-line arguments to determine the list of image URLs to fetch, the maximum number of concurrent connections, the cooldown time between requests, an optional JSON file containing URLs, and output options. It then fetches the images, saves them in the specified output directory, and prints the number of successfully downloaded images.
## Usage
Invoke the script with
```bash
python -m afetchx {args}
```
## Command-line arguments
- `-n`, `--max_connections`: Max number of concurrent connections (default: 5).
- `-c`, `--cooldown`: Cooldown time between requests in seconds (default: 1.0).
- `-j`, `--json`: Path to a JSON file containing URLs to fetch.
- `-o`, `--output-dir`: Directory to save downloaded images (default: `./downloaded` in the current working directory).
- `--filename-format`: Lambda function as a string to format filenames based on the URL (e.g., `"lambda url: url.split('/')[-1].split('.')[0] + '.png'"`).
- `urls`: List of image URLs to fetch (positional arguments).
### *Note*: You must satisfy exactly one of the following conditions
1. Use the `--json` (`-j`) argument to specify a JSON file containing URLs.
2. Provide one or more URLs directly as positional arguments.
## Examples
### Basic Usage
Download images from a JSON file with a cooldown of 1 second between each download and a maximum of 5 concurrent connections:
```bash
python -m afetchx -j urls.json -c 1.0 -n 5
```
### Specify Output Directory
Download images from URLs provided as positional arguments and save them to a custom directory:
```bash
python -m afetchx -c 1.0 -n 5 -o ./downloaded_pictures "https://unsplash.it/256.jpg" "https://unsplash.it/512.jpg"
```
### Use Custom Filename Format
Download images and apply a custom filename format based on the URL:
```bash
python -m afetchx -c 1.0 -n 5 --filename-format "lambda url: url.split('/')[-1].split('.')[0] + '.png'" "https://example.com/image1.png" "https://example.com/image2.jpg"
```
In this example, each downloaded image will be named according to the lambda function. The filename format option can be especially useful for sanitizing or modifying filenames before saving.
## Error Handling
If both URLs and a JSON file are provided, the script will print an error and exit. Likewise, if neither URLs nor a JSON file are provided, the script will print an error and exit. Ensure exactly one source for URLs.
Raw data
{
"_id": null,
"home_page": null,
"name": "afetchx",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.10",
"maintainer_email": null,
"keywords": null,
"author": "DJ Stomp",
"author_email": "85457381+DJStompZone@users.noreply.github.com",
"download_url": "https://files.pythonhosted.org/packages/da/df/b305a295265cbf478bb1d3ef23444a8800e2a5e88160fb27bfcbd8142012/afetchx-1.1.1.tar.gz",
"platform": null,
"description": "# afetchx\n\nAsynchronously fetch image URLs using httpx.\n\n`afetchx` parses command-line arguments to determine the list of image URLs to fetch, the maximum number of concurrent connections, the cooldown time between requests, an optional JSON file containing URLs, and output options. It then fetches the images, saves them in the specified output directory, and prints the number of successfully downloaded images.\n\n## Usage\n\nInvoke the script with\n\n```bash\npython -m afetchx {args}\n```\n\n## Command-line arguments\n\n- `-n`, `--max_connections`: Max number of concurrent connections (default: 5).\n- `-c`, `--cooldown`: Cooldown time between requests in seconds (default: 1.0).\n- `-j`, `--json`: Path to a JSON file containing URLs to fetch.\n- `-o`, `--output-dir`: Directory to save downloaded images (default: `./downloaded` in the current working directory).\n- `--filename-format`: Lambda function as a string to format filenames based on the URL (e.g., `\"lambda url: url.split('/')[-1].split('.')[0] + '.png'\"`).\n- `urls`: List of image URLs to fetch (positional arguments).\n\n### *Note*: You must satisfy exactly one of the following conditions\n\n1. Use the `--json` (`-j`) argument to specify a JSON file containing URLs.\n2. Provide one or more URLs directly as positional arguments.\n\n## Examples\n\n### Basic Usage\n\nDownload images from a JSON file with a cooldown of 1 second between each download and a maximum of 5 concurrent connections:\n\n```bash\npython -m afetchx -j urls.json -c 1.0 -n 5\n```\n\n### Specify Output Directory\n\nDownload images from URLs provided as positional arguments and save them to a custom directory:\n\n```bash\npython -m afetchx -c 1.0 -n 5 -o ./downloaded_pictures \"https://unsplash.it/256.jpg\" \"https://unsplash.it/512.jpg\"\n```\n\n### Use Custom Filename Format\n\nDownload images and apply a custom filename format based on the URL:\n\n```bash\npython -m afetchx -c 1.0 -n 5 --filename-format \"lambda url: url.split('/')[-1].split('.')[0] + '.png'\" \"https://example.com/image1.png\" \"https://example.com/image2.jpg\"\n```\n\nIn this example, each downloaded image will be named according to the lambda function. The filename format option can be especially useful for sanitizing or modifying filenames before saving.\n\n## Error Handling\n\nIf both URLs and a JSON file are provided, the script will print an error and exit. Likewise, if neither URLs nor a JSON file are provided, the script will print an error and exit. Ensure exactly one source for URLs.\n",
"bugtrack_url": null,
"license": null,
"summary": null,
"version": "1.1.1",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "de41505979237faa9925852a8a176cb310560fda2b221980c962173c39da3628",
"md5": "b16b4583a7fd9ba0e478b8b0ad4dc9b8",
"sha256": "0cfe72b57997097a230eea343e9743c95fd057146d1003e804d2e0329da6c6f3"
},
"downloads": -1,
"filename": "afetchx-1.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b16b4583a7fd9ba0e478b8b0ad4dc9b8",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.10",
"size": 4811,
"upload_time": "2024-11-03T16:55:37",
"upload_time_iso_8601": "2024-11-03T16:55:37.373195Z",
"url": "https://files.pythonhosted.org/packages/de/41/505979237faa9925852a8a176cb310560fda2b221980c962173c39da3628/afetchx-1.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "dadfb305a295265cbf478bb1d3ef23444a8800e2a5e88160fb27bfcbd8142012",
"md5": "0636390ba8925834b14edd6fdccac8fd",
"sha256": "9bf1db1d78be4ca8e99ee628cb5c29211a0b2ee7237f7093d467e88d9db1522c"
},
"downloads": -1,
"filename": "afetchx-1.1.1.tar.gz",
"has_sig": false,
"md5_digest": "0636390ba8925834b14edd6fdccac8fd",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.10",
"size": 4201,
"upload_time": "2024-11-03T16:55:38",
"upload_time_iso_8601": "2024-11-03T16:55:38.787374Z",
"url": "https://files.pythonhosted.org/packages/da/df/b305a295265cbf478bb1d3ef23444a8800e2a5e88160fb27bfcbd8142012/afetchx-1.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-03 16:55:38",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "afetchx"
}
