markdown-image-downloader

Name	markdown-image-downloader JSON
Version	0.1.3 JSON
	download
home_page	None
Summary	A tool to download and organize images referenced in markdown files
upload_time	2024-11-02 12:44:26
maintainer	None
docs_url	None
author	None
requires_python	>=3.9
license	The MIT License (MIT) Copyright 2024 mufidu Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
keywords	markdown image downloader automation
VCS
bugtrack_url
requirements	No requirements were recorded.
Travis-CI	No Travis.
coveralls test coverage	No coveralls.

            # Markdown Image Downloader

A Python script that automatically downloads and manages images referenced in markdown files, storing them locally in an `_attachments` folder. This script is particularly useful for maintaining local copies of images in markdown documentation and ensuring consistent image availability.

Or just for Obsidian's Readwise export, which I made this for.

Previously hosted on [GitHub Gist](https://gist.github.com/mufidu/f7b795f844f1ee4dc78e55123d5a398b). Moved here to allow for easier maintenance and contributions, if any. Also published to [PyPI](https://pypi.org/project/markdown-image-downloader) for convenience.

## Features

- Uses custom HTTP headers to avoid download blocks
- Downloads images from URLs referenced in markdown files
- Creates local copies of images in an `_attachments` directory
- Automatically updates links in the markdown files with new local image paths
- Compresses large images to reduce storage space
- Supports multithreaded concurrent downloads
- Uses rate limit to prevent server overload and download blocks
- Progress bar for tracking download status
- Maintains detailed logging of error operations
- Sanitizes filenames for cross-platform compatibility
- Supports for rerunning the script without re-downloading images

## Requirements

```
Python 3.9+
```

## Installation

Install directly from PyPI using pip:

```bash
pip install markdown-image-downloader
```

## Usage

Run the package from the command line, providing the folder containing your markdown files as an argument:

```bash
markdown-image-downloader <folder_name>
```

### Example
```bash
markdown-image-downloader ../Readwise/Articles
```

This will:
1. Scan all markdown files in the `../Readwise/Articles` folder
2. Download any images referenced in the markdown files
3. Store them in `../Readwise/Articles/_attachments`
4. Update the markdown files to reference the local copies

## How It Works

1. **Scanning**: The script scans all `.md` files in the specified folder for image references.
2. **Downloading**: For each image URL found:
   - Downloads the image if it's not already in `_attachments`
   - Compresses images larger than 500KB while maintaining quality
   - Generates unique filenames based on content hash
3. **Organization**: Creates an `_attachments` folder to store all images
4. **Updating**: Updates markdown files to reference the local copies in `_attachments`

## Features in Detail

### Image Compression
- Automatically compresses large images
- Maintains reasonable quality through progressive compression
- Converts RGBA images to RGB with white background

### Filename Handling
- Preserves original filenames
- Sanitizes filenames for cross-platform compatibility

### Concurrent Processing
- Uses ThreadPoolExecutor for parallel downloads
- Includes progress bar for tracking downloads
- Implements rate limiting to prevent server overload

### Error Handling
- Comprehensive logging of all operations
- Graceful handling of download failures
- Skips already processed images

## Logging

The script creates detailed logs in a `logs` directory:
- Location: `./logs/image_downloader.log`
- Includes timestamps, operation details, and error messages
- New log file created for each run

## Limitations

- Only processes image links in markdown format: `![alt](url)`
- Requires internet connection for downloading external images
- May be rate-limited or just straight denied by some servers
- SVG files are downloaded but not compressed

## Contributing

Feel free to submit issues, fork the repository, and create pull requests for any improvements.

## License

This project is available under the MIT License.

            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "markdown-image-downloader",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.9",
    "maintainer_email": null,
    "keywords": "markdown, image, downloader, automation",
    "author": null,
    "author_email": "mufidu <mufid.to@gmail.com>",
    "download_url": "https://files.pythonhosted.org/packages/b9/59/ea2dafb5f784ac41d7af2b7bdd72862ba297360d72ab4976bb1bf3f5233a/markdown_image_downloader-0.1.3.tar.gz",
    "platform": null,
    "description": "# Markdown Image Downloader\n\nA Python script that automatically downloads and manages images referenced in markdown files, storing them locally in an `_attachments` folder. This script is particularly useful for maintaining local copies of images in markdown documentation and ensuring consistent image availability.\n\nOr just for Obsidian's Readwise export, which I made this for.\n\nPreviously hosted on [GitHub Gist](https://gist.github.com/mufidu/f7b795f844f1ee4dc78e55123d5a398b). Moved here to allow for easier maintenance and contributions, if any. Also published to [PyPI](https://pypi.org/project/markdown-image-downloader) for convenience.\n\n## Features\n\n- Uses custom HTTP headers to avoid download blocks\n- Downloads images from URLs referenced in markdown files\n- Creates local copies of images in an `_attachments` directory\n- Automatically updates links in the markdown files with new local image paths\n- Compresses large images to reduce storage space\n- Supports multithreaded concurrent downloads\n- Uses rate limit to prevent server overload and download blocks\n- Progress bar for tracking download status\n- Maintains detailed logging of error operations\n- Sanitizes filenames for cross-platform compatibility\n- Supports for rerunning the script without re-downloading images\n\n## Requirements\n\n```\nPython 3.9+\n```\n\n## Installation\n\nInstall directly from PyPI using pip:\n\n```bash\npip install markdown-image-downloader\n```\n\n## Usage\n\nRun the package from the command line, providing the folder containing your markdown files as an argument:\n\n```bash\nmarkdown-image-downloader <folder_name>\n```\n\n### Example\n```bash\nmarkdown-image-downloader ../Readwise/Articles\n```\n\nThis will:\n1. Scan all markdown files in the `../Readwise/Articles` folder\n2. Download any images referenced in the markdown files\n3. Store them in `../Readwise/Articles/_attachments`\n4. Update the markdown files to reference the local copies\n\n## How It Works\n\n1. **Scanning**: The script scans all `.md` files in the specified folder for image references.\n2. **Downloading**: For each image URL found:\n   - Downloads the image if it's not already in `_attachments`\n   - Compresses images larger than 500KB while maintaining quality\n   - Generates unique filenames based on content hash\n3. **Organization**: Creates an `_attachments` folder to store all images\n4. **Updating**: Updates markdown files to reference the local copies in `_attachments`\n\n## Features in Detail\n\n### Image Compression\n- Automatically compresses large images\n- Maintains reasonable quality through progressive compression\n- Converts RGBA images to RGB with white background\n\n### Filename Handling\n- Preserves original filenames\n- Sanitizes filenames for cross-platform compatibility\n\n### Concurrent Processing\n- Uses ThreadPoolExecutor for parallel downloads\n- Includes progress bar for tracking downloads\n- Implements rate limiting to prevent server overload\n\n### Error Handling\n- Comprehensive logging of all operations\n- Graceful handling of download failures\n- Skips already processed images\n\n## Logging\n\nThe script creates detailed logs in a `logs` directory:\n- Location: `./logs/image_downloader.log`\n- Includes timestamps, operation details, and error messages\n- New log file created for each run\n\n## Limitations\n\n- Only processes image links in markdown format: `![alt](url)`\n- Requires internet connection for downloading external images\n- May be rate-limited or just straight denied by some servers\n- SVG files are downloaded but not compressed\n\n## Contributing\n\nFeel free to submit issues, fork the repository, and create pull requests for any improvements.\n\n## License\n\nThis project is available under the MIT License.\n",
    "bugtrack_url": null,
    "license": "The MIT License (MIT)  Copyright 2024 mufidu  Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:  The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.  THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.",
    "summary": "A tool to download and organize images referenced in markdown files",
    "version": "0.1.3",
    "project_urls": {
        "Homepage": "https://github.com/mufidu/markdown-image-downloader",
        "Source": "https://github.com/mufidu/markdown-image-downloader.git"
    },
    "split_keywords": [
        "markdown",
        " image",
        " downloader",
        " automation"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "313231ef16e5a7aba77e4827f034468567faad56366aaf57d79da70fd1368b80",
                "md5": "d405c476c6765baf58b0e2a83b2ea222",
                "sha256": "e92e942cb431f3cde591792f21636e65d500442be3fcb3173a4f337ca768a8b7"
            },
            "downloads": -1,
            "filename": "markdown_image_downloader-0.1.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "d405c476c6765baf58b0e2a83b2ea222",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.9",
            "size": 9085,
            "upload_time": "2024-11-02T12:44:24",
            "upload_time_iso_8601": "2024-11-02T12:44:24.909518Z",
            "url": "https://files.pythonhosted.org/packages/31/32/31ef16e5a7aba77e4827f034468567faad56366aaf57d79da70fd1368b80/markdown_image_downloader-0.1.3-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "b959ea2dafb5f784ac41d7af2b7bdd72862ba297360d72ab4976bb1bf3f5233a",
                "md5": "d1d868705452404ff8a14d898552de40",
                "sha256": "4965b1fbe3d7e85266096c04bb3fa9fa8a2c6c47d02cf9284346d634e3b73a96"
            },
            "downloads": -1,
            "filename": "markdown_image_downloader-0.1.3.tar.gz",
            "has_sig": false,
            "md5_digest": "d1d868705452404ff8a14d898552de40",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.9",
            "size": 8693,
            "upload_time": "2024-11-02T12:44:26",
            "upload_time_iso_8601": "2024-11-02T12:44:26.442313Z",
            "url": "https://files.pythonhosted.org/packages/b9/59/ea2dafb5f784ac41d7af2b7bdd72862ba297360d72ab4976bb1bf3f5233a/markdown_image_downloader-0.1.3.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-11-02 12:44:26",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "mufidu",
    "github_project": "markdown-image-downloader",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "lcname": "markdown-image-downloader"
}