<p align="center">
<a href="https://github.com/kdeldycke/mail-deduplicate/">
<img src="https://raw.githubusercontent.com/kdeldycke/mail-deduplicate/main/docs/assets/mail-deduplicate-logo-header.png" alt="Mail Deduplicate">
</a>
</p>
[](https://pypi.python.org/pypi/mail-deduplicate)
[](https://pypi.python.org/pypi/mail-deduplicate)
[](https://github.com/kdeldycke/mail-deduplicate/actions/workflows/tests.yaml?query=branch%3Amain)
[](https://github.com/kdeldycke/mail-deduplicate/actions/workflows/docs.yaml?query=branch%3Amain)
[](https://app.codecov.io/gh/kdeldycke/mail-deduplicate)
[](https://doi.org/10.5281/zenodo.7364256)
**What is Mail Deduplicate?**
Provides the `mdedup` CLI, an utility to deduplicate mails from a set of boxes.
<p align="center">
<img src="https://raw.githubusercontent.com/kdeldycke/mail-deduplicate/main/docs/assets/cli-coloured-header.png" alt="Mail Deduplicate">
</p>
## Features
- Duplicate detection based on cherry-picked and normalized mail
headers.
- Fetch mails from multiple sources.
- Reads and writes to `mbox`, `maildir`, `babyl`, `mh` and `mmdf`
formats.
- Deduplication strategies based on size, content, timestamp, file path
or random choice.
- Copy, move or delete the resulting set of duplicates.
- Dry-run mode.
- Protection against false-positives with safety checks on size and content differences.
- Supports macOS, Linux and Windows.
- [Standalone executables](#executables) for Linux, macOS and Windows.
- Shell auto-completion for Bash, Zsh and Fish.
> ⚠️ **Warning**: Performances
>
> `mdedup` implementation is quite naive at the moment and everything resides in memory.
>
> If this is good enough for a volume of a couple of gigabytes, the more emails `mdedup` try to parse, the closer you'll reach the memory limits of your machine. In which case [`mdedup` will exit abrubtly](https://github.com/kdeldycke/mail-deduplicate/issues/362#issuecomment-1266743045), zapped by the [OOM killer](https://en.wikipedia.org/wiki/Out_of_memory) of your OS. Of course your mileage may vary depending on your hardware.
>
> You can influence implementation of this feature with pull requests, or [purchase of business support 🤝 and sponsorship 🫶](https://github.com/sponsors/kdeldycke).
## Example
<p align="center">
<img src="https://raw.githubusercontent.com/kdeldycke/mail-deduplicate/main/docs/assets/cli-coloured-run.png">
</p>
## Installation
### From sources
Easiest way is to install `mdedup` from sources with [`pipx`](https://pypa.github.io/pipx/):
```shell-session
$ pipx install mail-deduplicate
```
Other
[alternatives installation methods](https://kdeldycke.github.io/mail-deduplicate/install.html)
are available in the documentation.
### Executables
Standalone executables of `mdedup`'s latest version are available for several platforms and architectures:
| Platform | `x86_64` |
| ----------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| **Linux** | [Download `mdedup-linux-x64.bin`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-linux-x64.bin) |
| **macOS** | [Download `mdedup-macos-x64.bin`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-macos-x64.bin) |
| **Windows** | [Download `mdedup-windows-x64.exe`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-windows-x64.exe) |
Raw data
{
"_id": null,
"home_page": "https://github.com/kdeldycke/mail-deduplicate",
"name": "mail-deduplicate",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4.0",
"maintainer_email": "",
"keywords": "CLI,mail,email,maildir,mbox,deduplication,dedupe,cleanup,mailbox,Babyl,MH,mbox,MMDF",
"author": "Kevin Deldycke",
"author_email": "kevin@deldycke.com",
"download_url": "https://files.pythonhosted.org/packages/08/ee/290ad9200a17ca48d86ac5576f21937cd5fb24905b57935a3255f2cc4e47/mail_deduplicate-7.3.0.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n <a href=\"https://github.com/kdeldycke/mail-deduplicate/\">\n <img src=\"https://raw.githubusercontent.com/kdeldycke/mail-deduplicate/main/docs/assets/mail-deduplicate-logo-header.png\" alt=\"Mail Deduplicate\">\n </a>\n</p>\n\n[](https://pypi.python.org/pypi/mail-deduplicate)\n[](https://pypi.python.org/pypi/mail-deduplicate)\n[](https://github.com/kdeldycke/mail-deduplicate/actions/workflows/tests.yaml?query=branch%3Amain)\n[](https://github.com/kdeldycke/mail-deduplicate/actions/workflows/docs.yaml?query=branch%3Amain)\n[](https://app.codecov.io/gh/kdeldycke/mail-deduplicate)\n[](https://doi.org/10.5281/zenodo.7364256)\n\n**What is Mail Deduplicate?**\n\nProvides the `mdedup` CLI, an utility to deduplicate mails from a set of boxes.\n\n<p align=\"center\">\n <img src=\"https://raw.githubusercontent.com/kdeldycke/mail-deduplicate/main/docs/assets/cli-coloured-header.png\" alt=\"Mail Deduplicate\">\n</p>\n\n## Features\n\n- Duplicate detection based on cherry-picked and normalized mail\n headers.\n- Fetch mails from multiple sources.\n- Reads and writes to `mbox`, `maildir`, `babyl`, `mh` and `mmdf`\n formats.\n- Deduplication strategies based on size, content, timestamp, file path\n or random choice.\n- Copy, move or delete the resulting set of duplicates.\n- Dry-run mode.\n- Protection against false-positives with safety checks on size and content differences.\n- Supports macOS, Linux and Windows.\n- [Standalone executables](#executables) for Linux, macOS and Windows.\n- Shell auto-completion for Bash, Zsh and Fish.\n\n> \u26a0\ufe0f **Warning**: Performances\n>\n> `mdedup` implementation is quite naive at the moment and everything resides in memory.\n>\n> If this is good enough for a volume of a couple of gigabytes, the more emails `mdedup` try to parse, the closer you'll reach the memory limits of your machine. In which case [`mdedup` will exit abrubtly](https://github.com/kdeldycke/mail-deduplicate/issues/362#issuecomment-1266743045), zapped by the [OOM killer](https://en.wikipedia.org/wiki/Out_of_memory) of your OS. Of course your mileage may vary depending on your hardware.\n>\n> You can influence implementation of this feature with pull requests, or [purchase of business support \ud83e\udd1d and sponsorship \ud83e\udef6](https://github.com/sponsors/kdeldycke).\n\n## Example\n\n<p align=\"center\">\n <img src=\"https://raw.githubusercontent.com/kdeldycke/mail-deduplicate/main/docs/assets/cli-coloured-run.png\">\n</p>\n\n## Installation\n\n### From sources\n\nEasiest way is to install `mdedup` from sources with [`pipx`](https://pypa.github.io/pipx/):\n\n```shell-session\n$ pipx install mail-deduplicate\n```\n\nOther\n[alternatives installation methods](https://kdeldycke.github.io/mail-deduplicate/install.html)\nare available in the documentation.\n\n### Executables\n\nStandalone executables of `mdedup`'s latest version are available for several platforms and architectures:\n\n| Platform | `x86_64` |\n| ----------- | ---------------------------------------------------------------------------------------------------------------------------------- |\n| **Linux** | [Download `mdedup-linux-x64.bin`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-linux-x64.bin) |\n| **macOS** | [Download `mdedup-macos-x64.bin`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-macos-x64.bin) |\n| **Windows** | [Download `mdedup-windows-x64.exe`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-windows-x64.exe) |\n\n",
"bugtrack_url": null,
"license": "GPL-2.0-or-later",
"summary": "\ud83d\udce7 CLI to deduplicate mails from mail boxes.",
"version": "7.3.0",
"project_urls": {
"Changelog": "https://kdeldycke.github.io/mail-deduplicate/changelog.html",
"Documentation": "https://kdeldycke.github.io/mail-deduplicate",
"Funding": "https://github.com/sponsors/kdeldycke",
"Homepage": "https://github.com/kdeldycke/mail-deduplicate",
"Issues": "https://github.com/kdeldycke/mail-deduplicate/issues",
"Repository": "https://github.com/kdeldycke/mail-deduplicate"
},
"split_keywords": [
"cli",
"mail",
"email",
"maildir",
"mbox",
"deduplication",
"dedupe",
"cleanup",
"mailbox",
"babyl",
"mh",
"mbox",
"mmdf"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "463aba7fc67621e0abdea6f967babdc496d2ae2afe17a1399b627891bafd0e41",
"md5": "32a8139e104fb86a5fd14e6be51ba2c4",
"sha256": "2b70de3fe817cc2ebdab97233623b0a9e481601077e96b4422bb525000ba78be"
},
"downloads": -1,
"filename": "mail_deduplicate-7.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "32a8139e104fb86a5fd14e6be51ba2c4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4.0",
"size": 40157,
"upload_time": "2023-11-15T12:15:50",
"upload_time_iso_8601": "2023-11-15T12:15:50.706901Z",
"url": "https://files.pythonhosted.org/packages/46/3a/ba7fc67621e0abdea6f967babdc496d2ae2afe17a1399b627891bafd0e41/mail_deduplicate-7.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "08ee290ad9200a17ca48d86ac5576f21937cd5fb24905b57935a3255f2cc4e47",
"md5": "86853cf40dcbefc54e2c58e78630cf82",
"sha256": "4e8b175d8e2f97dae1a0a57772e63e15e33d73761ced0ed3ff08abcd4056d185"
},
"downloads": -1,
"filename": "mail_deduplicate-7.3.0.tar.gz",
"has_sig": false,
"md5_digest": "86853cf40dcbefc54e2c58e78630cf82",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4.0",
"size": 29152,
"upload_time": "2023-11-15T12:15:52",
"upload_time_iso_8601": "2023-11-15T12:15:52.323481Z",
"url": "https://files.pythonhosted.org/packages/08/ee/290ad9200a17ca48d86ac5576f21937cd5fb24905b57935a3255f2cc4e47/mail_deduplicate-7.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-15 12:15:52",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "kdeldycke",
"github_project": "mail-deduplicate",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "mail-deduplicate"
}
JSON
