<p align="center">
<a href="https://github.com/kdeldycke/mail-deduplicate/">
<img src="https://raw.githubusercontent.com/kdeldycke/mail-deduplicate/main/docs/assets/mail-deduplicate-logo-header.png" alt="Mail Deduplicate">
</a>
</p>
[](https://pypi.python.org/pypi/mail-deduplicate)
[](https://pypi.python.org/pypi/mail-deduplicate)
[](https://github.com/kdeldycke/mail-deduplicate/actions/workflows/tests.yaml?query=branch%3Amain)
[](https://github.com/kdeldycke/mail-deduplicate/actions/workflows/docs.yaml?query=branch%3Amain)
[](https://app.codecov.io/gh/kdeldycke/mail-deduplicate)
[](https://doi.org/10.5281/zenodo.7364256)
**What is Mail Deduplicate?**
Provides the `mdedup` CLI, an utility to deduplicate mails from a set of boxes.
<p align="center">
<img src="https://raw.githubusercontent.com/kdeldycke/mail-deduplicate/main/docs/assets/cli-coloured-header.png" alt="Mail Deduplicate">
</p>
## Features
- Duplicate detection based on cherry-picked and normalized mail
headers.
- Fetch mails from multiple sources.
- Reads and writes to `mbox`, `maildir`, `babyl`, `mh` and `mmdf`
formats.
- Deduplication strategies based on size, content, timestamp, file path
or random choice.
- Copy, move or delete the resulting set of duplicates.
- Dry-run mode.
- Protection against false-positives with safety checks on size and content differences.
- Supports macOS, Linux and Windows.
- [Standalone executables](#executables) for Linux, macOS and Windows.
- Shell auto-completion for Bash, Zsh and Fish.
> ⚠️ **Warning**: Performances
>
> `mdedup` implementation is quite naive at the moment and everything resides in memory.
>
> If this is good enough for a volume of a couple of gigabytes, the more emails `mdedup` try to parse, the closer you'll reach the memory limits of your machine. In which case [`mdedup` will exit abrubtly](https://github.com/kdeldycke/mail-deduplicate/issues/362#issuecomment-1266743045), zapped by the [OOM killer](https://en.wikipedia.org/wiki/Out_of_memory) of your OS. Of course your mileage may vary depending on your hardware.
>
> You can influence implementation of this feature with pull requests, or [purchase of business support 🤝 and sponsorship 🫶](https://github.com/sponsors/kdeldycke).
## Example
<p align="center">
<img src="https://raw.githubusercontent.com/kdeldycke/mail-deduplicate/main/docs/assets/cli-coloured-run.png">
</p>
## Installation
### Python
Easiest way is to install `mdedup` from sources with [`pipx`](https://pipx.pypa.io):
```shell-session
$ pipx install mail-deduplicate
```
Other
[alternatives installation methods](https://kdeldycke.github.io/mail-deduplicate/install.html)
are available in the documentation.
### Executables
Standalone binaries of `mdedup`'s latest version are available for several platforms and architectures:
| Platform | `x86_64` | `arm64` |
| ----------- | ---------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| **Linux** | [Download `mdedup-linux-x64.bin`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-linux-x64.bin) | |
| **macOS** | [Download `mdedup-macos-x64.bin`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-macos-x64.bin) | [Download `mdedup-macos-arm64.bin`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-macos-arm64.bin) |
| **Windows** | [Download `mdedup-windows-x64.exe`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-windows-x64.exe) | |
Raw data
{
"_id": null,
"home_page": null,
"name": "mail-deduplicate",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "CLI, mail, email, maildir, mbox, deduplication, dedupe, cleanup, mailbox, Babyl, MH, mbox, MMDF",
"author": null,
"author_email": "Kevin Deldycke <kevin@deldycke.com>",
"download_url": "https://files.pythonhosted.org/packages/3d/d5/5149cebc32348416cc1fbd45d58258c33f596d5bfe2b564e6c7318e56433/mail_deduplicate-7.6.1.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n <a href=\"https://github.com/kdeldycke/mail-deduplicate/\">\n <img src=\"https://raw.githubusercontent.com/kdeldycke/mail-deduplicate/main/docs/assets/mail-deduplicate-logo-header.png\" alt=\"Mail Deduplicate\">\n </a>\n</p>\n\n[](https://pypi.python.org/pypi/mail-deduplicate)\n[](https://pypi.python.org/pypi/mail-deduplicate)\n[](https://github.com/kdeldycke/mail-deduplicate/actions/workflows/tests.yaml?query=branch%3Amain)\n[](https://github.com/kdeldycke/mail-deduplicate/actions/workflows/docs.yaml?query=branch%3Amain)\n[](https://app.codecov.io/gh/kdeldycke/mail-deduplicate)\n[](https://doi.org/10.5281/zenodo.7364256)\n\n**What is Mail Deduplicate?**\n\nProvides the `mdedup` CLI, an utility to deduplicate mails from a set of boxes.\n\n<p align=\"center\">\n <img src=\"https://raw.githubusercontent.com/kdeldycke/mail-deduplicate/main/docs/assets/cli-coloured-header.png\" alt=\"Mail Deduplicate\">\n</p>\n\n## Features\n\n- Duplicate detection based on cherry-picked and normalized mail\n headers.\n- Fetch mails from multiple sources.\n- Reads and writes to `mbox`, `maildir`, `babyl`, `mh` and `mmdf`\n formats.\n- Deduplication strategies based on size, content, timestamp, file path\n or random choice.\n- Copy, move or delete the resulting set of duplicates.\n- Dry-run mode.\n- Protection against false-positives with safety checks on size and content differences.\n- Supports macOS, Linux and Windows.\n- [Standalone executables](#executables) for Linux, macOS and Windows.\n- Shell auto-completion for Bash, Zsh and Fish.\n\n> \u26a0\ufe0f **Warning**: Performances\n>\n> `mdedup` implementation is quite naive at the moment and everything resides in memory.\n>\n> If this is good enough for a volume of a couple of gigabytes, the more emails `mdedup` try to parse, the closer you'll reach the memory limits of your machine. In which case [`mdedup` will exit abrubtly](https://github.com/kdeldycke/mail-deduplicate/issues/362#issuecomment-1266743045), zapped by the [OOM killer](https://en.wikipedia.org/wiki/Out_of_memory) of your OS. Of course your mileage may vary depending on your hardware.\n>\n> You can influence implementation of this feature with pull requests, or [purchase of business support \ud83e\udd1d and sponsorship \ud83e\udef6](https://github.com/sponsors/kdeldycke).\n\n## Example\n\n<p align=\"center\">\n <img src=\"https://raw.githubusercontent.com/kdeldycke/mail-deduplicate/main/docs/assets/cli-coloured-run.png\">\n</p>\n\n## Installation\n\n### Python\n\nEasiest way is to install `mdedup` from sources with [`pipx`](https://pipx.pypa.io):\n\n```shell-session\n$ pipx install mail-deduplicate\n```\n\nOther\n[alternatives installation methods](https://kdeldycke.github.io/mail-deduplicate/install.html)\nare available in the documentation.\n\n### Executables\n\nStandalone binaries of `mdedup`'s latest version are available for several platforms and architectures:\n\n| Platform | `x86_64` | `arm64` |\n| ----------- | ---------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |\n| **Linux** | [Download `mdedup-linux-x64.bin`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-linux-x64.bin) | |\n| **macOS** | [Download `mdedup-macos-x64.bin`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-macos-x64.bin) | [Download `mdedup-macos-arm64.bin`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-macos-arm64.bin) |\n| **Windows** | [Download `mdedup-windows-x64.exe`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-windows-x64.exe) | |\n",
"bugtrack_url": null,
"license": null,
"summary": "\ud83d\udce7 CLI to deduplicate mails from mail boxes",
"version": "7.6.1",
"project_urls": {
"Changelog": "https://github.com/kdeldycke/mail-deduplicate/blob/main/changelog.md",
"Documentation": "https://kdeldycke.github.io/mail-deduplicate",
"Funding": "https://github.com/sponsors/kdeldycke",
"Homepage": "https://github.com/kdeldycke/mail-deduplicate",
"Issues": "https://github.com/kdeldycke/mail-deduplicate/issues",
"Repository": "https://github.com/kdeldycke/mail-deduplicate"
},
"split_keywords": [
"cli",
" mail",
" email",
" maildir",
" mbox",
" deduplication",
" dedupe",
" cleanup",
" mailbox",
" babyl",
" mh",
" mbox",
" mmdf"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "7dc92025e5d64da63ad8ec9a18b3231bc999f46043bb71953445cf181d99c310",
"md5": "e33fcbfb82084db51e60e00120400d00",
"sha256": "e51e7997c2e3b89cfbc2dcca67385d6771e58a5727b56176b14537327c3470eb"
},
"downloads": -1,
"filename": "mail_deduplicate-7.6.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e33fcbfb82084db51e60e00120400d00",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 30882,
"upload_time": "2024-11-30T08:27:03",
"upload_time_iso_8601": "2024-11-30T08:27:03.508774Z",
"url": "https://files.pythonhosted.org/packages/7d/c9/2025e5d64da63ad8ec9a18b3231bc999f46043bb71953445cf181d99c310/mail_deduplicate-7.6.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "3dd55149cebc32348416cc1fbd45d58258c33f596d5bfe2b564e6c7318e56433",
"md5": "92a83ec575a4bdd27dd0ecf17aabfcf6",
"sha256": "48b9c955045838c342cb5c5f57ca1a8456e6d597a72464237a86d6aff8d3ad42"
},
"downloads": -1,
"filename": "mail_deduplicate-7.6.1.tar.gz",
"has_sig": false,
"md5_digest": "92a83ec575a4bdd27dd0ecf17aabfcf6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 30821,
"upload_time": "2024-11-30T08:27:27",
"upload_time_iso_8601": "2024-11-30T08:27:27.825452Z",
"url": "https://files.pythonhosted.org/packages/3d/d5/5149cebc32348416cc1fbd45d58258c33f596d5bfe2b564e6c7318e56433/mail_deduplicate-7.6.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-30 08:27:27",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "kdeldycke",
"github_project": "mail-deduplicate",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "mail-deduplicate"
}
JSON
