<p align="center">
<a href="https://github.com/kdeldycke/mail-deduplicate/">
<img src="https://raw.githubusercontent.com/kdeldycke/mail-deduplicate/main/docs/assets/mail-deduplicate-logo-header.png" alt="Mail Deduplicate">
</a>
</p>
[](https://pypi.python.org/pypi/mail-deduplicate)
[](https://pypi.python.org/pypi/mail-deduplicate)
[](https://github.com/kdeldycke/mail-deduplicate/actions/workflows/tests.yaml?query=branch%3Amain)
[](https://github.com/kdeldycke/mail-deduplicate/actions/workflows/docs.yaml?query=branch%3Amain)
[](https://app.codecov.io/gh/kdeldycke/mail-deduplicate)
[](https://doi.org/10.5281/zenodo.7364256)
**What is Mail Deduplicate?**
Provides the `mdedup` CLI, an utility to deduplicate mails from a set of boxes.
<p align="center">
<img src="https://raw.githubusercontent.com/kdeldycke/mail-deduplicate/main/docs/assets/cli-coloured-header.png" alt="Mail Deduplicate">
</p>
## Features
- Duplicate detection based on cherry-picked and normalized mail
headers.
- Fetch mails from multiple sources.
- Reads and writes to `mbox`, `maildir`, `babyl`, `mh` and `mmdf`
formats.
- Deduplication strategies based on size, content, timestamp, file path
or random choice.
- Copy, move or delete the resulting set of duplicates.
- Dry-run mode.
- Protection against false-positives with safety checks on size and content differences.
- Supports macOS, Linux and Windows.
- [Standalone executables](#executables) for Linux, macOS and Windows.
- Shell auto-completion for Bash, Zsh and Fish.
> ⚠️ **Warning**: Performances
>
> `mdedup` implementation is quite naive at the moment and everything resides in memory.
>
> If this is good enough for a volume of a couple of gigabytes, the more emails `mdedup` try to parse, the closer you'll reach the memory limits of your machine. In which case [`mdedup` will exit abrubtly](https://github.com/kdeldycke/mail-deduplicate/issues/362#issuecomment-1266743045), zapped by the [OOM killer](https://en.wikipedia.org/wiki/Out_of_memory) of your OS. Of course your mileage may vary depending on your hardware.
>
> You can influence implementation of this feature with pull requests, or [purchase of business support 🤝 and sponsorship 🫶](https://github.com/sponsors/kdeldycke).
## Example
<p align="center">
<img src="https://raw.githubusercontent.com/kdeldycke/mail-deduplicate/main/docs/assets/cli-coloured-run.png">
</p>
## Installation
### From sources
Easiest way is to install `mdedup` from sources with [`pipx`](https://pipx.pypa.io):
```shell-session
$ pipx install mail-deduplicate
```
Other
[alternatives installation methods](https://kdeldycke.github.io/mail-deduplicate/install.html)
are available in the documentation.
### Executables
Standalone executables of `mdedup`'s latest version are available for several platforms and architectures:
| Platform | `x86_64` |`arm64` |
| ----------- | ---------------------------------------------------------------------------------------------------------------------------------- |-------------------------------------------------------------------------------------------------------------------------------- |
| **Linux** | [Download `mdedup-linux-x64.bin`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-linux-x64.bin) | |
| **macOS** | [Download `mdedup-macos-x64.bin`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-macos-x64.bin) | [Download `mdedup-macos-arm64.bin`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-macos-arm64.bin) |
| **Windows** | [Download `mdedup-windows-x64.exe`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-windows-x64.exe) | |
Raw data
{
"_id": null,
"home_page": null,
"name": "mail-deduplicate",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "CLI, mail, email, maildir, mbox, deduplication, dedupe, cleanup, mailbox, Babyl, MH, mbox, MMDF",
"author": null,
"author_email": "Kevin Deldycke <kevin@deldycke.com>",
"download_url": "https://files.pythonhosted.org/packages/a0/83/8b84142e6f383d61497aa965124f233d8244c967f2ff58cb7cf4d056c38c/mail_deduplicate-7.5.0.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n <a href=\"https://github.com/kdeldycke/mail-deduplicate/\">\n <img src=\"https://raw.githubusercontent.com/kdeldycke/mail-deduplicate/main/docs/assets/mail-deduplicate-logo-header.png\" alt=\"Mail Deduplicate\">\n </a>\n</p>\n\n[](https://pypi.python.org/pypi/mail-deduplicate)\n[](https://pypi.python.org/pypi/mail-deduplicate)\n[](https://github.com/kdeldycke/mail-deduplicate/actions/workflows/tests.yaml?query=branch%3Amain)\n[](https://github.com/kdeldycke/mail-deduplicate/actions/workflows/docs.yaml?query=branch%3Amain)\n[](https://app.codecov.io/gh/kdeldycke/mail-deduplicate)\n[](https://doi.org/10.5281/zenodo.7364256)\n\n**What is Mail Deduplicate?**\n\nProvides the `mdedup` CLI, an utility to deduplicate mails from a set of boxes.\n\n<p align=\"center\">\n <img src=\"https://raw.githubusercontent.com/kdeldycke/mail-deduplicate/main/docs/assets/cli-coloured-header.png\" alt=\"Mail Deduplicate\">\n</p>\n\n## Features\n\n- Duplicate detection based on cherry-picked and normalized mail\n headers.\n- Fetch mails from multiple sources.\n- Reads and writes to `mbox`, `maildir`, `babyl`, `mh` and `mmdf`\n formats.\n- Deduplication strategies based on size, content, timestamp, file path\n or random choice.\n- Copy, move or delete the resulting set of duplicates.\n- Dry-run mode.\n- Protection against false-positives with safety checks on size and content differences.\n- Supports macOS, Linux and Windows.\n- [Standalone executables](#executables) for Linux, macOS and Windows.\n- Shell auto-completion for Bash, Zsh and Fish.\n\n> \u26a0\ufe0f **Warning**: Performances\n>\n> `mdedup` implementation is quite naive at the moment and everything resides in memory.\n>\n> If this is good enough for a volume of a couple of gigabytes, the more emails `mdedup` try to parse, the closer you'll reach the memory limits of your machine. In which case [`mdedup` will exit abrubtly](https://github.com/kdeldycke/mail-deduplicate/issues/362#issuecomment-1266743045), zapped by the [OOM killer](https://en.wikipedia.org/wiki/Out_of_memory) of your OS. Of course your mileage may vary depending on your hardware.\n>\n> You can influence implementation of this feature with pull requests, or [purchase of business support \ud83e\udd1d and sponsorship \ud83e\udef6](https://github.com/sponsors/kdeldycke).\n\n## Example\n\n<p align=\"center\">\n <img src=\"https://raw.githubusercontent.com/kdeldycke/mail-deduplicate/main/docs/assets/cli-coloured-run.png\">\n</p>\n\n## Installation\n\n### From sources\n\nEasiest way is to install `mdedup` from sources with [`pipx`](https://pipx.pypa.io):\n\n```shell-session\n$ pipx install mail-deduplicate\n```\n\nOther\n[alternatives installation methods](https://kdeldycke.github.io/mail-deduplicate/install.html)\nare available in the documentation.\n\n### Executables\n\nStandalone executables of `mdedup`'s latest version are available for several platforms and architectures:\n\n| Platform | `x86_64` |`arm64` |\n| ----------- | ---------------------------------------------------------------------------------------------------------------------------------- |-------------------------------------------------------------------------------------------------------------------------------- |\n| **Linux** | [Download `mdedup-linux-x64.bin`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-linux-x64.bin) | |\n| **macOS** | [Download `mdedup-macos-x64.bin`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-macos-x64.bin) | [Download `mdedup-macos-arm64.bin`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-macos-arm64.bin) |\n| **Windows** | [Download `mdedup-windows-x64.exe`](https://github.com/kdeldycke/mail-deduplicate/releases/latest/download/mdedup-windows-x64.exe) | |\n",
"bugtrack_url": null,
"license": null,
"summary": "\ud83d\udce7 CLI to deduplicate mails from mail boxes",
"version": "7.5.0",
"project_urls": {
"Changelog": "https://kdeldycke.github.io/mail-deduplicate/changelog.html",
"Documentation": "https://kdeldycke.github.io/mail-deduplicate",
"Funding": "https://github.com/sponsors/kdeldycke",
"Homepage": "https://github.com/kdeldycke/mail-deduplicate",
"Issues": "https://github.com/kdeldycke/mail-deduplicate/issues",
"Repository": "https://github.com/kdeldycke/mail-deduplicate"
},
"split_keywords": [
"cli",
" mail",
" email",
" maildir",
" mbox",
" deduplication",
" dedupe",
" cleanup",
" mailbox",
" babyl",
" mh",
" mbox",
" mmdf"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "4972b346c7bad1549e0714a24595da9f0c4d1ca4d5b27a455b9ab3c4845348d1",
"md5": "a822d6cb4dcb32ae570fe0809e73d1e2",
"sha256": "fce0a56c90070c7c34d5de4f9c1ab36ab7db4efeef153ddd83e9b30b9701b9c6"
},
"downloads": -1,
"filename": "mail_deduplicate-7.5.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "a822d6cb4dcb32ae570fe0809e73d1e2",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 30860,
"upload_time": "2024-07-03T15:54:02",
"upload_time_iso_8601": "2024-07-03T15:54:02.501902Z",
"url": "https://files.pythonhosted.org/packages/49/72/b346c7bad1549e0714a24595da9f0c4d1ca4d5b27a455b9ab3c4845348d1/mail_deduplicate-7.5.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a0838b84142e6f383d61497aa965124f233d8244c967f2ff58cb7cf4d056c38c",
"md5": "79b951099d413458ec608be31191fcce",
"sha256": "e6a93d7ed70b4176e427c15c90b735619275d8eca67be5995339f7b8076eb48c"
},
"downloads": -1,
"filename": "mail_deduplicate-7.5.0.tar.gz",
"has_sig": false,
"md5_digest": "79b951099d413458ec608be31191fcce",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 30801,
"upload_time": "2024-07-03T15:54:03",
"upload_time_iso_8601": "2024-07-03T15:54:03.779868Z",
"url": "https://files.pythonhosted.org/packages/a0/83/8b84142e6f383d61497aa965124f233d8244c967f2ff58cb7cf4d056c38c/mail_deduplicate-7.5.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-03 15:54:03",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "sponsors",
"github_project": "kdeldycke",
"github_not_found": true,
"lcname": "mail-deduplicate"
}
JSON
