# 由python实现的一系列高效的过滤器算法实现
[](https://pypi.org/project/pyfilters/)
- 基于redis和memory
- 低时间复杂度
## Useage
```python
from pyfilters import MemoryBloomFilter
bf = MemoryBloomFilter(10000, 0.00001)
for i in range(1000):
bf.add(i)
for i in range(1000):
assert i in bf
assert 1001 not in bf
```
## Advanced usage
- 计数形布隆过滤器，可以删除数据
```python
from pyfilters import CountMemoryBloomFilter
cbf = CountMemoryBloomFilter(10000, 0.00001)
for i in range(1000):
cbf.add(i)
for i in range(1000):
assert i in cbf
cbf.remove(1)
assert 1 not in cbf
```
- redis分块布隆过滤器，避免单key过大
```python
from redis import Redis
from pyfilters import ChunkedRedisBloomFilter
bf = ChunkedRedisBloomFilter(Redis(), "test_bloomfilter", 10000, 0.00001)
for i in range(1000):
bf.add(i)
for i in range(1000):
assert i in bf
assert 1001 not in bf
```
- 分块计数形redis布隆过滤器,可以删除数据
```python
from redis import Redis
from pyfilters import CountRedisBloomFilter
rcbf = CountRedisBloomFilter(Redis(), "test_countbloomfilter", 10000, 0.00001)
for i in range(1000):
rcbf.add(i)
for i in range(1000):
assert i in rcbf
rcbf.remove(1)
assert 1 not in rcbf
```
# asyncio兼容
在pyfilters.asyncio包
```python
import asyncio
from redis.asyncio import Redis
from pyfilters.asyncio import CountRedisBloomFilter
async def main():
rcbf = CountRedisBloomFilter(Redis(), "test_countbloomfilter", 10000, 0.00001)
for i in range(1000):
await rcbf.add(i)
for i in range(1000):
assert await rcbf.contains(i)
await rcbf.remove(1)
assert not await rcbf.contains(1)
asyncio.run(main())
```
Raw data
{
"_id": null,
"home_page": "https://github.com/synodriver/pyfilters",
"name": "pyfilters",
"maintainer": "v-vinson",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "bloomfilter",
"author": "synodriver",
"author_email": "diguohuangjiajinweijun@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/e7/26/c9692e1f65a17145e55eceaca7b08710bd314b8ccd9e46432ca735750d6b/pyfilters-0.1.5.tar.gz",
"platform": null,
"description": "# \u7531python\u5b9e\u73b0\u7684\u4e00\u7cfb\u5217\u9ad8\u6548\u7684\u8fc7\u6ee4\u5668\u7b97\u6cd5\u5b9e\u73b0\n[](https://pypi.org/project/pyfilters/)\n\n\n\n\n\n\n- \u57fa\u4e8eredis\u548cmemory\n- \u4f4e\u65f6\u95f4\u590d\u6742\u5ea6\n\n## Useage\n\n```python\nfrom pyfilters import MemoryBloomFilter\n\nbf = MemoryBloomFilter(10000, 0.00001)\nfor i in range(1000):\n bf.add(i)\nfor i in range(1000):\n assert i in bf\nassert 1001 not in bf\n```\n\n## Advanced usage\n\n\n- \u8ba1\u6570\u5f62\u5e03\u9686\u8fc7\u6ee4\u5668\uff0c\u53ef\u4ee5\u5220\u9664\u6570\u636e\n\n```python\nfrom pyfilters import CountMemoryBloomFilter\n\ncbf = CountMemoryBloomFilter(10000, 0.00001)\nfor i in range(1000):\n cbf.add(i)\nfor i in range(1000):\n assert i in cbf\ncbf.remove(1)\nassert 1 not in cbf\n```\n\n- redis\u5206\u5757\u5e03\u9686\u8fc7\u6ee4\u5668\uff0c\u907f\u514d\u5355key\u8fc7\u5927\n\n```python\nfrom redis import Redis\nfrom pyfilters import ChunkedRedisBloomFilter\n\nbf = ChunkedRedisBloomFilter(Redis(), \"test_bloomfilter\", 10000, 0.00001)\nfor i in range(1000):\n bf.add(i)\nfor i in range(1000):\n assert i in bf\nassert 1001 not in bf\n```\n\n\n- \u5206\u5757\u8ba1\u6570\u5f62redis\u5e03\u9686\u8fc7\u6ee4\u5668,\u53ef\u4ee5\u5220\u9664\u6570\u636e\n\n```python\nfrom redis import Redis\nfrom pyfilters import CountRedisBloomFilter\n\nrcbf = CountRedisBloomFilter(Redis(), \"test_countbloomfilter\", 10000, 0.00001)\nfor i in range(1000):\n rcbf.add(i)\nfor i in range(1000):\n assert i in rcbf\nrcbf.remove(1)\nassert 1 not in rcbf\n```\n\n# asyncio\u517c\u5bb9\n\u5728pyfilters.asyncio\u5305\n```python\nimport asyncio\n\nfrom redis.asyncio import Redis\nfrom pyfilters.asyncio import CountRedisBloomFilter\n\nasync def main():\n rcbf = CountRedisBloomFilter(Redis(), \"test_countbloomfilter\", 10000, 0.00001)\n for i in range(1000):\n await rcbf.add(i)\n for i in range(1000):\n assert await rcbf.contains(i)\n await rcbf.remove(1)\n assert not await rcbf.contains(1)\n\nasyncio.run(main())\n```\n",
"bugtrack_url": null,
"license": "GPLv3",
"summary": "Bloom filters and other filter algorithms based on memory and redis",
"version": "0.1.5",
"project_urls": {
"Bug Tracker": "https://github.com/synodriver/pyfilters/issues",
"Homepage": "https://github.com/synodriver/pyfilters"
},
"split_keywords": [
"bloomfilter"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "208e82e769cac03199c3a69bb8f1d4c6a6d65b2cc15a92aa5bb3d03b95b9d9db",
"md5": "913a7165feb68f49ae9a8cae75ff8e73",
"sha256": "17473c37a1d1266fd9db6be61daaf49a49501d0a6ed813fb8cacbb17e7c876c8"
},
"downloads": -1,
"filename": "pyfilters-0.1.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "913a7165feb68f49ae9a8cae75ff8e73",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 11352,
"upload_time": "2024-09-17T08:28:03",
"upload_time_iso_8601": "2024-09-17T08:28:03.387018Z",
"url": "https://files.pythonhosted.org/packages/20/8e/82e769cac03199c3a69bb8f1d4c6a6d65b2cc15a92aa5bb3d03b95b9d9db/pyfilters-0.1.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e726c9692e1f65a17145e55eceaca7b08710bd314b8ccd9e46432ca735750d6b",
"md5": "0488dc7e679192f24d3b4f63873e4cc2",
"sha256": "eaafe5948e0cd0c26e9dc26da38987d26d4313f4adcacd6daca640cc206d98c4"
},
"downloads": -1,
"filename": "pyfilters-0.1.5.tar.gz",
"has_sig": false,
"md5_digest": "0488dc7e679192f24d3b4f63873e4cc2",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 9578,
"upload_time": "2024-09-17T08:28:04",
"upload_time_iso_8601": "2024-09-17T08:28:04.355471Z",
"url": "https://files.pythonhosted.org/packages/e7/26/c9692e1f65a17145e55eceaca7b08710bd314b8ccd9e46432ca735750d6b/pyfilters-0.1.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-17 08:28:04",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "synodriver",
"github_project": "pyfilters",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "mmh3",
"specs": []
},
{
"name": "bitarray",
"specs": []
},
{
"name": "redis",
"specs": []
}
],
"lcname": "pyfilters"
}
JSON
