# PyRedLight
[](https://github.com/cocreators-ee/pyredlight/actions/workflows/publish.yaml)
[](https://github.com/astral-sh/ruff)
[](https://github.com/PyCQA/bandit)
[](https://github.com/cocreators-ee/pyredlight/blob/master/.pre-commit-config.yaml)
[](https://pypi.org/project/pyredlight/)
[](https://pypi.org/project/pyredlight/)
[](https://opensource.org/licenses/BSD-3-Clause)
Rate limiting for Python with fast Redis transactions.
Simply put, instead of running multiple individual operations leading to multiple round-trips to your Redis server, we
execute a small transaction consisting of the following commands (for a limit of `60/10s`)
```
SET key 60 EX 10 NX # Create key and set expiration time, if it does not exist
DECR key # Decrement value and return
TTL key # Return time until key expires
```
Then from the results we parse how many requests you're still allowed to perform within the limit (if any), and what is
the expiration time for the request window.
Also contains utilities and examples for convenient use with FastAPI.
## Installation
It's a Python library, what do you expect?
```bash
pip install pyredlight
# OR
poetry add pyredlight
```
## Usage
If you want to test the examples make sure you replace `redis://your.redis.server` with an actual connect string.
Limits can be defined using the format `{requests}/{time}[hms]`, so `100/10s` = 100 requests / 10 seconds, `60/1m` = 60
requests / minute, `5000/6h` = 5000 requests / 6 hours.
Small example of how you can use this library (also in [example.py](./example.py), test
with `poetry install && poetry run python example.py`):
```python
import asyncio
import redis.asyncio as redis
from pyredlight import limit, set_redis
requests_per_minute = limit("60/60s")
def get_key(request):
return f"rate_limit_example_{request['client_ip']}"
async def handle_request(request):
key = get_key(request)
ok, remaining, expires = await requests_per_minute.is_ok(key)
if not ok:
return {
"status": 429,
"rate_limit_remaining": remaining, # Always 0
"rate_limit_expires": expires,
}
else:
return {
"status": 200,
"rate_limit_remaining": remaining,
"rate_limit_expires": expires,
}
async def main():
for _ in range(10):
print(await handle_request({"client_ip": "127.0.0.1"}))
if __name__ == "__main__":
set_redis(redis.from_url("redis://your.redis.server"))
asyncio.run(main())
```
Or in FastAPI (also in [fastapi_example](./fastapi_example), test
with `cd fastapi_example && poetry install && poetry run python example.py`):
```python
from fastapi import FastAPI, Request, APIRouter, Depends
from starlette.responses import JSONResponse
from pyredlight import limit, set_redis
from pyredlight.fastapi import make_depends
import redis.asyncio as redis
per_minute_limit = limit("60/60s")
def get_rate_limit_key(request: Request):
return request.client.host + "custom"
per_minute_depend = make_depends(per_minute_limit)
custom_key_example = make_depends(per_minute_limit, get_key=get_rate_limit_key)
router = APIRouter()
@router.get("/")
async def get_data(_=Depends(per_minute_depend)):
return JSONResponse({"status": "ok"})
@router.post("/")
async def set_data(_=Depends(custom_key_example)):
return JSONResponse({"status": "ok"})
app = FastAPI()
app.include_router(router)
@app.middleware("http")
async def rate_limit_headers(request: Request, call_next):
response = await call_next(request)
rate_limit_remaining = request.scope.get("rate_limit_remaining", None)
if rate_limit_remaining is not None:
response.headers["X-Rate-Limit-Remains"] = str(request.scope["rate_limit_remaining"])
response.headers["X-Rate-Limit-Expires"] = str(request.scope["rate_limit_expires"])
return response
@app.on_event("startup")
async def setup():
set_redis(redis.from_url("redis://your.redis.server"))
```
For the rare cases you might want to reset a limit, you can also `.clear(key)` instead of `.is_ok(key)`.
You may also occasionally find it useful to merge limits, to e.g. check `20/1s` and `50/1m` at the same time, for that
there's a helper:
```python
from pyredlight import limit, merge_limits
per_second_limit = limit("20/1s")
per_minute_limit = limit("50/1m")
combined_limit = merge_limits([per_second_limit, per_minute_limit])
async def example():
ok, remaining, expires = await combined_limit.is_ok("key")
```
The return value from `merge_limits` implements the same interface as `limit`, so you can also `.clear()` keys - which
will then be cleared from all the limits.
## Performance
You should really not expect much extra latency beyond a single network RTT to your Redis server for each check, as long
as your Redis server is capable of handling the requests. With a very simple Redis server in the same LAN
as [benchmark.py](./benchmark.py) it seems each call is taking approx 110-150μsec.
## Development
Issues and PRs are welcome!
Please open an issue first to discuss the idea before sending a PR so that you know if it would be wanted or needs
re-thinking or if you should just make a fork for yourself.
For local development, make sure you install [pre-commit](https://pre-commit.com/#install), then run:
```bash
pre-commit install
poetry install
poetry run pytest-watch
poetry run python example.py
cd fastapi_example
poetry run python example.py
```
## License
The code is released under the BSD 3-Clause license. Details in the [LICENSE.md](./LICENSE.md) file.
# Financial support
This project has been made possible thanks to [Cocreators](https://cocreators.ee) and [Lietu](https://lietu.net). You
can help us continue our open source work by supporting us
on [Buy me a coffee](https://www.buymeacoffee.com/cocreators).
[](https://www.buymeacoffee.com/cocreators)
Raw data
{
"_id": null,
"home_page": "https://github.com/cocreators-ee/pyredlight/",
"name": "pyredlight",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4",
"maintainer_email": "",
"keywords": "rate,limit,limiter,limiting,async",
"author": "Janne Enberg",
"author_email": "janne.enberg@lietu.net",
"download_url": "https://files.pythonhosted.org/packages/24/8b/f8393576645eb08dc3db4eb534de294e18273db66a4d1eea9fa405bce6bb/pyredlight-0.2.0.tar.gz",
"platform": null,
"description": "# PyRedLight\n\n[](https://github.com/cocreators-ee/pyredlight/actions/workflows/publish.yaml)\n[](https://github.com/astral-sh/ruff)\n[](https://github.com/PyCQA/bandit)\n[](https://github.com/cocreators-ee/pyredlight/blob/master/.pre-commit-config.yaml)\n[](https://pypi.org/project/pyredlight/)\n[](https://pypi.org/project/pyredlight/)\n[](https://opensource.org/licenses/BSD-3-Clause)\n\nRate limiting for Python with fast Redis transactions.\n\nSimply put, instead of running multiple individual operations leading to multiple round-trips to your Redis server, we\nexecute a small transaction consisting of the following commands (for a limit of `60/10s`)\n\n```\nSET key 60 EX 10 NX # Create key and set expiration time, if it does not exist\nDECR key # Decrement value and return\nTTL key # Return time until key expires\n```\n\nThen from the results we parse how many requests you're still allowed to perform within the limit (if any), and what is\nthe expiration time for the request window.\n\nAlso contains utilities and examples for convenient use with FastAPI.\n\n## Installation\n\nIt's a Python library, what do you expect?\n\n```bash\npip install pyredlight\n# OR\npoetry add pyredlight\n```\n\n## Usage\n\nIf you want to test the examples make sure you replace `redis://your.redis.server` with an actual connect string.\n\nLimits can be defined using the format `{requests}/{time}[hms]`, so `100/10s` = 100 requests / 10 seconds, `60/1m` = 60\nrequests / minute, `5000/6h` = 5000 requests / 6 hours.\n\nSmall example of how you can use this library (also in [example.py](./example.py), test\nwith `poetry install && poetry run python example.py`):\n\n```python\nimport asyncio\n\nimport redis.asyncio as redis\n\nfrom pyredlight import limit, set_redis\n\nrequests_per_minute = limit(\"60/60s\")\n\n\ndef get_key(request):\n return f\"rate_limit_example_{request['client_ip']}\"\n\n\nasync def handle_request(request):\n key = get_key(request)\n ok, remaining, expires = await requests_per_minute.is_ok(key)\n if not ok:\n return {\n \"status\": 429,\n \"rate_limit_remaining\": remaining, # Always 0\n \"rate_limit_expires\": expires,\n }\n else:\n return {\n \"status\": 200,\n \"rate_limit_remaining\": remaining,\n \"rate_limit_expires\": expires,\n }\n\n\nasync def main():\n for _ in range(10):\n print(await handle_request({\"client_ip\": \"127.0.0.1\"}))\n\n\nif __name__ == \"__main__\":\n set_redis(redis.from_url(\"redis://your.redis.server\"))\n asyncio.run(main())\n```\n\nOr in FastAPI (also in [fastapi_example](./fastapi_example), test\nwith `cd fastapi_example && poetry install && poetry run python example.py`):\n\n```python\nfrom fastapi import FastAPI, Request, APIRouter, Depends\nfrom starlette.responses import JSONResponse\n\nfrom pyredlight import limit, set_redis\nfrom pyredlight.fastapi import make_depends\n\nimport redis.asyncio as redis\n\nper_minute_limit = limit(\"60/60s\")\n\n\ndef get_rate_limit_key(request: Request):\n return request.client.host + \"custom\"\n\n\nper_minute_depend = make_depends(per_minute_limit)\ncustom_key_example = make_depends(per_minute_limit, get_key=get_rate_limit_key)\n\nrouter = APIRouter()\n\n\n@router.get(\"/\")\nasync def get_data(_=Depends(per_minute_depend)):\n return JSONResponse({\"status\": \"ok\"})\n\n\n@router.post(\"/\")\nasync def set_data(_=Depends(custom_key_example)):\n return JSONResponse({\"status\": \"ok\"})\n\n\napp = FastAPI()\napp.include_router(router)\n\n\n@app.middleware(\"http\")\nasync def rate_limit_headers(request: Request, call_next):\n response = await call_next(request)\n rate_limit_remaining = request.scope.get(\"rate_limit_remaining\", None)\n if rate_limit_remaining is not None:\n response.headers[\"X-Rate-Limit-Remains\"] = str(request.scope[\"rate_limit_remaining\"])\n response.headers[\"X-Rate-Limit-Expires\"] = str(request.scope[\"rate_limit_expires\"])\n return response\n\n\n@app.on_event(\"startup\")\nasync def setup():\n set_redis(redis.from_url(\"redis://your.redis.server\"))\n```\n\nFor the rare cases you might want to reset a limit, you can also `.clear(key)` instead of `.is_ok(key)`.\n\nYou may also occasionally find it useful to merge limits, to e.g. check `20/1s` and `50/1m` at the same time, for that\nthere's a helper:\n\n```python\nfrom pyredlight import limit, merge_limits\n\nper_second_limit = limit(\"20/1s\")\nper_minute_limit = limit(\"50/1m\")\ncombined_limit = merge_limits([per_second_limit, per_minute_limit])\n\n\nasync def example():\n ok, remaining, expires = await combined_limit.is_ok(\"key\")\n```\n\nThe return value from `merge_limits` implements the same interface as `limit`, so you can also `.clear()` keys - which\nwill then be cleared from all the limits.\n\n## Performance\n\nYou should really not expect much extra latency beyond a single network RTT to your Redis server for each check, as long\nas your Redis server is capable of handling the requests. With a very simple Redis server in the same LAN\nas [benchmark.py](./benchmark.py) it seems each call is taking approx 110-150\u03bcsec.\n\n## Development\n\nIssues and PRs are welcome!\n\nPlease open an issue first to discuss the idea before sending a PR so that you know if it would be wanted or needs\nre-thinking or if you should just make a fork for yourself.\n\nFor local development, make sure you install [pre-commit](https://pre-commit.com/#install), then run:\n\n```bash\npre-commit install\npoetry install\npoetry run pytest-watch\npoetry run python example.py\n\ncd fastapi_example\npoetry run python example.py\n```\n\n## License\n\nThe code is released under the BSD 3-Clause license. Details in the [LICENSE.md](./LICENSE.md) file.\n\n# Financial support\n\nThis project has been made possible thanks to [Cocreators](https://cocreators.ee) and [Lietu](https://lietu.net). You\ncan help us continue our open source work by supporting us\non [Buy me a coffee](https://www.buymeacoffee.com/cocreators).\n\n[](https://www.buymeacoffee.com/cocreators)\n\n",
"bugtrack_url": null,
"license": "BSD-3-Clause",
"summary": "Redis transaction based rate limiter",
"version": "0.2.0",
"project_urls": {
"Documentation": "https://github.com/cocreators-ee/pyredlight/",
"Homepage": "https://github.com/cocreators-ee/pyredlight/",
"Repository": "https://github.com/cocreators-ee/pyredlight/"
},
"split_keywords": [
"rate",
"limit",
"limiter",
"limiting",
"async"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a0fa103bc70bdff281e168fe312fc4fadc2e5714e672c6328103a8e631a8e5e2",
"md5": "ad3a1cd7391f47cf984c54063ff14aeb",
"sha256": "0b34ce3c9043f706419589a778130e8971fdbb239978c2e0312256bc45e78386"
},
"downloads": -1,
"filename": "pyredlight-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ad3a1cd7391f47cf984c54063ff14aeb",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4",
"size": 6866,
"upload_time": "2024-02-03T08:18:30",
"upload_time_iso_8601": "2024-02-03T08:18:30.197987Z",
"url": "https://files.pythonhosted.org/packages/a0/fa/103bc70bdff281e168fe312fc4fadc2e5714e672c6328103a8e631a8e5e2/pyredlight-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "248bf8393576645eb08dc3db4eb534de294e18273db66a4d1eea9fa405bce6bb",
"md5": "3beacec4e16e2de47efbbdc00a3d321c",
"sha256": "e07303f3fcc8d87565c224b0f0f6aa383998386581809a70cb0061f24221ac1a"
},
"downloads": -1,
"filename": "pyredlight-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "3beacec4e16e2de47efbbdc00a3d321c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4",
"size": 5856,
"upload_time": "2024-02-03T08:18:32",
"upload_time_iso_8601": "2024-02-03T08:18:32.212601Z",
"url": "https://files.pythonhosted.org/packages/24/8b/f8393576645eb08dc3db4eb534de294e18273db66a4d1eea9fa405bce6bb/pyredlight-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-03 08:18:32",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cocreators-ee",
"github_project": "pyredlight",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "pyredlight"
}
JSON
