# octopus-api
## About
Octopus-api is a python library for performing client-based optimized connections requests and limit rates set by the endpoint contract.
Octopus-api is simple; it combines the [asyncio](https://docs.python.org/3/library/asyncio.html) and [aiohttp](https://docs.aiohttp.org/en/stable/) library's functionality and makes sure the requests follows the constraints set by the contract.
## Installation
`pip install octopus-api`
## PyPi
https://pypi.org/project/octopus-api/
## Get started
To start Octopus, you first initiate the client, setting your constraints.
```python
client = OctopusApi(rate=30, resolution="minute", retries=10)
client = OctopusApi(rate=5, resolution="sec", retries=3)
client = OctopusApi(connections=100, retries=5)
```
After that, you will specify what you want to perform on the endpoint response. This is done within a user-defined function.
```python
async def patch_data(session: TentacleSession, request: Dict):
async with session.patch(url=request["url"], data=requests["data"], params=request["params"]) as response:
body = await response.json()
return body["id"]
```
As Octopus `TentacleSession` uses [aiohttp](https://docs.aiohttp.org/en/stable/) under the hood, the resulting way to write
**POST**, **GET**, **PUT** and **PATCH** for aiohttp will be the same for octopus. The only difference is the added functionality of
retries and optional rate limit.
Finally, you finish everything up with the `execute` call for the octopus client, where you provide the list of requests dicts and the user function.
The execute call will then return a list of the return values defined in user function. As the requests list is a bounded stream we return the result in order.
```python
result: List = client.execute(requests_list=[
{
"url": "http://localhost:3000",
"data": {"id": "a", "first_name": "filip"},
"params": {"id": "a"}
},
{
"url": "http://localhost:3000",
"data": {"id": "b", "first_name": "morris"},
"params": {"id": "b"}
}
] , func=patch_data)
```
### Examples
Optimize the request based on max connections constraints:
```python
from octopus_api import TentacleSession, OctopusApi
from typing import Dict, List
if __name__ == '__main__':
async def get_text(session: TentacleSession, request: Dict):
async with session.get(url=request["url"], params=request["params"]) as response:
body = await response.text()
return body
client = OctopusApi(connections=100)
result: List = client.execute(requests_list=[{
"url": "http://google.com",
"params": {}}] * 100, func=get_text)
print(result)
```
Optimize the request based on rate limit constraints:
```python
from octopus_api import TentacleSession, OctopusApi
from typing import Dict, List
if __name__ == '__main__':
async def get_ethereum_id(session: TentacleSession, request: Dict):
async with session.get(url=request["url"], params=request["params"]) as response:
body = await response.json()
return body["id"]
client = OctopusApi(rate=30, resolution="minute")
result: List = client.execute(requests_list=[{
"url": "http://api.coingecko.com/api/v3/coins/ethereum?tickers=false&localization=false&market_data=false",
"params": {}}] * 100, func=get_ethereum_id)
print(result)
```
Optimize the request based on rate limit and connections limit:
```python
from octopus_api import TentacleSession, OctopusApi
from typing import Dict, List
if __name__ == '__main__':
async def get_ethereum(session: TentacleSession, request: Dict):
async with session.get(url=request["url"], params=request["params"]) as response:
body = await response.json()
return body
client = OctopusApi(rate=50, resolution="sec", connections=6)
result: List = client.execute(requests_list=[{
"url": "https://api.pro.coinbase.com/products/ETH-EUR/candles?granularity=900&start=2021-12-04T00:00:00Z&end=2021-12-04T00:00:00Z",
"params": {}}] * 1000, func=get_ethereum)
print(result)
```
Raw data
{
"_id": null,
"home_page": "https://github.com/FilipByren/octopus-api",
"name": "octopus-api",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7,<4.0",
"maintainer_email": "",
"keywords": "rate limiting,concurrent api,python,https://docs.aiohttp.org",
"author": "Filip Byr\u00e9n",
"author_email": "filip.j.byren@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/90/dd/169e492417bc92187d4ec52a343dc14029a3793a7c9b15eb2161cd72c4d2/octopus-api-2.1.0.tar.gz",
"platform": null,
"description": "# octopus-api\n\n## About\nOctopus-api is a python library for performing client-based optimized connections requests and limit rates set by the endpoint contract.\n\nOctopus-api is simple; it combines the [asyncio](https://docs.python.org/3/library/asyncio.html) and [aiohttp](https://docs.aiohttp.org/en/stable/) library's functionality and makes sure the requests follows the constraints set by the contract.\n\n## Installation\n`pip install octopus-api`\n\n## PyPi\nhttps://pypi.org/project/octopus-api/\n\n\n## Get started\nTo start Octopus, you first initiate the client, setting your constraints. \n```python\nclient = OctopusApi(rate=30, resolution=\"minute\", retries=10)\nclient = OctopusApi(rate=5, resolution=\"sec\", retries=3)\nclient = OctopusApi(connections=100, retries=5)\n```\nAfter that, you will specify what you want to perform on the endpoint response. This is done within a user-defined function.\n```python\nasync def patch_data(session: TentacleSession, request: Dict):\n async with session.patch(url=request[\"url\"], data=requests[\"data\"], params=request[\"params\"]) as response:\n body = await response.json()\n return body[\"id\"]\n```\n\nAs Octopus `TentacleSession` uses [aiohttp](https://docs.aiohttp.org/en/stable/) under the hood, the resulting way to write \n**POST**, **GET**, **PUT** and **PATCH** for aiohttp will be the same for octopus. The only difference is the added functionality of \nretries and optional rate limit.\n\nFinally, you finish everything up with the `execute` call for the octopus client, where you provide the list of requests dicts and the user function.\nThe execute call will then return a list of the return values defined in user function. As the requests list is a bounded stream we return the result in order.\n\n\n```python\nresult: List = client.execute(requests_list=[\n {\n \"url\": \"http://localhost:3000\",\n \"data\": {\"id\": \"a\", \"first_name\": \"filip\"},\n \"params\": {\"id\": \"a\"}\n },\n {\n \"url\": \"http://localhost:3000\",\n \"data\": {\"id\": \"b\", \"first_name\": \"morris\"},\n \"params\": {\"id\": \"b\"} \n }\n ] , func=patch_data)\n```\n\n\n### Examples\n\nOptimize the request based on max connections constraints:\n```python\nfrom octopus_api import TentacleSession, OctopusApi\nfrom typing import Dict, List\n\nif __name__ == '__main__':\n async def get_text(session: TentacleSession, request: Dict):\n async with session.get(url=request[\"url\"], params=request[\"params\"]) as response:\n body = await response.text()\n return body\n\n\n client = OctopusApi(connections=100)\n result: List = client.execute(requests_list=[{\n \"url\": \"http://google.com\",\n \"params\": {}}] * 100, func=get_text)\n print(result)\n\n```\nOptimize the request based on rate limit constraints:\n```python\nfrom octopus_api import TentacleSession, OctopusApi\nfrom typing import Dict, List\n\nif __name__ == '__main__':\n async def get_ethereum_id(session: TentacleSession, request: Dict):\n async with session.get(url=request[\"url\"], params=request[\"params\"]) as response:\n body = await response.json()\n return body[\"id\"]\n\n client = OctopusApi(rate=30, resolution=\"minute\")\n result: List = client.execute(requests_list=[{\n \"url\": \"http://api.coingecko.com/api/v3/coins/ethereum?tickers=false&localization=false&market_data=false\",\n \"params\": {}}] * 100, func=get_ethereum_id)\n print(result)\n\n```\nOptimize the request based on rate limit and connections limit:\n```python\nfrom octopus_api import TentacleSession, OctopusApi\nfrom typing import Dict, List\n\nif __name__ == '__main__':\n async def get_ethereum(session: TentacleSession, request: Dict):\n async with session.get(url=request[\"url\"], params=request[\"params\"]) as response:\n body = await response.json()\n return body\n\n client = OctopusApi(rate=50, resolution=\"sec\", connections=6)\n result: List = client.execute(requests_list=[{\n \"url\": \"https://api.pro.coinbase.com/products/ETH-EUR/candles?granularity=900&start=2021-12-04T00:00:00Z&end=2021-12-04T00:00:00Z\",\n \"params\": {}}] * 1000, func=get_ethereum)\n print(result)\n```\n",
"bugtrack_url": null,
"license": "GNU GENERAL PUBLIC LICENSE",
"summary": "Octopus-api is a python library for performing client-based optimized connections requests and limit rates set by the endpoint contract.",
"version": "2.1.0",
"split_keywords": [
"rate limiting",
"concurrent api",
"python",
"https://docs.aiohttp.org"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "11d85eab718d80224f41e578cbff4124f6e46bfdf8b99b2b81bd6e881eb4980b",
"md5": "063db99a0ceeb084df69524f9396b24e",
"sha256": "1cc3bace1c711ded2ce124dc586ddf87c807122c950a89e682b75880457c8f2f"
},
"downloads": -1,
"filename": "octopus_api-2.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "063db99a0ceeb084df69524f9396b24e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7,<4.0",
"size": 16551,
"upload_time": "2023-03-10T11:54:38",
"upload_time_iso_8601": "2023-03-10T11:54:38.993451Z",
"url": "https://files.pythonhosted.org/packages/11/d8/5eab718d80224f41e578cbff4124f6e46bfdf8b99b2b81bd6e881eb4980b/octopus_api-2.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "90dd169e492417bc92187d4ec52a343dc14029a3793a7c9b15eb2161cd72c4d2",
"md5": "dcde09fd8c107d36dd1d964561675968",
"sha256": "db75db1ab6396d3c812ad616624dc9f0cf6584fb820e2f025b59ee2a9b4515d3"
},
"downloads": -1,
"filename": "octopus-api-2.1.0.tar.gz",
"has_sig": false,
"md5_digest": "dcde09fd8c107d36dd1d964561675968",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7,<4.0",
"size": 16642,
"upload_time": "2023-03-10T11:54:36",
"upload_time_iso_8601": "2023-03-10T11:54:36.871846Z",
"url": "https://files.pythonhosted.org/packages/90/dd/169e492417bc92187d4ec52a343dc14029a3793a7c9b15eb2161cd72c4d2/octopus-api-2.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-03-10 11:54:36",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "FilipByren",
"github_project": "octopus-api",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "octopus-api"
}
JSON
