=======================
generic-connection-pool
=======================
.. image:: https://static.pepy.tech/personalized-badge/generic-connection-pool?period=month&units=international_system&left_color=grey&right_color=orange&left_text=Downloads/month
:target: https://pepy.tech/project/generic-connection-pool
:alt: Downloads/month
.. image:: https://github.com/dapper91/generic-connection-pool/actions/workflows/test.yml/badge.svg?branch=master
:target: https://github.com/dapper91/generic-connection-pool/actions/workflows/test.yml
:alt: Build status
.. image:: https://img.shields.io/pypi/l/generic-connection-pool.svg
:target: https://pypi.org/project/generic-connection-pool
:alt: License
.. image:: https://img.shields.io/pypi/pyversions/generic-connection-pool.svg
:target: https://pypi.org/project/generic-connection-pool
:alt: Supported Python versions
.. image:: https://codecov.io/gh/dapper91/generic-connection-pool/branch/master/graph/badge.svg
:target: https://codecov.io/gh/dapper91/generic-connection-pool
:alt: Code coverage
.. image:: https://readthedocs.org/projects/generic-connection-pool/badge/?version=stable&style=flat
:alt: ReadTheDocs status
:target: https://generic-connection-pool.readthedocs.io/en/stable/
``generic-connection-pool`` is an extensible connection pool agnostic to the connection type it is managing.
It can be used for TCP, http, database or ssh connections.
Features
--------
- **generic nature**: can be used for any connection you desire (TCP, http, database, ssh, etc.)
- **runtime agnostic**: synchronous and asynchronous runtime supported
- **flexibility**: flexable connection retention and recycling policy
- **fully-typed**: mypy type-checker compatible
Getting started
---------------
Connection pool supports the following configurations:
* **background_collector**: if ``True`` starts a background worker that disposes expired and idle connections
maintaining requested pool state. If ``False`` the connections will be disposed on each connection release.
* **dispose_batch_size**: maximum number of expired and idle connections to be disposed on connection release
(if background collector is started the parameter is ignored).
* **idle_timeout**: inactivity time (in seconds) after which an extra connection will be disposed
(a connection considered as extra if the number of endpoint connection exceeds ``min_idle``).
* **max_lifetime**: number of seconds after which any connection will be disposed.
* **min_idle**: minimum number of connections in each endpoint the pool tries to hold. Connections that exceed
that number will be considered as extra and disposed after ``idle_timeout`` seconds of inactivity.
* **max_size**: maximum number of endpoint connections.
* **total_max_size**: maximum number of all connections in the pool.
.. image:: /static/connection-pool.svg
:width: 1024
:alt: Connection Pool parameters
The following example illustrates how to create https pool:
.. code-block:: python
import socket
import ssl
import urllib.parse
from http.client import HTTPResponse
from typing import Tuple
from generic_connection_pool.contrib.socket import SslSocketConnectionManager
from generic_connection_pool.threading import ConnectionPool
Hostname = str
Port = int
Endpoint = Tuple[Hostname, Port]
Connection = socket.socket
http_pool = ConnectionPool[Endpoint, Connection](
SslSocketConnectionManager(ssl.create_default_context()),
idle_timeout=30.0,
max_lifetime=600.0,
min_idle=3,
max_size=20,
total_max_size=100,
background_collector=True,
)
def fetch(url: str, timeout: float = 5.0) -> None:
url = urllib.parse.urlsplit(url)
port = url.port or 443 if url.scheme == 'https' else 80
with http_pool.connection(endpoint=(url.hostname, port), timeout=timeout) as sock:
request = (
'GET {path} HTTP/1.1\r\n'
'Host: {host}\r\n'
'\r\n'
'\r\n'
).format(host=url.hostname, path=url.path)
sock.write(request.encode())
response = HTTPResponse(sock)
response.begin()
status, body = response.getcode(), response.read(response.length)
print(status)
print(body)
try:
fetch('https://en.wikipedia.org/wiki/HTTP') # http connection opened
fetch('https://en.wikipedia.org/wiki/Python_(programming_language)') # http connection reused
finally:
http_pool.close()
... or database one
.. code-block:: python
import psycopg2.extensions
from generic_connection_pool.contrib.psycopg2 import DbConnectionManager
from generic_connection_pool.threading import ConnectionPool
Endpoint = str
Connection = psycopg2.extensions.connection
dsn_params = dict(dbname='postgres', user='postgres', password='secret')
pg_pool = ConnectionPool[Endpoint, Connection](
DbConnectionManager(
dsn_params={
'master': dict(dsn_params, host='db-master.local'),
'replica-1': dict(dsn_params, host='db-replica-1.local'),
'replica-2': dict(dsn_params, host='db-replica-2.local'),
},
),
acquire_timeout=2.0,
idle_timeout=60.0,
max_lifetime=600.0,
min_idle=3,
max_size=10,
total_max_size=15,
background_collector=True,
)
try:
# connection opened
with pg_pool.connection(endpoint='master') as conn:
cur = conn.cursor()
cur.execute("SELECT * FROM pg_stats;")
print(cur.fetchone())
# connection opened
with pg_pool.connection(endpoint='replica-1') as conn:
cur = conn.cursor()
cur.execute("SELECT * FROM pg_stats;")
print(cur.fetchone())
# connection reused
with pg_pool.connection(endpoint='master') as conn:
cur = conn.cursor()
cur.execute("SELECT * FROM pg_stats;")
print(cur.fetchone())
finally:
pg_pool.close()
See `documentation <https://generic-connection-pool.readthedocs.io/en/latest/>`_ for more details.
Raw data
{
"_id": null,
"home_page": "https://github.com/dapper91/generic-connection-pool",
"name": "generic-connection-pool",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "",
"keywords": "pool,connection-pool,asyncio,socket,tcp",
"author": "Dmitry Pershin",
"author_email": "dapper1291@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/a2/b4/e3f95e47529fdab31bf5b6f16f5cdb2d7761bbb89a46007c36a456bc83b8/generic_connection_pool-0.7.0.tar.gz",
"platform": null,
"description": "=======================\ngeneric-connection-pool\n=======================\n\n.. image:: https://static.pepy.tech/personalized-badge/generic-connection-pool?period=month&units=international_system&left_color=grey&right_color=orange&left_text=Downloads/month\n :target: https://pepy.tech/project/generic-connection-pool\n :alt: Downloads/month\n.. image:: https://github.com/dapper91/generic-connection-pool/actions/workflows/test.yml/badge.svg?branch=master\n :target: https://github.com/dapper91/generic-connection-pool/actions/workflows/test.yml\n :alt: Build status\n.. image:: https://img.shields.io/pypi/l/generic-connection-pool.svg\n :target: https://pypi.org/project/generic-connection-pool\n :alt: License\n.. image:: https://img.shields.io/pypi/pyversions/generic-connection-pool.svg\n :target: https://pypi.org/project/generic-connection-pool\n :alt: Supported Python versions\n.. image:: https://codecov.io/gh/dapper91/generic-connection-pool/branch/master/graph/badge.svg\n :target: https://codecov.io/gh/dapper91/generic-connection-pool\n :alt: Code coverage\n.. image:: https://readthedocs.org/projects/generic-connection-pool/badge/?version=stable&style=flat\n :alt: ReadTheDocs status\n :target: https://generic-connection-pool.readthedocs.io/en/stable/\n\n\n``generic-connection-pool`` is an extensible connection pool agnostic to the connection type it is managing.\nIt can be used for TCP, http, database or ssh connections.\n\nFeatures\n--------\n\n- **generic nature**: can be used for any connection you desire (TCP, http, database, ssh, etc.)\n- **runtime agnostic**: synchronous and asynchronous runtime supported\n- **flexibility**: flexable connection retention and recycling policy\n- **fully-typed**: mypy type-checker compatible\n\n\nGetting started\n---------------\n\nConnection pool supports the following configurations:\n\n* **background_collector**: if ``True`` starts a background worker that disposes expired and idle connections\n maintaining requested pool state. If ``False`` the connections will be disposed on each connection release.\n* **dispose_batch_size**: maximum number of expired and idle connections to be disposed on connection release\n (if background collector is started the parameter is ignored).\n* **idle_timeout**: inactivity time (in seconds) after which an extra connection will be disposed\n (a connection considered as extra if the number of endpoint connection exceeds ``min_idle``).\n* **max_lifetime**: number of seconds after which any connection will be disposed.\n* **min_idle**: minimum number of connections in each endpoint the pool tries to hold. Connections that exceed\n that number will be considered as extra and disposed after ``idle_timeout`` seconds of inactivity.\n* **max_size**: maximum number of endpoint connections.\n* **total_max_size**: maximum number of all connections in the pool.\n\n\n.. image:: /static/connection-pool.svg\n :width: 1024\n :alt: Connection Pool parameters\n\n\nThe following example illustrates how to create https pool:\n\n.. code-block:: python\n\n import socket\n import ssl\n import urllib.parse\n from http.client import HTTPResponse\n from typing import Tuple\n\n from generic_connection_pool.contrib.socket import SslSocketConnectionManager\n from generic_connection_pool.threading import ConnectionPool\n\n Hostname = str\n Port = int\n Endpoint = Tuple[Hostname, Port]\n Connection = socket.socket\n\n\n http_pool = ConnectionPool[Endpoint, Connection](\n SslSocketConnectionManager(ssl.create_default_context()),\n idle_timeout=30.0,\n max_lifetime=600.0,\n min_idle=3,\n max_size=20,\n total_max_size=100,\n background_collector=True,\n )\n\n\n def fetch(url: str, timeout: float = 5.0) -> None:\n url = urllib.parse.urlsplit(url)\n port = url.port or 443 if url.scheme == 'https' else 80\n\n with http_pool.connection(endpoint=(url.hostname, port), timeout=timeout) as sock:\n request = (\n 'GET {path} HTTP/1.1\\r\\n'\n 'Host: {host}\\r\\n'\n '\\r\\n'\n '\\r\\n'\n ).format(host=url.hostname, path=url.path)\n\n sock.write(request.encode())\n\n response = HTTPResponse(sock)\n response.begin()\n status, body = response.getcode(), response.read(response.length)\n\n print(status)\n print(body)\n\n\n try:\n fetch('https://en.wikipedia.org/wiki/HTTP') # http connection opened\n fetch('https://en.wikipedia.org/wiki/Python_(programming_language)') # http connection reused\n finally:\n http_pool.close()\n\n... or database one\n\n.. code-block:: python\n\n import psycopg2.extensions\n\n from generic_connection_pool.contrib.psycopg2 import DbConnectionManager\n from generic_connection_pool.threading import ConnectionPool\n\n Endpoint = str\n Connection = psycopg2.extensions.connection\n\n\n dsn_params = dict(dbname='postgres', user='postgres', password='secret')\n\n pg_pool = ConnectionPool[Endpoint, Connection](\n DbConnectionManager(\n dsn_params={\n 'master': dict(dsn_params, host='db-master.local'),\n 'replica-1': dict(dsn_params, host='db-replica-1.local'),\n 'replica-2': dict(dsn_params, host='db-replica-2.local'),\n },\n ),\n acquire_timeout=2.0,\n idle_timeout=60.0,\n max_lifetime=600.0,\n min_idle=3,\n max_size=10,\n total_max_size=15,\n background_collector=True,\n )\n\n try:\n # connection opened\n with pg_pool.connection(endpoint='master') as conn:\n cur = conn.cursor()\n cur.execute(\"SELECT * FROM pg_stats;\")\n print(cur.fetchone())\n\n # connection opened\n with pg_pool.connection(endpoint='replica-1') as conn:\n cur = conn.cursor()\n cur.execute(\"SELECT * FROM pg_stats;\")\n print(cur.fetchone())\n\n # connection reused\n with pg_pool.connection(endpoint='master') as conn:\n cur = conn.cursor()\n cur.execute(\"SELECT * FROM pg_stats;\")\n print(cur.fetchone())\n\n finally:\n pg_pool.close()\n\n\nSee `documentation <https://generic-connection-pool.readthedocs.io/en/latest/>`_ for more details.\n",
"bugtrack_url": null,
"license": "Unlicense",
"summary": "generic connection pool",
"version": "0.7.0",
"project_urls": {
"Documentation": "https://generic-connection-pool.readthedocs.io",
"Homepage": "https://github.com/dapper91/generic-connection-pool",
"Repository": "https://github.com/dapper91/generic-connection-pool"
},
"split_keywords": [
"pool",
"connection-pool",
"asyncio",
"socket",
"tcp"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "9f6ca6bb3f839b14b38db1ed3216d4eb0088f725842f0962e64c19f871c2516a",
"md5": "1b6d8295e70d70f1572720d420f4ad50",
"sha256": "b6d7b36649084559b40e0796a713d9a91ca82680582f4840801a4f7d79588034"
},
"downloads": -1,
"filename": "generic_connection_pool-0.7.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1b6d8295e70d70f1572720d420f4ad50",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 28535,
"upload_time": "2023-12-09T09:03:55",
"upload_time_iso_8601": "2023-12-09T09:03:55.291514Z",
"url": "https://files.pythonhosted.org/packages/9f/6c/a6bb3f839b14b38db1ed3216d4eb0088f725842f0962e64c19f871c2516a/generic_connection_pool-0.7.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a2b4e3f95e47529fdab31bf5b6f16f5cdb2d7761bbb89a46007c36a456bc83b8",
"md5": "02b9a943033143bf0a78e3dab6921ded",
"sha256": "e8b5d3cd220953313bf5009fea28e205b2e7e4318bfb5cd199aac815cc64aa00"
},
"downloads": -1,
"filename": "generic_connection_pool-0.7.0.tar.gz",
"has_sig": false,
"md5_digest": "02b9a943033143bf0a78e3dab6921ded",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 22134,
"upload_time": "2023-12-09T09:03:58",
"upload_time_iso_8601": "2023-12-09T09:03:58.963367Z",
"url": "https://files.pythonhosted.org/packages/a2/b4/e3f95e47529fdab31bf5b6f16f5cdb2d7761bbb89a46007c36a456bc83b8/generic_connection_pool-0.7.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-12-09 09:03:58",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "dapper91",
"github_project": "generic-connection-pool",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "generic-connection-pool"
}
JSON
