async_stagger: Happy Eyeballs in ``asyncio``
############################################
Quick, what's the situation?
============================
To get all the benefits of Happy Eyeballs connection establishment algorithm,
simply use ``async_stagger.open_connection`` like you would use
``asyncio.open_connection``::
reader, writer = await async_stagger.open_connection('www.example.com', 80)
Now your connections are more dual-stack friendly and will complete faster!
A replacement for ``loop.create_connection`` is also provided.
The long version
================
What is Happy Eyeballs, and why should I use it?
------------------------------------------------
Happy Eyeballs is an algorithm for establishing TCP connections to destinations
specified by host names. It is described in :rfc:`6555` and :rfc:`8305`. The
primary benefit is that when host name resolution returns multiple addresses,
and some of the address are unreachable, Happy Eyeballs will establish the
connection much faster than conventional algorithms. For more information,
check the `Wikipedia article on Happy Eyeballs`_.
.. _Wikipedia article on Happy Eyeballs: https://en.wikipedia.org/wiki/Happy_Eyeballs
Python's standard library provides several high-level methods of establishing
TCP connections towards a host name: The **socket** module has
``socket.create_connection``,
and **asyncio** has ``loop.create_connection`` and ``asyncio.open_connection``.
By default,
these methods have the same behavior when a host name resolves to several IP
addresses: they try to connect to the first address in the list,
and only after the attempt fails (which may take tens of seconds) will
the second one be tried, and so on. In contrast, the Happy Eyeballs algorithm
will start an attempt with the second IP address in parallel to the first one
hasn't completed after some time, typically around 300 milliseconds.
As a result several attempts may be in flight at the same time, and whenever
one of the attempts succeed, all other connections are cancelled, and the
winning connection is used.
This means a much shorter wait before one of the IP addresses connect
successfully.
Happy Eyeballs is particularly important for dual-stack clients, when some hosts
may have resolvable IPv6 addresses that are somehow unreachable.
Starting from Python 3.8, stock **asyncio** also supports Happy Eyeballs.
See below for a comparison.
What does ``async_stagger`` has to offer?
-----------------------------------------
``async_stagger`` provides ``open_connection`` and
``create_connection`` with Happy Eyeballs support. They are mostly drop-in
replacements for their ``asyncio`` counterparts, and support most existing
arguments.
(There are small differences: ``create_connection`` takes
a *loop* argument instead of being a method on an event loop.
Also, these two methods do not support the *sock* argument.)
Another public coroutine ``create_connected_sock`` returns a connected
``socket.socket`` object.
Check the documentation for details.
These methods implements many features specified in :rfc:`8305` Happy Eyeballs
v2, which extends and obsoletes :rfc:`6555`. In particular, asynchronous
address resolution, destination address interleaving by family and staggered
connection attempts are implemented.
Happy Eyeballs sounds great! I want to use similar logic somewhere else!
------------------------------------------------------------------------
You're in luck! ``async_stagger`` actually exposes the underlying scheduling
logic as a reusable block: ``staggered_race``. It can be use when:
* There are several ways to achieve one goal. Some of the ways may fail, but
you have to try it to find out.
* Making attempts strictly in sequence is too slow.
* You want to parallelize, but also don't want to start the attempts all
at the same time. Maybe you want to give preference to some of the attempts,
so they should be started earlier and given more time to complete. Maybe you
want to avoid straining the system with simultaneous attempts.
* An attempt done half-way can be rolled back safely.
Where can I get it?
-------------------
``async_stagger`` requires Python 3.6 or later.
(v0.2.0 onwards uses more new features in 3.6 such as async generators and
async comprehensions, so it will probably require more than cosmetic changes
to make it run on 3.5.)
It does not have any external dependencies.
Install it from PyPI the usual way::
pip install async-stagger
The documentation can be found here:
http://async-stagger.readthedocs.io/en/latest/
This project is under active development, and APIs may change in the future.
Check out the Changelog in the documentation.
This project is licensed under the MIT license.
Python 3.8 Has Native Happy Eyeballs Now
========================================
I contributed an implementation of Happy Eyeballs to upstream **asyncio**,
and it landed in Python 3.8: see `the docs`__ for details.
__ https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.create_connection
That implementation is essentially an early version of this package,
so it lacks these features:
* Async address resolution
* Detailed exception report
* The ``local_addrs`` argument (as opposed to ``local_addr``)
Still, it should be sufficient for most scenarios, and it's right there in the standard library.
Miscellaneous Remarks
=====================
Asynchronous address resolution is added in v0.2.1. With that, I feel that
the package should be fairly feature-complete.
I have implemented Happy Eyeballs-like algorithms in some of my other projects,
and this module reflects the things I have learned. However I have yet to
eat my own dog food and actually import this module from those other projects.
I would love to hear people's experience using this module in real world
conditions.
`bpo-31861 <https://bugs.python.org/issue31861>`_ talks about adding native
``aiter`` and ``anext`` functions either as builtins or to the ``operator``
module. Well, I want them NAO!!!one!!!eleventy!! So I borrowed the
implementations from that bpo and put them in the ``aitertools`` submodule.
I have only kept the one-argument forms; In particular, the two-argument
``iter`` function is so disparate from the one-argument version, that I don't
think they belong to the same function at all, and there really shouldn't be
a need for ``aiter`` to emulate that behavior.
Acknowledgments
===============
The Happy Eyeballs scheduling algorithm implementation is inspired by
`the implementation in trio`__.
__ https://github.com/python-trio/trio/pull/145/files
Raw data
{
"_id": null,
"home_page": "https://github.com/twisteroidambassador/async_stagger",
"name": "async-stagger",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "happy-eyeballs dual-stack tcp",
"author": "twisteroid ambassador",
"author_email": "twisteroid.ambassador@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/f0/3e/1bbe65201027f05d973bf9cecf1e0141553f6ce4b8e78116c065fc59786a/async_stagger-0.3.1.tar.gz",
"platform": "",
"description": "async_stagger: Happy Eyeballs in ``asyncio``\n############################################\n\nQuick, what's the situation?\n============================\n\nTo get all the benefits of Happy Eyeballs connection establishment algorithm,\nsimply use ``async_stagger.open_connection`` like you would use\n``asyncio.open_connection``::\n\n reader, writer = await async_stagger.open_connection('www.example.com', 80)\n\nNow your connections are more dual-stack friendly and will complete faster!\nA replacement for ``loop.create_connection`` is also provided.\n\nThe long version\n================\n\nWhat is Happy Eyeballs, and why should I use it?\n------------------------------------------------\n\nHappy Eyeballs is an algorithm for establishing TCP connections to destinations\nspecified by host names. It is described in :rfc:`6555` and :rfc:`8305`. The\nprimary benefit is that when host name resolution returns multiple addresses,\nand some of the address are unreachable, Happy Eyeballs will establish the\nconnection much faster than conventional algorithms. For more information,\ncheck the `Wikipedia article on Happy Eyeballs`_.\n\n.. _Wikipedia article on Happy Eyeballs: https://en.wikipedia.org/wiki/Happy_Eyeballs\n\nPython's standard library provides several high-level methods of establishing\nTCP connections towards a host name: The **socket** module has\n``socket.create_connection``,\nand **asyncio** has ``loop.create_connection`` and ``asyncio.open_connection``.\nBy default,\nthese methods have the same behavior when a host name resolves to several IP\naddresses: they try to connect to the first address in the list,\nand only after the attempt fails (which may take tens of seconds) will\nthe second one be tried, and so on. In contrast, the Happy Eyeballs algorithm\nwill start an attempt with the second IP address in parallel to the first one\nhasn't completed after some time, typically around 300 milliseconds.\nAs a result several attempts may be in flight at the same time, and whenever\none of the attempts succeed, all other connections are cancelled, and the\nwinning connection is used.\nThis means a much shorter wait before one of the IP addresses connect\nsuccessfully.\n\nHappy Eyeballs is particularly important for dual-stack clients, when some hosts\nmay have resolvable IPv6 addresses that are somehow unreachable.\n\nStarting from Python 3.8, stock **asyncio** also supports Happy Eyeballs.\nSee below for a comparison.\n\n\nWhat does ``async_stagger`` has to offer?\n-----------------------------------------\n\n``async_stagger`` provides ``open_connection`` and\n``create_connection`` with Happy Eyeballs support. They are mostly drop-in\nreplacements for their ``asyncio`` counterparts, and support most existing\narguments.\n(There are small differences: ``create_connection`` takes\na *loop* argument instead of being a method on an event loop.\nAlso, these two methods do not support the *sock* argument.)\nAnother public coroutine ``create_connected_sock`` returns a connected\n``socket.socket`` object.\nCheck the documentation for details.\n\nThese methods implements many features specified in :rfc:`8305` Happy Eyeballs\nv2, which extends and obsoletes :rfc:`6555`. In particular, asynchronous\naddress resolution, destination address interleaving by family and staggered\nconnection attempts are implemented.\n\n\nHappy Eyeballs sounds great! I want to use similar logic somewhere else!\n------------------------------------------------------------------------\n\nYou're in luck! ``async_stagger`` actually exposes the underlying scheduling\nlogic as a reusable block: ``staggered_race``. It can be use when:\n\n* There are several ways to achieve one goal. Some of the ways may fail, but\n you have to try it to find out.\n\n* Making attempts strictly in sequence is too slow.\n\n* You want to parallelize, but also don't want to start the attempts all\n at the same time. Maybe you want to give preference to some of the attempts,\n so they should be started earlier and given more time to complete. Maybe you\n want to avoid straining the system with simultaneous attempts.\n\n* An attempt done half-way can be rolled back safely.\n\n\nWhere can I get it?\n-------------------\n\n``async_stagger`` requires Python 3.6 or later.\n(v0.2.0 onwards uses more new features in 3.6 such as async generators and\nasync comprehensions, so it will probably require more than cosmetic changes\nto make it run on 3.5.)\nIt does not have any external dependencies.\nInstall it from PyPI the usual way::\n\n pip install async-stagger\n\nThe documentation can be found here:\nhttp://async-stagger.readthedocs.io/en/latest/\n\nThis project is under active development, and APIs may change in the future.\nCheck out the Changelog in the documentation.\n\nThis project is licensed under the MIT license.\n\n\nPython 3.8 Has Native Happy Eyeballs Now\n========================================\n\nI contributed an implementation of Happy Eyeballs to upstream **asyncio**,\nand it landed in Python 3.8: see `the docs`__ for details.\n\n__ https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.create_connection\n\nThat implementation is essentially an early version of this package,\nso it lacks these features:\n\n* Async address resolution\n* Detailed exception report\n* The ``local_addrs`` argument (as opposed to ``local_addr``)\n\nStill, it should be sufficient for most scenarios, and it's right there in the standard library.\n\n\nMiscellaneous Remarks\n=====================\n\nAsynchronous address resolution is added in v0.2.1. With that, I feel that\nthe package should be fairly feature-complete.\n\nI have implemented Happy Eyeballs-like algorithms in some of my other projects,\nand this module reflects the things I have learned. However I have yet to\neat my own dog food and actually import this module from those other projects.\nI would love to hear people's experience using this module in real world\nconditions.\n\n`bpo-31861 <https://bugs.python.org/issue31861>`_ talks about adding native\n``aiter`` and ``anext`` functions either as builtins or to the ``operator``\nmodule. Well, I want them NAO!!!one!!!eleventy!! So I borrowed the\nimplementations from that bpo and put them in the ``aitertools`` submodule.\nI have only kept the one-argument forms; In particular, the two-argument\n``iter`` function is so disparate from the one-argument version, that I don't\nthink they belong to the same function at all, and there really shouldn't be\na need for ``aiter`` to emulate that behavior.\n\n\nAcknowledgments\n===============\n\nThe Happy Eyeballs scheduling algorithm implementation is inspired by\n`the implementation in trio`__.\n\n__ https://github.com/python-trio/trio/pull/145/files\n\n\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Happy eyeballs and underlying scheduling algorithm in asyncio",
"version": "0.3.1",
"project_urls": {
"Documentation": "http://async_stagger.readthedocs.io",
"Homepage": "https://github.com/twisteroidambassador/async_stagger"
},
"split_keywords": [
"happy-eyeballs",
"dual-stack",
"tcp"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2054ac8d5d9a82e9f9ae6fe68aca82471b41fddd3aa9a64376ac30e57d2ba9da",
"md5": "acbb11f8e9471feed28bd97caa25cf66",
"sha256": "4d9e238ee1845de12ab310e8cee03296325973fc9a0b219d1e5be052ba852cd0"
},
"downloads": -1,
"filename": "async_stagger-0.3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "acbb11f8e9471feed28bd97caa25cf66",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 25328,
"upload_time": "2020-10-20T03:08:14",
"upload_time_iso_8601": "2020-10-20T03:08:14.368719Z",
"url": "https://files.pythonhosted.org/packages/20/54/ac8d5d9a82e9f9ae6fe68aca82471b41fddd3aa9a64376ac30e57d2ba9da/async_stagger-0.3.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f03e1bbe65201027f05d973bf9cecf1e0141553f6ce4b8e78116c065fc59786a",
"md5": "72be6e13938a0b74d746ecdc2d97fd4a",
"sha256": "aa9f357e9efd277e9a516a94bde8124ad5e467e9bc65c9fadaaac98e956a43d6"
},
"downloads": -1,
"filename": "async_stagger-0.3.1.tar.gz",
"has_sig": false,
"md5_digest": "72be6e13938a0b74d746ecdc2d97fd4a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 23403,
"upload_time": "2020-10-20T03:08:15",
"upload_time_iso_8601": "2020-10-20T03:08:15.679959Z",
"url": "https://files.pythonhosted.org/packages/f0/3e/1bbe65201027f05d973bf9cecf1e0141553f6ce4b8e78116c065fc59786a/async_stagger-0.3.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2020-10-20 03:08:15",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "twisteroidambassador",
"github_project": "async_stagger",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "async-stagger"
}
JSON
