pyramid_forksafe
================
Build Status: 
This package creates standardized Pyramid events for various forking hooks in popular deployment containers.
Using `pyramid_forksafe` allows a developer to write generic routines for forking events, allowing them to easily swap containers during deployment or development.
Each event is invoked with the application's `registry`, through which one can access `registry.settings`
## Why?
Pyramid is Thread Safe, which is different than Fork Safe.
Several popular libraries are not fork-safe:
* SqlAlchemy's connection pool is not fork-safe. Your deployment *must* call `engine.dispose()` after a fork.
* PyMongo's connections and locks are not fork-safe. The entire client must be replaced after a fork.
* PyCrypto's Random generator will only work correctly if Random.atfork() is called.
In some situations, a developer may need to access the registry and/or settings during postfork actions. Getting this information into a custom hook can be a hassle, as one will need to write against each container's API instead of Pyramid's.
This blog posting describes the difference between fork-safe and thread-safe pretty well http://www.dctrwatson.com/2010/09/python-thread-safe-does-not-mean-fork-safe/
## Usage - Generic
Define a GENERIC hook.
from pyramid_forksafe.events import ApplicationPostFork
def post_fork_hook(event):
"""
The event has an attribute for the Pyramid Application's `registry`
`event.registry`
""""
cyrpto_atfork()
models.engine.dispose()
config.add_subscriber(post_fork_hook, ApplicationPostFork)
You can import the generic package in your `environment.ini` file (or main config), and this will try to enable services if possible:
# development.ini
pyramid.includes = pyramid_forksafe
or you may wish to import a SPECIFIC container package in your `environment.ini` file (or main config)
# development.ini
pyramid.includes = pyramid_forksafe.containers.uwsgi
Currently, this approach only works for `uWSGI`. `gunicorn` requires another approach.
## Usage - uWSGI
simply include the package and uwsgi will be automatically enabled:
in your `__init__.py`:
config.include('pyramid_forksafe')
or your `{environment}.ini`
pyramid.includes = pyramid_forksafe
important note:
you MUST run uWSGI with the `--master` argument.
## Usage - gunicorn
`gunicorn` will need some hooks imported into it's python configuration file
assuming you invoke gunicorn like this:
gunicorn --paste production.ini -c config.py
then your `config.py` just needs to import the container hooks:
from pyramid_forksafe.containers.gunicorn import (
pre_fork,
post_fork,
post_worker_init,
)
those hooks are written to the `gunicorn` api, and will invoke the notification
you can also update the debug tool by running after configuration:
from pyramid_forksafe.containers.gunicorn import mark_configured
mark_configured(config.registry)
## Container Support
Currently `uwsgi` and `gunicorn` are supported with the hooks outlined below. Celery integration is planned. Pull requests are very welcome.
| container | pyramid\_forksafe event | container hook |
|-----------|-----------------------------|------|
| uWSGI | `ApplicationPostFork` | [`postfork`](http://uwsgi-docs.readthedocs.io/en/latest/PythonDecorators.html#uwsgidecorators.postfork) |
| gunicorn | `ApplicationPostFork` | [`post_fork`](http://docs.gunicorn.org/en/latest/settings.html#post-fork) |
| gunicorn | `ApplicationPreFork` | [`pre_fork`](http://docs.gunicorn.org/en/latest/settings.html#pre-fork) |
| gunicorn | `ApplicationPostWorkerInit` | [`post_worker_init`](http://docs.gunicorn.org/en/latest/settings.html#post-worker-init) |
## The Debug Object
including this package will put an informative dict into `registry.pyramid_forksafe`
under waitress, it will look like this:
[('status', 'attempting auto-configure'),
('environment', None),
('autoconfigure.log', ['uWSGI not available']),
('executed_hooks', set([]))]
under uWSGI without master, it will look like this:
[('status', 'uWSGI error'),
('environment', None),
('autoconfigure.log',
['uWSGI error: you have to enable the uWSGI master process to use this module']),
('executed_hooks', set([]))]
under uWSGI properly configured, it will look like this:
[('status', 'uWSGI hook configured'),
('environment', 'uWSGI'),
('autoconfigure.log', ['uWSGI available', 'uWSGI hook configured']),
('executed_hooks',
set([('containers.uwsgi.post_fork_hook', 'ApplicationPostFork')]))]
## Debugtoolbar support
to enable the debugtoobar support, you can configure your `development.ini` with:
debugtoolbar.includes = pyramid_forksafe.debugtoolbar
The toolbar just shows the debug object `request.registry.pyramid_forksafe` on the toolbar
This should always show an error, because the debugtoolbar does not run under forking servers.
## Status
2019.05.01 - debugtoolbar
2019.04.30 - debug object
2019.04.29 - Python3 Support. This has been production safe for uWSGI for a while now.
2016.11.09 - this is experimental
Raw data
{
"_id": null,
"home_page": "https://github.com/jvanasco/pyramid_forksafe",
"name": "pyramid-forksafe",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "web pyramid fork uwsgi nginx",
"author": "Jonathan Vanasco",
"author_email": "jonathan@findmeon.com",
"download_url": "https://files.pythonhosted.org/packages/a4/c1/191520d35efb8a7f5979e71704616f5ae53d007b9d790b9ebb87b02a286b/pyramid_forksafe-0.2.0.tar.gz",
"platform": null,
"description": "pyramid_forksafe\n================\n\nBuild Status: \n\nThis package creates standardized Pyramid events for various forking hooks in popular deployment containers.\n\nUsing `pyramid_forksafe` allows a developer to write generic routines for forking events, allowing them to easily swap containers during deployment or development.\n\nEach event is invoked with the application's `registry`, through which one can access `registry.settings`\n\n## Why?\n\nPyramid is Thread Safe, which is different than Fork Safe.\n\nSeveral popular libraries are not fork-safe:\n\n* SqlAlchemy's connection pool is not fork-safe. Your deployment *must* call `engine.dispose()` after a fork.\n* PyMongo's connections and locks are not fork-safe. The entire client must be replaced after a fork.\n* PyCrypto's Random generator will only work correctly if Random.atfork() is called.\n\nIn some situations, a developer may need to access the registry and/or settings during postfork actions. Getting this information into a custom hook can be a hassle, as one will need to write against each container's API instead of Pyramid's. \n\nThis blog posting describes the difference between fork-safe and thread-safe pretty well http://www.dctrwatson.com/2010/09/python-thread-safe-does-not-mean-fork-safe/\n\n\n## Usage - Generic\n\nDefine a GENERIC hook. \n\n\tfrom pyramid_forksafe.events import ApplicationPostFork\n\n def post_fork_hook(event):\n \t\"\"\"\n \tThe event has an attribute for the Pyramid Application's `registry`\n \t\t`event.registry`\n \t\"\"\"\"\n cyrpto_atfork()\n models.engine.dispose()\n\n config.add_subscriber(post_fork_hook, ApplicationPostFork)\n\nYou can import the generic package in your `environment.ini` file (or main config), and this will try to enable services if possible:\n\n\t# development.ini\n pyramid.includes = pyramid_forksafe\n\nor you may wish to import a SPECIFIC container package in your `environment.ini` file (or main config)\n\n\t# development.ini\n pyramid.includes = pyramid_forksafe.containers.uwsgi\n\nCurrently, this approach only works for `uWSGI`. `gunicorn` requires another approach.\n\n\n## Usage - uWSGI\n\nsimply include the package and uwsgi will be automatically enabled:\n\nin your `__init__.py`:\n\n config.include('pyramid_forksafe')\n\nor your `{environment}.ini`\n\n pyramid.includes = pyramid_forksafe\n\nimportant note:\n\nyou MUST run uWSGI with the `--master` argument.\n\n\n## Usage - gunicorn\n\n`gunicorn` will need some hooks imported into it's python configuration file\n\nassuming you invoke gunicorn like this:\n\n\tgunicorn --paste production.ini -c config.py\n\nthen your `config.py` just needs to import the container hooks:\n\n from pyramid_forksafe.containers.gunicorn import (\n pre_fork,\n post_fork,\n post_worker_init,\n )\n\nthose hooks are written to the `gunicorn` api, and will invoke the notification\n\nyou can also update the debug tool by running after configuration:\n\n\tfrom pyramid_forksafe.containers.gunicorn import mark_configured\n\tmark_configured(config.registry)\n\n\n## Container Support\n\nCurrently `uwsgi` and `gunicorn` are supported with the hooks outlined below. Celery integration is planned. Pull requests are very welcome.\n\n\n| container | pyramid\\_forksafe event | container hook |\n|-----------|-----------------------------|------|\n| uWSGI | `ApplicationPostFork` | [`postfork`](http://uwsgi-docs.readthedocs.io/en/latest/PythonDecorators.html#uwsgidecorators.postfork) |\n| gunicorn | `ApplicationPostFork` | [`post_fork`](http://docs.gunicorn.org/en/latest/settings.html#post-fork) |\n| gunicorn | `ApplicationPreFork` | [`pre_fork`](http://docs.gunicorn.org/en/latest/settings.html#pre-fork) |\n| gunicorn | `ApplicationPostWorkerInit` | [`post_worker_init`](http://docs.gunicorn.org/en/latest/settings.html#post-worker-init) |\n\n\n## The Debug Object\n\nincluding this package will put an informative dict into `registry.pyramid_forksafe`\n\nunder waitress, it will look like this:\n\n\t[('status', 'attempting auto-configure'),\n\t ('environment', None),\n\t ('autoconfigure.log', ['uWSGI not available']),\n\t ('executed_hooks', set([]))]\n\nunder uWSGI without master, it will look like this:\n\n\t[('status', 'uWSGI error'),\n\t ('environment', None),\n\t ('autoconfigure.log',\n\t ['uWSGI error: you have to enable the uWSGI master process to use this module']),\n\t ('executed_hooks', set([]))]\n\nunder uWSGI properly configured, it will look like this:\n\n\t[('status', 'uWSGI hook configured'),\n\t ('environment', 'uWSGI'),\n\t ('autoconfigure.log', ['uWSGI available', 'uWSGI hook configured']),\n\t ('executed_hooks',\n\t set([('containers.uwsgi.post_fork_hook', 'ApplicationPostFork')]))]\n\n\n## Debugtoolbar support\n\nto enable the debugtoobar support, you can configure your `development.ini` with:\n\n\tdebugtoolbar.includes = pyramid_forksafe.debugtoolbar\n\nThe toolbar just shows the debug object `request.registry.pyramid_forksafe` on the toolbar\n\nThis should always show an error, because the debugtoolbar does not run under forking servers.\n\n\n\n## Status\n2019.05.01 - debugtoolbar\n2019.04.30 - debug object\n2019.04.29 - Python3 Support. This has been production safe for uWSGI for a while now.\n2016.11.09 - this is experimental\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Standardizes server `fork` events into Pyrmamid events",
"version": "0.2.0",
"project_urls": {
"Homepage": "https://github.com/jvanasco/pyramid_forksafe"
},
"split_keywords": [
"web",
"pyramid",
"fork",
"uwsgi",
"nginx"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a4c1191520d35efb8a7f5979e71704616f5ae53d007b9d790b9ebb87b02a286b",
"md5": "fdc82239dcea0984dfd0131dc7431286",
"sha256": "1a378c03cd690fa687c0ec18116224c3315b254a17caff7486c63843576b5359"
},
"downloads": -1,
"filename": "pyramid_forksafe-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "fdc82239dcea0984dfd0131dc7431286",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 12187,
"upload_time": "2023-06-13T21:02:27",
"upload_time_iso_8601": "2023-06-13T21:02:27.479799Z",
"url": "https://files.pythonhosted.org/packages/a4/c1/191520d35efb8a7f5979e71704616f5ae53d007b9d790b9ebb87b02a286b/pyramid_forksafe-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-13 21:02:27",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "jvanasco",
"github_project": "pyramid_forksafe",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "pyramid-forksafe"
}
JSON
