|Tests Status| |Coverage Status|
Patroni: A Template for PostgreSQL HA with ZooKeeper, etcd or Consul
--------------------------------------------------------------------
You can find a version of this documentation that is searchable and also easier to navigate at `patroni.readthedocs.io <https://patroni.readthedocs.io>`__.
There are many ways to run high availability with PostgreSQL; for a list, see the `PostgreSQL Documentation <https://wiki.postgresql.org/wiki/Replication,_Clustering,_and_Connection_Pooling>`__.
Patroni is a template for high availability (HA) PostgreSQL solutions using Python. For maximum accessibility, Patroni supports a variety of distributed configuration stores like `ZooKeeper <https://zookeeper.apache.org/>`__, `etcd <https://github.com/coreos/etcd>`__, `Consul <https://github.com/hashicorp/consul>`__ or `Kubernetes <https://kubernetes.io>`__. Database engineers, DBAs, DevOps engineers, and SREs who are looking to quickly deploy HA PostgreSQL in datacenters - or anywhere else - will hopefully find it useful.
We call Patroni a "template" because it is far from being a one-size-fits-all or plug-and-play replication system. It will have its own caveats. Use wisely.
Currently supported PostgreSQL versions: 9.3 to 16.
**Note to Citus users**: Starting from 3.0 Patroni nicely integrates with the `Citus <https://github.com/citusdata/citus>`__ database extension to Postgres. Please check the `Citus support page <https://github.com/zalando/patroni/blob/master/docs/citus.rst>`__ in the Patroni documentation for more info about how to use Patroni high availability together with a Citus distributed cluster.
**Note to Kubernetes users**: Patroni can run natively on top of Kubernetes. Take a look at the `Kubernetes <https://github.com/zalando/patroni/blob/master/docs/kubernetes.rst>`__ chapter of the Patroni documentation.
.. contents::
:local:
:depth: 1
:backlinks: none
=================
How Patroni Works
=================
Patroni originated as a fork of `Governor <https://github.com/compose/governor>`__, the project from Compose. It includes plenty of new features.
For an example of a Docker-based deployment with Patroni, see `Spilo <https://github.com/zalando/spilo>`__, currently in use at Zalando.
For additional background info, see:
* `Elephants on Automatic: HA Clustered PostgreSQL with Helm <https://www.youtube.com/watch?v=CftcVhFMGSY>`_, talk by Josh Berkus and Oleksii Kliukin at KubeCon Berlin 2017
* `PostgreSQL HA with Kubernetes and Patroni <https://www.youtube.com/watch?v=iruaCgeG7qs>`__, talk by Josh Berkus at KubeCon 2016 (video)
* `Feb. 2016 Zalando Tech blog post <https://tech.zalando.de/blog/zalandos-patroni-a-template-for-high-availability-postgresql/>`__
==================
Development Status
==================
Patroni is in active development and accepts contributions. See our `Contributing <https://github.com/zalando/patroni/blob/master/docs/CONTRIBUTING.rst>`__ section below for more details.
We report new releases information `here <https://github.com/zalando/patroni/releases>`__.
=========
Community
=========
There are two places to connect with the Patroni community: `on github <https://github.com/zalando/patroni>`__, via Issues and PRs, and on channel `#patroni <https://postgresteam.slack.com/archives/C9XPYG92A>`__ in the `PostgreSQL Slack <https://pgtreats.info/slack-invite>`__. If you're using Patroni, or just interested, please join us.
===================================
Technical Requirements/Installation
===================================
**Pre-requirements for Mac OS**
To install requirements on a Mac, run the following:
::
brew install postgresql etcd haproxy libyaml python
**Psycopg**
Starting from `psycopg2-2.8 <http://initd.org/psycopg/articles/2019/04/04/psycopg-28-released/>`__ the binary version of psycopg2 will no longer be installed by default. Installing it from the source code requires C compiler and postgres+python dev packages.
Since in the python world it is not possible to specify dependency as ``psycopg2 OR psycopg2-binary`` you will have to decide how to install it.
There are a few options available:
1. Use the package manager from your distro
::
sudo apt-get install python3-psycopg2 # install psycopg2 module on Debian/Ubuntu
sudo yum install python3-psycopg2 # install psycopg2 on RedHat/Fedora/CentOS
2. Specify one of `psycopg`, `psycopg2`, or `psycopg2-binary` in the list of dependencies when installing Patroni with pip (see below).
**General installation for pip**
Patroni can be installed with pip:
::
pip install patroni[dependencies]
where dependencies can be either empty, or consist of one or more of the following:
etcd or etcd3
`python-etcd` module in order to use Etcd as DCS
consul
`python-consul` module in order to use Consul as DCS
zookeeper
`kazoo` module in order to use Zookeeper as DCS
exhibitor
`kazoo` module in order to use Exhibitor as DCS (same dependencies as for Zookeeper)
kubernetes
`kubernetes` module in order to use Kubernetes as DCS in Patroni
raft
`pysyncobj` module in order to use python Raft implementation as DCS
aws
`boto3` in order to use AWS callbacks
all
all of the above (except psycopg family)
psycopg3
`psycopg[binary]>=3.0.0` module
psycopg2
`psycopg2>=2.5.4` module
psycopg2-binary
`psycopg2-binary` module
For example, the command in order to install Patroni together with psycopg3, dependencies for Etcd as a DCS, and AWS callbacks is:
::
pip install patroni[psycopg3,etcd3,aws]
Note that external tools to call in the replica creation or custom bootstrap scripts (i.e. WAL-E) should be installed independently of Patroni.
=======================
Running and Configuring
=======================
To get started, do the following from different terminals:
::
> etcd --data-dir=data/etcd --enable-v2=true
> ./patroni.py postgres0.yml
> ./patroni.py postgres1.yml
You will then see a high-availability cluster start up. Test different settings in the YAML files to see how the cluster's behavior changes. Kill some of the components to see how the system behaves.
Add more ``postgres*.yml`` files to create an even larger cluster.
Patroni provides an `HAProxy <http://www.haproxy.org/>`__ configuration, which will give your application a single endpoint for connecting to the cluster's leader. To configure,
run:
::
> haproxy -f haproxy.cfg
::
> psql --host 127.0.0.1 --port 5000 postgres
==================
YAML Configuration
==================
Go `here <https://github.com/zalando/patroni/blob/master/docs/SETTINGS.rst>`__ for comprehensive information about settings for etcd, consul, and ZooKeeper. And for an example, see `postgres0.yml <https://github.com/zalando/patroni/blob/master/postgres0.yml>`__.
=========================
Environment Configuration
=========================
Go `here <https://github.com/zalando/patroni/blob/master/docs/ENVIRONMENT.rst>`__ for comprehensive information about configuring(overriding) settings via environment variables.
===================
Replication Choices
===================
Patroni uses Postgres' streaming replication, which is asynchronous by default. Patroni's asynchronous replication configuration allows for ``maximum_lag_on_failover`` settings. This setting ensures failover will not occur if a follower is more than a certain number of bytes behind the leader. This setting should be increased or decreased based on business requirements. It's also possible to use synchronous replication for better durability guarantees. See `replication modes documentation <https://github.com/zalando/patroni/blob/master/docs/replication_modes.rst>`__ for details.
======================================
Applications Should Not Use Superusers
======================================
When connecting from an application, always use a non-superuser. Patroni requires access to the database to function properly. By using a superuser from an application, you can potentially use the entire connection pool, including the connections reserved for superusers, with the ``superuser_reserved_connections`` setting. If Patroni cannot access the Primary because the connection pool is full, behavior will be undesirable.
.. |Tests Status| image:: https://github.com/zalando/patroni/actions/workflows/tests.yaml/badge.svg
:target: https://github.com/zalando/patroni/actions/workflows/tests.yaml?query=branch%3Amaster
.. |Coverage Status| image:: https://coveralls.io/repos/zalando/patroni/badge.svg?branch=master
:target: https://coveralls.io/github/zalando/patroni?branch=master
Raw data
{
"_id": null,
"home_page": "https://github.com/zalando/patroni",
"name": "patroni",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "etcd governor patroni postgresql postgres ha haproxy confd zookeeper exhibitor consul streaming replication kubernetes k8s",
"author": "Alexander Kukushkin, Polina Bungina",
"author_email": "akukushkin@microsoft.com, polina.bungina@zalando.de",
"download_url": "https://files.pythonhosted.org/packages/ba/b5/a7e439aeab6d1964cefc942346934906dd7eecc329d7297b3e890d23f40e/patroni-3.2.1.tar.gz",
"platform": null,
"description": "|Tests Status| |Coverage Status|\n\nPatroni: A Template for PostgreSQL HA with ZooKeeper, etcd or Consul\n--------------------------------------------------------------------\n\nYou can find a version of this documentation that is searchable and also easier to navigate at `patroni.readthedocs.io <https://patroni.readthedocs.io>`__.\n\n\nThere are many ways to run high availability with PostgreSQL; for a list, see the `PostgreSQL Documentation <https://wiki.postgresql.org/wiki/Replication,_Clustering,_and_Connection_Pooling>`__.\n\nPatroni is a template for high availability (HA) PostgreSQL solutions using Python. For maximum accessibility, Patroni supports a variety of distributed configuration stores like `ZooKeeper <https://zookeeper.apache.org/>`__, `etcd <https://github.com/coreos/etcd>`__, `Consul <https://github.com/hashicorp/consul>`__ or `Kubernetes <https://kubernetes.io>`__. Database engineers, DBAs, DevOps engineers, and SREs who are looking to quickly deploy HA PostgreSQL in datacenters - or anywhere else - will hopefully find it useful.\n\nWe call Patroni a \"template\" because it is far from being a one-size-fits-all or plug-and-play replication system. It will have its own caveats. Use wisely.\n\nCurrently supported PostgreSQL versions: 9.3 to 16.\n\n**Note to Citus users**: Starting from 3.0 Patroni nicely integrates with the `Citus <https://github.com/citusdata/citus>`__ database extension to Postgres. Please check the `Citus support page <https://github.com/zalando/patroni/blob/master/docs/citus.rst>`__ in the Patroni documentation for more info about how to use Patroni high availability together with a Citus distributed cluster.\n\n**Note to Kubernetes users**: Patroni can run natively on top of Kubernetes. Take a look at the `Kubernetes <https://github.com/zalando/patroni/blob/master/docs/kubernetes.rst>`__ chapter of the Patroni documentation.\n\n.. contents::\n :local:\n :depth: 1\n :backlinks: none\n\n=================\nHow Patroni Works\n=================\n\nPatroni originated as a fork of `Governor <https://github.com/compose/governor>`__, the project from Compose. It includes plenty of new features.\n\nFor an example of a Docker-based deployment with Patroni, see `Spilo <https://github.com/zalando/spilo>`__, currently in use at Zalando.\n\nFor additional background info, see:\n\n* `Elephants on Automatic: HA Clustered PostgreSQL with Helm <https://www.youtube.com/watch?v=CftcVhFMGSY>`_, talk by Josh Berkus and Oleksii Kliukin at KubeCon Berlin 2017\n* `PostgreSQL HA with Kubernetes and Patroni <https://www.youtube.com/watch?v=iruaCgeG7qs>`__, talk by Josh Berkus at KubeCon 2016 (video)\n* `Feb. 2016 Zalando Tech blog post <https://tech.zalando.de/blog/zalandos-patroni-a-template-for-high-availability-postgresql/>`__\n\n==================\nDevelopment Status\n==================\n\nPatroni is in active development and accepts contributions. See our `Contributing <https://github.com/zalando/patroni/blob/master/docs/CONTRIBUTING.rst>`__ section below for more details.\n\nWe report new releases information `here <https://github.com/zalando/patroni/releases>`__.\n\n=========\nCommunity\n=========\n\nThere are two places to connect with the Patroni community: `on github <https://github.com/zalando/patroni>`__, via Issues and PRs, and on channel `#patroni <https://postgresteam.slack.com/archives/C9XPYG92A>`__ in the `PostgreSQL Slack <https://pgtreats.info/slack-invite>`__. If you're using Patroni, or just interested, please join us.\n\n===================================\nTechnical Requirements/Installation\n===================================\n\n**Pre-requirements for Mac OS**\n\nTo install requirements on a Mac, run the following:\n\n::\n\n brew install postgresql etcd haproxy libyaml python\n\n**Psycopg**\n\nStarting from `psycopg2-2.8 <http://initd.org/psycopg/articles/2019/04/04/psycopg-28-released/>`__ the binary version of psycopg2 will no longer be installed by default. Installing it from the source code requires C compiler and postgres+python dev packages.\nSince in the python world it is not possible to specify dependency as ``psycopg2 OR psycopg2-binary`` you will have to decide how to install it.\n\nThere are a few options available:\n\n1. Use the package manager from your distro\n\n::\n\n sudo apt-get install python3-psycopg2 # install psycopg2 module on Debian/Ubuntu\n sudo yum install python3-psycopg2 # install psycopg2 on RedHat/Fedora/CentOS\n\n2. Specify one of `psycopg`, `psycopg2`, or `psycopg2-binary` in the list of dependencies when installing Patroni with pip (see below).\n\n\n**General installation for pip**\n\nPatroni can be installed with pip:\n\n::\n\n pip install patroni[dependencies]\n\nwhere dependencies can be either empty, or consist of one or more of the following:\n\netcd or etcd3\n `python-etcd` module in order to use Etcd as DCS\nconsul\n `python-consul` module in order to use Consul as DCS\nzookeeper\n `kazoo` module in order to use Zookeeper as DCS\nexhibitor\n `kazoo` module in order to use Exhibitor as DCS (same dependencies as for Zookeeper)\nkubernetes\n `kubernetes` module in order to use Kubernetes as DCS in Patroni\nraft\n `pysyncobj` module in order to use python Raft implementation as DCS\naws\n `boto3` in order to use AWS callbacks\nall\n all of the above (except psycopg family)\npsycopg3\n `psycopg[binary]>=3.0.0` module\npsycopg2\n `psycopg2>=2.5.4` module\npsycopg2-binary\n `psycopg2-binary` module\n\nFor example, the command in order to install Patroni together with psycopg3, dependencies for Etcd as a DCS, and AWS callbacks is:\n\n::\n\n pip install patroni[psycopg3,etcd3,aws]\n\nNote that external tools to call in the replica creation or custom bootstrap scripts (i.e. WAL-E) should be installed independently of Patroni.\n\n=======================\nRunning and Configuring\n=======================\n\nTo get started, do the following from different terminals:\n::\n\n > etcd --data-dir=data/etcd --enable-v2=true\n > ./patroni.py postgres0.yml\n > ./patroni.py postgres1.yml\n\nYou will then see a high-availability cluster start up. Test different settings in the YAML files to see how the cluster's behavior changes. Kill some of the components to see how the system behaves.\n\nAdd more ``postgres*.yml`` files to create an even larger cluster.\n\nPatroni provides an `HAProxy <http://www.haproxy.org/>`__ configuration, which will give your application a single endpoint for connecting to the cluster's leader. To configure,\nrun:\n\n::\n\n > haproxy -f haproxy.cfg\n\n::\n\n > psql --host 127.0.0.1 --port 5000 postgres\n\n==================\nYAML Configuration\n==================\n\nGo `here <https://github.com/zalando/patroni/blob/master/docs/SETTINGS.rst>`__ for comprehensive information about settings for etcd, consul, and ZooKeeper. And for an example, see `postgres0.yml <https://github.com/zalando/patroni/blob/master/postgres0.yml>`__.\n\n=========================\nEnvironment Configuration\n=========================\n\nGo `here <https://github.com/zalando/patroni/blob/master/docs/ENVIRONMENT.rst>`__ for comprehensive information about configuring(overriding) settings via environment variables.\n\n===================\nReplication Choices\n===================\n\nPatroni uses Postgres' streaming replication, which is asynchronous by default. Patroni's asynchronous replication configuration allows for ``maximum_lag_on_failover`` settings. This setting ensures failover will not occur if a follower is more than a certain number of bytes behind the leader. This setting should be increased or decreased based on business requirements. It's also possible to use synchronous replication for better durability guarantees. See `replication modes documentation <https://github.com/zalando/patroni/blob/master/docs/replication_modes.rst>`__ for details.\n\n======================================\nApplications Should Not Use Superusers\n======================================\n\nWhen connecting from an application, always use a non-superuser. Patroni requires access to the database to function properly. By using a superuser from an application, you can potentially use the entire connection pool, including the connections reserved for superusers, with the ``superuser_reserved_connections`` setting. If Patroni cannot access the Primary because the connection pool is full, behavior will be undesirable.\n\n.. |Tests Status| image:: https://github.com/zalando/patroni/actions/workflows/tests.yaml/badge.svg\n :target: https://github.com/zalando/patroni/actions/workflows/tests.yaml?query=branch%3Amaster\n.. |Coverage Status| image:: https://coveralls.io/repos/zalando/patroni/badge.svg?branch=master\n :target: https://coveralls.io/github/zalando/patroni?branch=master\n",
"bugtrack_url": null,
"license": "The MIT License",
"summary": "PostgreSQL High-Available orchestrator and CLI",
"version": "3.2.1",
"project_urls": {
"Homepage": "https://github.com/zalando/patroni"
},
"split_keywords": [
"etcd",
"governor",
"patroni",
"postgresql",
"postgres",
"ha",
"haproxy",
"confd",
"zookeeper",
"exhibitor",
"consul",
"streaming",
"replication",
"kubernetes",
"k8s"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "32c367897010a3b6f4d151ea33db5bf7f899d8523f06225b17f2e5b82d19eecf",
"md5": "cd841f1a16992ea119825985d6143414",
"sha256": "ffce5a540ee30f94cfb4242e9aefcbf73b49875bf11de34a0e3ec14c575bec2a"
},
"downloads": -1,
"filename": "patroni-3.2.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "cd841f1a16992ea119825985d6143414",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 317293,
"upload_time": "2023-11-30T16:28:59",
"upload_time_iso_8601": "2023-11-30T16:28:59.124758Z",
"url": "https://files.pythonhosted.org/packages/32/c3/67897010a3b6f4d151ea33db5bf7f899d8523f06225b17f2e5b82d19eecf/patroni-3.2.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "bab5a7e439aeab6d1964cefc942346934906dd7eecc329d7297b3e890d23f40e",
"md5": "0333bcb74c8b7bad830e4bbe7f91af48",
"sha256": "522215ccb3a8e92ca2bc1cab28ddd25aff6de6f32f5bb654a0b7d6f19faf7abc"
},
"downloads": -1,
"filename": "patroni-3.2.1.tar.gz",
"has_sig": false,
"md5_digest": "0333bcb74c8b7bad830e4bbe7f91af48",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 375298,
"upload_time": "2023-11-30T16:29:01",
"upload_time_iso_8601": "2023-11-30T16:29:01.730513Z",
"url": "https://files.pythonhosted.org/packages/ba/b5/a7e439aeab6d1964cefc942346934906dd7eecc329d7297b3e890d23f40e/patroni-3.2.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-30 16:29:01",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "zalando",
"github_project": "patroni",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"tox": true,
"lcname": "patroni"
}
JSON
