[](https://app.travis-ci.com/github/postgrespro/testgres/branches)
[](https://codecov.io/gh/postgrespro/testgres)
[](https://badge.fury.io/py/testgres)
[Documentation](https://postgrespro.github.io/testgres/)
# testgres
PostgreSQL testing utility. Both Python 2.7 and 3.3+ are supported.
## Installation
To install `testgres`, run:
```
pip install testgres
```
We encourage you to use `virtualenv` for your testing environment.
## Usage
### Environment
> Note: by default testgres runs `initdb`, `pg_ctl`, `psql` provided by `PATH`.
There are several ways to specify a custom postgres installation:
* export `PG_CONFIG` environment variable pointing to the `pg_config` executable;
* export `PG_BIN` environment variable pointing to the directory with executable files.
Example:
```bash
export PG_BIN=$HOME/pg_10/bin
python my_tests.py
```
### Examples
Here is an example of what you can do with `testgres`:
```python
# create a node with random name, port, etc
with testgres.get_new_node() as node:
# run inidb
node.init()
# start PostgreSQL
node.start()
# execute a query in a default DB
print(node.execute('select 1'))
# ... node stops and its files are about to be removed
```
There are four API methods for runnig queries:
| Command | Description |
|----------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|
| `node.psql(query, ...)` | Runs query via `psql` command and returns tuple `(error code, stdout, stderr)`. |
| `node.safe_psql(query, ...)` | Same as `psql()` except that it returns only `stdout`. If an error occures during the execution, an exception will be thrown. |
| `node.execute(query, ...)` | Connects to PostgreSQL using `psycopg2` or `pg8000` (depends on which one is installed in your system) and returns two-dimensional array with data. |
| `node.connect(dbname, ...)` | Returns connection wrapper (`NodeConnection`) capable of running several queries within a single transaction. |
The last one is the most powerful: you can use `begin(isolation_level)`, `commit()` and `rollback()`:
```python
with node.connect() as con:
con.begin('serializable')
print(con.execute('select %s', 1))
con.rollback()
```
### Logging
By default, `cleanup()` removes all temporary files (DB files, logs etc) that were created by testgres' API methods.
If you'd like to keep logs, execute `configure_testgres(node_cleanup_full=False)` before running any tests.
> Note: context managers (aka `with`) call `stop()` and `cleanup()` automatically.
`testgres` supports [python logging](https://docs.python.org/3.6/library/logging.html),
which means that you can aggregate logs from several nodes into one file:
```python
import logging
# write everything to /tmp/testgres.log
logging.basicConfig(filename='/tmp/testgres.log')
# enable logging, and create two different nodes
testgres.configure_testgres(use_python_logging=True)
node1 = testgres.get_new_node().init().start()
node2 = testgres.get_new_node().init().start()
# execute a few queries
node1.execute('select 1')
node2.execute('select 2')
# disable logging
testgres.configure_testgres(use_python_logging=False)
```
Look at `tests/test_simple.py` file for a complete example of the logging
configuration.
### Backup & replication
It's quite easy to create a backup and start a new replica:
```python
with testgres.get_new_node('master') as master:
master.init().start()
# create a backup
with master.backup() as backup:
# create and start a new replica
replica = backup.spawn_replica('replica').start()
# catch up with master node
replica.catchup()
# execute a dummy query
print(replica.execute('postgres', 'select 1'))
```
### Benchmarks
`testgres` is also capable of running benchmarks using `pgbench`:
```python
with testgres.get_new_node('master') as master:
# start a new node
master.init().start()
# initialize default DB and run bench for 10 seconds
res = master.pgbench_init(scale=2).pgbench_run(time=10)
print(res)
```
### Custom configuration
It's often useful to extend default configuration provided by `testgres`.
`testgres` has `default_conf()` function that helps control some basic
options. The `append_conf()` function can be used to add custom
lines to configuration lines:
```python
ext_conf = "shared_preload_libraries = 'postgres_fdw'"
# initialize a new node
with testgres.get_new_node().init() as master:
# ... do something ...
# reset main config file
master.default_conf(fsync=True,
allow_streaming=True)
# add a new config line
master.append_conf('postgresql.conf', ext_conf)
```
Note that `default_conf()` is called by `init()` function; both of them overwrite
the configuration file, which means that they should be called before `append_conf()`.
### Remote mode
Testgres supports the creation of PostgreSQL nodes on a remote host. This is useful when you want to run distributed tests involving multiple nodes spread across different machines.
To use this feature, you need to use the RemoteOperations class. This feature is only supported with Linux.
Here is an example of how you might set this up:
```python
from testgres import ConnectionParams, RemoteOperations, TestgresConfig, get_remote_node
# Set up connection params
conn_params = ConnectionParams(
host='your_host', # replace with your host
username='user_name', # replace with your username
ssh_key='path_to_ssh_key' # replace with your SSH key path
)
os_ops = RemoteOperations(conn_params)
# Add remote testgres config before test
TestgresConfig.set_os_ops(os_ops=os_ops)
# Proceed with your test
def test_basic_query(self):
with get_remote_node(conn_params=conn_params) as node:
node.init().start()
res = node.execute('SELECT 1')
self.assertEqual(res, [(1,)])
```
## Authors
[Ildar Musin](https://github.com/zilder)
[Dmitry Ivanov](https://github.com/funbringer)
[Ildus Kurbangaliev](https://github.com/ildus)
[Yury Zhuravlev](https://github.com/stalkerg)
Raw data
{
"_id": null,
"home_page": "https://github.com/postgrespro/testgres",
"name": "testgres",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "test, testing, postgresql",
"author": "Postgres Professional",
"author_email": "testgres@postgrespro.ru",
"download_url": "https://files.pythonhosted.org/packages/65/55/881156fd940f23ca5d69af764bc6e60eff2b77570c9f792473a79deeb278/testgres-1.10.2.tar.gz",
"platform": null,
"description": "[](https://app.travis-ci.com/github/postgrespro/testgres/branches)\n[](https://codecov.io/gh/postgrespro/testgres)\n[](https://badge.fury.io/py/testgres)\n\n[Documentation](https://postgrespro.github.io/testgres/)\n\n# testgres\n\nPostgreSQL testing utility. Both Python 2.7 and 3.3+ are supported.\n\n\n## Installation\n\nTo install `testgres`, run:\n\n```\npip install testgres\n```\n\nWe encourage you to use `virtualenv` for your testing environment.\n\n\n## Usage\n\n### Environment\n\n> Note: by default testgres runs `initdb`, `pg_ctl`, `psql` provided by `PATH`.\n\nThere are several ways to specify a custom postgres installation:\n\n* export `PG_CONFIG` environment variable pointing to the `pg_config` executable;\n* export `PG_BIN` environment variable pointing to the directory with executable files.\n\nExample:\n\n```bash\nexport PG_BIN=$HOME/pg_10/bin\npython my_tests.py\n```\n\n\n### Examples\n\nHere is an example of what you can do with `testgres`:\n\n```python\n# create a node with random name, port, etc\nwith testgres.get_new_node() as node:\n\n # run inidb\n node.init()\n\n # start PostgreSQL\n node.start()\n\n # execute a query in a default DB\n print(node.execute('select 1'))\n\n# ... node stops and its files are about to be removed\n```\n\nThere are four API methods for runnig queries:\n\n| Command | Description |\n|----------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|\n| `node.psql(query, ...)` | Runs query via `psql` command and returns tuple `(error code, stdout, stderr)`. |\n| `node.safe_psql(query, ...)` | Same as `psql()` except that it returns only `stdout`. If an error occures during the execution, an exception will be thrown. |\n| `node.execute(query, ...)` | Connects to PostgreSQL using `psycopg2` or `pg8000` (depends on which one is installed in your system) and returns two-dimensional array with data. |\n| `node.connect(dbname, ...)` | Returns connection wrapper (`NodeConnection`) capable of running several queries within a single transaction. |\n\nThe last one is the most powerful: you can use `begin(isolation_level)`, `commit()` and `rollback()`:\n```python\nwith node.connect() as con:\n con.begin('serializable')\n print(con.execute('select %s', 1))\n con.rollback()\n```\n\n\n### Logging\n\nBy default, `cleanup()` removes all temporary files (DB files, logs etc) that were created by testgres' API methods.\nIf you'd like to keep logs, execute `configure_testgres(node_cleanup_full=False)` before running any tests.\n\n> Note: context managers (aka `with`) call `stop()` and `cleanup()` automatically.\n\n`testgres` supports [python logging](https://docs.python.org/3.6/library/logging.html),\nwhich means that you can aggregate logs from several nodes into one file:\n\n```python\nimport logging\n\n# write everything to /tmp/testgres.log\nlogging.basicConfig(filename='/tmp/testgres.log')\n\n# enable logging, and create two different nodes\ntestgres.configure_testgres(use_python_logging=True)\nnode1 = testgres.get_new_node().init().start()\nnode2 = testgres.get_new_node().init().start()\n\n# execute a few queries\nnode1.execute('select 1')\nnode2.execute('select 2')\n\n# disable logging\ntestgres.configure_testgres(use_python_logging=False)\n```\n\nLook at `tests/test_simple.py` file for a complete example of the logging\nconfiguration.\n\n\n### Backup & replication\n\nIt's quite easy to create a backup and start a new replica:\n\n```python\nwith testgres.get_new_node('master') as master:\n master.init().start()\n\n # create a backup\n with master.backup() as backup:\n\n # create and start a new replica\n replica = backup.spawn_replica('replica').start()\n\n # catch up with master node\n replica.catchup()\n\n # execute a dummy query\n print(replica.execute('postgres', 'select 1'))\n```\n\n### Benchmarks\n\n`testgres` is also capable of running benchmarks using `pgbench`:\n\n```python\nwith testgres.get_new_node('master') as master:\n # start a new node\n master.init().start()\n\n # initialize default DB and run bench for 10 seconds\n res = master.pgbench_init(scale=2).pgbench_run(time=10)\n print(res)\n```\n\n\n### Custom configuration\n\nIt's often useful to extend default configuration provided by `testgres`.\n\n`testgres` has `default_conf()` function that helps control some basic\noptions. The `append_conf()` function can be used to add custom\nlines to configuration lines:\n\n```python\next_conf = \"shared_preload_libraries = 'postgres_fdw'\"\n\n# initialize a new node\nwith testgres.get_new_node().init() as master:\n\n # ... do something ...\n\t\n # reset main config file\n master.default_conf(fsync=True,\n allow_streaming=True)\n\n # add a new config line\n master.append_conf('postgresql.conf', ext_conf)\n```\n\nNote that `default_conf()` is called by `init()` function; both of them overwrite\nthe configuration file, which means that they should be called before `append_conf()`.\n\n### Remote mode\nTestgres supports the creation of PostgreSQL nodes on a remote host. This is useful when you want to run distributed tests involving multiple nodes spread across different machines.\n\nTo use this feature, you need to use the RemoteOperations class. This feature is only supported with Linux.\nHere is an example of how you might set this up:\n\n```python\nfrom testgres import ConnectionParams, RemoteOperations, TestgresConfig, get_remote_node\n\n# Set up connection params\nconn_params = ConnectionParams(\n host='your_host', # replace with your host\n username='user_name', # replace with your username\n ssh_key='path_to_ssh_key' # replace with your SSH key path\n)\nos_ops = RemoteOperations(conn_params)\n\n# Add remote testgres config before test\nTestgresConfig.set_os_ops(os_ops=os_ops)\n\n# Proceed with your test\ndef test_basic_query(self):\n with get_remote_node(conn_params=conn_params) as node:\n node.init().start()\n res = node.execute('SELECT 1')\n self.assertEqual(res, [(1,)])\n```\n\n## Authors\n\n[Ildar Musin](https://github.com/zilder) \n[Dmitry Ivanov](https://github.com/funbringer) \n[Ildus Kurbangaliev](https://github.com/ildus) \n[Yury Zhuravlev](https://github.com/stalkerg) \n",
"bugtrack_url": null,
"license": "PostgreSQL",
"summary": "Testing utility for PostgreSQL and its extensions",
"version": "1.10.2",
"project_urls": {
"Homepage": "https://github.com/postgrespro/testgres"
},
"split_keywords": [
"test",
" testing",
" postgresql"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "54b0e38931e1b49f4a75a39b0b92ceeccbf002a8aa34da7ad94ffe59f97e053c",
"md5": "506addbf76b6b301350b7109243ba025",
"sha256": "7dd0a6201403e6c0fdb14ad39c889afa0f8ea960163f8f0fbb00ba54e9da0ffd"
},
"downloads": -1,
"filename": "testgres-1.10.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "506addbf76b6b301350b7109243ba025",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 45682,
"upload_time": "2024-09-27T14:46:12",
"upload_time_iso_8601": "2024-09-27T14:46:12.701386Z",
"url": "https://files.pythonhosted.org/packages/54/b0/e38931e1b49f4a75a39b0b92ceeccbf002a8aa34da7ad94ffe59f97e053c/testgres-1.10.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "6555881156fd940f23ca5d69af764bc6e60eff2b77570c9f792473a79deeb278",
"md5": "b6bc7afdc3f3be43d758d3a4cf1eb995",
"sha256": "50cf2a52d4662f162c5f81d628bd5cf1d30cf79ad5cb1b5ddb0ca694ec83d522"
},
"downloads": -1,
"filename": "testgres-1.10.2.tar.gz",
"has_sig": false,
"md5_digest": "b6bc7afdc3f3be43d758d3a4cf1eb995",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 4472342,
"upload_time": "2024-09-27T14:46:16",
"upload_time_iso_8601": "2024-09-27T14:46:16.366319Z",
"url": "https://files.pythonhosted.org/packages/65/55/881156fd940f23ca5d69af764bc6e60eff2b77570c9f792473a79deeb278/testgres-1.10.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-27 14:46:16",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "postgrespro",
"github_project": "testgres",
"travis_ci": true,
"coveralls": true,
"github_actions": false,
"lcname": "testgres"
}
JSON
