| Name | pg-stage JSON |
| Version |
0.3.3
JSON |
| download |
| home_page | None |
| Summary | Anonymization of data in pg_dump |
| upload_time | 2024-03-21 06:23:06 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.7 |
| license | None |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# pg_stage
A utility for generating a database dump, the data in which will be obfuscated. This dump can be used in development and
stage servers without fear of their theft.
## Content
- [How does it work?](#how-does-it-work)
- [Usage example](#usage-example)
- [Supported types of obfuscation](#supported-types-of-obfuscation)
- [Why did I write my utility?](#why-did-i-write-my-utility)
- [Thanks for the inspiration](#thanks-for-the-inspiration)
## How does it work?
The utility processes the output of the pg_dump command line by line and decides whether to obfuscate data at the level
of comments to a table or column.
## Usage example
1. You need to create a file with approximately the following contents:
```python
# main.py
from pg_stage.obfuscator import Obfuscator
obfuscator = Obfuscator(locale='ru_RU')
obfuscator.run()
```
2. Add comments to a column or table:
```sql
COMMENT ON COLUMN table_1.first_name IS 'anon: [{"mutation_name": "first_name"}]';
```
3. Run pg_dump and redirect the stream to the running script process:
```bash
pg_dump -d database | python3 main.py > dump.sql
```
4. After that you will get the obfuscated data in the table
## Supported types of obfuscation
You can see the current list [here](https://github.com/froOzzy/pg_stage/blob/main/src/pg_stage/mutator.py).
## Why did I write my utility?
I also adhere to the rule that you do not need to place third-party plugins in the working database for its security
(most utilities are in the form of database extensions).
Also, in similar utilities, I could not find the functionality for uniform obfuscation of data in related tables.
This prompted me to write my own utility that will be able to obfuscate data in related tables with the same result
by a foreign key.
Example:
```sql
COMMENT ON COLUMN table_1.first_name IS 'anon: [{"mutation_name": "first_name", "relations": [{"table_name": "table_1", "column_name": "last_name", "from_column_name": "id", "to_column_name": "id"}]}]';
```
where `relations` - links on tables where it is necessary to obfuscate fields according to the current field.
## Thanks for the inspiration
- [triki](https://github.com/josacar/triki)
- [fake_pipe](https://github.com/ddrscott/fake_pipe)
Raw data
{
"_id": null,
"home_page": null,
"name": "pg-stage",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": null,
"author": null,
"author_email": "froOzzy <ieme5gll6u78@mail.ru>",
"download_url": "https://files.pythonhosted.org/packages/25/1d/4133145d69b01f186d882b5b097ef26316af434fd47b4d5db886e627111f/pg_stage-0.3.3.tar.gz",
"platform": null,
"description": "# pg_stage\n\nA utility for generating a database dump, the data in which will be obfuscated. This dump can be used in development and \nstage servers without fear of their theft.\n\n## Content\n\n- [How does it work?](#how-does-it-work)\n- [Usage example](#usage-example)\n- [Supported types of obfuscation](#supported-types-of-obfuscation)\n- [Why did I write my utility?](#why-did-i-write-my-utility)\n- [Thanks for the inspiration](#thanks-for-the-inspiration)\n\n## How does it work?\n\nThe utility processes the output of the pg_dump command line by line and decides whether to obfuscate data at the level \nof comments to a table or column.\n\n## Usage example\n\n1. You need to create a file with approximately the following contents:\n\n```python\n# main.py\nfrom pg_stage.obfuscator import Obfuscator\n\n\nobfuscator = Obfuscator(locale='ru_RU')\nobfuscator.run()\n```\n\n2. Add comments to a column or table:\n\n```sql\nCOMMENT ON COLUMN table_1.first_name IS 'anon: [{\"mutation_name\": \"first_name\"}]';\n```\n\n3. Run pg_dump and redirect the stream to the running script process:\n\n```bash\npg_dump -d database | python3 main.py > dump.sql\n```\n\n4. After that you will get the obfuscated data in the table\n\n## Supported types of obfuscation\n\nYou can see the current list [here](https://github.com/froOzzy/pg_stage/blob/main/src/pg_stage/mutator.py).\n\n## Why did I write my utility?\n\nI also adhere to the rule that you do not need to place third-party plugins in the working database for its security \n(most utilities are in the form of database extensions).\n\nAlso, in similar utilities, I could not find the functionality for uniform obfuscation of data in related tables. \nThis prompted me to write my own utility that will be able to obfuscate data in related tables with the same result \nby a foreign key.\n\nExample:\n\n```sql\nCOMMENT ON COLUMN table_1.first_name IS 'anon: [{\"mutation_name\": \"first_name\", \"relations\": [{\"table_name\": \"table_1\", \"column_name\": \"last_name\", \"from_column_name\": \"id\", \"to_column_name\": \"id\"}]}]';\n```\n\nwhere `relations` - links on tables where it is necessary to obfuscate fields according to the current field.\n\n## Thanks for the inspiration\n\n- [triki](https://github.com/josacar/triki)\n- [fake_pipe](https://github.com/ddrscott/fake_pipe)\n",
"bugtrack_url": null,
"license": null,
"summary": "Anonymization of data in pg_dump",
"version": "0.3.3",
"project_urls": {
"Homepage": "https://github.com/froOzzy/pg_stage"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6cad3952574ae976559e1d0845deccd95edb197d41721a103f7d48ead319992b",
"md5": "fc134a6b571af9d977d3fa219e6d96f5",
"sha256": "7e3af1be0795c76dd3757d87d74029b84f6cf4866ee8009ca2496bc8807de4de"
},
"downloads": -1,
"filename": "pg_stage-0.3.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "fc134a6b571af9d977d3fa219e6d96f5",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 9951,
"upload_time": "2024-03-21T06:23:04",
"upload_time_iso_8601": "2024-03-21T06:23:04.481193Z",
"url": "https://files.pythonhosted.org/packages/6c/ad/3952574ae976559e1d0845deccd95edb197d41721a103f7d48ead319992b/pg_stage-0.3.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "251d4133145d69b01f186d882b5b097ef26316af434fd47b4d5db886e627111f",
"md5": "50661720ffda62519b965913e426a7f1",
"sha256": "e02e58ed81977ae323497ec185b9da2097cd65722de6f9b943e26c2ceff2418e"
},
"downloads": -1,
"filename": "pg_stage-0.3.3.tar.gz",
"has_sig": false,
"md5_digest": "50661720ffda62519b965913e426a7f1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 16274,
"upload_time": "2024-03-21T06:23:06",
"upload_time_iso_8601": "2024-03-21T06:23:06.702398Z",
"url": "https://files.pythonhosted.org/packages/25/1d/4133145d69b01f186d882b5b097ef26316af434fd47b4d5db886e627111f/pg_stage-0.3.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-21 06:23:06",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "froOzzy",
"github_project": "pg_stage",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "pg-stage"
}
