.. _readme:
elastic-pii-redacter
====================
Did you find PII (Personally Identifiable Information) in your Elasticsearch
indices that doesn't belong there? This is the tool for you!
The elastic-pii-redacter can help you redact information from even Searchable
Snapshot mounted indices. It works with deeply nested fields, too!
Client Configuration
--------------------
The tool connects using the ``es_client`` Python module.
You can use command-line options, or a YAML configuration file to configure the client connection.
If using a configuration file is desired, the configuration file structure requires
``elasticsearch`` at the root level as follows::
---
elasticsearch:
client:
hosts: https://10.11.12.13:9200
cloud_id:
bearer_auth:
opaque_id:
request_timeout: 60
http_compress:
verify_certs:
ca_certs:
client_cert:
client_key:
ssl_assert_hostname:
ssl_assert_fingerprint:
ssl_version:
other_settings:
master_only:
skip_version_test:
username:
password:
api_key:
id:
api_key:
token:
logging:
loglevel: INFO
logfile: /path/to/file.log
logformat: default
blacklist: []
`REDACTIONS_FILE` Configuration
-------------------------------
::
---
redactions:
- job_name_20230731_redact_hot:
pattern: hot-*
query: {'match': {'message': 'message1'}}
fields: ['message']
message: REDACTED
expected_docs: 1
restore_settings: {'index.routing.allocation.include._tier_preference': 'data_warm,data_hot,data_content'}
- job_name_20230731_redact_cold:
pattern: restored-cold-*
query: {'match': {'nested.key': 'nested19'}}
fields: ['nested.key']
message: REDACTED
expected_docs: 1
restore_settings: {'index.routing.allocation.include._tier_preference': 'data_warm,data_hot,data_content'}
forcemerge:
max_num_segments: 1
- job_name_20230731_redact_frozen:
pattern: partial-frozen-*
query: {'range': {'number': {'gte': 8, 'lte': 11}}}
fields: ['deep.l1.l2.l3']
message: REDACTED
expected_docs: 4
forcemerge:
only_expunge_deletes: True
Raw data
{
"_id": null,
"home_page": null,
"name": "elasticsearch-pii-redacter",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "elasticsearch, index, pii, redact",
"author": null,
"author_email": "Elastic <info@elastic.co>",
"download_url": "https://files.pythonhosted.org/packages/0f/43/0e097735fcff66b42b9d415ca5b81bf4130748c5893edf641511fe870ce6/elasticsearch_pii_redacter-1.10.1.tar.gz",
"platform": null,
"description": ".. _readme:\n\nelastic-pii-redacter\n====================\n\nDid you find PII (Personally Identifiable Information) in your Elasticsearch\nindices that doesn't belong there? This is the tool for you!\n\nThe elastic-pii-redacter can help you redact information from even Searchable\nSnapshot mounted indices. It works with deeply nested fields, too!\n\n\nClient Configuration\n--------------------\n\nThe tool connects using the ``es_client`` Python module.\n\nYou can use command-line options, or a YAML configuration file to configure the client connection.\nIf using a configuration file is desired, the configuration file structure requires\n``elasticsearch`` at the root level as follows::\n\n ---\n elasticsearch:\n client:\n hosts: https://10.11.12.13:9200\n cloud_id:\n bearer_auth:\n opaque_id:\n request_timeout: 60\n http_compress:\n verify_certs:\n ca_certs:\n client_cert:\n client_key:\n ssl_assert_hostname:\n ssl_assert_fingerprint:\n ssl_version:\n other_settings:\n master_only:\n skip_version_test:\n username:\n password:\n api_key:\n id:\n api_key:\n token:\n\n logging:\n loglevel: INFO\n logfile: /path/to/file.log\n logformat: default\n blacklist: []\n\n\n`REDACTIONS_FILE` Configuration\n-------------------------------\n\n::\n\n ---\n redactions:\n - job_name_20230731_redact_hot:\n pattern: hot-*\n query: {'match': {'message': 'message1'}}\n fields: ['message']\n message: REDACTED\n expected_docs: 1\n restore_settings: {'index.routing.allocation.include._tier_preference': 'data_warm,data_hot,data_content'}\n - job_name_20230731_redact_cold:\n pattern: restored-cold-*\n query: {'match': {'nested.key': 'nested19'}}\n fields: ['nested.key']\n message: REDACTED\n expected_docs: 1\n restore_settings: {'index.routing.allocation.include._tier_preference': 'data_warm,data_hot,data_content'}\n forcemerge:\n max_num_segments: 1\n - job_name_20230731_redact_frozen:\n pattern: partial-frozen-*\n query: {'range': {'number': {'gte': 8, 'lte': 11}}}\n fields: ['deep.l1.l2.l3']\n message: REDACTED\n expected_docs: 4\n forcemerge:\n only_expunge_deletes: True\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Redacting field data from your Elasticsearch indices and Searchable Snapshots",
"version": "1.10.1",
"project_urls": {
"Bug Tracker": "https://github.com/elastic/elastic-pii-redacter/issues",
"Homepage": "https://github.com/elastic/elastic-pii-redacter"
},
"split_keywords": [
"elasticsearch",
" index",
" pii",
" redact"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1936c9c7df9934cfd0c0c1183f1cf7d439014d81fb3f489ab46ae2b8b75b34dd",
"md5": "46b3bd27c15b0c6642126898cfc5dbaa",
"sha256": "f8b983f4086bb1aec0a18b5729d581a801744920879ebbfc25d3b569bebd58fc"
},
"downloads": -1,
"filename": "elasticsearch_pii_redacter-1.10.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "46b3bd27c15b0c6642126898cfc5dbaa",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 29479,
"upload_time": "2024-03-26T20:36:35",
"upload_time_iso_8601": "2024-03-26T20:36:35.290343Z",
"url": "https://files.pythonhosted.org/packages/19/36/c9c7df9934cfd0c0c1183f1cf7d439014d81fb3f489ab46ae2b8b75b34dd/elasticsearch_pii_redacter-1.10.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "0f430e097735fcff66b42b9d415ca5b81bf4130748c5893edf641511fe870ce6",
"md5": "0e6dba900a39a47e9dc7b102e4093312",
"sha256": "b2abb2e685d39c5dc53cf9dfa54eb6f5a12d31ca3322efe4be8c2195a21b5dc7"
},
"downloads": -1,
"filename": "elasticsearch_pii_redacter-1.10.1.tar.gz",
"has_sig": false,
"md5_digest": "0e6dba900a39a47e9dc7b102e4093312",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 24628,
"upload_time": "2024-03-26T20:36:37",
"upload_time_iso_8601": "2024-03-26T20:36:37.346921Z",
"url": "https://files.pythonhosted.org/packages/0f/43/0e097735fcff66b42b9d415ca5b81bf4130748c5893edf641511fe870ce6/elasticsearch_pii_redacter-1.10.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-26 20:36:37",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "elastic",
"github_project": "elastic-pii-redacter",
"github_not_found": true,
"lcname": "elasticsearch-pii-redacter"
}
JSON
