# es-fieldusage
[](https://pypi.org/project/es-fieldusage)
[](https://pypi.org/project/es-fieldusage)
-----
**Table of Contents**
- [es-fieldusage](#es-fieldusage)
- [Installation](#installation)
- [Description](#description)
- [Top-level help output](#top-level-help-output)
- [Command: `stdout` help output](#command-stdout-help-output)
- [Command `file` help output](#command-file-help-output)
- [Command `show-indices` help output](#command-show-indices-help-output)
- [Docker usage](#docker-usage)
- [Docker build](#docker-build)
- [Docker run](#docker-run)
- [License](#license)
## Installation
```console
pip install es-fieldusage
```
## Description
Determine which fields are being used, how much, for a given index.
### Top-level help output
```
$ es-fieldusage --help
Usage: es-fieldusage [OPTIONS] COMMAND [ARGS]...
Elasticsearch Index Field Usage Reporting Tool
Sum all field query/request access for one or more indices using the Elastic Field Usage API (https://ela.st/usagestats)
Generate a report at the command-line with the stdout command for all indices in INDEX_PATTERN:
$ es-fieldusage stdout INDEX_PATTERN
To avoid errors, be sure to encapsulate wildcards in single-quotes:
$ es-fieldusage stdout 'index-*'
Options:
--config PATH Path to configuration file.
--hosts TEXT Elasticsearch URL to connect to.
--cloud_id TEXT Elastic Cloud instance id
--api_token TEXT The base64 encoded API Key token
--id TEXT API Key "id" value
--api_key TEXT API Key "api_key" value
--username TEXT Elasticsearch username
--password TEXT Elasticsearch password
--request_timeout FLOAT Request timeout in seconds
--verify_certs / --no-verify_certs
Verify SSL/TLS certificate(s)
--ca_certs TEXT Path to CA certificate file or directory
--client_cert TEXT Path to client certificate file
--client_key TEXT Path to client key file
--loglevel [DEBUG|INFO|WARNING|ERROR|CRITICAL]
Log level
--logfile TEXT Log file
--logformat [default|ecs] Log output format
--blacklist TEXT Named entities will not be logged
-v, --version Show the version and exit.
-h, --help Show this message and exit.
Commands:
show-all-options Show all configuration options
stdout Output field usage information to the console
Learn more at https://github.com/untergeek/es-fieldusage
```
### Command: `stdout` help output
```
$ es-fieldusage stdout --help
Usage: es-fieldusage stdout [OPTIONS] SEARCH_PATTERN
Display field usage information on the console for SEARCH_PATTERN
$ es-fieldusage stdout [OPTIONS] SEARCH_PATTERN
Options:
--show-report / --hide-report Show a summary report [default: show-report]
--show-headers / --hide-headers
Show block headers for un|accessed fields [default: show-headers]
--show-accessed / --hide-accessed
Show accessed fields [default: hide-accessed]
--show-unaccessed / --hide-unaccessed
Show unaccessed fields [default: hide-unaccessed]
--show-counts / --hide-counts Show field access counts [default: hide-counts]
--delimiter TEXT Value delimiter if access counts are shown [default: ,]
-h, --help Show this message and exit.
Learn more at https://github.com/untergeek/es-fieldusage
```
### Command `file` help output
```
$ es-fieldusage file --help
Usage: es-fieldusage file [OPTIONS] SEARCH_PATTERN
Write field usage information to file for SEARCH_PATTERN
$ es-fieldusage file [OPTIONS] SEARCH_PATTERN
Options:
--show-report / --hide-report Show a summary report [default: show-report]
--show-headers / --hide-headers
Show block headers for un|accessed fields [default: show-headers]
--show-accessed / --hide-accessed
Show accessed fields [default: hide-accessed]
--show-unaccessed / --hide-unaccessed
Show unaccessed fields [default: hide-unaccessed]
--show-counts / --hide-counts Show field access counts [default: hide-counts]
--per_index Create one file per index found
--delimiter TEXT Value delimiter if access counts are shown [default: ,]
-h, --help Show this message and exit.
Learn more at https://github.com/untergeek/es-fieldusage
```
### Command `show-indices` help output
```
$ es-fieldusage show-indices --help
Usage: es-fieldusage show-indices SEARCH_PATTERN
Show indices on the console matching SEARCH_PATTERN
$ es-fieldusage show_indices SEARCH_PATTERN
This is included as a way to ensure you are seeing the indices you expect before using the file or stdout commands.
Options:
-h, --help Show this message and exit.
Learn more at https://github.com/untergeek/es-fieldusage
```
## Docker usage
### Docker build
From the path which contains `Dockerfile`:
```
$ ./docker build . -t reponame/es-fieldusage:x.y.z
```
e.g.
```
docker build . -t untergeek/es-fieldusage:1.0.0
```
You can also build for x86_64 and arm64 if you have the appropriate buildx image setup:
```
docker buildx build --platform linux/amd64,linux/arm64 -t untergeek/es-fieldusage:1.0.0 --push .
```
### Docker run
**Example:**
```
docker run -t --rm --name es-fieldusage -v /path/to/configfile/:/.esfieldusage -v $(pwd)/:/fileoutput untergeek/es-fieldusage:1.0.0 --config /.esfieldusage/config.yml show-indices 'index-*'
```
**Explanation:**
* The `-t` flag indicates that you are interacting with a terminal application
* `--rm` deletes the created Docker image after the run. Omitting this will result in a lot of created images that run once. Using `--name` will prevent this collision if `--rm` is omitted by reminding you that there is already an image with the same name.
* `--name` is the name of the image to create. This is optional.
* `-v` sets up volumes. `/path/to/configfile/` is the local file path to where you have a YAML configuration file, if you choose to use one. The `:/.esfieldusage` portion of the volume map is the directory where that configuration file is expected in the Docker image. The second volume mapping is `$(pwd):/fileoutput`. `/fileoutput` is the dedicated file path on the Docker image where output from the [file](#command-file-help-output) command will be written. By using `$(pwd)` it will map your present working directory so the files will appear there. Otherwise you can map another path here.
* `untergeek/es-fieldusage:1.0.0` is the `repository/image:version` to run.
* `--config /.esfieldusage/config.yml`, as stated previously, if you intend to use a YAML configuration file, the path needs to be mapped as a volume, and then accessed this way. The filename should match whatever you actually have, and not necessarily `config.yml`
* `show-indices 'index-*'` Everything after here is available as regular options and commands for es-fieldusage.
## License
`es-fieldusage` is distributed under the terms of the [Apache 2.0](http://www.apache.org/licenses/LICENSE-2.0) license.
Raw data
{
"_id": null,
"home_page": null,
"name": "es-fieldusage",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "elasticsearch, field, index, stats, usage",
"author": null,
"author_email": "Elastic <info@elastic.co>",
"download_url": "https://files.pythonhosted.org/packages/50/17/18be5f21ef794ad7d7acf72511f50d82d62415f07a3719a50f889f107665/es_fieldusage-1.1.2.tar.gz",
"platform": null,
"description": "# es-fieldusage\n\n[](https://pypi.org/project/es-fieldusage)\n[](https://pypi.org/project/es-fieldusage)\n\n-----\n\n**Table of Contents**\n\n- [es-fieldusage](#es-fieldusage)\n - [Installation](#installation)\n - [Description](#description)\n - [Top-level help output](#top-level-help-output)\n - [Command: `stdout` help output](#command-stdout-help-output)\n - [Command `file` help output](#command-file-help-output)\n - [Command `show-indices` help output](#command-show-indices-help-output)\n - [Docker usage](#docker-usage)\n - [Docker build](#docker-build)\n - [Docker run](#docker-run)\n - [License](#license)\n\n## Installation\n\n```console\npip install es-fieldusage\n```\n\n## Description\n\nDetermine which fields are being used, how much, for a given index.\n\n### Top-level help output\n```\n$ es-fieldusage --help\n\nUsage: es-fieldusage [OPTIONS] COMMAND [ARGS]...\n\n Elasticsearch Index Field Usage Reporting Tool\n\n Sum all field query/request access for one or more indices using the Elastic Field Usage API (https://ela.st/usagestats)\n\n Generate a report at the command-line with the stdout command for all indices in INDEX_PATTERN:\n\n $ es-fieldusage stdout INDEX_PATTERN\n\n To avoid errors, be sure to encapsulate wildcards in single-quotes:\n\n $ es-fieldusage stdout 'index-*'\n\nOptions:\n --config PATH Path to configuration file.\n --hosts TEXT Elasticsearch URL to connect to.\n --cloud_id TEXT Elastic Cloud instance id\n --api_token TEXT The base64 encoded API Key token\n --id TEXT API Key \"id\" value\n --api_key TEXT API Key \"api_key\" value\n --username TEXT Elasticsearch username\n --password TEXT Elasticsearch password\n --request_timeout FLOAT Request timeout in seconds\n --verify_certs / --no-verify_certs\n Verify SSL/TLS certificate(s)\n --ca_certs TEXT Path to CA certificate file or directory\n --client_cert TEXT Path to client certificate file\n --client_key TEXT Path to client key file\n --loglevel [DEBUG|INFO|WARNING|ERROR|CRITICAL]\n Log level\n --logfile TEXT Log file\n --logformat [default|ecs] Log output format\n --blacklist TEXT Named entities will not be logged\n -v, --version Show the version and exit.\n -h, --help Show this message and exit.\n\nCommands:\n show-all-options Show all configuration options\n stdout Output field usage information to the console\n\n Learn more at https://github.com/untergeek/es-fieldusage\n```\n\n### Command: `stdout` help output\n\n```\n$ es-fieldusage stdout --help\nUsage: es-fieldusage stdout [OPTIONS] SEARCH_PATTERN\n\n Display field usage information on the console for SEARCH_PATTERN\n\n $ es-fieldusage stdout [OPTIONS] SEARCH_PATTERN\n\nOptions:\n --show-report / --hide-report Show a summary report [default: show-report]\n --show-headers / --hide-headers\n Show block headers for un|accessed fields [default: show-headers]\n --show-accessed / --hide-accessed\n Show accessed fields [default: hide-accessed]\n --show-unaccessed / --hide-unaccessed\n Show unaccessed fields [default: hide-unaccessed]\n --show-counts / --hide-counts Show field access counts [default: hide-counts]\n --delimiter TEXT Value delimiter if access counts are shown [default: ,]\n -h, --help Show this message and exit.\n\n Learn more at https://github.com/untergeek/es-fieldusage\n ```\n\n### Command `file` help output\n\n```\n$ es-fieldusage file --help\nUsage: es-fieldusage file [OPTIONS] SEARCH_PATTERN\n\n Write field usage information to file for SEARCH_PATTERN\n\n $ es-fieldusage file [OPTIONS] SEARCH_PATTERN\n\nOptions:\n --show-report / --hide-report Show a summary report [default: show-report]\n --show-headers / --hide-headers\n Show block headers for un|accessed fields [default: show-headers]\n --show-accessed / --hide-accessed\n Show accessed fields [default: hide-accessed]\n --show-unaccessed / --hide-unaccessed\n Show unaccessed fields [default: hide-unaccessed]\n --show-counts / --hide-counts Show field access counts [default: hide-counts]\n --per_index Create one file per index found\n --delimiter TEXT Value delimiter if access counts are shown [default: ,]\n -h, --help Show this message and exit.\n\n Learn more at https://github.com/untergeek/es-fieldusage\n```\n\n### Command `show-indices` help output\n\n```\n$ es-fieldusage show-indices --help\nUsage: es-fieldusage show-indices SEARCH_PATTERN\n\n Show indices on the console matching SEARCH_PATTERN\n\n $ es-fieldusage show_indices SEARCH_PATTERN\n\n This is included as a way to ensure you are seeing the indices you expect before using the file or stdout commands.\n\nOptions:\n -h, --help Show this message and exit.\n\n Learn more at https://github.com/untergeek/es-fieldusage\n```\n\n## Docker usage\n\n### Docker build\n\nFrom the path which contains `Dockerfile`:\n\n```\n$ ./docker build . -t reponame/es-fieldusage:x.y.z\n```\n\ne.g.\n\n```\ndocker build . -t untergeek/es-fieldusage:1.0.0\n```\n\nYou can also build for x86_64 and arm64 if you have the appropriate buildx image setup:\n\n```\ndocker buildx build --platform linux/amd64,linux/arm64 -t untergeek/es-fieldusage:1.0.0 --push .\n```\n\n### Docker run\n\n**Example:**\n\n```\ndocker run -t --rm --name es-fieldusage -v /path/to/configfile/:/.esfieldusage -v $(pwd)/:/fileoutput untergeek/es-fieldusage:1.0.0 --config /.esfieldusage/config.yml show-indices 'index-*'\n```\n\n**Explanation:**\n\n * The `-t` flag indicates that you are interacting with a terminal application\n * `--rm` deletes the created Docker image after the run. Omitting this will result in a lot of created images that run once. Using `--name` will prevent this collision if `--rm` is omitted by reminding you that there is already an image with the same name.\n * `--name` is the name of the image to create. This is optional.\n * `-v` sets up volumes. `/path/to/configfile/` is the local file path to where you have a YAML configuration file, if you choose to use one. The `:/.esfieldusage` portion of the volume map is the directory where that configuration file is expected in the Docker image. The second volume mapping is `$(pwd):/fileoutput`. `/fileoutput` is the dedicated file path on the Docker image where output from the [file](#command-file-help-output) command will be written. By using `$(pwd)` it will map your present working directory so the files will appear there. Otherwise you can map another path here.\n * `untergeek/es-fieldusage:1.0.0` is the `repository/image:version` to run.\n * `--config /.esfieldusage/config.yml`, as stated previously, if you intend to use a YAML configuration file, the path needs to be mapped as a volume, and then accessed this way. The filename should match whatever you actually have, and not necessarily `config.yml`\n * `show-indices 'index-*'` Everything after here is available as regular options and commands for es-fieldusage.\n\n## License\n\n`es-fieldusage` is distributed under the terms of the [Apache 2.0](http://www.apache.org/licenses/LICENSE-2.0) license.\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Helping you determine field usage by index",
"version": "1.1.2",
"project_urls": {
"Documentation": "https://github.com/untergeek/es-fieldusage#readme",
"Issues": "https://github.com/untergeek/es-fieldusage/issues",
"Source": "https://github.com/untergeek/es-fieldusage"
},
"split_keywords": [
"elasticsearch",
" field",
" index",
" stats",
" usage"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "82cf14d1b38921288d018429d6daa0f2c7ca7ce1d6a50cd6de428a1d98b8793e",
"md5": "7e176c69f9c8baf824dde200ed283304",
"sha256": "3f1078dcd4a6348335e073fa2b8d41446fb2ea7f477995205dbb33b85bb83758"
},
"downloads": -1,
"filename": "es_fieldusage-1.1.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7e176c69f9c8baf824dde200ed283304",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 14125,
"upload_time": "2024-08-26T18:07:28",
"upload_time_iso_8601": "2024-08-26T18:07:28.806178Z",
"url": "https://files.pythonhosted.org/packages/82/cf/14d1b38921288d018429d6daa0f2c7ca7ce1d6a50cd6de428a1d98b8793e/es_fieldusage-1.1.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "501718be5f21ef794ad7d7acf72511f50d82d62415f07a3719a50f889f107665",
"md5": "45e7de3e3a392483532199bd8d5bf0d5",
"sha256": "834812102231b52e4d94fb6620c7605678fdd5060b1edad2f0ef1e9b7d4329b6"
},
"downloads": -1,
"filename": "es_fieldusage-1.1.2.tar.gz",
"has_sig": false,
"md5_digest": "45e7de3e3a392483532199bd8d5bf0d5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 14386,
"upload_time": "2024-08-26T18:07:29",
"upload_time_iso_8601": "2024-08-26T18:07:29.829650Z",
"url": "https://files.pythonhosted.org/packages/50/17/18be5f21ef794ad7d7acf72511f50d82d62415f07a3719a50f889f107665/es_fieldusage-1.1.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-26 18:07:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "untergeek",
"github_project": "es-fieldusage#readme",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "es-fieldusage"
}
JSON
