esrt


Nameesrt JSON
Version 3.3.0 PyPI version JSON
download
home_pageNone
SummaryNone
upload_time2024-08-12 04:50:19
maintainerNone
docs_urlNone
authorNone
requires_python>=3.9
licenseNone
keywords
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # esrt - Elasticsearch Request Tool

[![pypi](https://img.shields.io/pypi/v/esrt.svg)](https://pypi.python.org/pypi/esrt)

```sh
pip install pipx
pipx install esrt -f
esrt --install-completion  # Install completion for the current shell.
```

## Commands

- `e`: search
- `s`: scan / scroll
- `r`: request / api / a
- `t`: transmit / bulk / b
- `sql`: sql / query / q

---

## Example

You can start an es service with docker.

```sh
esrt_es_name="esrt-es"
docker run --name $esrt_es_name --rm -itd --platform=linux/amd64 -p 9200:9200 elasticsearch:5.6.9-alpine

# install sql command and restart container:
docker exec $esrt_es_name elasticsearch-plugin install https://github.com/NLPchina/elasticsearch-sql/releases/download/5.6.9.0/elasticsearch-sql-5.6.9.0.zip
docker restart $esrt_es_name
```

---

## `r` - Send a request

Check server:

```sh
esrt r localhost -X HEAD
# ->
# true
```

Create a index:

```sh
esrt r localhost -X PUT /my-index
# ->
# {"acknowledged": true, "shards_acknowledged": true, "index": "my-index"}
```

*If you want to `esrt` quote url path for you, add flag: `-Q`(`--quote-url`)*

Cat it:

```sh
esrt r localhost -X GET _cat/indices -p 'v&format=json' -p 's=index'
# ->
# [{"health": "yellow", "status": "open", "index": "my-index", "uuid": "avrhX1hzQLyfEGvXsA96NA", "pri": "5", "rep": "1", "docs.count": "0", "docs.deleted": "0", "store.size": "324b", "pri.store.size": "324b"}]
```

*`esrt` doesn't keep `-p pretty` format, but you can use `jq`.*

```sh
esrt r localhost -X GET _cat/indices -p 'v&format=json' -p 's=index' | jq
# ->
# [
#   {
#     "health": "yellow",
#     "status": "open",
#     "index": "my-index",
#     "uuid": "avrhX1hzQLyfEGvXsA96NA",
#     "pri": "5",
#     "rep": "1",
#     "docs.count": "0",
#     "docs.deleted": "0",
#     "store.size": "810b",
#     "pri.store.size": "810b"
#   }
# ]
```

---

## `t` - Transmit data (`streaming_bulk`)

Bulk with data from file `examples/bulk.ndjson`:

```json
{ "_op_type": "index",  "_index": "my-index", "_type": "type1", "_id": "1", "field1": "ii" }
{ "_op_type": "delete", "_index": "my-index", "_type": "type1", "_id": "1" }
{ "_op_type": "create", "_index": "my-index", "_type": "type1", "_id": "1", "field1": "cc" }
{ "_op_type": "update", "_index": "my-index", "_type": "type1", "_id": "1", "doc": {"field2": "uu"} }
```

```sh
esrt t localhost -f examples/bulk.ndjson
# ->
# <Client([{'host': 'localhost', 'port': 9200}])>
# streaming_bulk  [####################################]  4

# success = 4
# failed = 0
```

---

Read payload from `stdin`. And `-d` can be omitted.

```sh
esrt t localhost <<EOF
{ "_op_type": "index",  "_index": "my-index-2", "_type": "type1", "_id": "1", "field1": "11" }
{ "_op_type": "index",  "_index": "my-index-2", "_type": "type1", "_id": "2", "field1": "22" }
{ "_op_type": "index",  "_index": "my-index-2", "_type": "type1", "_id": "3", "field1": "33" }
EOF
# ->
# <Client([{'host': 'localhost', 'port': 9200}])>
# streaming_bulk  [####################################]  3

# success = 3
# failed = 0
```

Piping `heredoc` also works.

```sh
cat <<EOF | esrt t localhost
{ "_op_type": "index",  "_index": "my-index-2", "_type": "type1", "_id": "1", "field1": "11" }
{ "_op_type": "index",  "_index": "my-index-2", "_type": "type1", "_id": "2", "field1": "22" }
{ "_op_type": "index",  "_index": "my-index-2", "_type": "type1", "_id": "3", "field1": "33" }
EOF
```

---

Pipe `_search` result and update `_index` with `customized handler` to do more operations before bulk!

```sh
alias jq_es_hits="jq '.hits.hits.[]'"
```

```sh
esrt r localhost -X GET /my-index-2/_search | jq_es_hits -c | esrt t localhost -w examples.my-handlers:MyHandler  # <- `examples/my-handlers.py`
# ->
# <Client([{'host': 'localhost', 'port': 9200}])>
# streaming_bulk  [####################################]  3

# success = 3
# failed = 0
```

```py
# examples/my-handlers.py
import json
import typing as t

from esrt import DocHandler


# function style
def my_handler(actions: t.Iterable[str]):
    for action in actions:
        obj = json.loads(action)
        prefix = 'new-'
        if not t.cast(str, obj['_index']).startswith(prefix):
            obj['_index'] = prefix + obj['_index']
        yield obj


# class style
class MyHandler(DocHandler):
    def handle(self, actions: t.Iterable[str]):
        for action in actions:
            yield self.handle_one(action)

    def handle_one(self, action: str):
        obj = json.loads(action)
        prefix = 'new-'
        if not t.cast(str, obj['_index']).startswith(prefix):
            obj['_index'] = prefix + obj['_index']
        return obj
```

---

## `e` Search docs

```sh
esrt e localhost | jq_es_hits -c
# ->
# {"_index":"my-index-2","_type":"type1","_id":"2","_score":1.0,"_source":{"field1":"22"}}
# {"_index":"new-my-index-2","_type":"type1","_id":"2","_score":1.0,"_source":{"field1":"22"}}
# {"_index":"my-index","_type":"type1","_id":"1","_score":1.0,"_source":{"field1":"cc","field2":"uu"}}
# {"_index":"my-index-2","_type":"type1","_id":"1","_score":1.0,"_source":{"field1":"11"}}
# {"_index":"new-my-index-2","_type":"type1","_id":"1","_score":1.0,"_source":{"field1":"11"}}
# {"_index":"my-index-2","_type":"type1","_id":"3","_score":1.0,"_source":{"field1":"33"}}
# {"_index":"new-my-index-2","_type":"type1","_id":"3","_score":1.0,"_source":{"field1":"33"}}
```

```sh
esrt e localhost -f - <<EOF | jq_es_hits -c
{"query": {"term": {"_index": "new-my-index-2"}}}
EOF
# ->
# {"_index":"new-my-index-2","_type":"type1","_id":"2","_score":1.0,"_source":{"field1":"22"}}
# {"_index":"new-my-index-2","_type":"type1","_id":"1","_score":1.0,"_source":{"field1":"11"}}
# {"_index":"new-my-index-2","_type":"type1","_id":"3","_score":1.0,"_source":{"field1":"33"}}
```

## `s` - Search and `Scroll`

```sh
esrt s localhost
# ->
# total = 7
# {"_index": "my-index-2", "_type": "type1", "_id": "2", "_score": null, "_source": {"field1": "22"}, "sort": [0]}
# {"_index": "new-my-index-2", "_type": "type1", "_id": "2", "_score": null, "_source": {"field1": "22"}, "sort": [0]}
# {"_index": "my-index", "_type": "type1", "_id": "1", "_score": null, "_source": {"field1": "cc", "field2": "uu"}, "sort": [0]}
# {"_index": "my-index-2", "_type": "type1", "_id": "1", "_score": null, "_source": {"field1": "11"}, "sort": [0]}
# {"_index": "new-my-index-2", "_type": "type1", "_id": "1", "_score": null, "_source": {"field1": "11"}, "sort": [0]}
# {"_index": "my-index-2", "_type": "type1", "_id": "3", "_score": null, "_source": {"field1": "33"}, "sort": [0]}
# {"_index": "new-my-index-2", "_type": "type1", "_id": "3", "_score": null, "_source": {"field1": "33"}, "sort": [0]}
```

```sh
esrt s localhost -f - <<EOF
{"query": {"term": {"field1": "cc"}}}
EOF
# ->
# total = 1
# {"_index": "my-index", "_type": "type1", "_id": "1", "_score": null, "_source": {"field1": "cc", "field2": "uu"}, "sort": [0]}
```

## `sql` - Elasticsearch SQL

```sh
# Elasticsearch v6
export ESRT_SQL_API=_xpack/sql
```

```sh
esrt sql localhost -f - <<EOF | jq_es_hits -c
SELECT * from new-my-index-2
EOF
# ->
# {"_index":"new-my-index-2","_type":"type1","_id":"2","_score":1.0,"_source":{"field1":"22"}}
# {"_index":"new-my-index-2","_type":"type1","_id":"1","_score":1.0,"_source":{"field1":"11"}}
# {"_index":"new-my-index-2","_type":"type1","_id":"3","_score":1.0,"_source":{"field1":"33"}}
```

---

## Other Examples

```py
# examples/create-massive-docs.py
import json
import uuid


if __name__ == '__main__':
    for i, _ in enumerate(range(654321), start=1):
        d = {
            '_index': 'my-index-a',
            '_id': i,
            '_type': 'type1',
            '_source': {'field1': str(uuid.uuid4())},
        }
        print(json.dumps(d))
```

```sh
python examples/create-massive-docs.py | tee -a _.ndjson | esrt t localhost -c 10000
# ->
# <Client([{'host': 'localhost', 'port': 9200}])>
# streaming_bulk  [####################################]  654321

# success = 654321
# failed = 0

cat _.ndjson  # <- 79M
# ->
# {"_index": "my-index-a", "_id": 1, "_type": "type1", "_source": {"field1": "7e6a3924-1258-4e44-a19b-15395e802b1b"}}
# {"_index": "my-index-a", "_id": 2, "_type": "type1", "_source": {"field1": "9a05ea11-349b-452f-b771-a1aa168bdca9"}}
# {"_index": "my-index-a", "_id": 3, "_type": "type1", "_source": {"field1": "2e4d2d6a-54e3-4160-adbb-d0c52759bb89"}}
# {"_index": "my-index-a", "_id": 4, "_type": "type1", "_source": {"field1": "72cbc979-ed03-4653-8bb6-9f3dd723a2c8"}}
# {"_index": "my-index-a", "_id": 5, "_type": "type1", "_source": {"field1": "61a0acce-e415-4ac7-8417-66ccfe0f7932"}}
# {"_index": "my-index-a", "_id": 6, "_type": "type1", "_source": {"field1": "ba84e4b9-881c-4042-bf39-a449766f9e4b"}}
# {"_index": "my-index-a", "_id": 7, "_type": "type1", "_source": {"field1": "e92b2d83-97ae-4d5e-9797-b9ade4841f87"}}
# {"_index": "my-index-a", "_id": 8, "_type": "type1", "_source": {"field1": "c36acdb2-ea4e-4716-ad16-166171fa181d"}}
# {"_index": "my-index-a", "_id": 9, "_type": "type1", "_source": {"field1": "f17d588b-cbd0-4f72-8a47-040eb1203e35"}}
# {"_index": "my-index-a", "_id": 10, "_type": "type1", "_source": {"field1": "ac5d00fd-6443-4380-8d1b-72595d3f890c"}}
# {"_index": "my-index-a", "_id": 11, "_type": "type1", "_source": {"field1": "5d997ac9-e2c0-4347-9415-9d981f40f856"}}
# {"_index": "my-index-a", "_id": 12, "_type": "type1", "_source": {"field1": "7cf7ef75-9d95-4736-851b-5099dd11d1d6"}}
# {"_index": "my-index-a", "_id": 13, "_type": "type1", "_source": {"field1": "a7db50d4-65da-499f-84d2-5e27f719b3a7"}}
# {"_index": "my-index-a", "_id": 14, "_type": "type1", "_source": {"field1": "fd48cc37-520c-41be-a3e4-6e242bf91fed"}}
# {"_index": "my-index-a", "_id": 15, "_type": "type1", "_source": {"field1": "767286bb-5590-4265-b6f5-ce789f5f2848"}}
# {"_index": "my-index-a", "_id": 16, "_type": "type1", "_source": {"field1": "eca18f61-8189-46bc-b455-520a5c0a26d3"}}
# {"_index": "my-index-a", "_id": 17, "_type": "type1", "_source": {"field1": "61508630-c2b2-4f93-a91b-056c69208c34"}}
# {"_index": "my-index-a", "_id": 18, "_type": "type1", "_source": {"field1": "c6f1df60-9652-4102-98f8-df3c3e5d966b"}}
# {"_index": "my-index-a", "_id": 19, "_type": "type1", "_source": {"field1": "63c85746-baea-4fc5-a3c1-2638c2d3b9ed"}}
# {"_index": "my-index-a", "_id": 20, "_type": "type1", "_source": {"field1": "bd47ee1f-198c-4da8-8c3d-458814d547b9"}}
# ......
```

---

```py
# examples/copy-more-docs.py
from copy import deepcopy
import json
import typing as t
import uuid


if __name__ == '__main__':
    for i, _ in enumerate(range(54321), start=1):
        d = {
            '_index': 'my-index-b',
            '_id': i,
            '_type': 'type1',
            '_source': {'field1': str(uuid.uuid4())},
        }
        print(json.dumps(d))


def handle(actions: t.Iterable[str]):
    for action in actions:
        d: dict[str, t.Any] = json.loads(action)
        yield d
        d2 = deepcopy(d)
        d2['_source']['field1'] += '!!!'
        d2['_source']['field2'] = str(uuid.uuid4())
        yield d2
```

```sh
python examples/copy-more-docs.py | esrt t localhost -w examples.copy-more-docs:handle
# ->
# <Client([{'host': 'localhost', 'port': 9200}])>
# streaming_bulk  [####################################]  108642

# success = 108642
# failed = 0
```

            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "esrt",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.9",
    "maintainer_email": null,
    "keywords": null,
    "author": null,
    "author_email": null,
    "download_url": "https://files.pythonhosted.org/packages/77/e2/37a236b89fa1f1e868ba731bd9f481a51cdbf67f0d9d7cb67d484f916feb/esrt-3.3.0.tar.gz",
    "platform": null,
    "description": "# esrt - Elasticsearch Request Tool\n\n[![pypi](https://img.shields.io/pypi/v/esrt.svg)](https://pypi.python.org/pypi/esrt)\n\n```sh\npip install pipx\npipx install esrt -f\nesrt --install-completion  # Install completion for the current shell.\n```\n\n## Commands\n\n- `e`: search\n- `s`: scan / scroll\n- `r`: request / api / a\n- `t`: transmit / bulk / b\n- `sql`: sql / query / q\n\n---\n\n## Example\n\nYou can start an es service with docker.\n\n```sh\nesrt_es_name=\"esrt-es\"\ndocker run --name $esrt_es_name --rm -itd --platform=linux/amd64 -p 9200:9200 elasticsearch:5.6.9-alpine\n\n# install sql command and restart container:\ndocker exec $esrt_es_name elasticsearch-plugin install https://github.com/NLPchina/elasticsearch-sql/releases/download/5.6.9.0/elasticsearch-sql-5.6.9.0.zip\ndocker restart $esrt_es_name\n```\n\n---\n\n## `r` - Send a request\n\nCheck server:\n\n```sh\nesrt r localhost -X HEAD\n# ->\n# true\n```\n\nCreate a index:\n\n```sh\nesrt r localhost -X PUT /my-index\n# ->\n# {\"acknowledged\": true, \"shards_acknowledged\": true, \"index\": \"my-index\"}\n```\n\n*If you want to `esrt` quote url path for you, add flag: `-Q`(`--quote-url`)*\n\nCat it:\n\n```sh\nesrt r localhost -X GET _cat/indices -p 'v&format=json' -p 's=index'\n# ->\n# [{\"health\": \"yellow\", \"status\": \"open\", \"index\": \"my-index\", \"uuid\": \"avrhX1hzQLyfEGvXsA96NA\", \"pri\": \"5\", \"rep\": \"1\", \"docs.count\": \"0\", \"docs.deleted\": \"0\", \"store.size\": \"324b\", \"pri.store.size\": \"324b\"}]\n```\n\n*`esrt` doesn't keep `-p pretty` format, but you can use `jq`.*\n\n```sh\nesrt r localhost -X GET _cat/indices -p 'v&format=json' -p 's=index' | jq\n# ->\n# [\n#   {\n#     \"health\": \"yellow\",\n#     \"status\": \"open\",\n#     \"index\": \"my-index\",\n#     \"uuid\": \"avrhX1hzQLyfEGvXsA96NA\",\n#     \"pri\": \"5\",\n#     \"rep\": \"1\",\n#     \"docs.count\": \"0\",\n#     \"docs.deleted\": \"0\",\n#     \"store.size\": \"810b\",\n#     \"pri.store.size\": \"810b\"\n#   }\n# ]\n```\n\n---\n\n## `t` - Transmit data (`streaming_bulk`)\n\nBulk with data from file `examples/bulk.ndjson`:\n\n```json\n{ \"_op_type\": \"index\",  \"_index\": \"my-index\", \"_type\": \"type1\", \"_id\": \"1\", \"field1\": \"ii\" }\n{ \"_op_type\": \"delete\", \"_index\": \"my-index\", \"_type\": \"type1\", \"_id\": \"1\" }\n{ \"_op_type\": \"create\", \"_index\": \"my-index\", \"_type\": \"type1\", \"_id\": \"1\", \"field1\": \"cc\" }\n{ \"_op_type\": \"update\", \"_index\": \"my-index\", \"_type\": \"type1\", \"_id\": \"1\", \"doc\": {\"field2\": \"uu\"} }\n```\n\n```sh\nesrt t localhost -f examples/bulk.ndjson\n# ->\n# <Client([{'host': 'localhost', 'port': 9200}])>\n# streaming_bulk  [####################################]  4\n\n# success = 4\n# failed = 0\n```\n\n---\n\nRead payload from `stdin`. And `-d` can be omitted.\n\n```sh\nesrt t localhost <<EOF\n{ \"_op_type\": \"index\",  \"_index\": \"my-index-2\", \"_type\": \"type1\", \"_id\": \"1\", \"field1\": \"11\" }\n{ \"_op_type\": \"index\",  \"_index\": \"my-index-2\", \"_type\": \"type1\", \"_id\": \"2\", \"field1\": \"22\" }\n{ \"_op_type\": \"index\",  \"_index\": \"my-index-2\", \"_type\": \"type1\", \"_id\": \"3\", \"field1\": \"33\" }\nEOF\n# ->\n# <Client([{'host': 'localhost', 'port': 9200}])>\n# streaming_bulk  [####################################]  3\n\n# success = 3\n# failed = 0\n```\n\nPiping `heredoc` also works.\n\n```sh\ncat <<EOF | esrt t localhost\n{ \"_op_type\": \"index\",  \"_index\": \"my-index-2\", \"_type\": \"type1\", \"_id\": \"1\", \"field1\": \"11\" }\n{ \"_op_type\": \"index\",  \"_index\": \"my-index-2\", \"_type\": \"type1\", \"_id\": \"2\", \"field1\": \"22\" }\n{ \"_op_type\": \"index\",  \"_index\": \"my-index-2\", \"_type\": \"type1\", \"_id\": \"3\", \"field1\": \"33\" }\nEOF\n```\n\n---\n\nPipe `_search` result and update `_index` with `customized handler` to do more operations before bulk!\n\n```sh\nalias jq_es_hits=\"jq '.hits.hits.[]'\"\n```\n\n```sh\nesrt r localhost -X GET /my-index-2/_search | jq_es_hits -c | esrt t localhost -w examples.my-handlers:MyHandler  # <- `examples/my-handlers.py`\n# ->\n# <Client([{'host': 'localhost', 'port': 9200}])>\n# streaming_bulk  [####################################]  3\n\n# success = 3\n# failed = 0\n```\n\n```py\n# examples/my-handlers.py\nimport json\nimport typing as t\n\nfrom esrt import DocHandler\n\n\n# function style\ndef my_handler(actions: t.Iterable[str]):\n    for action in actions:\n        obj = json.loads(action)\n        prefix = 'new-'\n        if not t.cast(str, obj['_index']).startswith(prefix):\n            obj['_index'] = prefix + obj['_index']\n        yield obj\n\n\n# class style\nclass MyHandler(DocHandler):\n    def handle(self, actions: t.Iterable[str]):\n        for action in actions:\n            yield self.handle_one(action)\n\n    def handle_one(self, action: str):\n        obj = json.loads(action)\n        prefix = 'new-'\n        if not t.cast(str, obj['_index']).startswith(prefix):\n            obj['_index'] = prefix + obj['_index']\n        return obj\n```\n\n---\n\n## `e` Search docs\n\n```sh\nesrt e localhost | jq_es_hits -c\n# ->\n# {\"_index\":\"my-index-2\",\"_type\":\"type1\",\"_id\":\"2\",\"_score\":1.0,\"_source\":{\"field1\":\"22\"}}\n# {\"_index\":\"new-my-index-2\",\"_type\":\"type1\",\"_id\":\"2\",\"_score\":1.0,\"_source\":{\"field1\":\"22\"}}\n# {\"_index\":\"my-index\",\"_type\":\"type1\",\"_id\":\"1\",\"_score\":1.0,\"_source\":{\"field1\":\"cc\",\"field2\":\"uu\"}}\n# {\"_index\":\"my-index-2\",\"_type\":\"type1\",\"_id\":\"1\",\"_score\":1.0,\"_source\":{\"field1\":\"11\"}}\n# {\"_index\":\"new-my-index-2\",\"_type\":\"type1\",\"_id\":\"1\",\"_score\":1.0,\"_source\":{\"field1\":\"11\"}}\n# {\"_index\":\"my-index-2\",\"_type\":\"type1\",\"_id\":\"3\",\"_score\":1.0,\"_source\":{\"field1\":\"33\"}}\n# {\"_index\":\"new-my-index-2\",\"_type\":\"type1\",\"_id\":\"3\",\"_score\":1.0,\"_source\":{\"field1\":\"33\"}}\n```\n\n```sh\nesrt e localhost -f - <<EOF | jq_es_hits -c\n{\"query\": {\"term\": {\"_index\": \"new-my-index-2\"}}}\nEOF\n# ->\n# {\"_index\":\"new-my-index-2\",\"_type\":\"type1\",\"_id\":\"2\",\"_score\":1.0,\"_source\":{\"field1\":\"22\"}}\n# {\"_index\":\"new-my-index-2\",\"_type\":\"type1\",\"_id\":\"1\",\"_score\":1.0,\"_source\":{\"field1\":\"11\"}}\n# {\"_index\":\"new-my-index-2\",\"_type\":\"type1\",\"_id\":\"3\",\"_score\":1.0,\"_source\":{\"field1\":\"33\"}}\n```\n\n## `s` - Search and `Scroll`\n\n```sh\nesrt s localhost\n# ->\n# total = 7\n# {\"_index\": \"my-index-2\", \"_type\": \"type1\", \"_id\": \"2\", \"_score\": null, \"_source\": {\"field1\": \"22\"}, \"sort\": [0]}\n# {\"_index\": \"new-my-index-2\", \"_type\": \"type1\", \"_id\": \"2\", \"_score\": null, \"_source\": {\"field1\": \"22\"}, \"sort\": [0]}\n# {\"_index\": \"my-index\", \"_type\": \"type1\", \"_id\": \"1\", \"_score\": null, \"_source\": {\"field1\": \"cc\", \"field2\": \"uu\"}, \"sort\": [0]}\n# {\"_index\": \"my-index-2\", \"_type\": \"type1\", \"_id\": \"1\", \"_score\": null, \"_source\": {\"field1\": \"11\"}, \"sort\": [0]}\n# {\"_index\": \"new-my-index-2\", \"_type\": \"type1\", \"_id\": \"1\", \"_score\": null, \"_source\": {\"field1\": \"11\"}, \"sort\": [0]}\n# {\"_index\": \"my-index-2\", \"_type\": \"type1\", \"_id\": \"3\", \"_score\": null, \"_source\": {\"field1\": \"33\"}, \"sort\": [0]}\n# {\"_index\": \"new-my-index-2\", \"_type\": \"type1\", \"_id\": \"3\", \"_score\": null, \"_source\": {\"field1\": \"33\"}, \"sort\": [0]}\n```\n\n```sh\nesrt s localhost -f - <<EOF\n{\"query\": {\"term\": {\"field1\": \"cc\"}}}\nEOF\n# ->\n# total = 1\n# {\"_index\": \"my-index\", \"_type\": \"type1\", \"_id\": \"1\", \"_score\": null, \"_source\": {\"field1\": \"cc\", \"field2\": \"uu\"}, \"sort\": [0]}\n```\n\n## `sql` - Elasticsearch SQL\n\n```sh\n# Elasticsearch v6\nexport ESRT_SQL_API=_xpack/sql\n```\n\n```sh\nesrt sql localhost -f - <<EOF | jq_es_hits -c\nSELECT * from new-my-index-2\nEOF\n# ->\n# {\"_index\":\"new-my-index-2\",\"_type\":\"type1\",\"_id\":\"2\",\"_score\":1.0,\"_source\":{\"field1\":\"22\"}}\n# {\"_index\":\"new-my-index-2\",\"_type\":\"type1\",\"_id\":\"1\",\"_score\":1.0,\"_source\":{\"field1\":\"11\"}}\n# {\"_index\":\"new-my-index-2\",\"_type\":\"type1\",\"_id\":\"3\",\"_score\":1.0,\"_source\":{\"field1\":\"33\"}}\n```\n\n---\n\n## Other Examples\n\n```py\n# examples/create-massive-docs.py\nimport json\nimport uuid\n\n\nif __name__ == '__main__':\n    for i, _ in enumerate(range(654321), start=1):\n        d = {\n            '_index': 'my-index-a',\n            '_id': i,\n            '_type': 'type1',\n            '_source': {'field1': str(uuid.uuid4())},\n        }\n        print(json.dumps(d))\n```\n\n```sh\npython examples/create-massive-docs.py | tee -a _.ndjson | esrt t localhost -c 10000\n# ->\n# <Client([{'host': 'localhost', 'port': 9200}])>\n# streaming_bulk  [####################################]  654321\n\n# success = 654321\n# failed = 0\n\ncat _.ndjson  # <- 79M\n# ->\n# {\"_index\": \"my-index-a\", \"_id\": 1, \"_type\": \"type1\", \"_source\": {\"field1\": \"7e6a3924-1258-4e44-a19b-15395e802b1b\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 2, \"_type\": \"type1\", \"_source\": {\"field1\": \"9a05ea11-349b-452f-b771-a1aa168bdca9\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 3, \"_type\": \"type1\", \"_source\": {\"field1\": \"2e4d2d6a-54e3-4160-adbb-d0c52759bb89\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 4, \"_type\": \"type1\", \"_source\": {\"field1\": \"72cbc979-ed03-4653-8bb6-9f3dd723a2c8\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 5, \"_type\": \"type1\", \"_source\": {\"field1\": \"61a0acce-e415-4ac7-8417-66ccfe0f7932\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 6, \"_type\": \"type1\", \"_source\": {\"field1\": \"ba84e4b9-881c-4042-bf39-a449766f9e4b\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 7, \"_type\": \"type1\", \"_source\": {\"field1\": \"e92b2d83-97ae-4d5e-9797-b9ade4841f87\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 8, \"_type\": \"type1\", \"_source\": {\"field1\": \"c36acdb2-ea4e-4716-ad16-166171fa181d\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 9, \"_type\": \"type1\", \"_source\": {\"field1\": \"f17d588b-cbd0-4f72-8a47-040eb1203e35\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 10, \"_type\": \"type1\", \"_source\": {\"field1\": \"ac5d00fd-6443-4380-8d1b-72595d3f890c\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 11, \"_type\": \"type1\", \"_source\": {\"field1\": \"5d997ac9-e2c0-4347-9415-9d981f40f856\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 12, \"_type\": \"type1\", \"_source\": {\"field1\": \"7cf7ef75-9d95-4736-851b-5099dd11d1d6\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 13, \"_type\": \"type1\", \"_source\": {\"field1\": \"a7db50d4-65da-499f-84d2-5e27f719b3a7\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 14, \"_type\": \"type1\", \"_source\": {\"field1\": \"fd48cc37-520c-41be-a3e4-6e242bf91fed\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 15, \"_type\": \"type1\", \"_source\": {\"field1\": \"767286bb-5590-4265-b6f5-ce789f5f2848\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 16, \"_type\": \"type1\", \"_source\": {\"field1\": \"eca18f61-8189-46bc-b455-520a5c0a26d3\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 17, \"_type\": \"type1\", \"_source\": {\"field1\": \"61508630-c2b2-4f93-a91b-056c69208c34\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 18, \"_type\": \"type1\", \"_source\": {\"field1\": \"c6f1df60-9652-4102-98f8-df3c3e5d966b\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 19, \"_type\": \"type1\", \"_source\": {\"field1\": \"63c85746-baea-4fc5-a3c1-2638c2d3b9ed\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 20, \"_type\": \"type1\", \"_source\": {\"field1\": \"bd47ee1f-198c-4da8-8c3d-458814d547b9\"}}\n# ......\n```\n\n---\n\n```py\n# examples/copy-more-docs.py\nfrom copy import deepcopy\nimport json\nimport typing as t\nimport uuid\n\n\nif __name__ == '__main__':\n    for i, _ in enumerate(range(54321), start=1):\n        d = {\n            '_index': 'my-index-b',\n            '_id': i,\n            '_type': 'type1',\n            '_source': {'field1': str(uuid.uuid4())},\n        }\n        print(json.dumps(d))\n\n\ndef handle(actions: t.Iterable[str]):\n    for action in actions:\n        d: dict[str, t.Any] = json.loads(action)\n        yield d\n        d2 = deepcopy(d)\n        d2['_source']['field1'] += '!!!'\n        d2['_source']['field2'] = str(uuid.uuid4())\n        yield d2\n```\n\n```sh\npython examples/copy-more-docs.py | esrt t localhost -w examples.copy-more-docs:handle\n# ->\n# <Client([{'host': 'localhost', 'port': 9200}])>\n# streaming_bulk  [####################################]  108642\n\n# success = 108642\n# failed = 0\n```\n",
    "bugtrack_url": null,
    "license": null,
    "summary": null,
    "version": "3.3.0",
    "project_urls": null,
    "split_keywords": [],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "b8867a2e8d94c8e3845327ecba1362adea7802ffdaa4274b9d11ba93f625dcc4",
                "md5": "2b34c5ea0e64aabd7c0b13d17a2f80f0",
                "sha256": "04a81a5f0f465ccf496e899c122a8be6596eed14f58379afa5a6eeffa90b3d01"
            },
            "downloads": -1,
            "filename": "esrt-3.3.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "2b34c5ea0e64aabd7c0b13d17a2f80f0",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.9",
            "size": 13694,
            "upload_time": "2024-08-12T04:50:17",
            "upload_time_iso_8601": "2024-08-12T04:50:17.638626Z",
            "url": "https://files.pythonhosted.org/packages/b8/86/7a2e8d94c8e3845327ecba1362adea7802ffdaa4274b9d11ba93f625dcc4/esrt-3.3.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "77e237a236b89fa1f1e868ba731bd9f481a51cdbf67f0d9d7cb67d484f916feb",
                "md5": "c9001483f3b70b69753e5a4865e23f65",
                "sha256": "bab09a7c8d8df802ab68cf13c688095ca610fa4171a9c2a016aa142b297a9b6e"
            },
            "downloads": -1,
            "filename": "esrt-3.3.0.tar.gz",
            "has_sig": false,
            "md5_digest": "c9001483f3b70b69753e5a4865e23f65",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.9",
            "size": 12886,
            "upload_time": "2024-08-12T04:50:19",
            "upload_time_iso_8601": "2024-08-12T04:50:19.215510Z",
            "url": "https://files.pythonhosted.org/packages/77/e2/37a236b89fa1f1e868ba731bd9f481a51cdbf67f0d9d7cb67d484f916feb/esrt-3.3.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-08-12 04:50:19",
    "github": false,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "lcname": "esrt"
}
        
Elapsed time: 0.31360s