| Name | esrt JSON |
| Version |
2.2.0
JSON |
| download |
| home_page | None |
| Summary | None |
| upload_time | 2024-07-10 04:23:11 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.9 |
| license | None |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# esrt - Elasticsearch Request Tool
[](https://pypi.python.org/pypi/esrt)
```sh
pip install pipx
pipx install esrt -f
esrt --install-completion # Install completion for the current shell.
```
## Commands
- `e`: search
- `s`: scan / scroll
- `r`: request / api / a
- `t`: transmit / bulk / b
- `sql`: sql / query / q
---
## Example
You can start an es service with docker.
```sh
esrt_es_name="esrt-es"
docker run --name $esrt_es_name --rm -itd --platform=linux/amd64 -p 9200:9200 elasticsearch:5.6.9-alpine
# install sql command and restart container:
docker exec $esrt_es_name elasticsearch-plugin install https://github.com/NLPchina/elasticsearch-sql/releases/download/5.6.9.0/elasticsearch-sql-5.6.9.0.zip
docker restart $esrt_es_name
```
---
## `r` - Send a request
Check server:
```sh
esrt r localhost -X HEAD
# ->
# true
```
Create a index:
```sh
esrt r localhost -X PUT /my-index
# ->
# {"acknowledged": true, "shards_acknowledged": true, "index": "my-index"}
```
*If you want to `esrt` quote url path for you, add flag: `-Q`(`--quote-url`)*
Cat it:
```sh
esrt r localhost -X GET _cat/indices -p 'v&format=json' -p 's=index'
# ->
# [{"health": "yellow", "status": "open", "index": "my-index", "uuid": "avrhX1hzQLyfEGvXsA96NA", "pri": "5", "rep": "1", "docs.count": "0", "docs.deleted": "0", "store.size": "324b", "pri.store.size": "324b"}]
```
*`esrt` doesn't keep `-p pretty` format, but you can use `jq`.*
```sh
esrt r localhost -X GET _cat/indices -p 'v&format=json' -p 's=index' | jq
# ->
# [
# {
# "health": "yellow",
# "status": "open",
# "index": "my-index",
# "uuid": "avrhX1hzQLyfEGvXsA96NA",
# "pri": "5",
# "rep": "1",
# "docs.count": "0",
# "docs.deleted": "0",
# "store.size": "810b",
# "pri.store.size": "810b"
# }
# ]
```
---
## `t` - Transmit data (`streaming_bulk`)
Bulk with data from file `examples/bulk.ndjson`:
```json
{ "_op_type": "index", "_index": "my-index", "_type": "type1", "_id": "1", "field1": "ii" }
{ "_op_type": "delete", "_index": "my-index", "_type": "type1", "_id": "1" }
{ "_op_type": "create", "_index": "my-index", "_type": "type1", "_id": "1", "field1": "cc" }
{ "_op_type": "update", "_index": "my-index", "_type": "type1", "_id": "1", "doc": {"field2": "uu"} }
```
```sh
esrt t localhost -f examples/bulk.ndjson
# ->
# <Client([{'host': 'localhost', 'port': 9200}])>
# streaming_bulk [####################################] 4
# success = 4
# failed = 0
```
---
Read payload from `stdin`. And `-d` can be omitted.
```sh
esrt t localhost <<EOF
{ "_op_type": "index", "_index": "my-index-2", "_type": "type1", "_id": "1", "field1": "11" }
{ "_op_type": "index", "_index": "my-index-2", "_type": "type1", "_id": "2", "field1": "22" }
{ "_op_type": "index", "_index": "my-index-2", "_type": "type1", "_id": "3", "field1": "33" }
EOF
# ->
# <Client([{'host': 'localhost', 'port': 9200}])>
# streaming_bulk [####################################] 3
# success = 3
# failed = 0
```
Piping `heredoc` also works.
```sh
cat <<EOF | esrt t localhost
{ "_op_type": "index", "_index": "my-index-2", "_type": "type1", "_id": "1", "field1": "11" }
{ "_op_type": "index", "_index": "my-index-2", "_type": "type1", "_id": "2", "field1": "22" }
{ "_op_type": "index", "_index": "my-index-2", "_type": "type1", "_id": "3", "field1": "33" }
EOF
```
---
Pipe `_search` result and update `_index` with `customized handler` to do more operations before bulk!
```sh
alias jq_es_hits="jq '.hits.hits.[]'"
```
```sh
esrt r localhost -X GET /my-index-2/_search | jq_es_hits -c | esrt t localhost -w examples.my-handlers:MyHandler # <- `examples/my-handlers.py`
# ->
# <Client([{'host': 'localhost', 'port': 9200}])>
# streaming_bulk [####################################] 3
# success = 3
# failed = 0
```
```py
# examples/my-handlers.py
import json
import typing as t
from esrt import DocHandler
# function style
def my_handler(actions: t.Iterable[str]):
for action in actions:
obj = json.loads(action)
prefix = 'new-'
if not t.cast(str, obj['_index']).startswith(prefix):
obj['_index'] = prefix + obj['_index']
yield obj
# class style
class MyHandler(DocHandler):
def handle(self, actions: t.Iterable[str]):
for action in actions:
yield self.handle_one(action)
def handle_one(self, action: str):
obj = json.loads(action)
prefix = 'new-'
if not t.cast(str, obj['_index']).startswith(prefix):
obj['_index'] = prefix + obj['_index']
return obj
```
---
## `e` Search docs
```sh
esrt e localhost | jq_es_hits -c
# ->
# {"_index":"my-index-2","_type":"type1","_id":"2","_score":1.0,"_source":{"field1":"22"}}
# {"_index":"new-my-index-2","_type":"type1","_id":"2","_score":1.0,"_source":{"field1":"22"}}
# {"_index":"my-index","_type":"type1","_id":"1","_score":1.0,"_source":{"field1":"cc","field2":"uu"}}
# {"_index":"my-index-2","_type":"type1","_id":"1","_score":1.0,"_source":{"field1":"11"}}
# {"_index":"new-my-index-2","_type":"type1","_id":"1","_score":1.0,"_source":{"field1":"11"}}
# {"_index":"my-index-2","_type":"type1","_id":"3","_score":1.0,"_source":{"field1":"33"}}
# {"_index":"new-my-index-2","_type":"type1","_id":"3","_score":1.0,"_source":{"field1":"33"}}
```
```sh
esrt e localhost -f - <<EOF | jq_es_hits -c
{"query": {"term": {"_index": "new-my-index-2"}}}
EOF
# ->
# {"_index":"new-my-index-2","_type":"type1","_id":"2","_score":1.0,"_source":{"field1":"22"}}
# {"_index":"new-my-index-2","_type":"type1","_id":"1","_score":1.0,"_source":{"field1":"11"}}
# {"_index":"new-my-index-2","_type":"type1","_id":"3","_score":1.0,"_source":{"field1":"33"}}
```
## `s` - Search and `Scroll`
```sh
esrt s localhost
# ->
# total = 7
# {"_index": "my-index-2", "_type": "type1", "_id": "2", "_score": null, "_source": {"field1": "22"}, "sort": [0]}
# {"_index": "new-my-index-2", "_type": "type1", "_id": "2", "_score": null, "_source": {"field1": "22"}, "sort": [0]}
# {"_index": "my-index", "_type": "type1", "_id": "1", "_score": null, "_source": {"field1": "cc", "field2": "uu"}, "sort": [0]}
# {"_index": "my-index-2", "_type": "type1", "_id": "1", "_score": null, "_source": {"field1": "11"}, "sort": [0]}
# {"_index": "new-my-index-2", "_type": "type1", "_id": "1", "_score": null, "_source": {"field1": "11"}, "sort": [0]}
# {"_index": "my-index-2", "_type": "type1", "_id": "3", "_score": null, "_source": {"field1": "33"}, "sort": [0]}
# {"_index": "new-my-index-2", "_type": "type1", "_id": "3", "_score": null, "_source": {"field1": "33"}, "sort": [0]}
```
```sh
esrt s localhost -f - <<EOF
{"query": {"term": {"field1": "cc"}}}
EOF
# ->
# total = 1
# {"_index": "my-index", "_type": "type1", "_id": "1", "_score": null, "_source": {"field1": "cc", "field2": "uu"}, "sort": [0]}
```
## `sql` - Elasticsearch SQL
```sh
# Elasticsearch v6
export ESRT_SQL_API=_xpack/sql
```
```sh
esrt sql localhost -f - <<EOF | jq_es_hits -c
SELECT * from new-my-index-2
EOF
# ->
# {"_index":"new-my-index-2","_type":"type1","_id":"2","_score":1.0,"_source":{"field1":"22"}}
# {"_index":"new-my-index-2","_type":"type1","_id":"1","_score":1.0,"_source":{"field1":"11"}}
# {"_index":"new-my-index-2","_type":"type1","_id":"3","_score":1.0,"_source":{"field1":"33"}}
```
---
## Other Examples
```py
# examples/create-massive-docs.py
import json
import uuid
if __name__ == '__main__':
for i, _ in enumerate(range(654321), start=1):
d = {
'_index': 'my-index-a',
'_id': i,
'_type': 'type1',
'_source': {'field1': str(uuid.uuid4())},
}
print(json.dumps(d))
```
```sh
python examples/create-massive-docs.py | tee -a _.ndjson | esrt t localhost -c 10000
# ->
# <Client([{'host': 'localhost', 'port': 9200}])>
# streaming_bulk [####################################] 654321
# success = 654321
# failed = 0
cat _.ndjson # <- 79M
# ->
# {"_index": "my-index-a", "_id": 1, "_type": "type1", "_source": {"field1": "7e6a3924-1258-4e44-a19b-15395e802b1b"}}
# {"_index": "my-index-a", "_id": 2, "_type": "type1", "_source": {"field1": "9a05ea11-349b-452f-b771-a1aa168bdca9"}}
# {"_index": "my-index-a", "_id": 3, "_type": "type1", "_source": {"field1": "2e4d2d6a-54e3-4160-adbb-d0c52759bb89"}}
# {"_index": "my-index-a", "_id": 4, "_type": "type1", "_source": {"field1": "72cbc979-ed03-4653-8bb6-9f3dd723a2c8"}}
# {"_index": "my-index-a", "_id": 5, "_type": "type1", "_source": {"field1": "61a0acce-e415-4ac7-8417-66ccfe0f7932"}}
# {"_index": "my-index-a", "_id": 6, "_type": "type1", "_source": {"field1": "ba84e4b9-881c-4042-bf39-a449766f9e4b"}}
# {"_index": "my-index-a", "_id": 7, "_type": "type1", "_source": {"field1": "e92b2d83-97ae-4d5e-9797-b9ade4841f87"}}
# {"_index": "my-index-a", "_id": 8, "_type": "type1", "_source": {"field1": "c36acdb2-ea4e-4716-ad16-166171fa181d"}}
# {"_index": "my-index-a", "_id": 9, "_type": "type1", "_source": {"field1": "f17d588b-cbd0-4f72-8a47-040eb1203e35"}}
# {"_index": "my-index-a", "_id": 10, "_type": "type1", "_source": {"field1": "ac5d00fd-6443-4380-8d1b-72595d3f890c"}}
# {"_index": "my-index-a", "_id": 11, "_type": "type1", "_source": {"field1": "5d997ac9-e2c0-4347-9415-9d981f40f856"}}
# {"_index": "my-index-a", "_id": 12, "_type": "type1", "_source": {"field1": "7cf7ef75-9d95-4736-851b-5099dd11d1d6"}}
# {"_index": "my-index-a", "_id": 13, "_type": "type1", "_source": {"field1": "a7db50d4-65da-499f-84d2-5e27f719b3a7"}}
# {"_index": "my-index-a", "_id": 14, "_type": "type1", "_source": {"field1": "fd48cc37-520c-41be-a3e4-6e242bf91fed"}}
# {"_index": "my-index-a", "_id": 15, "_type": "type1", "_source": {"field1": "767286bb-5590-4265-b6f5-ce789f5f2848"}}
# {"_index": "my-index-a", "_id": 16, "_type": "type1", "_source": {"field1": "eca18f61-8189-46bc-b455-520a5c0a26d3"}}
# {"_index": "my-index-a", "_id": 17, "_type": "type1", "_source": {"field1": "61508630-c2b2-4f93-a91b-056c69208c34"}}
# {"_index": "my-index-a", "_id": 18, "_type": "type1", "_source": {"field1": "c6f1df60-9652-4102-98f8-df3c3e5d966b"}}
# {"_index": "my-index-a", "_id": 19, "_type": "type1", "_source": {"field1": "63c85746-baea-4fc5-a3c1-2638c2d3b9ed"}}
# {"_index": "my-index-a", "_id": 20, "_type": "type1", "_source": {"field1": "bd47ee1f-198c-4da8-8c3d-458814d547b9"}}
# ......
```
---
```py
# examples/copy-more-docs.py
from copy import deepcopy
import json
import typing as t
import uuid
if __name__ == '__main__':
for i, _ in enumerate(range(54321), start=1):
d = {
'_index': 'my-index-b',
'_id': i,
'_type': 'type1',
'_source': {'field1': str(uuid.uuid4())},
}
print(json.dumps(d))
def handle(actions: t.Iterable[str]):
for action in actions:
d: dict[str, t.Any] = json.loads(action)
yield d
d2 = deepcopy(d)
d2['_source']['field1'] += '!!!'
d2['_source']['field2'] = str(uuid.uuid4())
yield d2
```
```sh
python examples/copy-more-docs.py | esrt t localhost -w examples.copy-more-docs:handle
# ->
# <Client([{'host': 'localhost', 'port': 9200}])>
# streaming_bulk [####################################] 108642
# success = 108642
# failed = 0
```
Raw data
{
"_id": null,
"home_page": null,
"name": "esrt",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": null,
"author": null,
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/9f/ba/717d75301dd0c28930b5bcd3e81cf547bd28b4400c08f172d618b4cd297f/esrt-2.2.0.tar.gz",
"platform": null,
"description": "# esrt - Elasticsearch Request Tool\n\n[](https://pypi.python.org/pypi/esrt)\n\n```sh\npip install pipx\npipx install esrt -f\nesrt --install-completion # Install completion for the current shell.\n```\n\n## Commands\n\n- `e`: search\n- `s`: scan / scroll\n- `r`: request / api / a\n- `t`: transmit / bulk / b\n- `sql`: sql / query / q\n\n---\n\n## Example\n\nYou can start an es service with docker.\n\n```sh\nesrt_es_name=\"esrt-es\"\ndocker run --name $esrt_es_name --rm -itd --platform=linux/amd64 -p 9200:9200 elasticsearch:5.6.9-alpine\n\n# install sql command and restart container:\ndocker exec $esrt_es_name elasticsearch-plugin install https://github.com/NLPchina/elasticsearch-sql/releases/download/5.6.9.0/elasticsearch-sql-5.6.9.0.zip\ndocker restart $esrt_es_name\n```\n\n---\n\n## `r` - Send a request\n\nCheck server:\n\n```sh\nesrt r localhost -X HEAD\n# ->\n# true\n```\n\nCreate a index:\n\n```sh\nesrt r localhost -X PUT /my-index\n# ->\n# {\"acknowledged\": true, \"shards_acknowledged\": true, \"index\": \"my-index\"}\n```\n\n*If you want to `esrt` quote url path for you, add flag: `-Q`(`--quote-url`)*\n\nCat it:\n\n```sh\nesrt r localhost -X GET _cat/indices -p 'v&format=json' -p 's=index'\n# ->\n# [{\"health\": \"yellow\", \"status\": \"open\", \"index\": \"my-index\", \"uuid\": \"avrhX1hzQLyfEGvXsA96NA\", \"pri\": \"5\", \"rep\": \"1\", \"docs.count\": \"0\", \"docs.deleted\": \"0\", \"store.size\": \"324b\", \"pri.store.size\": \"324b\"}]\n```\n\n*`esrt` doesn't keep `-p pretty` format, but you can use `jq`.*\n\n```sh\nesrt r localhost -X GET _cat/indices -p 'v&format=json' -p 's=index' | jq\n# ->\n# [\n# {\n# \"health\": \"yellow\",\n# \"status\": \"open\",\n# \"index\": \"my-index\",\n# \"uuid\": \"avrhX1hzQLyfEGvXsA96NA\",\n# \"pri\": \"5\",\n# \"rep\": \"1\",\n# \"docs.count\": \"0\",\n# \"docs.deleted\": \"0\",\n# \"store.size\": \"810b\",\n# \"pri.store.size\": \"810b\"\n# }\n# ]\n```\n\n---\n\n## `t` - Transmit data (`streaming_bulk`)\n\nBulk with data from file `examples/bulk.ndjson`:\n\n```json\n{ \"_op_type\": \"index\", \"_index\": \"my-index\", \"_type\": \"type1\", \"_id\": \"1\", \"field1\": \"ii\" }\n{ \"_op_type\": \"delete\", \"_index\": \"my-index\", \"_type\": \"type1\", \"_id\": \"1\" }\n{ \"_op_type\": \"create\", \"_index\": \"my-index\", \"_type\": \"type1\", \"_id\": \"1\", \"field1\": \"cc\" }\n{ \"_op_type\": \"update\", \"_index\": \"my-index\", \"_type\": \"type1\", \"_id\": \"1\", \"doc\": {\"field2\": \"uu\"} }\n```\n\n```sh\nesrt t localhost -f examples/bulk.ndjson\n# ->\n# <Client([{'host': 'localhost', 'port': 9200}])>\n# streaming_bulk [####################################] 4\n\n# success = 4\n# failed = 0\n```\n\n---\n\nRead payload from `stdin`. And `-d` can be omitted.\n\n```sh\nesrt t localhost <<EOF\n{ \"_op_type\": \"index\", \"_index\": \"my-index-2\", \"_type\": \"type1\", \"_id\": \"1\", \"field1\": \"11\" }\n{ \"_op_type\": \"index\", \"_index\": \"my-index-2\", \"_type\": \"type1\", \"_id\": \"2\", \"field1\": \"22\" }\n{ \"_op_type\": \"index\", \"_index\": \"my-index-2\", \"_type\": \"type1\", \"_id\": \"3\", \"field1\": \"33\" }\nEOF\n# ->\n# <Client([{'host': 'localhost', 'port': 9200}])>\n# streaming_bulk [####################################] 3\n\n# success = 3\n# failed = 0\n```\n\nPiping `heredoc` also works.\n\n```sh\ncat <<EOF | esrt t localhost\n{ \"_op_type\": \"index\", \"_index\": \"my-index-2\", \"_type\": \"type1\", \"_id\": \"1\", \"field1\": \"11\" }\n{ \"_op_type\": \"index\", \"_index\": \"my-index-2\", \"_type\": \"type1\", \"_id\": \"2\", \"field1\": \"22\" }\n{ \"_op_type\": \"index\", \"_index\": \"my-index-2\", \"_type\": \"type1\", \"_id\": \"3\", \"field1\": \"33\" }\nEOF\n```\n\n---\n\nPipe `_search` result and update `_index` with `customized handler` to do more operations before bulk!\n\n```sh\nalias jq_es_hits=\"jq '.hits.hits.[]'\"\n```\n\n```sh\nesrt r localhost -X GET /my-index-2/_search | jq_es_hits -c | esrt t localhost -w examples.my-handlers:MyHandler # <- `examples/my-handlers.py`\n# ->\n# <Client([{'host': 'localhost', 'port': 9200}])>\n# streaming_bulk [####################################] 3\n\n# success = 3\n# failed = 0\n```\n\n```py\n# examples/my-handlers.py\nimport json\nimport typing as t\n\nfrom esrt import DocHandler\n\n\n# function style\ndef my_handler(actions: t.Iterable[str]):\n for action in actions:\n obj = json.loads(action)\n prefix = 'new-'\n if not t.cast(str, obj['_index']).startswith(prefix):\n obj['_index'] = prefix + obj['_index']\n yield obj\n\n\n# class style\nclass MyHandler(DocHandler):\n def handle(self, actions: t.Iterable[str]):\n for action in actions:\n yield self.handle_one(action)\n\n def handle_one(self, action: str):\n obj = json.loads(action)\n prefix = 'new-'\n if not t.cast(str, obj['_index']).startswith(prefix):\n obj['_index'] = prefix + obj['_index']\n return obj\n```\n\n---\n\n## `e` Search docs\n\n```sh\nesrt e localhost | jq_es_hits -c\n# ->\n# {\"_index\":\"my-index-2\",\"_type\":\"type1\",\"_id\":\"2\",\"_score\":1.0,\"_source\":{\"field1\":\"22\"}}\n# {\"_index\":\"new-my-index-2\",\"_type\":\"type1\",\"_id\":\"2\",\"_score\":1.0,\"_source\":{\"field1\":\"22\"}}\n# {\"_index\":\"my-index\",\"_type\":\"type1\",\"_id\":\"1\",\"_score\":1.0,\"_source\":{\"field1\":\"cc\",\"field2\":\"uu\"}}\n# {\"_index\":\"my-index-2\",\"_type\":\"type1\",\"_id\":\"1\",\"_score\":1.0,\"_source\":{\"field1\":\"11\"}}\n# {\"_index\":\"new-my-index-2\",\"_type\":\"type1\",\"_id\":\"1\",\"_score\":1.0,\"_source\":{\"field1\":\"11\"}}\n# {\"_index\":\"my-index-2\",\"_type\":\"type1\",\"_id\":\"3\",\"_score\":1.0,\"_source\":{\"field1\":\"33\"}}\n# {\"_index\":\"new-my-index-2\",\"_type\":\"type1\",\"_id\":\"3\",\"_score\":1.0,\"_source\":{\"field1\":\"33\"}}\n```\n\n```sh\nesrt e localhost -f - <<EOF | jq_es_hits -c\n{\"query\": {\"term\": {\"_index\": \"new-my-index-2\"}}}\nEOF\n# ->\n# {\"_index\":\"new-my-index-2\",\"_type\":\"type1\",\"_id\":\"2\",\"_score\":1.0,\"_source\":{\"field1\":\"22\"}}\n# {\"_index\":\"new-my-index-2\",\"_type\":\"type1\",\"_id\":\"1\",\"_score\":1.0,\"_source\":{\"field1\":\"11\"}}\n# {\"_index\":\"new-my-index-2\",\"_type\":\"type1\",\"_id\":\"3\",\"_score\":1.0,\"_source\":{\"field1\":\"33\"}}\n```\n\n## `s` - Search and `Scroll`\n\n```sh\nesrt s localhost\n# ->\n# total = 7\n# {\"_index\": \"my-index-2\", \"_type\": \"type1\", \"_id\": \"2\", \"_score\": null, \"_source\": {\"field1\": \"22\"}, \"sort\": [0]}\n# {\"_index\": \"new-my-index-2\", \"_type\": \"type1\", \"_id\": \"2\", \"_score\": null, \"_source\": {\"field1\": \"22\"}, \"sort\": [0]}\n# {\"_index\": \"my-index\", \"_type\": \"type1\", \"_id\": \"1\", \"_score\": null, \"_source\": {\"field1\": \"cc\", \"field2\": \"uu\"}, \"sort\": [0]}\n# {\"_index\": \"my-index-2\", \"_type\": \"type1\", \"_id\": \"1\", \"_score\": null, \"_source\": {\"field1\": \"11\"}, \"sort\": [0]}\n# {\"_index\": \"new-my-index-2\", \"_type\": \"type1\", \"_id\": \"1\", \"_score\": null, \"_source\": {\"field1\": \"11\"}, \"sort\": [0]}\n# {\"_index\": \"my-index-2\", \"_type\": \"type1\", \"_id\": \"3\", \"_score\": null, \"_source\": {\"field1\": \"33\"}, \"sort\": [0]}\n# {\"_index\": \"new-my-index-2\", \"_type\": \"type1\", \"_id\": \"3\", \"_score\": null, \"_source\": {\"field1\": \"33\"}, \"sort\": [0]}\n```\n\n```sh\nesrt s localhost -f - <<EOF\n{\"query\": {\"term\": {\"field1\": \"cc\"}}}\nEOF\n# ->\n# total = 1\n# {\"_index\": \"my-index\", \"_type\": \"type1\", \"_id\": \"1\", \"_score\": null, \"_source\": {\"field1\": \"cc\", \"field2\": \"uu\"}, \"sort\": [0]}\n```\n\n## `sql` - Elasticsearch SQL\n\n```sh\n# Elasticsearch v6\nexport ESRT_SQL_API=_xpack/sql\n```\n\n```sh\nesrt sql localhost -f - <<EOF | jq_es_hits -c\nSELECT * from new-my-index-2\nEOF\n# ->\n# {\"_index\":\"new-my-index-2\",\"_type\":\"type1\",\"_id\":\"2\",\"_score\":1.0,\"_source\":{\"field1\":\"22\"}}\n# {\"_index\":\"new-my-index-2\",\"_type\":\"type1\",\"_id\":\"1\",\"_score\":1.0,\"_source\":{\"field1\":\"11\"}}\n# {\"_index\":\"new-my-index-2\",\"_type\":\"type1\",\"_id\":\"3\",\"_score\":1.0,\"_source\":{\"field1\":\"33\"}}\n```\n\n---\n\n## Other Examples\n\n```py\n# examples/create-massive-docs.py\nimport json\nimport uuid\n\n\nif __name__ == '__main__':\n for i, _ in enumerate(range(654321), start=1):\n d = {\n '_index': 'my-index-a',\n '_id': i,\n '_type': 'type1',\n '_source': {'field1': str(uuid.uuid4())},\n }\n print(json.dumps(d))\n```\n\n```sh\npython examples/create-massive-docs.py | tee -a _.ndjson | esrt t localhost -c 10000\n# ->\n# <Client([{'host': 'localhost', 'port': 9200}])>\n# streaming_bulk [####################################] 654321\n\n# success = 654321\n# failed = 0\n\ncat _.ndjson # <- 79M\n# ->\n# {\"_index\": \"my-index-a\", \"_id\": 1, \"_type\": \"type1\", \"_source\": {\"field1\": \"7e6a3924-1258-4e44-a19b-15395e802b1b\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 2, \"_type\": \"type1\", \"_source\": {\"field1\": \"9a05ea11-349b-452f-b771-a1aa168bdca9\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 3, \"_type\": \"type1\", \"_source\": {\"field1\": \"2e4d2d6a-54e3-4160-adbb-d0c52759bb89\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 4, \"_type\": \"type1\", \"_source\": {\"field1\": \"72cbc979-ed03-4653-8bb6-9f3dd723a2c8\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 5, \"_type\": \"type1\", \"_source\": {\"field1\": \"61a0acce-e415-4ac7-8417-66ccfe0f7932\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 6, \"_type\": \"type1\", \"_source\": {\"field1\": \"ba84e4b9-881c-4042-bf39-a449766f9e4b\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 7, \"_type\": \"type1\", \"_source\": {\"field1\": \"e92b2d83-97ae-4d5e-9797-b9ade4841f87\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 8, \"_type\": \"type1\", \"_source\": {\"field1\": \"c36acdb2-ea4e-4716-ad16-166171fa181d\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 9, \"_type\": \"type1\", \"_source\": {\"field1\": \"f17d588b-cbd0-4f72-8a47-040eb1203e35\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 10, \"_type\": \"type1\", \"_source\": {\"field1\": \"ac5d00fd-6443-4380-8d1b-72595d3f890c\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 11, \"_type\": \"type1\", \"_source\": {\"field1\": \"5d997ac9-e2c0-4347-9415-9d981f40f856\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 12, \"_type\": \"type1\", \"_source\": {\"field1\": \"7cf7ef75-9d95-4736-851b-5099dd11d1d6\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 13, \"_type\": \"type1\", \"_source\": {\"field1\": \"a7db50d4-65da-499f-84d2-5e27f719b3a7\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 14, \"_type\": \"type1\", \"_source\": {\"field1\": \"fd48cc37-520c-41be-a3e4-6e242bf91fed\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 15, \"_type\": \"type1\", \"_source\": {\"field1\": \"767286bb-5590-4265-b6f5-ce789f5f2848\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 16, \"_type\": \"type1\", \"_source\": {\"field1\": \"eca18f61-8189-46bc-b455-520a5c0a26d3\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 17, \"_type\": \"type1\", \"_source\": {\"field1\": \"61508630-c2b2-4f93-a91b-056c69208c34\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 18, \"_type\": \"type1\", \"_source\": {\"field1\": \"c6f1df60-9652-4102-98f8-df3c3e5d966b\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 19, \"_type\": \"type1\", \"_source\": {\"field1\": \"63c85746-baea-4fc5-a3c1-2638c2d3b9ed\"}}\n# {\"_index\": \"my-index-a\", \"_id\": 20, \"_type\": \"type1\", \"_source\": {\"field1\": \"bd47ee1f-198c-4da8-8c3d-458814d547b9\"}}\n# ......\n```\n\n---\n\n```py\n# examples/copy-more-docs.py\nfrom copy import deepcopy\nimport json\nimport typing as t\nimport uuid\n\n\nif __name__ == '__main__':\n for i, _ in enumerate(range(54321), start=1):\n d = {\n '_index': 'my-index-b',\n '_id': i,\n '_type': 'type1',\n '_source': {'field1': str(uuid.uuid4())},\n }\n print(json.dumps(d))\n\n\ndef handle(actions: t.Iterable[str]):\n for action in actions:\n d: dict[str, t.Any] = json.loads(action)\n yield d\n d2 = deepcopy(d)\n d2['_source']['field1'] += '!!!'\n d2['_source']['field2'] = str(uuid.uuid4())\n yield d2\n```\n\n```sh\npython examples/copy-more-docs.py | esrt t localhost -w examples.copy-more-docs:handle\n# ->\n# <Client([{'host': 'localhost', 'port': 9200}])>\n# streaming_bulk [####################################] 108642\n\n# success = 108642\n# failed = 0\n```\n",
"bugtrack_url": null,
"license": null,
"summary": null,
"version": "2.2.0",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "33d79cf9f4f3bade51ec1c7284931f8f4903f63b342d3b877e0c5e1b8a1deb27",
"md5": "0f7130d0d1d9de05c06e3635be644176",
"sha256": "8c7dd3c782ca1833e6693d8101f6e1702455646bcd8e69b5c1146f6a954cb832"
},
"downloads": -1,
"filename": "esrt-2.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "0f7130d0d1d9de05c06e3635be644176",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 10221,
"upload_time": "2024-07-10T04:23:09",
"upload_time_iso_8601": "2024-07-10T04:23:09.945657Z",
"url": "https://files.pythonhosted.org/packages/33/d7/9cf9f4f3bade51ec1c7284931f8f4903f63b342d3b877e0c5e1b8a1deb27/esrt-2.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9fba717d75301dd0c28930b5bcd3e81cf547bd28b4400c08f172d618b4cd297f",
"md5": "f0e1f925565def2b3be4da73a92fde58",
"sha256": "9aa00478042a3349b1ac187e1f413318774d2a15bad5226869c4822c4b2d1c6a"
},
"downloads": -1,
"filename": "esrt-2.2.0.tar.gz",
"has_sig": false,
"md5_digest": "f0e1f925565def2b3be4da73a92fde58",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 11879,
"upload_time": "2024-07-10T04:23:11",
"upload_time_iso_8601": "2024-07-10T04:23:11.842635Z",
"url": "https://files.pythonhosted.org/packages/9f/ba/717d75301dd0c28930b5bcd3e81cf547bd28b4400c08f172d618b4cd297f/esrt-2.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-10 04:23:11",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "esrt"
}
