# evtx2es
[](LICENSE)
[](https://badge.fury.io/py/evtx2es)
[](https://pypi.org/project/evtx2es/)
[](https://github.com/sumeshi/evtx2es/actions/workflows/test.yml)
Fast import of Windows EventLogs(.evtx) into Elasticsearch.
Life is too short and there is not enough time to process **huge Windows EventLogs** with **pure-Python software**.
**evtx2es** uses Rust library [pyevtx-rs](https://github.com/omerbenamram/pyevtx-rs), so it runs much faster than traditional software.
## Usage
**evtx2es** can be executed from the command line or incorporated into a Python script.
```bash
$ evtx2es /path/to/your/file.evtx
```
```python
from evtx2es import evtx2es
if __name__ == '__main__':
filepath = '/path/to/your/file.evtx'
evtx2es(filepath)
```
### Arguments
evtx2es supports simultaneous import of multiple files.
```bash
$ evtx2es file1.evtx file2.evtx file3.evtx
```
Additionally, it also allows for recursive import under the specified directory.
```bash
$ tree .
evtxfiles/
├── file1.evtx
├── file2.evtx
├── file3.evtx
└── subdirectory/
├── file4.evtx
└── subsubdirectory/
├── file5.evtx
└── file6.evtx
$ evtx2es /evtxfiles/ # The Path is recursively expanded to file1~6.evtx.
```
### Options
```
--version, -v
--help, -h
--quiet, -q
Flag to suppress standard output
(default: False)
--multiprocess, -m:
Enable multiprocessing for faster execution
(default: False)
--size:
Chunk size for processing (default: 500)
--host:
ElasticSearch host address (default: localhost)
--port:
ElasticSearch port number (default: 9200)
--index:
Destination index name for importing (default: evtx2es)
--scheme:
Protocol scheme to use (http or https) (default: http)
--pipeline:
Elasticsearch Ingest Pipeline to use (default: )
--datasetdate:
Date of the latest record in the dataset, extracted from TimeCreated field (MM/DD/YYYY.HH:MM:SS) (default: 0)
--login:
The login to use if Elastic Security is enabled (default: )
--pwd:
The password associated with the provided login (default: )
```
### Examples
When using from the commandline interface:
```
$ evtx2es /path/to/your/file.evtx --host=localhost --port=9200 --index=foobar --size=500
```
When using from the python-script:
```py
if __name__ == '__main__':
evtx2es('/path/to/your/file.evtx', host=localhost, port=9200, index='foobar', size=500)
```
With credentials for Elastic Security:
```
$ evtx2es /path/to/your/file.evtx --host=localhost --port=9200 --index=foobar --login=elastic --pwd=******
```
Note: The current version does not verify the certificate.
## Appendix
### Evtx2json
An additional feature: :sushi: :sushi: :sushi:
Convert Windows Event Logs to a JSON file.
```bash
$ evtx2json /path/to/your/file.evtx /path/to/output/target.json
```
Convert Windows Event Logs to a Python List[dict] object.
```python
from evtx2es import evtx2json
if __name__ == '__main__':
filepath = '/path/to/your/file.evtx'
result: List[dict] = evtx2json(filepath)
```
## Output Format Example
Using the sample evtx file of [JPCERT/CC:LogonTracer](https://github.com/JPCERTCC/LogonTracer) as an example.
```
[
{
"event_record_id": 227559,
"timestamp": "2016-10-06 01:50:49.420927 UTC",
"winlog": {
"channel": "Security",
"computer_name": "WIN-WFBHIBE5GXZ.example.co.jp",
"event_id": 4624,
"opcode": 0,
"provider_guid": "{54849625-5478-4994-a5ba-3e3b0328c30d}",
"provider_name": "Microsoft-Windows-Security-Auditing",
"record_id": 227559,
"task": 12544,
"version": 0,
"process": {
"pid": 572,
"thread_id": 1244
},
"event_data": {
"AuthenticationPackageName": "Kerberos",
"IpAddress": "192.168.16.102",
"IpPort": "49220",
"KeyLength": 0,
"LmPackageName": "-",
"LogonGuid": "F4DC1C19-0544-BC52-0900-DFC19752C3C6",
"LogonProcessName": "Kerberos",
"LogonType": 3,
"ProcessId": 0,
"ProcessName": "-",
"SubjectDomainName": "-",
"SubjectLogonId": "0x0",
"SubjectUserName": "-",
"SubjectUserSid": "S-1-0-0",
"TargetDomainName": "EXAMPLE",
"TargetLogonId": "0x1fa0869",
"TargetUserName": "WIN7_64JP_02$",
"TargetUserSid": "S-1-5-21-1524084746-3249201829-3114449661-1107",
"TransmittedServices": "-",
"WorkstationName": "",
"Status": null
}
},
"log": {
"file": {
"name": "sample/Security.evtx"
}
},
"event": {
"code": 4624,
"created": "2016-10-06T01:50:49.420927Z"
},
"@timestamp": "2016-10-06T01:50:49.420927Z"
},
...
]
```
## Performance Evaluations
evtx2es was evaluated using the sample evtx file of [JPCERT/CC:LogonTracer](https://github.com/JPCERTCC/LogonTracer) (about 30MB binary data).
```.bash
$ time evtx2es ./Security.evtx
> 6.25 user 0.13 system 0:14.08 elapsed 45%CPU
```
See [Qiita](https://qiita.com/sumeshi/items/cb2fbafe59c2c83e3085) for more information.
### Running Environment
```
OS: Ubuntu 18.04
CPU: Intel Core i5-6500
RAM: DDR4 32GB
```
ElasticSearch 7.4 was running on the Docker version(Official Image).
https://hub.docker.com/_/elasticsearch
## Installation
### from PyPI
```
$ pip install evtx2es
```
### from GitHub Releases
The version compiled into a binary using Nuitka is also available for use.
```bash
$ chmod +x ./ntfsdump
$ ./ntfsdump {{options...}}
```
```powershell
> ntfsdump.exe {{options...}}
```
## Contributing
The source code for evtx2es is hosted at GitHub, and you may download, fork, and review it from this repository(https://github.com/sumeshi/evtx2es).
Please report issues and feature requests. :sushi: :sushi: :sushi:
## License
evtx2es is released under the [MIT](https://github.com/sumeshi/evtx2es/blob/master/LICENSE) License.
Powered by following libraries:
- [pyevtx-rs](https://github.com/omerbenamram/pyevtx-rs)
- [Nuitka](https://github.com/Nuitka/Nuitka)
Inspired by [EvtxtoElk](https://github.com/dgunter/evtxtoelk).
Raw data
{
"_id": null,
"home_page": "https://github.com/sumeshi/evtx2es",
"name": "evtx2es",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.11,<4.0",
"maintainer_email": "",
"keywords": "evtx,elasticsearch,json",
"author": "sumeshi",
"author_email": "sum3sh1@protonmail.com",
"download_url": "https://files.pythonhosted.org/packages/36/78/ee4aaaf1fb114bf7e8e16b0320500ea6d3da128f05e62ce7d5053aa9b2c1/evtx2es-1.6.0.tar.gz",
"platform": null,
"description": "# evtx2es\n\n[](LICENSE)\n[](https://badge.fury.io/py/evtx2es)\n[](https://pypi.org/project/evtx2es/)\n[](https://github.com/sumeshi/evtx2es/actions/workflows/test.yml)\n\n\n\nFast import of Windows EventLogs(.evtx) into Elasticsearch.\n\nLife is too short and there is not enough time to process **huge Windows EventLogs** with **pure-Python software**. \n**evtx2es** uses Rust library [pyevtx-rs](https://github.com/omerbenamram/pyevtx-rs), so it runs much faster than traditional software.\n\n## Usage\n\n**evtx2es** can be executed from the command line or incorporated into a Python script.\n\n```bash\n$ evtx2es /path/to/your/file.evtx\n```\n\n```python\nfrom evtx2es import evtx2es\n\nif __name__ == '__main__':\n filepath = '/path/to/your/file.evtx'\n evtx2es(filepath)\n```\n\n### Arguments\n\nevtx2es supports simultaneous import of multiple files.\n\n```bash\n$ evtx2es file1.evtx file2.evtx file3.evtx\n```\n\nAdditionally, it also allows for recursive import under the specified directory.\n\n```bash\n$ tree .\nevtxfiles/\n \u251c\u2500\u2500 file1.evtx\n \u251c\u2500\u2500 file2.evtx\n \u251c\u2500\u2500 file3.evtx\n \u2514\u2500\u2500 subdirectory/\n \u251c\u2500\u2500 file4.evtx\n \u2514\u2500\u2500 subsubdirectory/\n \u251c\u2500\u2500 file5.evtx\n \u2514\u2500\u2500 file6.evtx\n\n$ evtx2es /evtxfiles/ # The Path is recursively expanded to file1~6.evtx.\n```\n\n### Options\n\n```\n--version, -v\n\n--help, -h\n\n--quiet, -q\n Flag to suppress standard output\n (default: False)\n\n--multiprocess, -m:\n Enable multiprocessing for faster execution\n (default: False)\n\n--size:\n Chunk size for processing (default: 500)\n\n--host:\n ElasticSearch host address (default: localhost)\n\n--port:\n ElasticSearch port number (default: 9200)\n\n--index:\n Destination index name for importing (default: evtx2es)\n\n--scheme:\n Protocol scheme to use (http or https) (default: http)\n\n--pipeline:\n Elasticsearch Ingest Pipeline to use (default: )\n\n--datasetdate:\n Date of the latest record in the dataset, extracted from TimeCreated field (MM/DD/YYYY.HH:MM:SS) (default: 0)\n\n--login:\n The login to use if Elastic Security is enabled (default: )\n\n--pwd:\n The password associated with the provided login (default: )\n```\n\n### Examples\n\nWhen using from the commandline interface:\n\n```\n$ evtx2es /path/to/your/file.evtx --host=localhost --port=9200 --index=foobar --size=500\n```\n\nWhen using from the python-script:\n\n```py\nif __name__ == '__main__':\n evtx2es('/path/to/your/file.evtx', host=localhost, port=9200, index='foobar', size=500)\n```\n\nWith credentials for Elastic Security:\n\n```\n$ evtx2es /path/to/your/file.evtx --host=localhost --port=9200 --index=foobar --login=elastic --pwd=******\n```\n\nNote: The current version does not verify the certificate.\n\n\n## Appendix\n\n### Evtx2json\n\nAn additional feature: :sushi: :sushi: :sushi:\n\nConvert Windows Event Logs to a JSON file.\n\n```bash\n$ evtx2json /path/to/your/file.evtx /path/to/output/target.json\n```\n\nConvert Windows Event Logs to a Python List[dict] object.\n\n```python\nfrom evtx2es import evtx2json\n\nif __name__ == '__main__':\n filepath = '/path/to/your/file.evtx'\n result: List[dict] = evtx2json(filepath)\n```\n\n## Output Format Example\n\nUsing the sample evtx file of [JPCERT/CC:LogonTracer](https://github.com/JPCERTCC/LogonTracer) as an example.\n\n```\n[\n {\n \"event_record_id\": 227559,\n \"timestamp\": \"2016-10-06 01:50:49.420927 UTC\",\n \"winlog\": {\n \"channel\": \"Security\",\n \"computer_name\": \"WIN-WFBHIBE5GXZ.example.co.jp\",\n \"event_id\": 4624,\n \"opcode\": 0,\n \"provider_guid\": \"{54849625-5478-4994-a5ba-3e3b0328c30d}\",\n \"provider_name\": \"Microsoft-Windows-Security-Auditing\",\n \"record_id\": 227559,\n \"task\": 12544,\n \"version\": 0,\n \"process\": {\n \"pid\": 572,\n \"thread_id\": 1244\n },\n \"event_data\": {\n \"AuthenticationPackageName\": \"Kerberos\",\n \"IpAddress\": \"192.168.16.102\",\n \"IpPort\": \"49220\",\n \"KeyLength\": 0,\n \"LmPackageName\": \"-\",\n \"LogonGuid\": \"F4DC1C19-0544-BC52-0900-DFC19752C3C6\",\n \"LogonProcessName\": \"Kerberos\",\n \"LogonType\": 3,\n \"ProcessId\": 0,\n \"ProcessName\": \"-\",\n \"SubjectDomainName\": \"-\",\n \"SubjectLogonId\": \"0x0\",\n \"SubjectUserName\": \"-\",\n \"SubjectUserSid\": \"S-1-0-0\",\n \"TargetDomainName\": \"EXAMPLE\",\n \"TargetLogonId\": \"0x1fa0869\",\n \"TargetUserName\": \"WIN7_64JP_02$\",\n \"TargetUserSid\": \"S-1-5-21-1524084746-3249201829-3114449661-1107\",\n \"TransmittedServices\": \"-\",\n \"WorkstationName\": \"\",\n \"Status\": null\n }\n },\n \"log\": {\n \"file\": {\n \"name\": \"sample/Security.evtx\"\n }\n },\n \"event\": {\n \"code\": 4624,\n \"created\": \"2016-10-06T01:50:49.420927Z\"\n },\n \"@timestamp\": \"2016-10-06T01:50:49.420927Z\"\n },\n ...\n]\n```\n\n## Performance Evaluations\n\nevtx2es was evaluated using the sample evtx file of [JPCERT/CC:LogonTracer](https://github.com/JPCERTCC/LogonTracer) (about 30MB binary data).\n\n```.bash\n$ time evtx2es ./Security.evtx\n> 6.25 user 0.13 system 0:14.08 elapsed 45%CPU\n```\n\nSee [Qiita](https://qiita.com/sumeshi/items/cb2fbafe59c2c83e3085) for more information.\n\n### Running Environment\n\n```\nOS: Ubuntu 18.04\nCPU: Intel Core i5-6500\nRAM: DDR4 32GB\n```\n\nElasticSearch 7.4 was running on the Docker version(Official Image). \nhttps://hub.docker.com/_/elasticsearch\n\n## Installation\n\n### from PyPI\n\n```\n$ pip install evtx2es\n```\n\n### from GitHub Releases\n\nThe version compiled into a binary using Nuitka is also available for use.\n\n```bash\n$ chmod +x ./ntfsdump\n$ ./ntfsdump {{options...}}\n```\n\n```powershell\n> ntfsdump.exe {{options...}}\n```\n\n## Contributing\n\nThe source code for evtx2es is hosted at GitHub, and you may download, fork, and review it from this repository(https://github.com/sumeshi/evtx2es).\nPlease report issues and feature requests. :sushi: :sushi: :sushi:\n\n## License\n\nevtx2es is released under the [MIT](https://github.com/sumeshi/evtx2es/blob/master/LICENSE) License.\n\nPowered by following libraries:\n- [pyevtx-rs](https://github.com/omerbenamram/pyevtx-rs)\n- [Nuitka](https://github.com/Nuitka/Nuitka)\n\nInspired by [EvtxtoElk](https://github.com/dgunter/evtxtoelk).\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Fast import of Windows EventLogs(.evtx) into Elasticsearch.",
"version": "1.6.0",
"project_urls": {
"Homepage": "https://github.com/sumeshi/evtx2es",
"Repository": "https://github.com/sumeshi/evtx2es"
},
"split_keywords": [
"evtx",
"elasticsearch",
"json"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e12ac91b4384aa88c7260176e27536f151116f06ec7ca0debc4c8f4c8b99e734",
"md5": "8ebbebcb69c48832454359384aa2a8d0",
"sha256": "549d7682a2ecc1fc449f167bd69be78b168e3b0310fb51df4e19e1a40090fa66"
},
"downloads": -1,
"filename": "evtx2es-1.6.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8ebbebcb69c48832454359384aa2a8d0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11,<4.0",
"size": 13486,
"upload_time": "2023-11-26T03:10:00",
"upload_time_iso_8601": "2023-11-26T03:10:00.591464Z",
"url": "https://files.pythonhosted.org/packages/e1/2a/c91b4384aa88c7260176e27536f151116f06ec7ca0debc4c8f4c8b99e734/evtx2es-1.6.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3678ee4aaaf1fb114bf7e8e16b0320500ea6d3da128f05e62ce7d5053aa9b2c1",
"md5": "f2b8eb14438be79379e9aa798a546f40",
"sha256": "41e15b18123afecbedef53fcccc24fd94df74c518eb1ad262033e3b6a6dd85bb"
},
"downloads": -1,
"filename": "evtx2es-1.6.0.tar.gz",
"has_sig": false,
"md5_digest": "f2b8eb14438be79379e9aa798a546f40",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11,<4.0",
"size": 12077,
"upload_time": "2023-11-26T03:10:02",
"upload_time_iso_8601": "2023-11-26T03:10:02.806603Z",
"url": "https://files.pythonhosted.org/packages/36/78/ee4aaaf1fb114bf7e8e16b0320500ea6d3da128f05e62ce7d5053aa9b2c1/evtx2es-1.6.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-26 03:10:02",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "sumeshi",
"github_project": "evtx2es",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "evtx2es"
}
JSON
