pygrok |Build Status|
=====================
|Join the chat at https://gitter.im/garyelephant/pygrok|
A Python library to parse strings and extract information from
structured/unstructured data
What can I use Grok for?
------------------------
- parsing and matching patterns in a string(log, message etc.)
- relieving from complex regular expressions.
- extracting information from structured/unstructured data
Installation
------------
.. code:: Bash
$ pip install pygrok
or download, uncompress and install pygrok from
`here <https://github.com/garyelephant/pygrok/releases/latest>`__:
.. code:: Bash
$ tar zxvf pygrok-xx.tar.gz
$ cd pygrok_dir
$ sudo python setup.py install
Getting Started
---------------
.. code:: Python
from pygrok import Grok
text = 'gary is male, 25 years old and weighs 68.5 kilograms'
pattern = '%{WORD:name} is %{WORD:gender}, %{NUMBER:age} years old and weighs %{NUMBER:weight} kilograms'
grok = Grok(pattern)
print grok.match(text)
# {'gender': 'male', 'age': '25', 'name': 'gary', 'weight': '68.5'}
Pretty Cool !
Numbers can be converted from string to ``int`` or ``float`` if you use
``%{pattern:name:type}`` syntax, such as ``%{NUMBER:age:int}``
.. code:: Python
from pygrok import Grok
text = 'gary is male, 25 years old and weighs 68.5 kilograms'
pattern = '%{WORD:name} is %{WORD:gender}, %{NUMBER:age:int} years old and weighs %{NUMBER:weight:float} kilograms'
grok = Grok(pattern)
print grok.match(text, pattern)
# {'gender': 'male', 'age': 25, 'name': 'gary', 'weight': 68.5}
Now ``age`` is of type ``int`` and ``weight`` is of type ``float``.
Awesome !
Some of the pattern you can use are listed here:
::
`WORD` means \b\w+\b in regular expression.
`NUMBER` means (?:%{BASE10NUM})
`BASE10NUM` means (?<![0-9.+-])(?>[+-]?(?:(?:[0-9]+(?:\.[0-9]+)?)|(?:\.[0-9]+)))
other patterns such as `IP`, `HOSTNAME`, `URIPATH`, `DATE`, `TIMESTAMP_ISO8601`, `COMMONAPACHELOG`..
See All patterns `here <./pygrok/patterns>`__
You can also have custom pattern, see `these
codes <https://github.com/garyelephant/pygrok/blob/master/tests/test_pygrok.py#L97>`__.
More details
------------
Beause python re module does not support regular expression syntax
atomic grouping(?>),so pygrok requires
`regex <https://pypi.python.org/pypi/regex/2014.06.28>`__ to be
installed.
pygrok is inspired by `Grok <https://github.com/jordansissel/grok>`__
developed by Jordan Sissel. This is not a wrapper of Jordan Sissel's
Grok and totally implemented by me.
Grok is a simple software that allows you to easily parse strings, logs
and other files. With grok, you can turn unstructured log and event data
into structured data.Pygrok does the same thing.
I recommend you to have a look at `logstash filter
grok <https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html>`__,
it explains how Grok-like thing work.
pattern files come from `logstash filter grok's pattern
files <https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns>`__
Contribute
----------
- You are encouraged to
`fork <https://github.com/garyelephant/pygrok/fork>`__, improve the
code, then make a pull request.
- `Issue tracker <https://github.com/garyelephant/pygrok/issues>`__
Get Help
--------
::
mail:garygaowork@gmail.com
twitter:@garyelephant
Contributors
------------
Thanks to `all
contributors <https://github.com/garyelephant/pygrok/graphs/contributors>`__
.. |Build Status| image:: https://travis-ci.org/garyelephant/pygrok.svg?branch=master
:target: https://travis-ci.org/garyelephant/pygrok
.. |Join the chat at https://gitter.im/garyelephant/pygrok| image:: https://badges.gitter.im/Join%20Chat.svg
:target: https://gitter.im/garyelephant/pygrok?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge
Raw data
{
"_id": null,
"home_page": "https://github.com/garyelephant/pygrok",
"name": "pygrok",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "python grok,regex",
"author": "garyelephant",
"author_email": "garygaowork@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/ce/a5/963d78c4eda7edb0ea827679dbcf5f77e4d767562b59681bd23ea5913af6/pygrok-1.0.0.tar.gz",
"platform": "UNKNOWN",
"description": "pygrok |Build Status|\n=====================\n\n|Join the chat at https://gitter.im/garyelephant/pygrok|\n\nA Python library to parse strings and extract information from\nstructured/unstructured data\n\nWhat can I use Grok for?\n------------------------\n\n- parsing and matching patterns in a string(log, message etc.)\n- relieving from complex regular expressions.\n- extracting information from structured/unstructured data\n\nInstallation\n------------\n\n.. code:: Bash\n\n $ pip install pygrok\n\nor download, uncompress and install pygrok from\n`here <https://github.com/garyelephant/pygrok/releases/latest>`__:\n\n.. code:: Bash\n\n $ tar zxvf pygrok-xx.tar.gz\n $ cd pygrok_dir\n $ sudo python setup.py install\n\nGetting Started\n---------------\n\n.. code:: Python\n\n from pygrok import Grok\n text = 'gary is male, 25 years old and weighs 68.5 kilograms'\n pattern = '%{WORD:name} is %{WORD:gender}, %{NUMBER:age} years old and weighs %{NUMBER:weight} kilograms'\n grok = Grok(pattern)\n print grok.match(text)\n\n # {'gender': 'male', 'age': '25', 'name': 'gary', 'weight': '68.5'}\n\nPretty Cool !\n\nNumbers can be converted from string to ``int`` or ``float`` if you use\n``%{pattern:name:type}`` syntax, such as ``%{NUMBER:age:int}``\n\n.. code:: Python\n\n from pygrok import Grok\n text = 'gary is male, 25 years old and weighs 68.5 kilograms'\n pattern = '%{WORD:name} is %{WORD:gender}, %{NUMBER:age:int} years old and weighs %{NUMBER:weight:float} kilograms'\n grok = Grok(pattern)\n print grok.match(text, pattern)\n\n # {'gender': 'male', 'age': 25, 'name': 'gary', 'weight': 68.5}\n\nNow ``age`` is of type ``int`` and ``weight`` is of type ``float``.\n\nAwesome !\n\nSome of the pattern you can use are listed here:\n\n::\n\n `WORD` means \\b\\w+\\b in regular expression.\n `NUMBER` means (?:%{BASE10NUM})\n `BASE10NUM` means (?<![0-9.+-])(?>[+-]?(?:(?:[0-9]+(?:\\.[0-9]+)?)|(?:\\.[0-9]+)))\n\n other patterns such as `IP`, `HOSTNAME`, `URIPATH`, `DATE`, `TIMESTAMP_ISO8601`, `COMMONAPACHELOG`..\n\nSee All patterns `here <./pygrok/patterns>`__\n\nYou can also have custom pattern, see `these\ncodes <https://github.com/garyelephant/pygrok/blob/master/tests/test_pygrok.py#L97>`__.\n\nMore details\n------------\n\nBeause python re module does not support regular expression syntax\natomic grouping(?>),so pygrok requires\n`regex <https://pypi.python.org/pypi/regex/2014.06.28>`__ to be\ninstalled.\n\npygrok is inspired by `Grok <https://github.com/jordansissel/grok>`__\ndeveloped by Jordan Sissel. This is not a wrapper of Jordan Sissel's\nGrok and totally implemented by me.\n\nGrok is a simple software that allows you to easily parse strings, logs\nand other files. With grok, you can turn unstructured log and event data\ninto structured data.Pygrok does the same thing.\n\nI recommend you to have a look at `logstash filter\ngrok <https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html>`__,\nit explains how Grok-like thing work.\n\npattern files come from `logstash filter grok's pattern\nfiles <https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns>`__\n\nContribute\n----------\n\n- You are encouraged to\n `fork <https://github.com/garyelephant/pygrok/fork>`__, improve the\n code, then make a pull request.\n- `Issue tracker <https://github.com/garyelephant/pygrok/issues>`__\n\nGet Help\n--------\n\n::\n\n mail:garygaowork@gmail.com\n twitter:@garyelephant\n\nContributors\n------------\n\nThanks to `all\ncontributors <https://github.com/garyelephant/pygrok/graphs/contributors>`__\n\n.. |Build Status| image:: https://travis-ci.org/garyelephant/pygrok.svg?branch=master\n :target: https://travis-ci.org/garyelephant/pygrok\n.. |Join the chat at https://gitter.im/garyelephant/pygrok| image:: https://badges.gitter.im/Join%20Chat.svg\n :target: https://gitter.im/garyelephant/pygrok?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge",
"bugtrack_url": null,
"license": "MIT",
"summary": "A Python library to parse strings and extract information from structured/unstructured data",
"version": "1.0.0",
"split_keywords": [
"python grok",
"regex"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "cea5963d78c4eda7edb0ea827679dbcf5f77e4d767562b59681bd23ea5913af6",
"md5": "db490a696c46f4ffbd4057f5e2d4cd3f",
"sha256": "ae635e3c0ba0eab76aec9d86ae1bab70883e8e71505ec2d6cb8989e66f5810af"
},
"downloads": -1,
"filename": "pygrok-1.0.0.tar.gz",
"has_sig": false,
"md5_digest": "db490a696c46f4ffbd4057f5e2d4cd3f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 18997,
"upload_time": "2016-09-24T08:32:38",
"upload_time_iso_8601": "2016-09-24T08:32:38.164199Z",
"url": "https://files.pythonhosted.org/packages/ce/a5/963d78c4eda7edb0ea827679dbcf5f77e4d767562b59681bd23ea5913af6/pygrok-1.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2016-09-24 08:32:38",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "garyelephant",
"github_project": "pygrok",
"travis_ci": true,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "regex",
"specs": [
[
"==",
"2015.10.29"
]
]
}
],
"lcname": "pygrok"
}
JSON
