[](https://pepy.tech/project/ttp)
[](https://pypi.python.org/pypi/ttp/)
[](http://ttp.readthedocs.io/?badge=latest)
# Template Text Parser
TTP is a Python library for semi-structured text parsing using templates.
## Why?
To save ones time on transforming raw text into structured data and beyond.
## How?
Regexes, regexes everywhere... but, dynamically formed out of TTP templates with added capabilities to simplify the process of getting desired outcome.
## What?
In essence TTP can help to:
- Prepare, sort and load text data for parsing
- Parse text using regexes dynamically derived out of templates
- Process matches on the fly using broad set of built-in or custom functions
- Combine match results in a structure with arbitrary hierarchy
- Transform results in desired format to ease consumption by humans or machines
- Return results to various destinations for storage or further processing
Reference [documentation](https://ttp.readthedocs.io) for more information.
TTP [Networktocode Slack channel](https://networktocode.slack.com/archives/C018HMJQECB)
Collection of [TTP Templates](https://github.com/dmulyalin/ttp_templates)
## Example - as simple as it can be
Simple interfaces configuration parsing example
<details><summary>Code</summary>
```python
from ttp import ttp
import pprint
data = """
interface Loopback0
description Router-id-loopback
ip address 192.168.0.113/24
!
interface Vlan778
description CPE_Acces_Vlan
ip address 2002::fd37/124
ip vrf CPE1
!
"""
template = """
interface {{ interface }}
ip address {{ ip }}/{{ mask }}
description {{ description }}
ip vrf {{ vrf }}
"""
parser = ttp(data, template)
parser.parse()
pprint.pprint(parser.result(), width=100)
# prints:
# [[[{'description': 'Router-id-loopback',
# 'interface': 'Loopback0',
# 'ip': '192.168.0.113',
# 'mask': '24'},
# {'description': 'CPE_Acces_Vlan',
# 'interface': 'Vlan778',
# 'ip': '2002::fd37',
# 'mask': '124',
# 'vrf': 'CPE1'}]]]
```
</details>
## Example - a bit more complicated
For this example lets say we want to parse BGP peerings output, but combine state with configuration data, at the end we want to get pretty looking text table printed to screen.
<details><summary>Code</summary>
```python
template="""
<doc>
This template first parses "show bgp vrf CUST-1 vpnv4 unicast summary" commands
output, forming results for "bgp_state" dictionary, where peer ip is a key.
Following that, "show run | section bgp" output parsed by group "bgp_cfg". That
group uses nested groups to form results structure, including absolute path
"/bgp_peers*" with path formatter to produce a list of peers under "bgp_peers"
path.
For each peer "hostname" and local bgp "local_asn" added using previous matches.
Additionally, group lookup function used to lookup peer state from "bgp_state"
group results, adding found data to peer results.
Finally, "bgp_peers" section of results passed via "tabulate_outputter" to
from and print this table to terminal:
hostname local_asn vrf_name peer_ip peer_asn uptime state description afi rpl_in rpl_out
----------------- ----------- ---------- --------- ---------- -------- ------- ------------- ----- --------------- ---------------
ucs-core-switch-1 65100 CUST-1 192.0.2.1 65101 00:12:33 300 peer-1 ipv4 RPL-1-IMPORT-v4 RPL-1-EXPORT-V4
ucs-core-switch-1 65100 CUST-1 192.0.2.2 65102 03:55:01 idle peer-2 ipv4 RPL-2-IMPORT-V6 RPL-2-EXPORT-V6
Run this script with "python filename.py"
</doc>
<vars>
hostname="gethostname"
chain_1 = [
"set('vrf_name')",
"lookup('peer_ip', group='bgp_state', update=True)"
]
</vars>
<group name="bgp_state.{{ peer }}" input="bgp_state">
{{ peer }} 4 65101 20 21 43 0 0 {{ uptime }} {{ state }}
</group>
<group name="bgp_cfg" input="bgp_config">
router bgp {{ asn | record(asn) }}
<group name="vrfs.{{ vrf_name }}" record="vrf_name">
vrf {{ vrf_name }}
<group name="/bgp_peers*" chain="chain_1">
neighbor {{ peer_ip }}
{{ local_asn | set(asn) }}
{{ hostname | set(hostname) }}
remote-as {{ peer_asn }}
description {{ description }}
address-family {{ afi }} unicast
route-map {{ rpl_in }} in
route-map {{ rpl_out }} out
</group>
</group>
</group>
<output
name="tabulate_outputter"
format="tabulate"
path="bgp_peers"
returner="terminal"
headers="hostname, local_asn, vrf_name, peer_ip, peer_asn, uptime, state, description, afi, rpl_in, rpl_out"
/>
"""
data_bgp_state = """
ucs-core-switch-1#show bgp vrf CUST-1 vpnv4 unicast summary
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
192.0.2.1 4 65101 32 54 42 0 0 00:12:33 300
192.0.2.2 4 65101 11 45 99 0 0 03:55:01 idle
"""
data_bgp_config = """
ucs-core-switch-1#show run | section bgp
router bgp 65100
vrf CUST-1
neighbor 192.0.2.1
remote-as 65101
description peer-1
address-family ipv4 unicast
route-map RPL-1-IMPORT-v4 in
route-map RPL-1-EXPORT-V4 out
neighbor 192.0.2.2
remote-as 65102
description peer-2
address-family ipv4 unicast
route-map RPL-2-IMPORT-V6 in
route-map RPL-2-EXPORT-V6 out
"""
from ttp import ttp
parser = ttp()
parser.add_template(template)
parser.add_input(data=data_bgp_state, input_name="bgp_state")
parser.add_input(data=data_bgp_config, input_name="bgp_config")
parser.parse()
```
</details>
# Contributions
Feel free to submit an issue, report a bug or ask a question, feature requests are welcomed. Or [buy](https://paypal.me/dmulyalin) Author a coffee
# Additional resources
List of additional resources:
- Sandbox to test TTP templates - http://textfsm.nornir.tech/ by [tbotnz](https://github.com/tbotnz)
- Videos on TTP - https://pynet.twb-tech.com/videos/ttp/ttp.html by [Kirk Byers](https://github.com/ktbyers)
Raw data
{
"_id": null,
"home_page": "https://github.com/dmulyalin/ttp",
"name": "ttp",
"maintainer": "Denis Mulyalin",
"docs_url": null,
"requires_python": ">=2.7,<4.0",
"maintainer_email": "d.mulyalin@gmail.com",
"keywords": "Parsing,TTP,regex",
"author": "Denis Mulyalin",
"author_email": "d.mulyalin@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/6e/4f/b95e48d3b653e1b1494a3a5b15614db5a340826282c5389e38cb2248da39/ttp-0.9.5.tar.gz",
"platform": null,
"description": "[](https://pepy.tech/project/ttp)\n[](https://pypi.python.org/pypi/ttp/)\n[](http://ttp.readthedocs.io/?badge=latest)\n\n# Template Text Parser\n\nTTP is a Python library for semi-structured text parsing using templates.\n\n## Why?\n\nTo save ones time on transforming raw text into structured data and beyond.\n\n## How?\n\nRegexes, regexes everywhere... but, dynamically formed out of TTP templates with added capabilities to simplify the process of getting desired outcome.\n\n## What?\n\nIn essence TTP can help to:\n - Prepare, sort and load text data for parsing\n - Parse text using regexes dynamically derived out of templates\n - Process matches on the fly using broad set of built-in or custom functions\n - Combine match results in a structure with arbitrary hierarchy\n - Transform results in desired format to ease consumption by humans or machines\n - Return results to various destinations for storage or further processing\n\nReference [documentation](https://ttp.readthedocs.io) for more information.\n\nTTP [Networktocode Slack channel](https://networktocode.slack.com/archives/C018HMJQECB)\n\nCollection of [TTP Templates](https://github.com/dmulyalin/ttp_templates)\n\n## Example - as simple as it can be\n\nSimple interfaces configuration parsing example\n\n<details><summary>Code</summary>\n\n```python\nfrom ttp import ttp\nimport pprint\n\ndata = \"\"\"\ninterface Loopback0\n description Router-id-loopback\n ip address 192.168.0.113/24\n!\ninterface Vlan778\n description CPE_Acces_Vlan\n ip address 2002::fd37/124\n ip vrf CPE1\n!\n\"\"\"\n\ntemplate = \"\"\"\ninterface {{ interface }}\n ip address {{ ip }}/{{ mask }}\n description {{ description }}\n ip vrf {{ vrf }}\n\"\"\"\n\nparser = ttp(data, template)\nparser.parse()\npprint.pprint(parser.result(), width=100)\n\n# prints:\n# [[[{'description': 'Router-id-loopback',\n# 'interface': 'Loopback0',\n# 'ip': '192.168.0.113',\n# 'mask': '24'},\n# {'description': 'CPE_Acces_Vlan',\n# 'interface': 'Vlan778',\n# 'ip': '2002::fd37',\n# 'mask': '124',\n# 'vrf': 'CPE1'}]]]\n```\n</details>\n\n## Example - a bit more complicated\n\nFor this example lets say we want to parse BGP peerings output, but combine state with configuration data, at the end we want to get pretty looking text table printed to screen.\n\n<details><summary>Code</summary>\n\n```python\ntemplate=\"\"\"\n<doc>\nThis template first parses \"show bgp vrf CUST-1 vpnv4 unicast summary\" commands\noutput, forming results for \"bgp_state\" dictionary, where peer ip is a key.\n\nFollowing that, \"show run | section bgp\" output parsed by group \"bgp_cfg\". That\ngroup uses nested groups to form results structure, including absolute path\n\"/bgp_peers*\" with path formatter to produce a list of peers under \"bgp_peers\"\npath.\n\nFor each peer \"hostname\" and local bgp \"local_asn\" added using previous matches.\nAdditionally, group lookup function used to lookup peer state from \"bgp_state\"\ngroup results, adding found data to peer results.\n\nFinally, \"bgp_peers\" section of results passed via \"tabulate_outputter\" to\nfrom and print this table to terminal:\n\nhostname local_asn vrf_name peer_ip peer_asn uptime state description afi rpl_in rpl_out\n----------------- ----------- ---------- --------- ---------- -------- ------- ------------- ----- --------------- ---------------\nucs-core-switch-1 65100 CUST-1 192.0.2.1 65101 00:12:33 300 peer-1 ipv4 RPL-1-IMPORT-v4 RPL-1-EXPORT-V4\nucs-core-switch-1 65100 CUST-1 192.0.2.2 65102 03:55:01 idle peer-2 ipv4 RPL-2-IMPORT-V6 RPL-2-EXPORT-V6\n\nRun this script with \"python filename.py\"\n</doc>\n\n<vars>\nhostname=\"gethostname\"\nchain_1 = [\n \"set('vrf_name')\",\n \"lookup('peer_ip', group='bgp_state', update=True)\"\n]\n</vars>\n\n<group name=\"bgp_state.{{ peer }}\" input=\"bgp_state\">\n{{ peer }} 4 65101 20 21 43 0 0 {{ uptime }} {{ state }}\n</group>\n\n<group name=\"bgp_cfg\" input=\"bgp_config\">\nrouter bgp {{ asn | record(asn) }}\n <group name=\"vrfs.{{ vrf_name }}\" record=\"vrf_name\">\n vrf {{ vrf_name }}\n <group name=\"/bgp_peers*\" chain=\"chain_1\">\n neighbor {{ peer_ip }}\n {{ local_asn | set(asn) }}\n {{ hostname | set(hostname) }}\n remote-as {{ peer_asn }}\n description {{ description }}\n address-family {{ afi }} unicast\n route-map {{ rpl_in }} in\n route-map {{ rpl_out }} out\n\t</group>\n </group>\n</group>\n\n<output\nname=\"tabulate_outputter\"\nformat=\"tabulate\"\npath=\"bgp_peers\"\nreturner=\"terminal\"\nheaders=\"hostname, local_asn, vrf_name, peer_ip, peer_asn, uptime, state, description, afi, rpl_in, rpl_out\"\n/>\n\"\"\"\n\ndata_bgp_state = \"\"\"\nucs-core-switch-1#show bgp vrf CUST-1 vpnv4 unicast summary\nNeighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd\n192.0.2.1 4 65101 32 54 42 0 0 00:12:33 300\n192.0.2.2 4 65101 11 45 99 0 0 03:55:01 idle\n\"\"\"\n\ndata_bgp_config = \"\"\"\nucs-core-switch-1#show run | section bgp\nrouter bgp 65100\n vrf CUST-1\n neighbor 192.0.2.1\n remote-as 65101\n description peer-1\n address-family ipv4 unicast\n route-map RPL-1-IMPORT-v4 in\n route-map RPL-1-EXPORT-V4 out\n neighbor 192.0.2.2\n remote-as 65102\n description peer-2\n address-family ipv4 unicast\n route-map RPL-2-IMPORT-V6 in\n route-map RPL-2-EXPORT-V6 out\n\"\"\"\n\nfrom ttp import ttp\n\nparser = ttp()\nparser.add_template(template)\nparser.add_input(data=data_bgp_state, input_name=\"bgp_state\")\nparser.add_input(data=data_bgp_config, input_name=\"bgp_config\")\nparser.parse()\n```\n</details>\n\n# Contributions\nFeel free to submit an issue, report a bug or ask a question, feature requests are welcomed. Or [buy](https://paypal.me/dmulyalin) Author a coffee\n\n# Additional resources\n\nList of additional resources:\n\n- Sandbox to test TTP templates - http://textfsm.nornir.tech/ by [tbotnz](https://github.com/tbotnz)\n- Videos on TTP - https://pynet.twb-tech.com/videos/ttp/ttp.html by [Kirk Byers](https://github.com/ktbyers)\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Template Text Parser",
"version": "0.9.5",
"project_urls": {
"Documentation": "https://ttp.readthedocs.io/",
"Homepage": "https://github.com/dmulyalin/ttp",
"Repository": "https://github.com/dmulyalin/ttp"
},
"split_keywords": [
"parsing",
"ttp",
"regex"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0338d475e7bde8d192ca5d64ccc0988f3d58f36211bd68c32b6c5883332a8abf",
"md5": "b0a3431d7dc88c94c633de453a71230a",
"sha256": "2c9fcf560b3f696e9fdd3554dc8e4622cbb10cac1d4fca13a7cf608c4a7fd137"
},
"downloads": -1,
"filename": "ttp-0.9.5-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "b0a3431d7dc88c94c633de453a71230a",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=2.7,<4.0",
"size": 85763,
"upload_time": "2023-06-25T00:28:11",
"upload_time_iso_8601": "2023-06-25T00:28:11.949380Z",
"url": "https://files.pythonhosted.org/packages/03/38/d475e7bde8d192ca5d64ccc0988f3d58f36211bd68c32b6c5883332a8abf/ttp-0.9.5-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "6e4fb95e48d3b653e1b1494a3a5b15614db5a340826282c5389e38cb2248da39",
"md5": "43b9fc3bf54caa4f7cfffc96b36dca24",
"sha256": "234414f4d3039d2d1cde09993f89f8db1b34d447f76c6a402555cefac2e59c4e"
},
"downloads": -1,
"filename": "ttp-0.9.5.tar.gz",
"has_sig": false,
"md5_digest": "43b9fc3bf54caa4f7cfffc96b36dca24",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=2.7,<4.0",
"size": 69141,
"upload_time": "2023-06-25T00:28:14",
"upload_time_iso_8601": "2023-06-25T00:28:14.036976Z",
"url": "https://files.pythonhosted.org/packages/6e/4f/b95e48d3b653e1b1494a3a5b15614db5a340826282c5389e38cb2248da39/ttp-0.9.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-25 00:28:14",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "dmulyalin",
"github_project": "ttp",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "ttp"
}
JSON
