apertium-streamparser


Nameapertium-streamparser JSON
Version 5.0.2 PyPI version JSON
download
home_pagehttps://github.com/apertium/streamparser
SummaryPython library to parse Apertium stream format
upload_time2018-04-20 19:47:26
maintainer
docs_urlNone
authorSushain K. Cherivirala
requires_python>=3.4
licenseGPLv3+
keywords apertium parsing linguistics
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI
coveralls test coverage 
            # Apertium Streamparser

[![Build Status](https://travis-ci.org/apertium/streamparser.svg)](https://travis-ci.org/apertium/streamparser)
[![Coverage Status](https://coveralls.io/repos/github/apertium/streamparser/badge.svg?branch=master)](https://coveralls.io/github/apertium/streamparser?branch=master)
[![PyPI](https://img.shields.io/pypi/v/apertium-streamparser.svg)](https://pypi.org/project/apertium-streamparser/)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/apertium-streamparser.svg)]((https://pypi.org/project/apertium-streamparser/))
[![PyPI - Implementation](https://img.shields.io/pypi/implementation/apertium-streamparser.svg)]((https://pypi.org/project/apertium-streamparser/))

Python 3 library to parse [Apertium stream format][1], generating `LexicalUnit`s.

## Installation

Streamparser is available through [PyPi][2]:

    $ pip install apertium-streamparser
    $ apertium-streamparser
    $^vino/vino<n><m><sg>/venir<vblex><ifi><p3><sg>$
    [[SReading(baseform='vino', tags=['n', 'm', 'sg'])], [SReading(baseform='venir', tags=['vblex', 'ifi', 'p3', 'sg'])]]

Installation through PyPi will also install the `streamparser` module.

## Usage

### As a library

#### With string input

```python
>>> from streamparser import parse
>>> lexical_units = parse('^hypercholesterolemia/*hypercholesterolemia$\[\]\^\$[^ignoreme/yesreally$]^a\/s/a\/s<n><nt>$^vino/vino<n><m><sg>/venir<vblex><ifi><p3><sg>$.eefe^dímelo/decir<vblex><imp><p2><sg>+me<prn><enc><p1><mf><sg>+lo<prn><enc><p3><nt>/decir<vblex><imp><p2><sg>+me<prn><enc><p1><mf><sg>+lo<prn><enc><p3><m><sg>$')
>>> for lexical_unit in lexical_units:
        print('%s (%s) → %s' % (lexical_unit.wordform, lexical_unit.knownness, lexical_unit.readings))
```

    hypercholesterolemia (<class 'streamparser.unknown'>) → [[SReading(baseform='*hypercholesterolemia', tags=[])]]
    a\/s (<class 'streamparser.known'>) → [[SReading(baseform='a\\/s', tags=['n', 'nt'])]]
    vino (<class 'streamparser.known'>) → [[SReading(baseform='vino', tags=['n', 'm', 'sg'])], [SReading(baseform='venir', tags=['vblex', 'ifi', 'p3', 'sg'])]]
    dímelo (<class 'streamparser.known'>) → [[SReading(baseform='decir', tags=['vblex', 'imp', 'p2', 'sg']), SReading(baseform='me', tags=['prn', 'enc', 'p1', 'mf', 'sg']), SReading(baseform='lo', tags=['prn', 'enc', 'p3', 'nt'])], [SReading(baseform='decir', tags=['vblex', 'imp', 'p2', 'sg']), SReading(baseform='me', tags=['prn', 'enc', 'p1', 'mf', 'sg']), SReading(baseform='lo', tags=['prn', 'enc', 'p3', 'm', 'sg'])]]

#### With file input

```python
>>> from streamparser import parse_file
>>> lexical_units = parse_file(open('~/Downloads/analyzed.txt'))
>>> for lexical_unit in lexical_units:
        print('%s (%s) → %s' % (lexical_unit.wordform, lexical_unit.knownness, lexical_unit.readings))
```

    Høgre (<class 'streamparser.known'>) → [[SReading(baseform='Høgre', tags=['np'])], [SReading(baseform='høgre', tags=['n', 'nt', 'sp'])], [SReading(baseform='høg', tags=['un', 'sint', 'sp', 'comp', 'adj'])], [SReading(baseform='høgre', tags=['f', 'n', 'ind', 'sg'])], [SReading(baseform='høgre', tags=['f', 'n', 'ind', 'sg'])], [SReading(baseform='høgre', tags=['sg', 'nt', 'ind', 'posi', 'adj'])], [SReading(baseform='høgre', tags=['mf', 'sg', 'ind', 'posi', 'adj'])], [SReading(baseform='høgre', tags=['un', 'ind', 'pl', 'posi', 'adj'])], [SReading(baseform='høgre', tags=['un', 'def', 'sp', 'posi', 'adj'])]]
    kolonne (<class 'streamparser.known'>) → [[SReading(baseform='kolonne', tags=['m', 'n', 'ind', 'sg'])], [SReading(baseform='kolonne', tags=['m', 'n', 'ind', 'sg'])]]
    Grunnprinsipp (<class 'streamparser.known'>) → [[SReading(baseform='grunnprinsipp', tags=['n', 'nt', 'ind', 'sg'])], S[Reading(baseform='grunnprinsipp', tags=['n', 'nt', 'pl', 'ind'])], [SReading(baseform='grunnprinsipp', tags=['n', 'nt', 'ind', 'sg'])], [SReading(baseform='grunnprinsipp', tags=['n', 'nt', 'pl', 'ind'])]]
    7 (<class 'streamparser.known'>) → [[SReading(baseform='7', tags=['qnt', 'pl', 'det'])]]
    px (<class 'streamparser.unknown'>) → []

### From the terminal

#### With standard input

```bash
$ bzcat ~/corpora/nnclean2.txt.bz2 | apertium-deshtml | lt-proc -we /usr/share/apertium/apertium-nno/nno.automorf.bin | python3 streamparser.py
[[SReading(baseform='Høgre', tags=['np'])],
 [SReading(baseform='høgre', tags=['n', 'sp', 'nt'])],
 [SReading(baseform='høg', tags=['un', 'sp', 'adj', 'comp', 'sint'])],
 [SReading(baseform='høgre', tags=['n', 'f', 'ind', 'sg'])],
 [SReading(baseform='høgre', tags=['n', 'f', 'ind', 'sg'])],
 [SReading(baseform='høgre', tags=['posi', 'ind', 'adj', 'nt', 'sg'])],
 [SReading(baseform='høgre', tags=['posi', 'ind', 'adj', 'mf', 'sg'])],
 [SReading(baseform='høgre', tags=['posi', 'ind', 'adj', 'un', 'pl'])],
 [SReading(baseform='høgre', tags=['posi', 'def', 'sp', 'adj', 'un'])]]
[[SReading(baseform='kolonne', tags=['n', 'm', 'ind', 'sg'])],
 [SReading(baseform='kolonne', tags=['n', 'm', 'ind', 'sg'])]]
...
```

#### With file input in terminal

```bash
$ bzcat ~/corpora/nnclean2.txt.bz2 | apertium-deshtml | lt-proc -we /usr/share/apertium/apertium-nno/nno.automorf.bin > analyzed.txt
$ python3 streamparser.py analyzed.txt
[[SReading(baseform='Høgre', tags=['np'])],
 [SReading(baseform='høgre', tags=['n', 'sp', 'nt'])],
 [SReading(baseform='høg', tags=['un', 'sp', 'adj', 'comp', 'sint'])],
 [SReading(baseform='høgre', tags=['n', 'f', 'ind', 'sg'])],
 [SReading(baseform='høgre', tags=['n', 'f', 'ind', 'sg'])],
 [SReading(baseform='høgre', tags=['posi', 'ind', 'adj', 'nt', 'sg'])],
 [SReading(baseform='høgre', tags=['posi', 'ind', 'adj', 'mf', 'sg'])],
 [SReading(baseform='høgre', tags=['posi', 'ind', 'adj', 'un', 'pl'])],
 [SReading(baseform='høgre', tags=['posi', 'def', 'sp', 'adj', 'un'])]]
[[SReading(baseform='kolonne', tags=['n', 'm', 'ind', 'sg'])],
 [SReading(baseform='kolonne', tags=['n', 'm', 'ind', 'sg'])]]
...
```

## Contributing

Streamparser uses [TravisCI][3] for continous integration. Locally, use
`make test` to run the same checks it does. Use `pip install -r requirements.txt`
to install the requirements required for development, e.g. linters.

[1]: http://wiki.apertium.org/wiki/Apertium_stream_format
[2]: https://pypi.org/project/apertium-streamparser/
[3]: https://travis-ci.org/apertium/streamparser

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/apertium/streamparser",
    "name": "apertium-streamparser",
    "maintainer": "",
    "docs_url": null,
    "requires_python": ">=3.4",
    "maintainer_email": "",
    "keywords": "apertium parsing linguistics",
    "author": "Sushain K. Cherivirala",
    "author_email": "sushain@skc.name",
    "download_url": "https://files.pythonhosted.org/packages/6e/04/c85695308d203650dc0d9f550fbdc0e6a839364b7515d77a76f2e8e19de9/apertium-streamparser-5.0.2.tar.gz",
    "platform": "",
    "description": "# Apertium Streamparser\n\n[![Build Status](https://travis-ci.org/apertium/streamparser.svg)](https://travis-ci.org/apertium/streamparser)\n[![Coverage Status](https://coveralls.io/repos/github/apertium/streamparser/badge.svg?branch=master)](https://coveralls.io/github/apertium/streamparser?branch=master)\n[![PyPI](https://img.shields.io/pypi/v/apertium-streamparser.svg)](https://pypi.org/project/apertium-streamparser/)\n[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/apertium-streamparser.svg)]((https://pypi.org/project/apertium-streamparser/))\n[![PyPI - Implementation](https://img.shields.io/pypi/implementation/apertium-streamparser.svg)]((https://pypi.org/project/apertium-streamparser/))\n\nPython 3 library to parse [Apertium stream format][1], generating `LexicalUnit`s.\n\n## Installation\n\nStreamparser is available through [PyPi][2]:\n\n    $ pip install apertium-streamparser\n    $ apertium-streamparser\n    $^vino/vino<n><m><sg>/venir<vblex><ifi><p3><sg>$\n    [[SReading(baseform='vino', tags=['n', 'm', 'sg'])], [SReading(baseform='venir', tags=['vblex', 'ifi', 'p3', 'sg'])]]\n\nInstallation through PyPi will also install the `streamparser` module.\n\n## Usage\n\n### As a library\n\n#### With string input\n\n```python\n>>> from streamparser import parse\n>>> lexical_units = parse('^hypercholesterolemia/*hypercholesterolemia$\\[\\]\\^\\$[^ignoreme/yesreally$]^a\\/s/a\\/s<n><nt>$^vino/vino<n><m><sg>/venir<vblex><ifi><p3><sg>$.eefe^d\u00edmelo/decir<vblex><imp><p2><sg>+me<prn><enc><p1><mf><sg>+lo<prn><enc><p3><nt>/decir<vblex><imp><p2><sg>+me<prn><enc><p1><mf><sg>+lo<prn><enc><p3><m><sg>$')\n>>> for lexical_unit in lexical_units:\n        print('%s (%s) \u2192 %s' % (lexical_unit.wordform, lexical_unit.knownness, lexical_unit.readings))\n```\n\n    hypercholesterolemia (<class 'streamparser.unknown'>) \u2192 [[SReading(baseform='*hypercholesterolemia', tags=[])]]\n    a\\/s (<class 'streamparser.known'>) \u2192 [[SReading(baseform='a\\\\/s', tags=['n', 'nt'])]]\n    vino (<class 'streamparser.known'>) \u2192 [[SReading(baseform='vino', tags=['n', 'm', 'sg'])], [SReading(baseform='venir', tags=['vblex', 'ifi', 'p3', 'sg'])]]\n    d\u00edmelo (<class 'streamparser.known'>) \u2192 [[SReading(baseform='decir', tags=['vblex', 'imp', 'p2', 'sg']), SReading(baseform='me', tags=['prn', 'enc', 'p1', 'mf', 'sg']), SReading(baseform='lo', tags=['prn', 'enc', 'p3', 'nt'])], [SReading(baseform='decir', tags=['vblex', 'imp', 'p2', 'sg']), SReading(baseform='me', tags=['prn', 'enc', 'p1', 'mf', 'sg']), SReading(baseform='lo', tags=['prn', 'enc', 'p3', 'm', 'sg'])]]\n\n#### With file input\n\n```python\n>>> from streamparser import parse_file\n>>> lexical_units = parse_file(open('~/Downloads/analyzed.txt'))\n>>> for lexical_unit in lexical_units:\n        print('%s (%s) \u2192 %s' % (lexical_unit.wordform, lexical_unit.knownness, lexical_unit.readings))\n```\n\n    H\u00f8gre (<class 'streamparser.known'>) \u2192 [[SReading(baseform='H\u00f8gre', tags=['np'])], [SReading(baseform='h\u00f8gre', tags=['n', 'nt', 'sp'])], [SReading(baseform='h\u00f8g', tags=['un', 'sint', 'sp', 'comp', 'adj'])], [SReading(baseform='h\u00f8gre', tags=['f', 'n', 'ind', 'sg'])], [SReading(baseform='h\u00f8gre', tags=['f', 'n', 'ind', 'sg'])], [SReading(baseform='h\u00f8gre', tags=['sg', 'nt', 'ind', 'posi', 'adj'])], [SReading(baseform='h\u00f8gre', tags=['mf', 'sg', 'ind', 'posi', 'adj'])], [SReading(baseform='h\u00f8gre', tags=['un', 'ind', 'pl', 'posi', 'adj'])], [SReading(baseform='h\u00f8gre', tags=['un', 'def', 'sp', 'posi', 'adj'])]]\n    kolonne (<class 'streamparser.known'>) \u2192 [[SReading(baseform='kolonne', tags=['m', 'n', 'ind', 'sg'])], [SReading(baseform='kolonne', tags=['m', 'n', 'ind', 'sg'])]]\n    Grunnprinsipp (<class 'streamparser.known'>) \u2192 [[SReading(baseform='grunnprinsipp', tags=['n', 'nt', 'ind', 'sg'])], S[Reading(baseform='grunnprinsipp', tags=['n', 'nt', 'pl', 'ind'])], [SReading(baseform='grunnprinsipp', tags=['n', 'nt', 'ind', 'sg'])], [SReading(baseform='grunnprinsipp', tags=['n', 'nt', 'pl', 'ind'])]]\n    7 (<class 'streamparser.known'>) \u2192 [[SReading(baseform='7', tags=['qnt', 'pl', 'det'])]]\n    px (<class 'streamparser.unknown'>) \u2192 []\n\n### From the terminal\n\n#### With standard input\n\n```bash\n$ bzcat ~/corpora/nnclean2.txt.bz2 | apertium-deshtml | lt-proc -we /usr/share/apertium/apertium-nno/nno.automorf.bin | python3 streamparser.py\n[[SReading(baseform='H\u00f8gre', tags=['np'])],\n [SReading(baseform='h\u00f8gre', tags=['n', 'sp', 'nt'])],\n [SReading(baseform='h\u00f8g', tags=['un', 'sp', 'adj', 'comp', 'sint'])],\n [SReading(baseform='h\u00f8gre', tags=['n', 'f', 'ind', 'sg'])],\n [SReading(baseform='h\u00f8gre', tags=['n', 'f', 'ind', 'sg'])],\n [SReading(baseform='h\u00f8gre', tags=['posi', 'ind', 'adj', 'nt', 'sg'])],\n [SReading(baseform='h\u00f8gre', tags=['posi', 'ind', 'adj', 'mf', 'sg'])],\n [SReading(baseform='h\u00f8gre', tags=['posi', 'ind', 'adj', 'un', 'pl'])],\n [SReading(baseform='h\u00f8gre', tags=['posi', 'def', 'sp', 'adj', 'un'])]]\n[[SReading(baseform='kolonne', tags=['n', 'm', 'ind', 'sg'])],\n [SReading(baseform='kolonne', tags=['n', 'm', 'ind', 'sg'])]]\n...\n```\n\n#### With file input in terminal\n\n```bash\n$ bzcat ~/corpora/nnclean2.txt.bz2 | apertium-deshtml | lt-proc -we /usr/share/apertium/apertium-nno/nno.automorf.bin > analyzed.txt\n$ python3 streamparser.py analyzed.txt\n[[SReading(baseform='H\u00f8gre', tags=['np'])],\n [SReading(baseform='h\u00f8gre', tags=['n', 'sp', 'nt'])],\n [SReading(baseform='h\u00f8g', tags=['un', 'sp', 'adj', 'comp', 'sint'])],\n [SReading(baseform='h\u00f8gre', tags=['n', 'f', 'ind', 'sg'])],\n [SReading(baseform='h\u00f8gre', tags=['n', 'f', 'ind', 'sg'])],\n [SReading(baseform='h\u00f8gre', tags=['posi', 'ind', 'adj', 'nt', 'sg'])],\n [SReading(baseform='h\u00f8gre', tags=['posi', 'ind', 'adj', 'mf', 'sg'])],\n [SReading(baseform='h\u00f8gre', tags=['posi', 'ind', 'adj', 'un', 'pl'])],\n [SReading(baseform='h\u00f8gre', tags=['posi', 'def', 'sp', 'adj', 'un'])]]\n[[SReading(baseform='kolonne', tags=['n', 'm', 'ind', 'sg'])],\n [SReading(baseform='kolonne', tags=['n', 'm', 'ind', 'sg'])]]\n...\n```\n\n## Contributing\n\nStreamparser uses [TravisCI][3] for continous integration. Locally, use\n`make test` to run the same checks it does. Use `pip install -r requirements.txt`\nto install the requirements required for development, e.g. linters.\n\n[1]: http://wiki.apertium.org/wiki/Apertium_stream_format\n[2]: https://pypi.org/project/apertium-streamparser/\n[3]: https://travis-ci.org/apertium/streamparser",
    "bugtrack_url": null,
    "license": "GPLv3+",
    "summary": "Python library to parse Apertium stream format",
    "version": "5.0.2",
    "project_urls": {
        "Homepage": "https://github.com/apertium/streamparser"
    },
    "split_keywords": [
        "apertium",
        "parsing",
        "linguistics"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "ca4785027843345b1d7e4d0beca98c3c55ac8bb1b2d9069a126877b645be481c",
                "md5": "c78164e8f3a530bf1a53fc17888b4c04",
                "sha256": "e14e99f9a682725b6f8c0955f86d79319d7786d2e43b1dcaa50f4151b0410771"
            },
            "downloads": -1,
            "filename": "apertium_streamparser-5.0.2-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "c78164e8f3a530bf1a53fc17888b4c04",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.4",
            "size": 5659,
            "upload_time": "2018-04-20T19:48:02",
            "upload_time_iso_8601": "2018-04-20T19:48:02.726519Z",
            "url": "https://files.pythonhosted.org/packages/ca/47/85027843345b1d7e4d0beca98c3c55ac8bb1b2d9069a126877b645be481c/apertium_streamparser-5.0.2-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "6e04c85695308d203650dc0d9f550fbdc0e6a839364b7515d77a76f2e8e19de9",
                "md5": "bfd1392e541c7e51d4a69995c9277bc0",
                "sha256": "fd83d3d573d23c54b34339865cdd40cded3687311c18629d2d39c4e8ad1da597"
            },
            "downloads": -1,
            "filename": "apertium-streamparser-5.0.2.tar.gz",
            "has_sig": false,
            "md5_digest": "bfd1392e541c7e51d4a69995c9277bc0",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.4",
            "size": 19285,
            "upload_time": "2018-04-20T19:47:26",
            "upload_time_iso_8601": "2018-04-20T19:47:26.086199Z",
            "url": "https://files.pythonhosted.org/packages/6e/04/c85695308d203650dc0d9f550fbdc0e6a839364b7515d77a76f2e8e19de9/apertium-streamparser-5.0.2.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2018-04-20 19:47:26",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "apertium",
    "github_project": "streamparser",
    "travis_ci": true,
    "coveralls": true,
    "github_actions": false,
    "lcname": "apertium-streamparser"
}
Sushain K. Cherivirala
Elapsed time: 0.07176s