| Name | parsetc JSON |
| Version |
0.4.1
JSON |
| download |
| home_page | |
| Summary | Parse and convert between different Teochew phonetic spelling schemes. |
| upload_time | 2024-02-18 10:03:33 |
| maintainer | |
| docs_url | None |
| author | Brandon Seah |
| requires_python | >=3.6 |
| license | The MIT License (MIT) Copyright (c) 2023 Brandon Seah Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
chinese
teochew
romanization
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
Parsing and conversion of Teochew Chinese romanizations
=======================================================
[](https://pypi.python.org/pypi/parsetc/)
Parse and convert between different Teochew romanized spelling schemes.
`parsetc` represents romanization schemes as context-free grammars, and
implements parsers and translators using the
[`lark`](https://lark-parser.readthedocs.io/en/stable/index.html) Python
library. The aim is to represent romanized text as an abstract parse tree
representing the phonology, which can then be converted to other romanization
schemes in a consistent, rule-based way.
Available romanization schemes for Teochew
------------------------------------------
### Input
* Geng'dang Pêng'im 廣東拼音 (`gdpi`)
* Gaginang Peng'im 家己儂拼音 (`ggn`)
* Gaginang Peng'im with coda `-n` allowed (nasalization written with `ñ`
instead) (`ggnn`)
* Dieghv 潮語 (`dieghv`);
[(source)](https://kahaani.github.io/gatian/appendix1/index.html)
* Tie-tsiann-hue 潮正會, also known as Tie-lo 潮羅 (`tlo`);
[(source)](http://library.hiteo.pw/book/wagpzbkv.html)
* Duffus system (`duffus`), also known as Pe̍h-ūe-jī (PUJ);
[(source)](https://archive.org/details/englishchinesev00duffgoog)
### Output
`gdpi`, `ggnn`, `tlo`, `duffus`, `dieghv`, plus:
* Teochew Sinwenz (`sinwz`);
[(source)](http://eresources.nlb.gov.sg/newspapers/Digitised/Page/nysp19391115-1.1.22)
* Traditional initial-final categories (`15`) from 《彙集雅俗通十五音》 also
known as 《擊木知音》, based on the analysis by 徐宇航
「《擊木知音》音系之再研究」 (2014)
Other languages
---------------
This tool was originally developed for Teochew, but can be extended in the
future to other dialects or languages. An experimental Cantonese module
(`--language Cantonese`) with Jyutping (`jp`) and Cantonese Pinyin (`cpy`) is
available.
Orthographic requirements for input text
----------------------------------------
* Text must be in lower case, otherwise will be converted to lower case
* Syllables may be written with or without tone numbers
* Syllables may be combined into words for legibility, i.e. [word-segmented
writing](https://en.wikipedia.org/wiki/Chinese_word-segmented_writing)
* If syllables are combined into words, they must either have tone numbers
(e.g. `diê5ziu1`), or use a syllable separator character (e.g. `diê-ziu` or
`pêng'im`, or `diê5-ziu1`). The separator character is either a hyphen or
single apostrophe. This is because of ambiguous parsings, e.g. `pê-ngi-m`
instead of `pêng-im`, which in general can only be dealt with by usage
frequency, which is not available.
* If syllable separator characters are used, they must be used consistently.
Mixing conventions may cause unexpected parsing errors.
Installation
------------
`parsetc` requires Python 3 and [`lark`](https://lark-parser.readthedocs.io/en/latest/) v1.1.
Install latest release with `pip` from PyPI:
```bash
pip install parsetc
```
If you are interested in latest development version, you can clone this
repository and checkout the `dev` branch, then install with `pip` from source
code:
```bash
pip install .
```
See help message:
```bash
parsetc --help
```
View available input and output schemes for Teochew:
```bash
parsetc --language Teochew --show_options
```
Usage
-----
### Command line tool
Input text is read line-by-line from STDIN. Output is written to STDOUT.
The language (`--language` or `-l`) is `Teochew` by default.
```bash
# Convert to Tie-lo
echo 'ua2 ain3 oh8 diê5ghe2, ain3 dan3 diê5ziu1 uê7.' | parsetc -i gdpi -o tlo
# Convert to all available output romanizations
echo 'ua2 ain3 oh8 diê5ghe2, ain3 dan3 diê5ziu1 uê7.' | parsetc -i gdpi --all
# Show parse tree (useful for debugging)
echo 'ua2 ain3 oh8 diê5ghe2, ain3 dan3 diê5ziu1 uê7.' | parsetc -i gdpi -p
```
Testing with sample texts in the `examples/` folder:
```bash
# Example with tone numbers but no syllable separators
cat examples/teochew.dieghv.tones.txt | parsetc -i dieghv --all
# Example with hyphens as syllable separators
cat examples/teochew.dieghv.sep.txt | parsetc -i dieghv --all
```
Try a Cantonese example (work in progress):
```bash
echo "ceon1 min4 bat1 gok3 hiu2" | parsetc -l Cantonese -i jp --all
cat examples/cantonese.cpy.txt | parsetc -l Cantonese -i cpy --all
```
### Python module
[ Work in progress. Updated docs coming soon... ]
Common functions and command line script are in the `parsetc.parsetc`
submodule. Each language has a dedicated subpackage with the following
structure (using Teochew as an example):
```
src/parsetc/Teochew/
├── shared.lark # lark grammar for shared parser rules
├── extends.json # lark %extend statements specific to individual schemes
├── terminals.json # dictionary of terminals for each scheme
├── mergers.json # dictionary of phonological mergers specific to individual schemes
├── parser.py # preprocessing and parser functions
└── translit.py # Translator classes
```
The lark rules and json files are the data required for the parsing and
translation functions in `parser.py` and `translit.py`.
Q & A
-----
* Q: Can I convert Chinese characters to romanization with `parsetc`?
* A: No, it is for converting between different romanization systems only.
* Q: `parsetc` makes a mistake, or it crashes when parsing a text. Can you fix it?
* A: Please check first if you are using the latest version of `parsetc` (run
`parsetc --version`). If the error still persists, please file a bug report
on the Github [issues](https://github.com/learn-teochew/parsetc/issues) page,
with the input text that caused the crash/error.
* Q: Can I contribute code to this project?
* A: You are welcome to look into the source code and fork/modfiy it. However,
I currently do not have the capacity to review pull requests and
contributions in detail, so please understand if I decline them or take a
long time to respond.
Related projects
----------------
### From Learn-Teochew
We use `parsetc` to convert Teochew opera transcriptions into different
romanization systems for the [Learn Teochew
Opera](https://learn-teochew.github.io/tc-opera/transcriptions) website.
How to type the special diacritics like `ṳ, o̍, o͘` in some romanization systems?
If you use MacOS X, check out our [custom keyboard
layouts](https://github.com/learn-teochew/POJ-variants-keyboard)
### Others
* [taibun](https://github.com/andreihar/taibun) - Transliterator and tokenizer
for Taiwanese Hokkien by Andrei Harbachov
Raw data
{
"_id": null,
"home_page": "",
"name": "parsetc",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "Chinese,Teochew,romanization",
"author": "Brandon Seah",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/06/d8/bb7aa7ceb33033dd26a6d13acf140ee58efef969e989c7d03511aeef8395/parsetc-0.4.1.tar.gz",
"platform": null,
"description": "Parsing and conversion of Teochew Chinese romanizations\n=======================================================\n\n[](https://pypi.python.org/pypi/parsetc/)\n\n\nParse and convert between different Teochew romanized spelling schemes.\n\n`parsetc` represents romanization schemes as context-free grammars, and\nimplements parsers and translators using the\n[`lark`](https://lark-parser.readthedocs.io/en/stable/index.html) Python\nlibrary. The aim is to represent romanized text as an abstract parse tree\nrepresenting the phonology, which can then be converted to other romanization\nschemes in a consistent, rule-based way.\n\n\nAvailable romanization schemes for Teochew\n------------------------------------------\n\n### Input\n\n * Geng'dang P\u00eang'im \u5ee3\u6771\u62fc\u97f3 (`gdpi`)\n * Gaginang Peng'im \u5bb6\u5df1\u5102\u62fc\u97f3 (`ggn`)\n * Gaginang Peng'im with coda `-n` allowed (nasalization written with `\u00f1`\n instead) (`ggnn`)\n * Dieghv \u6f6e\u8a9e (`dieghv`);\n [(source)](https://kahaani.github.io/gatian/appendix1/index.html)\n * Tie-tsiann-hue \u6f6e\u6b63\u6703, also known as Tie-lo \u6f6e\u7f85 (`tlo`);\n [(source)](http://library.hiteo.pw/book/wagpzbkv.html)\n * Duffus system (`duffus`), also known as Pe\u030dh-\u016be-j\u012b (PUJ);\n [(source)](https://archive.org/details/englishchinesev00duffgoog)\n\n\n### Output\n\n`gdpi`, `ggnn`, `tlo`, `duffus`, `dieghv`, plus:\n\n * Teochew Sinwenz (`sinwz`);\n [(source)](http://eresources.nlb.gov.sg/newspapers/Digitised/Page/nysp19391115-1.1.22)\n * Traditional initial-final categories (`15`) from \u300a\u5f59\u96c6\u96c5\u4fd7\u901a\u5341\u4e94\u97f3\u300b also\n known as \u300a\u64ca\u6728\u77e5\u97f3\u300b, based on the analysis by \u5f90\u5b87\u822a\n \u300c\u300a\u64ca\u6728\u77e5\u97f3\u300b\u97f3\u7cfb\u4e4b\u518d\u7814\u7a76\u300d (2014)\n\n\nOther languages\n---------------\n\nThis tool was originally developed for Teochew, but can be extended in the\nfuture to other dialects or languages. An experimental Cantonese module\n(`--language Cantonese`) with Jyutping (`jp`) and Cantonese Pinyin (`cpy`) is\navailable.\n\n\nOrthographic requirements for input text\n----------------------------------------\n\n * Text must be in lower case, otherwise will be converted to lower case\n * Syllables may be written with or without tone numbers\n * Syllables may be combined into words for legibility, i.e. [word-segmented\n writing](https://en.wikipedia.org/wiki/Chinese_word-segmented_writing)\n * If syllables are combined into words, they must either have tone numbers\n (e.g. `di\u00ea5ziu1`), or use a syllable separator character (e.g. `di\u00ea-ziu` or\n `p\u00eang'im`, or `di\u00ea5-ziu1`). The separator character is either a hyphen or\n single apostrophe. This is because of ambiguous parsings, e.g. `p\u00ea-ngi-m`\n instead of `p\u00eang-im`, which in general can only be dealt with by usage\n frequency, which is not available.\n * If syllable separator characters are used, they must be used consistently.\n Mixing conventions may cause unexpected parsing errors.\n\n\nInstallation\n------------\n\n`parsetc` requires Python 3 and [`lark`](https://lark-parser.readthedocs.io/en/latest/) v1.1.\n\nInstall latest release with `pip` from PyPI:\n\n```bash\npip install parsetc\n```\n\nIf you are interested in latest development version, you can clone this\nrepository and checkout the `dev` branch, then install with `pip` from source\ncode:\n\n```bash\npip install .\n```\n\nSee help message:\n\n```bash\nparsetc --help\n```\n\nView available input and output schemes for Teochew:\n\n```bash\nparsetc --language Teochew --show_options\n```\n\n\nUsage\n-----\n\n### Command line tool\n\nInput text is read line-by-line from STDIN. Output is written to STDOUT.\n\nThe language (`--language` or `-l`) is `Teochew` by default.\n\n```bash\n# Convert to Tie-lo\necho 'ua2 ain3 oh8 di\u00ea5ghe2, ain3 dan3 di\u00ea5ziu1 u\u00ea7.' | parsetc -i gdpi -o tlo\n# Convert to all available output romanizations\necho 'ua2 ain3 oh8 di\u00ea5ghe2, ain3 dan3 di\u00ea5ziu1 u\u00ea7.' | parsetc -i gdpi --all\n# Show parse tree (useful for debugging)\necho 'ua2 ain3 oh8 di\u00ea5ghe2, ain3 dan3 di\u00ea5ziu1 u\u00ea7.' | parsetc -i gdpi -p\n```\n\nTesting with sample texts in the `examples/` folder:\n\n```bash\n# Example with tone numbers but no syllable separators\ncat examples/teochew.dieghv.tones.txt | parsetc -i dieghv --all\n# Example with hyphens as syllable separators\ncat examples/teochew.dieghv.sep.txt | parsetc -i dieghv --all\n```\n\nTry a Cantonese example (work in progress):\n\n```bash\necho \"ceon1 min4 bat1 gok3 hiu2\" | parsetc -l Cantonese -i jp --all\ncat examples/cantonese.cpy.txt | parsetc -l Cantonese -i cpy --all\n```\n\n\n### Python module\n\n[ Work in progress. Updated docs coming soon... ]\n\nCommon functions and command line script are in the `parsetc.parsetc`\nsubmodule. Each language has a dedicated subpackage with the following\nstructure (using Teochew as an example):\n\n```\nsrc/parsetc/Teochew/\n\u251c\u2500\u2500 shared.lark # lark grammar for shared parser rules\n\u251c\u2500\u2500 extends.json # lark %extend statements specific to individual schemes\n\u251c\u2500\u2500 terminals.json # dictionary of terminals for each scheme\n\u251c\u2500\u2500 mergers.json # dictionary of phonological mergers specific to individual schemes\n\u251c\u2500\u2500 parser.py # preprocessing and parser functions\n\u2514\u2500\u2500 translit.py # Translator classes\n```\n\nThe lark rules and json files are the data required for the parsing and\ntranslation functions in `parser.py` and `translit.py`.\n\n\nQ & A\n-----\n\n* Q: Can I convert Chinese characters to romanization with `parsetc`?\n* A: No, it is for converting between different romanization systems only.\n\n* Q: `parsetc` makes a mistake, or it crashes when parsing a text. Can you fix it?\n* A: Please check first if you are using the latest version of `parsetc` (run\n `parsetc --version`). If the error still persists, please file a bug report\n on the Github [issues](https://github.com/learn-teochew/parsetc/issues) page,\n with the input text that caused the crash/error.\n\n* Q: Can I contribute code to this project?\n* A: You are welcome to look into the source code and fork/modfiy it. However,\n I currently do not have the capacity to review pull requests and\n contributions in detail, so please understand if I decline them or take a\n long time to respond.\n\n\nRelated projects\n----------------\n\n### From Learn-Teochew\n\nWe use `parsetc` to convert Teochew opera transcriptions into different\nromanization systems for the [Learn Teochew\nOpera](https://learn-teochew.github.io/tc-opera/transcriptions) website.\n\nHow to type the special diacritics like `u\u0324, o\u030d, o\u0358` in some romanization systems?\nIf you use MacOS X, check out our [custom keyboard\nlayouts](https://github.com/learn-teochew/POJ-variants-keyboard)\n\n\n### Others\n\n* [taibun](https://github.com/andreihar/taibun) - Transliterator and tokenizer\n for Taiwanese Hokkien by Andrei Harbachov\n",
"bugtrack_url": null,
"license": " The MIT License (MIT) Copyright (c) 2023 Brandon Seah Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ",
"summary": "Parse and convert between different Teochew phonetic spelling schemes.",
"version": "0.4.1",
"project_urls": {
"homepage": "https://learn-teochew.github.io",
"source": "https://github.com/learn-teochew/parsetc"
},
"split_keywords": [
"chinese",
"teochew",
"romanization"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ec8878cb78fccd9f58cd8f69657fa1dab48ad29567eb7975d87bc9fce63e0b8c",
"md5": "f73ac20cd072b4c2df447535fed90f5a",
"sha256": "8ae01b1898dd5737aa71827df40b36e7c1279a724c77463c5b33a4135f4615ae"
},
"downloads": -1,
"filename": "parsetc-0.4.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f73ac20cd072b4c2df447535fed90f5a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 22140,
"upload_time": "2024-02-18T10:03:31",
"upload_time_iso_8601": "2024-02-18T10:03:31.405572Z",
"url": "https://files.pythonhosted.org/packages/ec/88/78cb78fccd9f58cd8f69657fa1dab48ad29567eb7975d87bc9fce63e0b8c/parsetc-0.4.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "06d8bb7aa7ceb33033dd26a6d13acf140ee58efef969e989c7d03511aeef8395",
"md5": "134266620745490a482794e8e2b64f07",
"sha256": "ecd1cfd99dd88080847fa86aa93ba10bff47007c9aa7d9450e0a6ce5c5d87a0b"
},
"downloads": -1,
"filename": "parsetc-0.4.1.tar.gz",
"has_sig": false,
"md5_digest": "134266620745490a482794e8e2b64f07",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 20796,
"upload_time": "2024-02-18T10:03:33",
"upload_time_iso_8601": "2024-02-18T10:03:33.052962Z",
"url": "https://files.pythonhosted.org/packages/06/d8/bb7aa7ceb33033dd26a6d13acf140ee58efef969e989c7d03511aeef8395/parsetc-0.4.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-18 10:03:33",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "learn-teochew",
"github_project": "parsetc",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "parsetc"
}
