neologdn
#########
|downloads| |pyversion| |version| |license|
neologdn is a Japanese text normalizer for `mecab-neologd <https://github.com/neologd/mecab-ipadic-neologd>`_.
The normalization is based on the neologd's rules:
https://github.com/neologd/mecab-ipadic-neologd/wiki/Regexp.ja
Contributions are welcome!
NOTE: Installing this module requires C++11 compiler.
Installation
*************
::
$ pip install neologdn
Usage
******
.. code:: python
import neologdn
neologdn.normalize("ﾊﾝｶｸｶﾅ")
# => 'ハンカクカナ'
neologdn.normalize("全角記号！？＠＃")
# => '全角記号!?@#'
neologdn.normalize("全角記号例外「・」")
# => '全角記号例外「・」'
neologdn.normalize("長音短縮ウェーーーーイ")
# => '長音短縮ウェーイ'
neologdn.normalize("チルダ削除ウェ~∼∾〜〰～イ")
# => 'チルダ削除ウェイ'
neologdn.normalize("いろんなハイフン˗֊‐‑‒–⁃⁻₋−")
# => 'いろんなハイフン-'
neologdn.normalize(" ＰＲＭＬ 副 読 本 ")
# => 'PRML副読本'
neologdn.normalize(" Natural Language Processing ")
# => 'Natural Language Processing'
neologdn.normalize("かわいいいいいいいいい", repeat=6)
# => 'かわいいいいいい'
neologdn.normalize("無駄無駄無駄無駄ァ", repeat=1)
# => '無駄ァ'
neologdn.normalize("1995〜2001年", tilde="normalize")
# => '1995~2001年'
neologdn.normalize("1995~2001年", tilde="normalize_zenkaku")
# => '1995〜2001年'
neologdn.normalize("1995〜2001年", tilde="ignore") # Don't convert tilde
# => '1995〜2001年'
neologdn.normalize("1995〜2001年", tilde="remove")
# => '19952001年'
neologdn.normalize("1995〜2001年") # Default parameter
# => '19952001年'
Benchmark
**********
.. code:: python
# Sample code from
# https://github.com/neologd/mecab-ipadic-neologd/wiki/Regexp.ja#python-written-by-hideaki-t--overlast
import normalize_neologd
%timeit normalize(normalize_neologd.normalize_neologd)
# => 9.55 s ± 29.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
import neologdn
%timeit normalize(neologdn.normalize)
# => 6.66 s ± 35.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
neologdn is about x1.43 faster than sample code.
details are described as the below notebook:
https://github.com/ikegami-yukino/neologdn/blob/master/benchmark/benchmark.ipynb
License
*********
Apache Software License.
Contribution
*************
Contributions are welcome! See: https://github.com/ikegami-yukino/neologdn/blob/master/.github/CONTRIBUTING.md
Cited by
**********
Book
========
山本 和英. テキスト処理の要素技術. 近代科学者. P.41. 2021.
Blog
========
- 【ライブラリ紹介】テキスト正規化ライブラリ neologdn: https://diatonic.codes/blog/neologdn/
- 日本語テキストの前処理：neologdn、大文字小文字、Unicode正規化 - tuttieee’s blog: https://tuttieee.hatenablog.com/entry/ja-nlp-preprocess
- ▲本日の関数==neologdn.normalize()== - TPTブログ: https://ds-blog.tbtech.co.jp/entry/2020/05/11/%E2%96%B2%E6%9C%AC%E6%97%A5%E3%81%AE%E9%96%A2%E6%95%B0%3D%3Dneologdn_normalize%28%29%3D%3D
- NLPについて学ぶ: https://zenn.dev/panyoriokome/scraps/d67f68ab50c0c1
- テキスト正規化用PythonライブラリをMATLABからコール #Python - Qiita: https://qiita.com/aoimidori/items/ab5a4383b5a7bb307bad
- 自然言語処理の前処理手順をPythonコード付きでご紹介 | AI活用・AI導入事例の紹介 | AI活用・AI導入事例の紹介: https://www.matrixflow.net/case-study/75/
- pythonによる日本語前処理備忘録 | DATUM STUDIO株式会社: https://datumstudio.jp/blog/python%E3%81%AB%E3%82%88%E3%82%8B%E6%97%A5%E6%9C%AC%E8%AA%9E%E5%89%8D%E5%87%A6%E7%90%86%E5%82%99%E5%BF%98%E9%8C%B2/
- 前処理、前処理、そして、前処理 （自然言語処理：日本語編）｜narudesu: https://note.com/narudesu/n/na35de30a583a
- ショートカットキーでneologd.normalize: https://scrapbox.io/nishio/%E3%82%B7%E3%83%A7%E3%83%BC%E3%83%88%E3%82%AB%E3%83%83%E3%83%88%E3%82%AD%E3%83%BC%E3%81%A7neologd.normalize
- Pythonで自然言語処理を行うための環境構築 #Python - Qiita: https://qiita.com/lawyer_alpaca/items/86b0deda984170203467
- Python normalize Examples: https://python.hotexamples.com/examples/neologdn/-/normalize/python-normalize-function-examples.html
- 株式会社ししまろ (ch-4) 潜在的ディリクレ配分(LDA)によるchABSAデータセットの分析: https://shishimaro.co.jp/blog/ai/538
- 形態素解析前の日本語文書の前処理 (Python) - け日記: https://ohke.hateblo.jp/entry/2019/02/09/141500
- 人工知能に言語を理解させる！？自然言語処理に重要なデータの前処理をPythonで徹底解説 | AI研究所: https://ai-kenkyujo.com/programming/make-ai-understand-the-language/
- 最新wikipediaを反映したMeCabユーザー辞書を作る - NEologd拡張 | ぷらこめ: https://purakome.net/mecab/addwiki/
- 【自然言語処理入門】文に対してストップワードと正規化から処理を施す | マイナビエンジニアブログ: https://engineerblog.mynavi.jp/technology/nlp_stopword/
- 表記統一 [自然言語処理の餅屋]: https://www.jnlp.org/nlp/%E6%A0%A1%E6%AD%A3/%E8%A1%A8%E8%A8%98%E7%B5%B1%E4%B8%80
- Pytorchを使ってテキスト生成モデルのT5を構築 〜Transformersでの転移学習による手軽な実践〜 - 見習いデータサイエンティストの隠れ家: https://www.dskomei.com/entry/2021/09/28/110016
- 象と散歩: Goolge Colabでお手軽テキストマイニング(日本語前処理): https://walking-elephant.blogspot.com/2023/07/text-mining-normalized.html
- 【Pythonで自然言語処理（NLP）を実装してみよう！】学ぶべき知識についても徹底解説！ - ベトナムオフショア開発の最前線 by Mattock inc.: https://mattock.jp/blog/artificial-intelligence/nlp/lets-implement-nlp-in-python/
- tools [Digital Humanities Japan: Resource Wiki]: https://dhjapan.org/wiki/doku.php?id=tools
- Pythonで現代の季語を調べてみた | Aidemy | 10秒で始めるAIプログラミング学習サービスAidemy［アイデミー］: https://aidemy.net/magazine/703/
.. |downloads| image:: https://static.pepy.tech/personalized-badge/neologdn?period=total&units=international_system&left_color=black&right_color=orange&left_text=Downloads
:target: https://pepy.tech/project/neologdn
.. |version| image:: https://img.shields.io/pypi/v/neologdn.svg
:target: http://pypi.python.org/pypi/neologdn/
:alt: latest version
.. |pyversion| image:: https://img.shields.io/pypi/pyversions/neologdn.svg
.. |license| image:: https://img.shields.io/pypi/l/neologdn.svg
:target: http://pypi.python.org/pypi/neologdn/
:alt: license
CHANGES
========
0.5.3 (2024-05-03)
----------------------------
- Support Python 3.12
0.5.2 (2023-08-03)
----------------------------
- Support Python 3.10 and 3.11 (Many thanks @polm)
0.5.1 (2021-05-02)
----------------------------
- Improve performance of shorten_repeat function (Many thanks @yskn67)
- Add tilde option to normalize function
0.4 (2018-12-06)
----------------------------
- Add shorten_repeat function, which shortening contiguous substring. For example: neologdn.normalize("無駄無駄無駄無駄ァ", repeat=1) -> 無駄ァ
0.3.2 (2018-05-17)
----------------------------
- Add option for suppression removal of spaces between Japanese characters
0.2.2 (2018-03-10)
----------------------------
- Fix bug (daku-ten & handaku-ten)
- Support mac osx 10.13 (Many thanks @r9y9)
0.2.1 (2017-01-23)
----------------------------
- Fix bug (Check if a previous character of daku-ten character is in maps) (Many thanks @unnonouno)
0.2 (2016-04-12)
----------------------------
- Add lengthened expression (repeating character) threshold
0.1.2 (2016-03-29)
----------------------------
- Fix installation bug
0.1.1.1 (2016-03-19)
----------------------------
- Support Windows
- Explicitly specify to -std=c++11 in build (Many thanks @id774)
0.1.1 (2015-10-10)
----------------------------
Initial release.
Raw data
{
"_id": null,
"home_page": "http://github.com/ikegami-yukino/neologdn",
"name": "neologdn",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "japanese, MeCab",
"author": "Yukino Ikegami",
"author_email": "yknikgm@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/fa/3c/266f9e4ec59d6d73e17f8434a43acd2575822e1c71fd40bbbe22a630d229/neologdn-0.5.3.tar.gz",
"platform": null,
"description": "neologdn\n#########\n\n|downloads| |pyversion| |version| |license|\n\nneologdn is a Japanese text normalizer for `mecab-neologd <https://github.com/neologd/mecab-ipadic-neologd>`_.\n\nThe normalization is based on the neologd's rules:\nhttps://github.com/neologd/mecab-ipadic-neologd/wiki/Regexp.ja\n\n\nContributions are welcome!\n\nNOTE: Installing this module requires C++11 compiler.\n\nInstallation\n*************\n\n::\n\n $ pip install neologdn\n\nUsage\n******\n\n.. code:: python\n\n import neologdn\n neologdn.normalize(\"\uff8a\uff9d\uff76\uff78\uff76\uff85\")\n # => '\u30cf\u30f3\u30ab\u30af\u30ab\u30ca'\n neologdn.normalize(\"\u5168\u89d2\u8a18\u53f7\uff01\uff1f\uff20\uff03\")\n # => '\u5168\u89d2\u8a18\u53f7!?@#'\n neologdn.normalize(\"\u5168\u89d2\u8a18\u53f7\u4f8b\u5916\u300c\u30fb\u300d\")\n # => '\u5168\u89d2\u8a18\u53f7\u4f8b\u5916\u300c\u30fb\u300d'\n neologdn.normalize(\"\u9577\u97f3\u77ed\u7e2e\u30a6\u30a7\u30fc\u30fc\u30fc\u30fc\u30a4\")\n # => '\u9577\u97f3\u77ed\u7e2e\u30a6\u30a7\u30fc\u30a4'\n neologdn.normalize(\"\u30c1\u30eb\u30c0\u524a\u9664\u30a6\u30a7~\u223c\u223e\u301c\u3030\uff5e\u30a4\")\n # => '\u30c1\u30eb\u30c0\u524a\u9664\u30a6\u30a7\u30a4'\n neologdn.normalize(\"\u3044\u308d\u3093\u306a\u30cf\u30a4\u30d5\u30f3\u02d7\u058a\u2010\u2011\u2012\u2013\u2043\u207b\u208b\u2212\")\n # => '\u3044\u308d\u3093\u306a\u30cf\u30a4\u30d5\u30f3-'\n neologdn.normalize(\"\u3000\u3000\u3000\uff30\uff32\uff2d\uff2c\u3000\u3000\u526f\u3000\u8aad\u3000\u672c\u3000\u3000\u3000\")\n # => 'PRML\u526f\u8aad\u672c'\n neologdn.normalize(\" Natural Language Processing \")\n # => 'Natural Language Processing'\n neologdn.normalize(\"\u304b\u308f\u3044\u3044\u3044\u3044\u3044\u3044\u3044\u3044\u3044\", repeat=6)\n # => '\u304b\u308f\u3044\u3044\u3044\u3044\u3044\u3044'\n neologdn.normalize(\"\u7121\u99c4\u7121\u99c4\u7121\u99c4\u7121\u99c4\u30a1\", repeat=1)\n # => '\u7121\u99c4\u30a1'\n neologdn.normalize(\"1995\u301c2001\u5e74\", tilde=\"normalize\")\n # => '1995~2001\u5e74'\n neologdn.normalize(\"1995~2001\u5e74\", tilde=\"normalize_zenkaku\")\n # => '1995\u301c2001\u5e74'\n neologdn.normalize(\"1995\u301c2001\u5e74\", tilde=\"ignore\") # Don't convert tilde\n # => '1995\u301c2001\u5e74'\n neologdn.normalize(\"1995\u301c2001\u5e74\", tilde=\"remove\")\n # => '19952001\u5e74'\n neologdn.normalize(\"1995\u301c2001\u5e74\") # Default parameter\n # => '19952001\u5e74'\n\n\nBenchmark\n**********\n\n.. code:: python\n\n # Sample code from\n # https://github.com/neologd/mecab-ipadic-neologd/wiki/Regexp.ja#python-written-by-hideaki-t--overlast\n import normalize_neologd\n\n %timeit normalize(normalize_neologd.normalize_neologd)\n # => 9.55 s \u00b1 29.4 ms per loop (mean \u00b1 std. dev. of 7 runs, 1 loop each)\n\n\n import neologdn\n %timeit normalize(neologdn.normalize)\n # => 6.66 s \u00b1 35.8 ms per loop (mean \u00b1 std. dev. of 7 runs, 1 loop each)\n\n\nneologdn is about x1.43 faster than sample code.\n\ndetails are described as the below notebook:\nhttps://github.com/ikegami-yukino/neologdn/blob/master/benchmark/benchmark.ipynb\n\n\nLicense\n*********\n\nApache Software License.\n\n\nContribution\n*************\n\nContributions are welcome! See: https://github.com/ikegami-yukino/neologdn/blob/master/.github/CONTRIBUTING.md\n\nCited by\n**********\nBook\n========\n\u5c71\u672c \u548c\u82f1. \u30c6\u30ad\u30b9\u30c8\u51e6\u7406\u306e\u8981\u7d20\u6280\u8853. \u8fd1\u4ee3\u79d1\u5b66\u8005. P.41. 2021.\n\nBlog\n========\n- \u3010\u30e9\u30a4\u30d6\u30e9\u30ea\u7d39\u4ecb\u3011\u30c6\u30ad\u30b9\u30c8\u6b63\u898f\u5316\u30e9\u30a4\u30d6\u30e9\u30ea neologdn: https://diatonic.codes/blog/neologdn/\n- \u65e5\u672c\u8a9e\u30c6\u30ad\u30b9\u30c8\u306e\u524d\u51e6\u7406\uff1aneologdn\u3001\u5927\u6587\u5b57\u5c0f\u6587\u5b57\u3001Unicode\u6b63\u898f\u5316 - tuttieee\u2019s blog: https://tuttieee.hatenablog.com/entry/ja-nlp-preprocess\n- \u25b2\u672c\u65e5\u306e\u95a2\u6570==neologdn.normalize()== - TPT\u30d6\u30ed\u30b0: https://ds-blog.tbtech.co.jp/entry/2020/05/11/%E2%96%B2%E6%9C%AC%E6%97%A5%E3%81%AE%E9%96%A2%E6%95%B0%3D%3Dneologdn_normalize%28%29%3D%3D\n- NLP\u306b\u3064\u3044\u3066\u5b66\u3076: https://zenn.dev/panyoriokome/scraps/d67f68ab50c0c1\n- \u30c6\u30ad\u30b9\u30c8\u6b63\u898f\u5316\u7528Python\u30e9\u30a4\u30d6\u30e9\u30ea\u3092MATLAB\u304b\u3089\u30b3\u30fc\u30eb #Python - Qiita: https://qiita.com/aoimidori/items/ab5a4383b5a7bb307bad\n- \u81ea\u7136\u8a00\u8a9e\u51e6\u7406\u306e\u524d\u51e6\u7406\u624b\u9806\u3092Python\u30b3\u30fc\u30c9\u4ed8\u304d\u3067\u3054\u7d39\u4ecb | AI\u6d3b\u7528\u30fbAI\u5c0e\u5165\u4e8b\u4f8b\u306e\u7d39\u4ecb | AI\u6d3b\u7528\u30fbAI\u5c0e\u5165\u4e8b\u4f8b\u306e\u7d39\u4ecb: https://www.matrixflow.net/case-study/75/\n- python\u306b\u3088\u308b\u65e5\u672c\u8a9e\u524d\u51e6\u7406\u5099\u5fd8\u9332 | DATUM STUDIO\u682a\u5f0f\u4f1a\u793e: https://datumstudio.jp/blog/python%E3%81%AB%E3%82%88%E3%82%8B%E6%97%A5%E6%9C%AC%E8%AA%9E%E5%89%8D%E5%87%A6%E7%90%86%E5%82%99%E5%BF%98%E9%8C%B2/\n- \u524d\u51e6\u7406\u3001\u524d\u51e6\u7406\u3001\u305d\u3057\u3066\u3001\u524d\u51e6\u7406 \uff08\u81ea\u7136\u8a00\u8a9e\u51e6\u7406\uff1a\u65e5\u672c\u8a9e\u7de8\uff09\uff5cnarudesu: https://note.com/narudesu/n/na35de30a583a\n- \u30b7\u30e7\u30fc\u30c8\u30ab\u30c3\u30c8\u30ad\u30fc\u3067neologd.normalize: https://scrapbox.io/nishio/%E3%82%B7%E3%83%A7%E3%83%BC%E3%83%88%E3%82%AB%E3%83%83%E3%83%88%E3%82%AD%E3%83%BC%E3%81%A7neologd.normalize\n- Python\u3067\u81ea\u7136\u8a00\u8a9e\u51e6\u7406\u3092\u884c\u3046\u305f\u3081\u306e\u74b0\u5883\u69cb\u7bc9 #Python - Qiita: https://qiita.com/lawyer_alpaca/items/86b0deda984170203467\n- Python normalize Examples: https://python.hotexamples.com/examples/neologdn/-/normalize/python-normalize-function-examples.html\n- \u682a\u5f0f\u4f1a\u793e\u3057\u3057\u307e\u308d (ch-4) \u6f5c\u5728\u7684\u30c7\u30a3\u30ea\u30af\u30ec\u914d\u5206(LDA)\u306b\u3088\u308bchABSA\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306e\u5206\u6790: https://shishimaro.co.jp/blog/ai/538\n- \u5f62\u614b\u7d20\u89e3\u6790\u524d\u306e\u65e5\u672c\u8a9e\u6587\u66f8\u306e\u524d\u51e6\u7406 (Python) - \u3051\u65e5\u8a18: https://ohke.hateblo.jp/entry/2019/02/09/141500\n- \u4eba\u5de5\u77e5\u80fd\u306b\u8a00\u8a9e\u3092\u7406\u89e3\u3055\u305b\u308b\uff01\uff1f\u81ea\u7136\u8a00\u8a9e\u51e6\u7406\u306b\u91cd\u8981\u306a\u30c7\u30fc\u30bf\u306e\u524d\u51e6\u7406\u3092Python\u3067\u5fb9\u5e95\u89e3\u8aac | AI\u7814\u7a76\u6240: https://ai-kenkyujo.com/programming/make-ai-understand-the-language/\n- \u6700\u65b0wikipedia\u3092\u53cd\u6620\u3057\u305fMeCab\u30e6\u30fc\u30b6\u30fc\u8f9e\u66f8\u3092\u4f5c\u308b - NEologd\u62e1\u5f35 | \u3077\u3089\u3053\u3081: https://purakome.net/mecab/addwiki/\n- \u3010\u81ea\u7136\u8a00\u8a9e\u51e6\u7406\u5165\u9580\u3011\u6587\u306b\u5bfe\u3057\u3066\u30b9\u30c8\u30c3\u30d7\u30ef\u30fc\u30c9\u3068\u6b63\u898f\u5316\u304b\u3089\u51e6\u7406\u3092\u65bd\u3059 | \u30de\u30a4\u30ca\u30d3\u30a8\u30f3\u30b8\u30cb\u30a2\u30d6\u30ed\u30b0: https://engineerblog.mynavi.jp/technology/nlp_stopword/\n- \u8868\u8a18\u7d71\u4e00 [\u81ea\u7136\u8a00\u8a9e\u51e6\u7406\u306e\u9905\u5c4b]: https://www.jnlp.org/nlp/%E6%A0%A1%E6%AD%A3/%E8%A1%A8%E8%A8%98%E7%B5%B1%E4%B8%80\n- Pytorch\u3092\u4f7f\u3063\u3066\u30c6\u30ad\u30b9\u30c8\u751f\u6210\u30e2\u30c7\u30eb\u306eT5\u3092\u69cb\u7bc9 \u301cTransformers\u3067\u306e\u8ee2\u79fb\u5b66\u7fd2\u306b\u3088\u308b\u624b\u8efd\u306a\u5b9f\u8df5\u301c - \u898b\u7fd2\u3044\u30c7\u30fc\u30bf\u30b5\u30a4\u30a8\u30f3\u30c6\u30a3\u30b9\u30c8\u306e\u96a0\u308c\u5bb6: https://www.dskomei.com/entry/2021/09/28/110016\n- \u8c61\u3068\u6563\u6b69: Goolge Colab\u3067\u304a\u624b\u8efd\u30c6\u30ad\u30b9\u30c8\u30de\u30a4\u30cb\u30f3\u30b0(\u65e5\u672c\u8a9e\u524d\u51e6\u7406): https://walking-elephant.blogspot.com/2023/07/text-mining-normalized.html\n- \u3010Python\u3067\u81ea\u7136\u8a00\u8a9e\u51e6\u7406\uff08NLP\uff09\u3092\u5b9f\u88c5\u3057\u3066\u307f\u3088\u3046\uff01\u3011\u5b66\u3076\u3079\u304d\u77e5\u8b58\u306b\u3064\u3044\u3066\u3082\u5fb9\u5e95\u89e3\u8aac\uff01 - \u30d9\u30c8\u30ca\u30e0\u30aa\u30d5\u30b7\u30e7\u30a2\u958b\u767a\u306e\u6700\u524d\u7dda by Mattock inc.: https://mattock.jp/blog/artificial-intelligence/nlp/lets-implement-nlp-in-python/\n- tools [Digital Humanities Japan: Resource Wiki]: https://dhjapan.org/wiki/doku.php?id=tools\n- Python\u3067\u73fe\u4ee3\u306e\u5b63\u8a9e\u3092\u8abf\u3079\u3066\u307f\u305f | Aidemy | 10\u79d2\u3067\u59cb\u3081\u308bAI\u30d7\u30ed\u30b0\u30e9\u30df\u30f3\u30b0\u5b66\u7fd2\u30b5\u30fc\u30d3\u30b9Aidemy\uff3b\u30a2\u30a4\u30c7\u30df\u30fc\uff3d: https://aidemy.net/magazine/703/\n\n\n.. |downloads| image:: https://static.pepy.tech/personalized-badge/neologdn?period=total&units=international_system&left_color=black&right_color=orange&left_text=Downloads\n :target: https://pepy.tech/project/neologdn\n\n.. |version| image:: https://img.shields.io/pypi/v/neologdn.svg\n :target: http://pypi.python.org/pypi/neologdn/\n :alt: latest version\n\n.. |pyversion| image:: https://img.shields.io/pypi/pyversions/neologdn.svg\n\n.. |license| image:: https://img.shields.io/pypi/l/neologdn.svg\n :target: http://pypi.python.org/pypi/neologdn/\n :alt: license\n\n\n\nCHANGES\n========\n\n0.5.3 (2024-05-03)\n----------------------------\n\n- Support Python 3.12\n\n0.5.2 (2023-08-03)\n----------------------------\n\n- Support Python 3.10 and 3.11 (Many thanks @polm)\n\n0.5.1 (2021-05-02)\n----------------------------\n\n- Improve performance of shorten_repeat function (Many thanks @yskn67)\n- Add tilde option to normalize function\n\n0.4 (2018-12-06)\n----------------------------\n\n- Add shorten_repeat function, which shortening contiguous substring. For example: neologdn.normalize(\"\u7121\u99c4\u7121\u99c4\u7121\u99c4\u7121\u99c4\u30a1\", repeat=1) -> \u7121\u99c4\u30a1\n\n0.3.2 (2018-05-17)\n----------------------------\n\n- Add option for suppression removal of spaces between Japanese characters\n\n0.2.2 (2018-03-10)\n----------------------------\n\n- Fix bug (daku-ten & handaku-ten)\n- Support mac osx 10.13 (Many thanks @r9y9)\n\n0.2.1 (2017-01-23)\n----------------------------\n\n- Fix bug (Check if a previous character of daku-ten character is in maps) (Many thanks @unnonouno)\n\n0.2 (2016-04-12)\n----------------------------\n\n- Add lengthened expression (repeating character) threshold\n\n0.1.2 (2016-03-29)\n----------------------------\n\n- Fix installation bug\n\n0.1.1.1 (2016-03-19)\n----------------------------\n\n- Support Windows\n- Explicitly specify to -std=c++11 in build (Many thanks @id774)\n\n0.1.1 (2015-10-10)\n----------------------------\n\nInitial release.\n",
"bugtrack_url": null,
"license": "Apache Software License",
"summary": "Japanese text normalizer for mecab-neologd",
"version": "0.5.3",
"project_urls": {
"Homepage": "http://github.com/ikegami-yukino/neologdn"
},
"split_keywords": [
"japanese",
" mecab"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "fa3c266f9e4ec59d6d73e17f8434a43acd2575822e1c71fd40bbbe22a630d229",
"md5": "4cb62a73bc02307314f592dc96f68403",
"sha256": "f3f31878f7e0576f3b5a4f28f90cc5dccfc8880e616a0481332d478fe75160a5"
},
"downloads": -1,
"filename": "neologdn-0.5.3.tar.gz",
"has_sig": false,
"md5_digest": "4cb62a73bc02307314f592dc96f68403",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 144601,
"upload_time": "2024-05-02T15:58:50",
"upload_time_iso_8601": "2024-05-02T15:58:50.952978Z",
"url": "https://files.pythonhosted.org/packages/fa/3c/266f9e4ec59d6d73e17f8434a43acd2575822e1c71fd40bbbe22a630d229/neologdn-0.5.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-02 15:58:50",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ikegami-yukino",
"github_project": "neologdn",
"travis_ci": true,
"coveralls": false,
"github_actions": true,
"lcname": "neologdn"
}
JSON
