| Name | rannet JSON |
| Version |
0.3.1
JSON |
| download |
| home_page | |
| Summary | Recurrent Attention Networks |
| upload_time | 2023-08-12 13:05:12 |
| maintainer | |
| docs_url | None |
| author | sean lee |
| requires_python | |
| license | |
| keywords |
rannet
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
<h1 align='center'><img src='assets/logo_min.png' style='height: 25px;'/> RAN: Recurrent Attention Network</h1>
<h4 align='center'> 📢 This project is still in the works in order to make long document modeling easier.</h4>
<h4 align="center">
<a href="https://github.com/4AI/RAN/blob/main/LICENSE">
<img src="https://img.shields.io/badge/License-MIT-blue.svg?style=flat-square" alt="RAN is released under the MIT license." />
</a>
<a href="https://pypi.org/project/rannet/">
<img src="https://img.shields.io/pypi/v/rannet?style=flat-square" alt="PyPI version" />
</a>
<a href="https://pypi.org/project/rannet/">
<img src="https://img.shields.io/pypi/dm/rannet?style=flat-square" alt="PyPI Downloads" />
</a>
<a href="http://makeapullrequest.com">
<img src="https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square" alt="http://makeapullrequest.com" />
</a>
<a href="https://arxiv.org/abs/2306.06843">
<img src="https://img.shields.io/badge/Arxiv-2306.06843-yellow.svg?style=flat-square" alt="https://arxiv.org/abs/2306.06843" />
</a>
</h4>
<p align="center">
<img src='assets/framework.png' alt='The framework of RAN' style='width: 520px;' />
</p>
# ⬇️ Installation
*stable*
```bash
python -m pip install -U rannet
```
*latest*
```bash
python -m pip install git+https://github.com/4AI/RAN.git
```
*environment*
- ⭐ tensorflow>2.0,<=2.10 🤗 `export TF_KERAS=1`
- tensorflow>=1.14,<2.0 🤗 Keras==2.3.1
## 🏛️ Pretrained Models
### V3 Models
🎯 compatible with: `rannet>0.2.1`
| Lang | Google Drive | Baidu NetDrive |
|------|--------------|----------------|
| EN | [base](https://drive.google.com/file/d/1CO1M_57U506_3mDBqtGo-5b1XXNpONln/view?usp=sharing) | [base](https://pan.baidu.com/s/1Z2wODILsIeZ3i8_9GEpl2g)\[code: udts\] |
Chinese Models are still pretraining...
### V2 Models
🎯 compatible with: `rannet<=0.2.1`
| Lang | Google Drive | Baidu NetDrive |
|------|--------------|----------------|
| EN | [base](https://drive.google.com/file/d/1mRabw0Hy9T5_EWbZshD6Uk-bvauNzG9R/view?usp=sharing) | [base](https://pan.baidu.com/s/18uhAkY46aIcy4ncwzXp5mA)\[code: djkj\] |
| CN | [base](https://drive.google.com/file/d/1_gmrulSU-ln_jElc2hktPTTQDzaeG1wU/view?usp=sharing) \| [small](https://drive.google.com/file/d/1D-FCxY_UMwZCkvcwl6hkRcl6VnCzRGIj/view?usp=sharing) | [base](https://pan.baidu.com/s/1WIcePgmqb7Ox0w1qigWQ_w)\[code: e47w\] \| [small](https://pan.baidu.com/s/17DAboL9w0mArcBBuiy3tGg)\[code: mdmg\] |
### V1 Models
V1 models are not open.
# 🚀 Quick Tour
## 🈶 w/ pretrained models
*Extract semantic feature*
set `return_sequences=False` to extract semantic feature.
```python
import numpy as np
from rannet import RanNet, RanNetWordPieceTokenizer
vocab_path = 'pretrained/vocab.txt'
ckpt_path = 'pretrained/model.ckpt'
config_path = 'pretrained/config.json'
tokenizer = RanNetWordPieceTokenizer(vocab_path, lowercase=True)
rannet, rannet_model = RanNet.load_rannet(
config_path=config_path,
checkpoint_path=ckpt_path,
return_sequences=False,
apply_cell_transform=False,
cell_pooling='mean'
)
text = 'input text'
tok = tokenizer.encode(text)
vec = rannet_model.predict(np.array([tok.ids]))
```
*For the classification task*
```python
from rannet import RanNet, RanNetWordPieceTokenizer
vocab_path = 'pretrained/vocab.txt'
ckpt_path = 'pretrained/model.ckpt'
config_path = 'pretrained/config.json'
tokenizer = RanNetWordPieceTokenizer(vocab_path, lowercase=True)
rannet, rannet_model = RanNet.load_rannet(
config_path=config_path, checkpoint_path=ckpt_path, return_sequences=False)
output = rannet_model.output # (B, D)
output = L.Dropout(0.1)(output)
output = L.Dense(2, activation='softmax')(output)
model = keras.models.Model(rannet_model.input, output)
model.summary()
```
*For the sequence task*
```python
from rannet import RanNet, RanNetWordPieceTokenizer
vocab_path = 'pretrained/vocab.txt'
ckpt_path = 'pretrained/model.ckpt'
config_path = 'pretrained/config.json'
tokenizer = RanNetWordPieceTokenizer(vocab_path, lowercase=True)
rannet, rannet_model = RanNet.load_rannet(
config_path=config_path, checkpoint_path=ckpt_path, return_cell=False)
output = rannet_model.output # (B, L, D)
rannet_model.summary()
```
## 🈚 w/o pretrained models
Embed the `RAN` (a Keras layer) into your network.
```python
from rannet import RAN
ran = RAN(head_num=8,
head_size=256,
window_size=256,
min_window_size=16,
activation='swish',
kernel_initializer='glorot_normal',
apply_lm_mask=False,
apply_seq2seq_mask=False,
apply_memory_review=True,
dropout_rate=0.0,
cell_initializer_type='zero')
output, cell = ran(X)
```
## w/ history
```python
import numpy as np
from rannet import RanNet, RanNetWordPieceTokenizer
vocab_path = 'pretrained/vocab.txt'
ckpt_path = 'pretrained/model.ckpt'
config_path = 'pretrained/config.json'
tokenizer = RanNetWordPieceTokenizer(vocab_path, lowercase=True)
rannet, rannet_model = RanNet.load_rannet(
config_path=config_path,
checkpoint_path=ckpt_path,
return_sequences=False,
apply_cell_transform=False,
return_history=True, # return history
cell_pooling='mean',
with_cell=True, # with cell input
)
rannet_model.summary()
text = 'sentence 1'
tok = tokenizer.encode(text)
init_cell = np.zeros((1, 768)) # 768 is embedding size
vec, history = rannet_model.predict([np.array([tok.ids]), init_cell])
text2 = 'sentence 2'
tok = tokenizer.encode(text2)
vec2, history = rannet_model.predict([np.array([tok.ids]), history]) # input history of sentence 1
```
# 📚 Citation
If you use our code in your research, please cite our work:
```
@inproceedings{li-etal-2023-recurrent,
title = "Recurrent Attention Networks for Long-text Modeling",
author = "Li, Xianming and
Li, Zongxi and
Luo, Xiaotian and
Xie, Haoran and
Lee, Xing and
Zhao, Yingbin and
Wang, Fu Lee and
Li, Qing",
booktitle = "Findings of the Association for Computational Linguistics: ACL 2023",
month = jul,
year = "2023",
publisher = "Association for Computational Linguistics",
pages = "3006--3019",
}
```
# 📬 Contact
Please contact us at 1) for code problems, create a GitHub issue; 2) for paper problems, email xmlee97@gmail.com
Raw data
{
"_id": null,
"home_page": "",
"name": "rannet",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "rannet",
"author": "sean lee",
"author_email": "xmlee97@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/52/42/d1b42a8ba33846934dd0ad077b210adb716e85335c3541c90b5538c4a1db/rannet-0.3.1.tar.gz",
"platform": null,
"description": "<h1 align='center'><img src='assets/logo_min.png' style='height: 25px;'/> RAN: Recurrent Attention Network</h1>\n\n<h4 align='center'> \ud83d\udce2 This project is still in the works in order to make long document modeling easier.</h4>\n\n<h4 align=\"center\">\n <a href=\"https://github.com/4AI/RAN/blob/main/LICENSE\">\n <img src=\"https://img.shields.io/badge/License-MIT-blue.svg?style=flat-square\" alt=\"RAN is released under the MIT license.\" />\n </a>\n <a href=\"https://pypi.org/project/rannet/\">\n <img src=\"https://img.shields.io/pypi/v/rannet?style=flat-square\" alt=\"PyPI version\" />\n </a>\n <a href=\"https://pypi.org/project/rannet/\">\n <img src=\"https://img.shields.io/pypi/dm/rannet?style=flat-square\" alt=\"PyPI Downloads\" />\n </a>\n <a href=\"http://makeapullrequest.com\">\n <img src=\"https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square\" alt=\"http://makeapullrequest.com\" />\n </a>\n <a href=\"https://arxiv.org/abs/2306.06843\">\n <img src=\"https://img.shields.io/badge/Arxiv-2306.06843-yellow.svg?style=flat-square\" alt=\"https://arxiv.org/abs/2306.06843\" />\n </a>\n</h4>\n\n\n<p align=\"center\">\n<img src='assets/framework.png' alt='The framework of RAN' style='width: 520px;' />\n</p>\n\n\n# \u2b07\ufe0f Installation\n\n*stable*\n\n```bash\npython -m pip install -U rannet\n```\n\n*latest*\n\n```bash\npython -m pip install git+https://github.com/4AI/RAN.git\n```\n\n*environment*\n- \u2b50 tensorflow>2.0,<=2.10 \ud83e\udd17 `export TF_KERAS=1`\n- tensorflow>=1.14,<2.0 \ud83e\udd17 Keras==2.3.1\n\n## \ud83c\udfdb\ufe0f Pretrained Models\n\n### V3 Models\n\n\ud83c\udfaf compatible with: `rannet>0.2.1`\n\n| Lang | Google Drive | Baidu NetDrive |\n|------|--------------|----------------|\n| EN | [base](https://drive.google.com/file/d/1CO1M_57U506_3mDBqtGo-5b1XXNpONln/view?usp=sharing) | [base](https://pan.baidu.com/s/1Z2wODILsIeZ3i8_9GEpl2g)\\[code: udts\\] |\n\nChinese Models are still pretraining...\n\n\n### V2 Models\n\n\ud83c\udfaf compatible with: `rannet<=0.2.1`\n\n| Lang | Google Drive | Baidu NetDrive |\n|------|--------------|----------------|\n| EN | [base](https://drive.google.com/file/d/1mRabw0Hy9T5_EWbZshD6Uk-bvauNzG9R/view?usp=sharing) | [base](https://pan.baidu.com/s/18uhAkY46aIcy4ncwzXp5mA)\\[code: djkj\\] |\n| CN | [base](https://drive.google.com/file/d/1_gmrulSU-ln_jElc2hktPTTQDzaeG1wU/view?usp=sharing) \\| [small](https://drive.google.com/file/d/1D-FCxY_UMwZCkvcwl6hkRcl6VnCzRGIj/view?usp=sharing) | [base](https://pan.baidu.com/s/1WIcePgmqb7Ox0w1qigWQ_w)\\[code: e47w\\] \\| [small](https://pan.baidu.com/s/17DAboL9w0mArcBBuiy3tGg)\\[code: mdmg\\] |\n\n### V1 Models\n\nV1 models are not open.\n\n\n# \ud83d\ude80 Quick Tour\n\n## \ud83c\ude36 w/ pretrained models\n\n*Extract semantic feature*\n\nset `return_sequences=False` to extract semantic feature.\n\n```python\nimport numpy as np\nfrom rannet import RanNet, RanNetWordPieceTokenizer\n\n\nvocab_path = 'pretrained/vocab.txt'\nckpt_path = 'pretrained/model.ckpt'\nconfig_path = 'pretrained/config.json'\ntokenizer = RanNetWordPieceTokenizer(vocab_path, lowercase=True)\n\nrannet, rannet_model = RanNet.load_rannet(\n config_path=config_path,\n checkpoint_path=ckpt_path,\n return_sequences=False,\n apply_cell_transform=False,\n cell_pooling='mean'\n)\ntext = 'input text'\ntok = tokenizer.encode(text)\nvec = rannet_model.predict(np.array([tok.ids]))\n```\n\n*For the classification task*\n\n```python\nfrom rannet import RanNet, RanNetWordPieceTokenizer\n\n\nvocab_path = 'pretrained/vocab.txt'\nckpt_path = 'pretrained/model.ckpt'\nconfig_path = 'pretrained/config.json'\ntokenizer = RanNetWordPieceTokenizer(vocab_path, lowercase=True)\n\nrannet, rannet_model = RanNet.load_rannet(\n config_path=config_path, checkpoint_path=ckpt_path, return_sequences=False)\noutput = rannet_model.output # (B, D)\noutput = L.Dropout(0.1)(output)\noutput = L.Dense(2, activation='softmax')(output)\nmodel = keras.models.Model(rannet_model.input, output)\nmodel.summary()\n```\n\n*For the sequence task*\n\n```python\nfrom rannet import RanNet, RanNetWordPieceTokenizer\n\n\nvocab_path = 'pretrained/vocab.txt'\nckpt_path = 'pretrained/model.ckpt'\nconfig_path = 'pretrained/config.json'\ntokenizer = RanNetWordPieceTokenizer(vocab_path, lowercase=True)\n\nrannet, rannet_model = RanNet.load_rannet(\n config_path=config_path, checkpoint_path=ckpt_path, return_cell=False)\noutput = rannet_model.output # (B, L, D)\nrannet_model.summary()\n```\n\n## \ud83c\ude1a w/o pretrained models\n\nEmbed the `RAN` (a Keras layer) into your network.\n\n```python\nfrom rannet import RAN\n\nran = RAN(head_num=8,\n head_size=256,\n window_size=256,\n min_window_size=16,\n activation='swish',\n kernel_initializer='glorot_normal',\n apply_lm_mask=False,\n apply_seq2seq_mask=False,\n apply_memory_review=True,\n dropout_rate=0.0,\n cell_initializer_type='zero')\noutput, cell = ran(X)\n```\n\n## w/ history\n\n```python\nimport numpy as np\nfrom rannet import RanNet, RanNetWordPieceTokenizer\n\n\nvocab_path = 'pretrained/vocab.txt'\nckpt_path = 'pretrained/model.ckpt'\nconfig_path = 'pretrained/config.json'\ntokenizer = RanNetWordPieceTokenizer(vocab_path, lowercase=True)\n\nrannet, rannet_model = RanNet.load_rannet(\n config_path=config_path,\n checkpoint_path=ckpt_path,\n return_sequences=False,\n apply_cell_transform=False,\n return_history=True, # return history\n cell_pooling='mean',\n with_cell=True, # with cell input\n)\nrannet_model.summary()\n\ntext = 'sentence 1'\ntok = tokenizer.encode(text)\ninit_cell = np.zeros((1, 768)) # 768 is embedding size\nvec, history = rannet_model.predict([np.array([tok.ids]), init_cell])\n\ntext2 = 'sentence 2'\ntok = tokenizer.encode(text2)\nvec2, history = rannet_model.predict([np.array([tok.ids]), history]) # input history of sentence 1\n```\n\n# \ud83d\udcda Citation\n\nIf you use our code in your research, please cite our work:\n\n```\n@inproceedings{li-etal-2023-recurrent,\n title = \"Recurrent Attention Networks for Long-text Modeling\",\n author = \"Li, Xianming and\n Li, Zongxi and\n Luo, Xiaotian and\n Xie, Haoran and\n Lee, Xing and\n Zhao, Yingbin and\n Wang, Fu Lee and\n Li, Qing\",\n booktitle = \"Findings of the Association for Computational Linguistics: ACL 2023\",\n month = jul,\n year = \"2023\",\n publisher = \"Association for Computational Linguistics\",\n pages = \"3006--3019\",\n}\n```\n\n# \ud83d\udcec Contact\n\nPlease contact us at 1) for code problems, create a GitHub issue; 2) for paper problems, email xmlee97@gmail.com\n",
"bugtrack_url": null,
"license": "",
"summary": "Recurrent Attention Networks",
"version": "0.3.1",
"project_urls": null,
"split_keywords": [
"rannet"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "66deca796f02398de73da07fdb248dba5dd01818bf3e188a9fdb61e72e770ed4",
"md5": "b7102e33650979a17c6ce3b699369761",
"sha256": "bf11113f72ef4027577489cc47a080ca8cc243c67850e2e4d2a6bad1c5be0e61"
},
"downloads": -1,
"filename": "rannet-0.3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b7102e33650979a17c6ce3b699369761",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 32385,
"upload_time": "2023-08-12T13:05:10",
"upload_time_iso_8601": "2023-08-12T13:05:10.742094Z",
"url": "https://files.pythonhosted.org/packages/66/de/ca796f02398de73da07fdb248dba5dd01818bf3e188a9fdb61e72e770ed4/rannet-0.3.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5242d1b42a8ba33846934dd0ad077b210adb716e85335c3541c90b5538c4a1db",
"md5": "91965aeca884c2b79aa70c628283687b",
"sha256": "26687a7c97ba90e339e60719d540794a5342b5f49911071d3c755ccea57a9273"
},
"downloads": -1,
"filename": "rannet-0.3.1.tar.gz",
"has_sig": false,
"md5_digest": "91965aeca884c2b79aa70c628283687b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 31807,
"upload_time": "2023-08-12T13:05:12",
"upload_time_iso_8601": "2023-08-12T13:05:12.239717Z",
"url": "https://files.pythonhosted.org/packages/52/42/d1b42a8ba33846934dd0ad077b210adb716e85335c3541c90b5538c4a1db/rannet-0.3.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-08-12 13:05:12",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "rannet"
}
