brc-pytorch
===========
PyTorch implementation of the bistable recurrent cell (BRC) from the paper [_A bio-inspired bistable recurrent cell allows for
long-lasting memory_ (Vecoven et al., 2020)](https://arxiv.org/abs/2006.05252).
Install
-------
```bash
pip install brc-pytorch
```
Usage
-----
```python
import torch
from brc_pytorch import utils
from brc_pytorch.modules import BRC, NBRC, StackedRNN
brc = StackedRNN(
cell=BRC, # NBRC for the neuromodulated version
input_size=128,
hidden_size=256,
num_layers=3
).cuda()
# [ seq_len, batch_size, dim ]
x = torch.randn(64, 32, 128).cuda()
init_hidden = brc.init_hidden(batch_size=32)
out, hidden = brc(x, init_hidden)
# Detach hidden for next call
hidden = utils.detach_hidden(hidden)
```
Performance
-----------
The implementation is written in TorchScript (a subset of Python) and makes use of the PyTorch JIT compiler for a more efficient execution.
In my not really statistically significant experiments, the implementation seems to be about half as fast as the cuDNN based reference LSTM implementation with modest batch sizes and sequence lengths which can be considered pretty solid for a non-CUDA implementation.
**Update (2023/07)**
A lot has happened since the last real update of this repository. [PyTorch 2.0](https://pytorch.org/blog/pytorch-2.0-release/) has introduced a new dynamic compilation system, which, to my understanding, should replace the old scripting API. After doing a quick test, it seems that the code is still working without issues with the scripting API and can in principle also be compiled with `torch.compile`. However, it does not seem to run faster, perhaps even slower, and would need an adaption to the new compilation system.
References
----------
```bibtex
@misc{vecoven2020bioinspired,
title={A bio-inspired bistable recurrent cell allows for long-lasting memory},
author={Nicolas Vecoven and Damien Ernst and Guillaume Drion},
year={2020},
eprint={2006.05252},
archivePrefix={arXiv},
primaryClass={cs.NE}
}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/kklemon/brc-pytorch",
"name": "brc",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "artificial intelligence,deep learning",
"author": "Kristian Klemon",
"author_email": "kristian.klemon@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/d5/8a/28f3e85122f2a139b9bcc512ad4a6abc17c3d77c0a80d586241a99151875/brc-0.1.2.tar.gz",
"platform": null,
"description": "brc-pytorch\n===========\n\nPyTorch implementation of the bistable recurrent cell (BRC) from the paper [_A bio-inspired bistable recurrent cell allows for\nlong-lasting memory_ (Vecoven et al., 2020)](https://arxiv.org/abs/2006.05252).\n\n\n\nInstall\n-------\n\n```bash\npip install brc-pytorch\n```\n\nUsage\n-----\n\n```python\nimport torch\n\nfrom brc_pytorch import utils\nfrom brc_pytorch.modules import BRC, NBRC, StackedRNN\n\nbrc = StackedRNN(\n cell=BRC, # NBRC for the neuromodulated version\n input_size=128,\n hidden_size=256,\n num_layers=3\n).cuda()\n\n# [ seq_len, batch_size, dim ]\nx = torch.randn(64, 32, 128).cuda()\n\ninit_hidden = brc.init_hidden(batch_size=32)\nout, hidden = brc(x, init_hidden)\n\n# Detach hidden for next call\nhidden = utils.detach_hidden(hidden)\n```\n\nPerformance\n-----------\n\nThe implementation is written in TorchScript (a subset of Python) and makes use of the PyTorch JIT compiler for a more efficient execution.\n\nIn my not really statistically significant experiments, the implementation seems to be about half as fast as the cuDNN based reference LSTM implementation with modest batch sizes and sequence lengths which can be considered pretty solid for a non-CUDA implementation.\n\n**Update (2023/07)**\n\nA lot has happened since the last real update of this repository. [PyTorch 2.0](https://pytorch.org/blog/pytorch-2.0-release/) has introduced a new dynamic compilation system, which, to my understanding, should replace the old scripting API. After doing a quick test, it seems that the code is still working without issues with the scripting API and can in principle also be compiled with `torch.compile`. However, it does not seem to run faster, perhaps even slower, and would need an adaption to the new compilation system.\n\nReferences\n----------\n\n```bibtex\n@misc{vecoven2020bioinspired,\n title={A bio-inspired bistable recurrent cell allows for long-lasting memory},\n author={Nicolas Vecoven and Damien Ernst and Guillaume Drion},\n year={2020},\n eprint={2006.05252},\n archivePrefix={arXiv},\n primaryClass={cs.NE}\n}\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Implementation of the bistable recurrent cell (BRC) in PyTorch",
"version": "0.1.2",
"project_urls": {
"Homepage": "https://github.com/kklemon/brc-pytorch"
},
"split_keywords": [
"artificial intelligence",
"deep learning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "fa4ac5838d50e9b7ac4e23e8114353d3dd83967b07fb863eaa53d9d0af553dd3",
"md5": "92c24380f3a0e946dd800b7febc68b2a",
"sha256": "100ef3badcafded18e627df843707de465dcf334da03845481ed55d8e294098d"
},
"downloads": -1,
"filename": "brc-0.1.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "92c24380f3a0e946dd800b7febc68b2a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 5515,
"upload_time": "2023-07-29T21:58:59",
"upload_time_iso_8601": "2023-07-29T21:58:59.370918Z",
"url": "https://files.pythonhosted.org/packages/fa/4a/c5838d50e9b7ac4e23e8114353d3dd83967b07fb863eaa53d9d0af553dd3/brc-0.1.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d58a28f3e85122f2a139b9bcc512ad4a6abc17c3d77c0a80d586241a99151875",
"md5": "a6d978971ce5a50836ddd6fc9f4b22bd",
"sha256": "bb75986cce4be8c654d381f5932914bacd1b3e1ec109537b66917d1e26ecf6f4"
},
"downloads": -1,
"filename": "brc-0.1.2.tar.gz",
"has_sig": false,
"md5_digest": "a6d978971ce5a50836ddd6fc9f4b22bd",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 6081,
"upload_time": "2023-07-29T21:59:00",
"upload_time_iso_8601": "2023-07-29T21:59:00.403534Z",
"url": "https://files.pythonhosted.org/packages/d5/8a/28f3e85122f2a139b9bcc512ad4a6abc17c3d77c0a80d586241a99151875/brc-0.1.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-29 21:59:00",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "kklemon",
"github_project": "brc-pytorch",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "brc"
}
JSON
