| Name | ml-rwkv JSON |
| Version |
0.0.6
JSON |
| download |
| home_page | https://github.com/codekansas/rwkv |
| Summary | RWKV implementation that is friendly with `ml-starter` |
| upload_time | 2023-06-23 02:50:59 |
| maintainer | |
| docs_url | None |
| author | Benjamin Bolte |
| requires_python | >=3.10 |
| license | |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# RWKV
This repository contains my implementation of the pretrained RWKV model, with numerically-stable Triton kernels and code for doing LoRA fine-tuning.
For an explanation of the math, see [this accompanying blog post](https://ben.bolte.cc/rwkv-model).
## Getting Started
Install the package:
```
pip install ml-rwkv
```
Generate code:
```
rwkv 169m "Scientists recently discovered a herd of Chinese-speaking goats. To their surprise,"
```
## WKV Computations
This repo currently contains 3 different implementations of the WKV computation:
1. Vanilla
2. Eps
3. Log
See the blog post linked above for more details about how each of these works.
## Speed
The following table shows measured tokens per second for generating a sequence on various devices using the `run.py` script. Take these numbers with a heap of salt though because the inference pass is so fast that the overhead of the Python script is significant (I only observed around 15% GPU utilization on my 4090). For really fast inference, I would suggest using [this implementation](https://github.com/saharNooby/rwkv.cpp), which uses GGML.
| Device | Vanilla | Eps | Log |
| ----------- | ------- | ----- | ----- |
| M2 (Metal) | 30.87 | 26.56 | 26.99 |
| M2 (CPU) | 61.25 | 58.90 | 60.69 |
| 4090 (BF16) | 101.94 | 99.28 | 93.11 |
| 4090 (Log) | 100.86 | 96.05 | 92.61 |
Raw data
{
"_id": null,
"home_page": "https://github.com/codekansas/rwkv",
"name": "ml-rwkv",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": "",
"keywords": "",
"author": "Benjamin Bolte",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/79/b5/074c608c332ee917f31be6b8d86227f45f62483fa18b8ab84e94fa1184d2/ml-rwkv-0.0.6.tar.gz",
"platform": null,
"description": "# RWKV\n\nThis repository contains my implementation of the pretrained RWKV model, with numerically-stable Triton kernels and code for doing LoRA fine-tuning.\n\nFor an explanation of the math, see [this accompanying blog post](https://ben.bolte.cc/rwkv-model).\n\n## Getting Started\n\nInstall the package:\n\n```\npip install ml-rwkv\n```\n\nGenerate code:\n\n```\nrwkv 169m \"Scientists recently discovered a herd of Chinese-speaking goats. To their surprise,\"\n```\n\n## WKV Computations\n\nThis repo currently contains 3 different implementations of the WKV computation:\n\n1. Vanilla\n2. Eps\n3. Log\n\nSee the blog post linked above for more details about how each of these works.\n\n## Speed\n\nThe following table shows measured tokens per second for generating a sequence on various devices using the `run.py` script. Take these numbers with a heap of salt though because the inference pass is so fast that the overhead of the Python script is significant (I only observed around 15% GPU utilization on my 4090). For really fast inference, I would suggest using [this implementation](https://github.com/saharNooby/rwkv.cpp), which uses GGML.\n\n| Device | Vanilla | Eps | Log |\n| ----------- | ------- | ----- | ----- |\n| M2 (Metal) | 30.87 | 26.56 | 26.99 |\n| M2 (CPU) | 61.25 | 58.90 | 60.69 |\n| 4090 (BF16) | 101.94 | 99.28 | 93.11 |\n| 4090 (Log) | 100.86 | 96.05 | 92.61 |\n",
"bugtrack_url": null,
"license": "",
"summary": "RWKV implementation that is friendly with `ml-starter`",
"version": "0.0.6",
"project_urls": {
"Homepage": "https://github.com/codekansas/rwkv"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "4e75263c6b2e547c558100c89e5c7597f63e1526243fa559553c541deb48a283",
"md5": "662d92479f37e72804272304ef7a0cb3",
"sha256": "a4ccb9be61ea35bbef436ae3eaf7fbb45c01ac18f1d1bbca705a8c3497a96ae9"
},
"downloads": -1,
"filename": "ml_rwkv-0.0.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "662d92479f37e72804272304ef7a0cb3",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 25068,
"upload_time": "2023-06-23T02:50:57",
"upload_time_iso_8601": "2023-06-23T02:50:57.821274Z",
"url": "https://files.pythonhosted.org/packages/4e/75/263c6b2e547c558100c89e5c7597f63e1526243fa559553c541deb48a283/ml_rwkv-0.0.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "79b5074c608c332ee917f31be6b8d86227f45f62483fa18b8ab84e94fa1184d2",
"md5": "e0fb8c9138d15e08a45948cf7602b2fb",
"sha256": "ffc3b108ff6771295c3dba47acf2c385fd3c88f09448e05ae79de289ca4b3084"
},
"downloads": -1,
"filename": "ml-rwkv-0.0.6.tar.gz",
"has_sig": false,
"md5_digest": "e0fb8c9138d15e08a45948cf7602b2fb",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 19248,
"upload_time": "2023-06-23T02:50:59",
"upload_time_iso_8601": "2023-06-23T02:50:59.244841Z",
"url": "https://files.pythonhosted.org/packages/79/b5/074c608c332ee917f31be6b8d86227f45f62483fa18b8ab84e94fa1184d2/ml-rwkv-0.0.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-23 02:50:59",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "codekansas",
"github_project": "rwkv",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "ml-rwkv"
}
