# SimplePE
Simple implementation of some Positional Encoding methods in PyTorch.
- [x] Sinusoidal Positional Encoding
- [x] Rotary Positional Encoding
## Installation
```bash
git clone https://github.com/donglinkang2021/simplepe.git
cd simplepe
pip install .
```
## Usage
```python
python example.py # pe value
python example2.py # attention score
```
You can see the following results:
| Method | Sinusoidal | Rotary |
| ------------ | ---------- | ------ |
| Dimension-Time_Step Heatmap |  |  |
| Value-Time_Step Line_Plot |  |  |
| Multi-Head Dimension-Time_Step Heatmap |  |  |
| Multi-Head Value-Time_Step Line_Plot |  |  |
| Method\Time_Step-Time_Step Heatmap | T=2048, D=32 | T=2048, D=64 | T=4096, D=512 |
| ------------ | ---------- | ------ | ------ |
| Sinusoidal |  |  |  |
| Rotary |  |  |  |
| Method\Attention_Score-Time_Step Line_Plot | T=2048, D=32 | T=2048, D=64 | T=4096, D=512 |
| ------------ | ---------- | ------ | ------ |
| Sinusoidal |  |  |  |
| Rotary |  |  |  |
Raw data
{
"_id": null,
"home_page": "https://github.com/donglinkang2021/simplepe",
"name": "simplepe",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "artificial intelligence, deep learning, positional embedding",
"author": "Dong Linkang",
"author_email": "donglinkang2021@163.com",
"download_url": "https://files.pythonhosted.org/packages/60/ca/847ebd6101d571b7edd2c56f862cf89657b51018fb4f5e26baca742d0780/simplepe-0.0.3.tar.gz",
"platform": null,
"description": "# SimplePE\n\nSimple implementation of some Positional Encoding methods in PyTorch.\n\n- [x] Sinusoidal Positional Encoding\n- [x] Rotary Positional Encoding\n\n## Installation\n\n```bash\ngit clone https://github.com/donglinkang2021/simplepe.git\ncd simplepe\npip install .\n```\n\n## Usage\n\n```python\npython example.py # pe value\npython example2.py # attention score\n```\n\nYou can see the following results:\n\n| Method | Sinusoidal | Rotary |\n| ------------ | ---------- | ------ |\n| Dimension-Time_Step Heatmap |  |  |\n| Value-Time_Step Line_Plot |  |  |\n| Multi-Head Dimension-Time_Step Heatmap |  |  |\n| Multi-Head Value-Time_Step Line_Plot |  |  |\n\n| Method\\Time_Step-Time_Step Heatmap | T=2048, D=32 | T=2048, D=64 | T=4096, D=512 |\n| ------------ | ---------- | ------ | ------ |\n| Sinusoidal |  |  |  |\n| Rotary |  |  |  |\n\n| Method\\Attention_Score-Time_Step Line_Plot | T=2048, D=32 | T=2048, D=64 | T=4096, D=512 |\n| ------------ | ---------- | ------ | ------ |\n| Sinusoidal |  |  |  |\n| Rotary |  |  |  |\n\n\n",
"bugtrack_url": null,
"license": null,
"summary": "A simple simplepe for positional encoding",
"version": "0.0.3",
"project_urls": {
"Homepage": "https://github.com/donglinkang2021/simplepe"
},
"split_keywords": [
"artificial intelligence",
" deep learning",
" positional embedding"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "edea013f7449aabcc3f20a31ddd5f2b006bf0da21fc4cab67d39447f0bbef7ea",
"md5": "84edc8f14c41dc9c19abad8af2db5790",
"sha256": "6def100b601e9f8f71f1a69807abd15c2f8f2d97e541762bee8991fe0a266fb8"
},
"downloads": -1,
"filename": "simplepe-0.0.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "84edc8f14c41dc9c19abad8af2db5790",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 4527,
"upload_time": "2025-01-08T05:49:31",
"upload_time_iso_8601": "2025-01-08T05:49:31.965794Z",
"url": "https://files.pythonhosted.org/packages/ed/ea/013f7449aabcc3f20a31ddd5f2b006bf0da21fc4cab67d39447f0bbef7ea/simplepe-0.0.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "60ca847ebd6101d571b7edd2c56f862cf89657b51018fb4f5e26baca742d0780",
"md5": "b6ccce1ecc1c7c3147f8391517929874",
"sha256": "9871208258d5b54d0df4eb406cd877062ce781dfa3e4180971fcd5f7639433b5"
},
"downloads": -1,
"filename": "simplepe-0.0.3.tar.gz",
"has_sig": false,
"md5_digest": "b6ccce1ecc1c7c3147f8391517929874",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 3745,
"upload_time": "2025-01-08T05:49:35",
"upload_time_iso_8601": "2025-01-08T05:49:35.471136Z",
"url": "https://files.pythonhosted.org/packages/60/ca/847ebd6101d571b7edd2c56f862cf89657b51018fb4f5e26baca742d0780/simplepe-0.0.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-08 05:49:35",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "donglinkang2021",
"github_project": "simplepe",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "simplepe"
}
JSON
