Afine

Name	Afine JSON
Version	0.0.2 JSON
	download
home_page	None
Summary	Afine is a gradient based PyTorch optimizer.
upload_time	2025-01-17 14:02:41
maintainer	None
docs_url	None
author	None
requires_python	>=3.8
license	None
keywords	affine afine pytorch optimizer ai neural nn
VCS
bugtrack_url
requirements	No requirements were recorded.
Travis-CI	No Travis.
coveralls test coverage	No coveralls.

            # Afine Optimizer
An optimizer that is superior to Adam. Based on @PyTorch

The training is more stable and reaches a more **elegant** optimal solution faster.

![image](rendering.png)
![image](renderingZoom.png)

## 思路说明
把每组参数看作高维向量. 优化步骤被分割为3个过程: 旋转、缩放、平移. 使用三个学习率分别控制三个过程的调整量. 

1.旋转
- 计算梯度对参数向量方向的影响.
- 根据旋转的稳定程度调整旋转的作用强度. 
- 使用近似旋转改变参数向量方向而不改变模长.

2.缩放
- 计算梯度对参数向量模长的影响.
- 根据模长改变的稳定程度调整缩放的作用强度.
- 权重衰减作用于这一步: 使用ArcSinh函数对向量放大带来阻力而促进向量缩小.
- 应用缩放变换.

3.平移
- 旋转和缩放对零向量无作用, 为使零初始化和模长较小的向量也能被训练, 需加入平移.
- 此处的平移带有惯性,且平移量与梯度稳定程度有关.

## Demo程序
demo程序在test.py文件中,直接运行即可得到此页面中Affine与RAdam的对比图.(依赖visdom)
demo中构建了一个神经网络用简陋的方式对一个包含动态随机参数的公式生成的周期性曲线进行预测.

## Install
```bash
pip install afine
```

## Use
```python
from Afine import Afine
...
optimizer=Affine(moduel.parameters(),lr_rotate = lr_rotate,lr_scale = lr_scale,lr_base = lr_base,weight_decay=weight_decay)
```

## HomePage
<https://github.com/PsycheHalo/Afine/>

            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "Afine",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.8",
    "maintainer_email": null,
    "keywords": "affine, Afine, PyTorch, optimizer, AI, neural, NN",
    "author": null,
    "author_email": "Psyche Halo <Psyche@WinsGod.com>",
    "download_url": "https://files.pythonhosted.org/packages/aa/54/7cbb388146f4d4aceda6a2285e8ab8e1f58556996c86b21ccb6b055ad23e/afine-0.0.2.tar.gz",
    "platform": null,
    "description": "# Afine Optimizer\nAn optimizer that is superior to Adam. Based on @PyTorch\n\nThe training is more stable and reaches a more **elegant** optimal solution faster.\n\n![image](rendering.png)\n![image](renderingZoom.png)\n\n## \u601d\u8def\u8bf4\u660e\n\u628a\u6bcf\u7ec4\u53c2\u6570\u770b\u4f5c\u9ad8\u7ef4\u5411\u91cf. \u4f18\u5316\u6b65\u9aa4\u88ab\u5206\u5272\u4e3a3\u4e2a\u8fc7\u7a0b: \u65cb\u8f6c\u3001\u7f29\u653e\u3001\u5e73\u79fb. \u4f7f\u7528\u4e09\u4e2a\u5b66\u4e60\u7387\u5206\u522b\u63a7\u5236\u4e09\u4e2a\u8fc7\u7a0b\u7684\u8c03\u6574\u91cf. \n\n1.\u65cb\u8f6c\n- \u8ba1\u7b97\u68af\u5ea6\u5bf9\u53c2\u6570\u5411\u91cf\u65b9\u5411\u7684\u5f71\u54cd.\n- \u6839\u636e\u65cb\u8f6c\u7684\u7a33\u5b9a\u7a0b\u5ea6\u8c03\u6574\u65cb\u8f6c\u7684\u4f5c\u7528\u5f3a\u5ea6. \n- \u4f7f\u7528\u8fd1\u4f3c\u65cb\u8f6c\u6539\u53d8\u53c2\u6570\u5411\u91cf\u65b9\u5411\u800c\u4e0d\u6539\u53d8\u6a21\u957f.\n\n2.\u7f29\u653e\n- \u8ba1\u7b97\u68af\u5ea6\u5bf9\u53c2\u6570\u5411\u91cf\u6a21\u957f\u7684\u5f71\u54cd.\n- \u6839\u636e\u6a21\u957f\u6539\u53d8\u7684\u7a33\u5b9a\u7a0b\u5ea6\u8c03\u6574\u7f29\u653e\u7684\u4f5c\u7528\u5f3a\u5ea6.\n- \u6743\u91cd\u8870\u51cf\u4f5c\u7528\u4e8e\u8fd9\u4e00\u6b65: \u4f7f\u7528ArcSinh\u51fd\u6570\u5bf9\u5411\u91cf\u653e\u5927\u5e26\u6765\u963b\u529b\u800c\u4fc3\u8fdb\u5411\u91cf\u7f29\u5c0f.\n- \u5e94\u7528\u7f29\u653e\u53d8\u6362.\n\n3.\u5e73\u79fb\n- \u65cb\u8f6c\u548c\u7f29\u653e\u5bf9\u96f6\u5411\u91cf\u65e0\u4f5c\u7528, \u4e3a\u4f7f\u96f6\u521d\u59cb\u5316\u548c\u6a21\u957f\u8f83\u5c0f\u7684\u5411\u91cf\u4e5f\u80fd\u88ab\u8bad\u7ec3, \u9700\u52a0\u5165\u5e73\u79fb.\n- \u6b64\u5904\u7684\u5e73\u79fb\u5e26\u6709\u60ef\u6027,\u4e14\u5e73\u79fb\u91cf\u4e0e\u68af\u5ea6\u7a33\u5b9a\u7a0b\u5ea6\u6709\u5173.\n\n## Demo\u7a0b\u5e8f\ndemo\u7a0b\u5e8f\u5728test.py\u6587\u4ef6\u4e2d,\u76f4\u63a5\u8fd0\u884c\u5373\u53ef\u5f97\u5230\u6b64\u9875\u9762\u4e2dAffine\u4e0eRAdam\u7684\u5bf9\u6bd4\u56fe.(\u4f9d\u8d56visdom)\ndemo\u4e2d\u6784\u5efa\u4e86\u4e00\u4e2a\u795e\u7ecf\u7f51\u7edc\u7528\u7b80\u964b\u7684\u65b9\u5f0f\u5bf9\u4e00\u4e2a\u5305\u542b\u52a8\u6001\u968f\u673a\u53c2\u6570\u7684\u516c\u5f0f\u751f\u6210\u7684\u5468\u671f\u6027\u66f2\u7ebf\u8fdb\u884c\u9884\u6d4b.\n\n## Install\n```bash\npip install afine\n```\n\n## Use\n```python\nfrom Afine import Afine\n...\noptimizer=Affine(moduel.parameters(),lr_rotate = lr_rotate,lr_scale = lr_scale,lr_base = lr_base,weight_decay=weight_decay)\n```\n\n## HomePage\n<https://github.com/PsycheHalo/Afine/>\n",
    "bugtrack_url": null,
    "license": null,
    "summary": "Afine is a gradient based PyTorch optimizer.",
    "version": "0.0.2",
    "project_urls": {
        "Homepage": "https://github.com/PsycheHalo/Afine",
        "Issues": "https://github.com/PsycheHalo/Afine/issues"
    },
    "split_keywords": [
        "affine",
        " afine",
        " pytorch",
        " optimizer",
        " ai",
        " neural",
        " nn"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "2253220ec07932588d5392f0bd6ffa69fc725863d34e30cb91eceaa193778af8",
                "md5": "c04008707e03985778636c6be0877da0",
                "sha256": "651a8dd60ccc5362c60624577e6829b92750d0e31bf2087fb3247651be806ee5"
            },
            "downloads": -1,
            "filename": "Afine-0.0.2-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "c04008707e03985778636c6be0877da0",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.8",
            "size": 11404,
            "upload_time": "2025-01-17T14:02:40",
            "upload_time_iso_8601": "2025-01-17T14:02:40.345004Z",
            "url": "https://files.pythonhosted.org/packages/22/53/220ec07932588d5392f0bd6ffa69fc725863d34e30cb91eceaa193778af8/Afine-0.0.2-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "aa547cbb388146f4d4aceda6a2285e8ab8e1f58556996c86b21ccb6b055ad23e",
                "md5": "b33f28d5411e6e1237f95f9a458d9a83",
                "sha256": "e1357e4e9483e80f35c3bee6daa1ef6ab488656391671a50d0691063f6a0b4aa"
            },
            "downloads": -1,
            "filename": "afine-0.0.2.tar.gz",
            "has_sig": false,
            "md5_digest": "b33f28d5411e6e1237f95f9a458d9a83",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.8",
            "size": 11070,
            "upload_time": "2025-01-17T14:02:41",
            "upload_time_iso_8601": "2025-01-17T14:02:41.777412Z",
            "url": "https://files.pythonhosted.org/packages/aa/54/7cbb388146f4d4aceda6a2285e8ab8e1f58556996c86b21ccb6b055ad23e/afine-0.0.2.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-01-17 14:02:41",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "PsycheHalo",
    "github_project": "Afine",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "afine"
}