| Name | mindpet JSON |
| Version |
1.0.4
JSON |
| download |
| home_page | |
| Summary | Parameter-Efficient Tuning |
| upload_time | 2024-03-12 11:36:03 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | |
| license | |
| keywords |
parameter-efficient
tuning
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# MindPet微调算法用户文档
## 一、MindPet简介
MindPet（Pet：Parameter-Efficient Tuning）是属于Mindspore领域的微调算法套件。随着计算算力不断增加，大模型无限的潜力也被挖掘出来。但随之在应用和训练上带来了巨大的花销，导致商业落地困难。因此，出现一种新的参数高效（parameter-efficient）算法，与标准的全参数微调相比，这些算法仅需要微调小部分参数，可以大大降低计算和存储成本，同时可媲美全参微调的性能。
## 二、环境准备
### 2.1 环境依赖
- Python 3.7至3.9版本
- MindSpore >= 1.8
### 2.2 软件安装
在代码仓根目录下运行以下命令，会生成dist文件夹以及whl包：
```shell
python set_up.py bdist_wheel
```
执行以下命令安装whl包：
```shell
pip install dist/mindpet-1.0.4-py3-none-any.whl
```
### 2.3 软件卸载
通过以下命令进行卸载：
```shell
pip uninstall mindpet
```
## 三、微调算法API
**目前MindPet已提供以下六种经典低参微调算法以及一种提升精度的微调算法的API接口，用户可快速适配原始大模型，提升下游任务微调性能和精度；**
| 微调算法 | 算法论文 | 使用说明 |
|----------------| ----------------------------------------------------------- |-----------------------------------------------------------------|
| LoRA | LoRA: Low-Rank Adaptation of Large Language Models | [MindPet_DeltaAlgorithm_README](doc/MindPet_DeltaAlgorithm_README.md) 第一章 |
| PrefixTuning | Prefix-Tuning: Optimizing Continuous Prompts for Generation | [MindPet_DeltaAlgorithm_README](doc/MindPet_DeltaAlgorithm_README.md) 第二章 |
| Adapter | Parameter-Efficient Transfer Learning for NLP | [MindPet_DeltaAlgorithm_README](doc/MindPet_DeltaAlgorithm_README.md) 第三章 |
| LowRankAdapter | Compacter: Efficient low-rank hypercom plex adapter layers | [MindPet_DeltaAlgorithm_README](doc/MindPet_DeltaAlgorithm_README.md) 第四章 |
| BitFit | BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models | [MindPet_DeltaAlgorithm_README](doc/MindPet_DeltaAlgorithm_README.md) 第五章 |
| R_Drop | R-Drop: Regularized Dropout for Neural Networks | [MindPet_DeltaAlgorithm_README](doc/MindPet_DeltaAlgorithm_README.md) 第六章 |
| P-Tuning v2 | P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks | [MindPet_DeltaAlgorithm_README](doc/MindPet_DeltaAlgorithm_README.md) 第七章 |
## 四、共性图操作API
### 4.1 冻结指定模块功能API
MindPet支持用户根据 微调算法 或 模块名 冻结网络中部分模块，提供调用接口和配置文件两种实现方式。
使用说明参考[MindPet_GraphOperation_README](doc/MindPet_GraphOperation_README.md) 第一章。
### 4.2 保存可训练参数功能API
MindPet支持用户单独保存训练中可更新的参数为ckpt文件，从而节省存储所用的物理资源。
使用说明参考[MindPet_GraphOperation_README](doc/MindPet_GraphOperation_README.md) 第二章。
Raw data
{
"_id": null,
"home_page": "",
"name": "mindpet",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "Parameter-Efficient Tuning",
"author": "",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/79/14/471d804f322e6b9c7090653e2b0a240cea94a86183296e35db1f516d59bc/mindpet-1.0.4.tar.gz",
"platform": null,
"description": "# MindPet\u5fae\u8c03\u7b97\u6cd5\u7528\u6237\u6587\u6863\n\n\n\n## \u4e00\u3001MindPet\u7b80\u4ecb\n\nMindPet\uff08Pet\uff1aParameter-Efficient Tuning\uff09\u662f\u5c5e\u4e8eMindspore\u9886\u57df\u7684\u5fae\u8c03\u7b97\u6cd5\u5957\u4ef6\u3002\u968f\u7740\u8ba1\u7b97\u7b97\u529b\u4e0d\u65ad\u589e\u52a0\uff0c\u5927\u6a21\u578b\u65e0\u9650\u7684\u6f5c\u529b\u4e5f\u88ab\u6316\u6398\u51fa\u6765\u3002\u4f46\u968f\u4e4b\u5728\u5e94\u7528\u548c\u8bad\u7ec3\u4e0a\u5e26\u6765\u4e86\u5de8\u5927\u7684\u82b1\u9500\uff0c\u5bfc\u81f4\u5546\u4e1a\u843d\u5730\u56f0\u96be\u3002\u56e0\u6b64\uff0c\u51fa\u73b0\u4e00\u79cd\u65b0\u7684\u53c2\u6570\u9ad8\u6548\uff08parameter-efficient\uff09\u7b97\u6cd5\uff0c\u4e0e\u6807\u51c6\u7684\u5168\u53c2\u6570\u5fae\u8c03\u76f8\u6bd4\uff0c\u8fd9\u4e9b\u7b97\u6cd5\u4ec5\u9700\u8981\u5fae\u8c03\u5c0f\u90e8\u5206\u53c2\u6570\uff0c\u53ef\u4ee5\u5927\u5927\u964d\u4f4e\u8ba1\u7b97\u548c\u5b58\u50a8\u6210\u672c\uff0c\u540c\u65f6\u53ef\u5ab2\u7f8e\u5168\u53c2\u5fae\u8c03\u7684\u6027\u80fd\u3002\n\n\n## \u4e8c\u3001\u73af\u5883\u51c6\u5907\n\n### 2.1 \u73af\u5883\u4f9d\u8d56\n\n- Python 3.7\u81f33.9\u7248\u672c\n- MindSpore >= 1.8\n\n\n\n### 2.2 \u8f6f\u4ef6\u5b89\u88c5\n\n\u5728\u4ee3\u7801\u4ed3\u6839\u76ee\u5f55\u4e0b\u8fd0\u884c\u4ee5\u4e0b\u547d\u4ee4\uff0c\u4f1a\u751f\u6210dist\u6587\u4ef6\u5939\u4ee5\u53cawhl\u5305\uff1a\n\n```shell\npython set_up.py bdist_wheel\n```\n\n\u6267\u884c\u4ee5\u4e0b\u547d\u4ee4\u5b89\u88c5whl\u5305\uff1a\n```shell\npip install dist/mindpet-1.0.4-py3-none-any.whl\n```\n\n\n### 2.3 \u8f6f\u4ef6\u5378\u8f7d\n\n\u901a\u8fc7\u4ee5\u4e0b\u547d\u4ee4\u8fdb\u884c\u5378\u8f7d\uff1a\n```shell\npip uninstall mindpet\n```\n\n\n\n## \u4e09\u3001\u5fae\u8c03\u7b97\u6cd5API\n\n**\u76ee\u524dMindPet\u5df2\u63d0\u4f9b\u4ee5\u4e0b\u516d\u79cd\u7ecf\u5178\u4f4e\u53c2\u5fae\u8c03\u7b97\u6cd5\u4ee5\u53ca\u4e00\u79cd\u63d0\u5347\u7cbe\u5ea6\u7684\u5fae\u8c03\u7b97\u6cd5\u7684API\u63a5\u53e3\uff0c\u7528\u6237\u53ef\u5feb\u901f\u9002\u914d\u539f\u59cb\u5927\u6a21\u578b\uff0c\u63d0\u5347\u4e0b\u6e38\u4efb\u52a1\u5fae\u8c03\u6027\u80fd\u548c\u7cbe\u5ea6\uff1b**\n\n| \u5fae\u8c03\u7b97\u6cd5 | \u7b97\u6cd5\u8bba\u6587 | \u4f7f\u7528\u8bf4\u660e |\n|----------------| ----------------------------------------------------------- |-----------------------------------------------------------------|\n| LoRA | LoRA: Low-Rank Adaptation of Large Language Models | [MindPet_DeltaAlgorithm_README](doc/MindPet_DeltaAlgorithm_README.md) \u7b2c\u4e00\u7ae0 |\n| PrefixTuning | Prefix-Tuning: Optimizing Continuous Prompts for Generation | [MindPet_DeltaAlgorithm_README](doc/MindPet_DeltaAlgorithm_README.md) \u7b2c\u4e8c\u7ae0 |\n| Adapter | Parameter-Efficient Transfer Learning for NLP | [MindPet_DeltaAlgorithm_README](doc/MindPet_DeltaAlgorithm_README.md) \u7b2c\u4e09\u7ae0 |\n| LowRankAdapter | Compacter: Efficient low-rank hypercom plex adapter layers | [MindPet_DeltaAlgorithm_README](doc/MindPet_DeltaAlgorithm_README.md) \u7b2c\u56db\u7ae0 |\n| BitFit | BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models | [MindPet_DeltaAlgorithm_README](doc/MindPet_DeltaAlgorithm_README.md) \u7b2c\u4e94\u7ae0 |\n| R_Drop | R-Drop: Regularized Dropout for Neural Networks | [MindPet_DeltaAlgorithm_README](doc/MindPet_DeltaAlgorithm_README.md) \u7b2c\u516d\u7ae0 |\n| P-Tuning v2 | P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks | [MindPet_DeltaAlgorithm_README](doc/MindPet_DeltaAlgorithm_README.md) \u7b2c\u4e03\u7ae0 |\n\n\n\n## \u56db\u3001\u5171\u6027\u56fe\u64cd\u4f5cAPI\n\n### 4.1 \u51bb\u7ed3\u6307\u5b9a\u6a21\u5757\u529f\u80fdAPI\n\nMindPet\u652f\u6301\u7528\u6237\u6839\u636e \u5fae\u8c03\u7b97\u6cd5 \u6216 \u6a21\u5757\u540d \u51bb\u7ed3\u7f51\u7edc\u4e2d\u90e8\u5206\u6a21\u5757\uff0c\u63d0\u4f9b\u8c03\u7528\u63a5\u53e3\u548c\u914d\u7f6e\u6587\u4ef6\u4e24\u79cd\u5b9e\u73b0\u65b9\u5f0f\u3002\n\n\u4f7f\u7528\u8bf4\u660e\u53c2\u8003[MindPet_GraphOperation_README](doc/MindPet_GraphOperation_README.md) \u7b2c\u4e00\u7ae0\u3002\n\n\n\n### 4.2 \u4fdd\u5b58\u53ef\u8bad\u7ec3\u53c2\u6570\u529f\u80fdAPI\n\nMindPet\u652f\u6301\u7528\u6237\u5355\u72ec\u4fdd\u5b58\u8bad\u7ec3\u4e2d\u53ef\u66f4\u65b0\u7684\u53c2\u6570\u4e3ackpt\u6587\u4ef6\uff0c\u4ece\u800c\u8282\u7701\u5b58\u50a8\u6240\u7528\u7684\u7269\u7406\u8d44\u6e90\u3002\n\n\u4f7f\u7528\u8bf4\u660e\u53c2\u8003[MindPet_GraphOperation_README](doc/MindPet_GraphOperation_README.md) \u7b2c\u4e8c\u7ae0\u3002\n\n\n",
"bugtrack_url": null,
"license": "",
"summary": "Parameter-Efficient Tuning",
"version": "1.0.4",
"project_urls": null,
"split_keywords": [
"parameter-efficient",
"tuning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "057c3266e061b7dd74c17ce7556dde55456cedb9a931959998d2ff30c2bd4e51",
"md5": "7af40340ed61fc1ebf09b61e89f59bf9",
"sha256": "1b81f5039864f06f9118887f5051de090b4334913c0d8ff1dacecbead8f867e5"
},
"downloads": -1,
"filename": "mindpet-1.0.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7af40340ed61fc1ebf09b61e89f59bf9",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 83904,
"upload_time": "2024-03-12T11:36:01",
"upload_time_iso_8601": "2024-03-12T11:36:01.532180Z",
"url": "https://files.pythonhosted.org/packages/05/7c/3266e061b7dd74c17ce7556dde55456cedb9a931959998d2ff30c2bd4e51/mindpet-1.0.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7914471d804f322e6b9c7090653e2b0a240cea94a86183296e35db1f516d59bc",
"md5": "afcd95d9fc2a4226ba347dfb274c9ac0",
"sha256": "626d34ba8e4adab213f91c905967e2b8341ca3cbd43a6a1f656f58e58e248256"
},
"downloads": -1,
"filename": "mindpet-1.0.4.tar.gz",
"has_sig": false,
"md5_digest": "afcd95d9fc2a4226ba347dfb274c9ac0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 60314,
"upload_time": "2024-03-12T11:36:03",
"upload_time_iso_8601": "2024-03-12T11:36:03.993069Z",
"url": "https://files.pythonhosted.org/packages/79/14/471d804f322e6b9c7090653e2b0a240cea94a86183296e35db1f516d59bc/mindpet-1.0.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-12 11:36:03",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "mindpet"
}
