[**🇨🇳中文**](https://github.com/shibing624/textgen/blob/main/README.md) | [**🌐English**](https://github.com/shibing624/textgen/blob/main/README_EN.md) | [**📖文档/Docs**](https://github.com/shibing624/textgen/wiki) | [**🤖模型/Models**](https://huggingface.co/shibing624)
<div align="center">
<a href="https://github.com/shibing624/textgen">
<img src="https://github.com/shibing624/textgen/blob/main/docs/logo.svg" alt="Logo">
</a>
</div>
-----------------
# TextGen: Implementation of Text Generation models
[](https://badge.fury.io/py/textgen)
[](https://pepy.tech/project/textgen)
[](CONTRIBUTING.md)
[](LICENSE)
[](requirements.txt)
[](https://github.com/shibing624/textgen/issues)
[](#Contact)
## 📖 Introduction
**TextGen**实现了多种文本生成模型，包括：LLaMA、ChatGLM、UDA、GPT2、Seq2Seq、BART、T5、SongNet等模型，开箱即用。
## 🔥 News
[2023/09/05] v1.1.1版本: 支持多卡推理，推理速度加倍，调库textgen做batch推理，多卡推理更方便、快速。详见[Release-v1.1.1](https://github.com/shibing624/textgen/releases/tag/1.1.1)
[2023/08/23] v1.1.0版本: 发布基于ShareGPT4数据集微调的中英文Vicuna-13B模型[shibing624/vicuna-baichuan-13b-chat](https://huggingface.co/shibing624/vicuna-baichuan-13b-chat)，和对应的LoRA模型[shibing624/vicuna-baichuan-13b-chat-lora](https://huggingface.co/shibing624/vicuna-baichuan-13b-chat-lora)，支持多轮对话，评测效果有提升，详见[Release-v1.1.0](https://github.com/shibing624/textgen/releases/tag/1.1.0)
[2023/08/02] v1.0.2版本: 新增支持ChatGLM2和LLaMA2模型的SFT微调训练，详见[Release-v1.0.2](https://github.com/shibing624/textgen/releases/tag/1.0.2)
[2023/06/15] v1.0.0版本: 新增ChatGLM/LLaMA/Bloom模型的多轮对话微调训练，并发布医疗问诊LoRA模型[shibing624/ziya-llama-13b-medical-lora](https://huggingface.co/shibing624/ziya-llama-13b-medical-lora)。详见[Release-v1.0.0](https://github.com/shibing624/textgen/releases/tag/1.0.0)
[2023/06/02] v0.2.7版本: 新增ChatGLM/LLaMA/Bloom模型的SFT微调训练，并发布适用于通用对话和中文纠错的LoRA模型。详见[Release-v0.2.7](https://github.com/shibing624/textgen/releases/tag/0.2.7)
## 😊 Feature
- [GPT](textgen/gpt)：本项目基于PyTorch实现了ChatGLM-6B/Baichuan/LLaMA2/BLOOM等GPT模型LoRA微调训练和预测，可以用于对话生成任务和领域微调训练
- [UDA/EDA](textgen/augment/word_level_augment.py)：本项目实现了UDA(非核心词替换)、EDA和Back Translation(回译)算法，基于TF-IDF将句子中部分不重要词替换为同义词，随机词插入、删除、替换等方法，产生新的文本，实现了文本扩增
- [Seq2Seq](textgen/seq2seq)：本项目基于PyTorch实现了Seq2Seq、ConvSeq2Seq、BART模型的训练和预测，可以用于文本翻译、对话生成、摘要生成等文本生成任务
- [T5](textgen/t5)：本项目基于PyTorch实现了T5和CopyT5模型训练和预测，可以用于文本翻译、对话生成、对联生成、文案撰写等文本生成任务
- [GPT2](textgen/language_modeling)：本项目基于PyTorch实现了GTP2模型训练和预测，可以用于文章生成、对联生成等文本生成任务
- [SongNet](textgen/language_modeling/songnet_model.py)：本项目基于PyTorch实现了SongNet模型训练和预测，可以用于规范格式的诗词、歌词等文本生成任务
- [TGLS](textgen/unsup_generation)：本项目实现了[TGLS](https://www.jiqizhixin.com/articles/2020-08-11-5)无监督相似文本生成模型，是一种“先搜索后学习”的文本生成方法，通过反复迭代学习候选集，最终模型能生成类似候选集的高质量相似文本
### Release Models
release基于`textgen`训练的中文模型，模型已经release到HuggingFace models，指定模型名称`textgen`会自动下载模型，可直接使用。
| Model | Arch | Introduction | Train Script | Predict Script |
|:----------------------------------------------------------------------------------------------------------|:-------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------|
| [shibing624/t5-chinese-couplet](https://huggingface.co/shibing624/t5-chinese-couplet) | T5 | fine-tuned中文对联后的模型 | [对联生成模型调研](https://github.com/shibing624/textgen/blob/main/docs/%E5%AF%B9%E8%81%94%E7%94%9F%E6%88%90%E6%A8%A1%E5%9E%8B%E5%AF%B9%E6%AF%94.md) | [predict script](https://github.com/shibing624/textgen/blob/main/examples/t5/t5_couplet_demo.py) |
| [shibing624/songnet-base-chinese-songci](https://huggingface.co/shibing624/songnet-base-chinese-songci) | SongNet | fine-tuned宋词后的模型 | [training script](https://github.com/shibing624/textgen/blob/main/examples/songnet/training_zh_songnet_demo.py) | [predict script](https://github.com/shibing624/textgen/blob/main/examples/songnet/songnet_songci_demo.py) |
| [shibing624/songnet-base-chinese-couplet](https://huggingface.co/shibing624/songnet-base-chinese-couplet) | SongNet | fine-tuned对联后的模型 | [training script](https://github.com/shibing624/textgen/blob/main/examples/songnet/training_zh_songnet_demo.py) | [predict script](https://github.com/shibing624/textgen/blob/main/examples/songnet/songnet_couplet_demo.py) |
| [shibing624/chatglm-6b-csc-zh-lora](https://huggingface.co/shibing624/chatglm-6b-csc-zh-lora) | ChatGLM-6B | 在27万中文拼写纠错数据[shibing624/CSC](https://huggingface.co/datasets/shibing624/CSC)上微调了一版ChatGLM-6B，纠错效果有提升，发布微调后的LoRA权重 | [training script](https://github.com/shibing624/textgen/blob/main/examples/chatglm/training_chatglm_csc_demo.py) | [predict script](https://github.com/shibing624/textgen/blob/main/examples/chatglm/csc_demo.py) |
| [shibing624/chatglm-6b-belle-zh-lora](https://huggingface.co/shibing624/chatglm-6b-belle-zh-lora) | ChatGLM-6B | 在100万条中文ChatGPT指令Belle数据集[BelleGroup/train_1M_CN](https://huggingface.co/datasets/BelleGroup/train_1M_CN)上微调了一版ChatGLM-6B，问答效果有提升，发布微调后的LoRA权重 | [training script](https://github.com/shibing624/textgen/blob/main/examples/chatglm/training_chatglm_hfdataset_demo.py) | [predict script](https://github.com/shibing624/textgen/blob/main/examples/chatglm/training_chatglm_hfdataset_demo.py) |
| [shibing624/llama-13b-belle-zh-lora](https://huggingface.co/shibing624/llama-13b-belle-zh-lora) | LLaMA-13B | 在100万条中文ChatGPT指令Belle数据集[BelleGroup/train_1M_CN](https://huggingface.co/datasets/BelleGroup/train_1M_CN)上微调了一版Llama-13B，问答效果有提升，发布微调后的LoRA权重 | [training script](https://github.com/shibing624/textgen/blob/main/examples/llama/training_llama_hfdataset_demo.py) | [predict script](https://github.com/shibing624/textgen/blob/main/examples/llama/training_llama_hfdataset_demo.py) |
| [shibing624/chinese-alpaca-plus-7b-hf](https://huggingface.co/shibing624/chinese-alpaca-plus-7b-hf) | LLaMA-7B | [中文LLaMA-Plus, Alpaca-Plus 7B版本](https://github.com/ymcui/Chinese-LLaMA-Alpaca/releases/tag/v3.0)，在LLaMA-7B上扩充了中文词表并继续预训练120G文本（通用领域），在4M指令数据集上微调后得到的中文Alpaca-plus模型 | [training script](https://github.com/shibing624/textgen/blob/main/examples/gpt/training_llama_demo.py) | [predict script](https://github.com/shibing624/textgen/blob/main/examples/gpt/training_llama_demo.py) |
| [shibing624/chinese-alpaca-plus-13b-hf](https://huggingface.co/shibing624/chinese-alpaca-plus-13b-hf) | LLaMA-13B | [中文LLaMA-Plus, Alpaca-Plus 13B版本](https://github.com/ymcui/Chinese-LLaMA-Alpaca/releases/tag/v3.1)，在LLaMA-13B上扩充了中文词表并继续预训练120G文本（通用领域），在4.3M指令数据集上微调后得到的中文Alpaca-plus模型 | [training script](https://github.com/shibing624/textgen/blob/main/examples/gpt/training_llama_demo.py) | [predict script](https://github.com/shibing624/textgen/blob/main/examples/gpt/training_llama_demo.py) |
| [shibing624/ziya-llama-13b-medical-lora](https://huggingface.co/shibing624/ziya-llama-13b-medical-lora) | LLaMA-13B | 在240万条中英文医疗数据集[shibing624/medical](https://huggingface.co/datasets/shibing624/medical)上微调了一版Ziya-LLaMA-13B模型，医疗问答效果有提升，发布微调后的LoRA权重 | [training script](https://github.com/shibing624/textgen/blob/main/examples/gpt/training_llama_demo.py) | [predict script](https://github.com/shibing624/textgen/blob/main/examples/gpt/training_llama_demo.py) |
| [shibing624/vicuna-baichuan-13b-chat](https://huggingface.co/shibing624/vicuna-baichuan-13b-chat) | Baichuan-13B-Chat | 在10万条多语言ShareGPT GPT4多轮对话数据集[shibing624/sharegpt_gpt4](https://huggingface.co/datasets/shibing624/sharegpt_gpt4)上SFT微调了一版baichuan-13b-chat多轮问答模型，日常问答和医疗问答效果有提升，发布微调后的完整模型权重 | [training script](https://github.com/shibing624/textgen/blob/main/examples/gpt/training_llama_demo.py) | [predict script](https://github.com/shibing624/textgen/blob/main/examples/gpt/training_llama_demo.py) |
### Evaluation
| Model | Arch | Introduction | Score |
|:--------------------------------------------------------------------------------------------------------------------------------------------|:-----------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------|
| [LLaMA-7B-Chinese-Alpaca](https://huggingface.co/ziqingyang/chinese-alpaca-lora-7b) | LLaMA-7B | 复用[ymcui/Chinese-LLaMA-Alpaca](https://github.com/ymcui/Chinese-LLaMA-Alpaca/blob/main/examples/README.md)的评估case和得分 | 4.92 |
| [LLaMA-13B-Chinese-Alpaca](https://huggingface.co/ziqingyang/chinese-alpaca-lora-13b) | LLaMA-13B | 复用[ymcui/Chinese-LLaMA-Alpaca](https://github.com/ymcui/Chinese-LLaMA-Alpaca/blob/main/examples/README.md)的评估case和得分 | 7.05 |
| [ChatGLM-6B](https://huggingface.co/THUDM/chatglm-6b) | ChatGLM-6B | 基于原生`THUDM/chatglm-6b`评估测试集得分 | 7.16 |
| [ChatGLM-6B-v1.1](https://huggingface.co/THUDM/chatglm-6b) | ChatGLM-6B | 基于原生`THUDM/chatglm-6b`v1.1英文优化版模型评估测试集得分 | **7.18** |
| [shibing624/chatglm-6b-belle-zh-lora](https://huggingface.co/shibing624/chatglm-6b-belle-zh-lora) | ChatGLM-6B | 基于`THUDM/chatglm-6b`加载`shibing624/chatglm-6b-belle-zh-lora`LoRA模型后评估测试集得分 | 7.03 |
| [facat/alpaca-lora-cn-13b](https://huggingface.co/facat/alpaca-lora-cn-13b) | LLaMA-13B | 基于`decapoda-research/llama-13b-hf`加载`facat/alpaca-lora-cn-13b`LoRA模型后评估测试集并标注得分 | 4.13 |
| [Chinese-Vicuna/Chinese-Vicuna-lora-13b-belle-and-guanaco](https://huggingface.co/Chinese-Vicuna/Chinese-Vicuna-lora-13b-belle-and-guanaco) | LLaMA-13B | 基于`decapoda-research/llama-13b-hf`加载`Chinese-Vicuna/Chinese-Vicuna-lora-13b-belle-and-guanaco`LoRA模型后评估测试集并标注得分 | 3.98 |
| [shibing624/chinese-alpaca-plus-7b-hf](https://huggingface.co/shibing624/chinese-alpaca-plus-7b-hf) | LLaMA-7B | 使用[ymcui/Chinese-LLaMA-Alpaca 合并模型方法](https://github.com/ymcui/Chinese-LLaMA-Alpaca/wiki/%E6%89%8B%E5%8A%A8%E6%A8%A1%E5%9E%8B%E5%90%88%E5%B9%B6%E4%B8%8E%E8%BD%AC%E6%8D%A2#%E5%A4%9Alora%E6%9D%83%E9%87%8D%E5%90%88%E5%B9%B6%E9%80%82%E7%94%A8%E4%BA%8Echinese-alpaca-plus)合并HF权重后，评估测试集并标注得分 | 6.93 |
| [shibing624/chinese-alpaca-plus-13b-hf](https://huggingface.co/shibing624/chinese-alpaca-plus-13b-hf) | LLaMA-13B | 使用[ymcui/Chinese-LLaMA-Alpaca 合并模型方法](https://github.com/ymcui/Chinese-LLaMA-Alpaca/wiki/%E6%89%8B%E5%8A%A8%E6%A8%A1%E5%9E%8B%E5%90%88%E5%B9%B6%E4%B8%8E%E8%BD%AC%E6%8D%A2#%E5%A4%9Alora%E6%9D%83%E9%87%8D%E5%90%88%E5%B9%B6%E9%80%82%E7%94%A8%E4%BA%8Echinese-alpaca-plus)合并HF权重后，评估测试集并标注得分 | 7.07 |
| [TheBloke/vicuna-13B-1.1-HF](https://huggingface.co/TheBloke/vicuna-13B-1.1-HF) | LLaMA-13B | 使用原生vicuna-13B-1.1合并后的模型，评估测试集并标注得分 | 5.13 |
| [IDEA-CCNL/Ziya-LLaMA-13B-v1](https://huggingface.co/IDEA-CCNL/Ziya-LLaMA-13B-v1) | LLaMA-13B | 使用姜子牙通用大模型V1，评估测试集并标注得分 | 6.63 |
说明：
- 评估case，详见在线文档：中文LLM-benchmark多任务评估集(腾讯文档) https://docs.qq.com/sheet/DUUpsREtWbFBsUVJE?tab=r7io7g 感谢韩俊明、[杨家铭](https://github.com/yangjiam)等同学的标注
- 评估任务类型包括：知识问答，开放式问答，数值计算，诗词、音乐、体育，娱乐，写文章，文本翻译，代码编程，伦理、拒答类，多轮问答，Score 评分是前100条（10分制）的平均分数，人工打分，越高越好
- 评估数量少，任务类型不够全面，评分之间的大小关系有一些参考价值，分数的绝对值没太大参考价值
- 评估脚本：[tests/test_benchmark.py](https://github.com/shibing624/textgen/blob/main/tests/test_benchmark.py) ，使用fp16预测，无int量化处理，运行脚本可复现评估结果，但生成结果具有随机性，受解码超参、随机种子等因素影响。评测并非绝对严谨，测试结果仅供晾晒参考
- 结论：ChatGLM-6B、LLaMA-13B的中文衍生模型（包括alpaca-plus, vicuna, ziya）的表现属于第一梯队，原版LLaMA-7B的表现整体稍差些
- LLaMA-13B-Chinese-Alpaca是在原版LLaMA上扩充了中文词表，并融入了约20G的通用中文语料后的指令微调模型，表明了LLaMA的底座优秀，具有强大的语言迁移能力
- ChatGLM这种原生的中文预训练模型更理解中文语义，且在中文知识问答、开放式问答得分高
- LLaMA系列模型数值计算、中英翻译、代码编程类得分高
- 经过中文预训练和SFT微调后的Chinese-LLaMA模型在中文诗词、娱乐、伦理类得分相较原版LLaMA有提升
## 🚀 Demo
HuggingFace Demo: https://huggingface.co/spaces/shibing624/chinese-couplet-generate
run example: [examples/T5/gradio_demo.py](examples/T5/gradio_demo.py) to see the demo:
```shell
python examples/T5/gradio_demo.py
```
model trained by [examples/t5/T5_Finetune_Chinese_Couplet.ipynb](https://github.com/shibing624/textgen/blob/main/examples/t5/T5_Finetune_Chinese_Couplet.ipynb)
## 💾 Install
```shell
pip install -U textgen
```
or
install develop version:
```shell
pip install torch # conda install pytorch
git clone https://github.com/shibing624/textgen.git
cd textgen
python setup.py install
```
## ▶️ Usage
### ChatGLM-6B 模型
#### 使用 ChatGLM-6B 微调后的模型
example: [examples/gpt/inference_demo.py](https://github.com/shibing624/textgen/blob/main/examples/gpt/inference_demo.py)
```python
from textgen import GptModel
model = GptModel("chatglm", "THUDM/chatglm-6b", peft_name="shibing624/chatglm-6b-csc-zh-lora")
r = model.predict(["介绍下北京"])
print(r) # ['北京是中国的首都...']
```
#### 训练 ChatGLM-6B 微调模型
1. 支持自定义训练数据集和训练参数，数据集格式参考[examples/data/sharegpt_zh_100_format.jsonl](https://github.com/shibing624/textgen/blob/main/examples/data/sharegpt_zh_100_format.jsonl)
2. 支持QLoRA、AdaLoRA、LoRA、P_Tuning、Prefix_Tuning等部分参数微调方法，也支持全参微调
3. 支持多卡训练，支持混合精度训练
4. 支持多卡推理
example: [examples/gpt/training_chatglm_demo.py](https://github.com/shibing624/textgen/blob/main/examples/gpt/training_chatglm_demo.py)
单卡训练：
```shell
cd examples/gpt
CUDA_VISIBLE_DEVICES=0 python training_chatglm_demo.py --do_train --do_predict --num_epochs 1 --output_dir outputs_chatglm_v1
```
多卡训练：
```shell
cd examples/gpt
CUDA_VISIBLE_DEVICES=0,1 torchrun --nproc_per_node 2 training_chatglm_demo.py --do_train --do_predict --num_epochs 20 --output_dir outputs_chatglm_v1
```
多卡推理：
```shell
cd examples/gpt
CUDA_VISIBLE_DEVICES=0,1 torchrun --nproc_per_node 2 inference_multigpu_demo.py --model_type chatglm --base_model THUDM/chatglm-6b
```
### LLaMA 模型
#### 使用 LLaMA 微调后的模型
example: [examples/gpt/inference_demo.py](https://github.com/shibing624/textgen/blob/main/examples/gpt/inference_demo.py)
<details>
<summary>show code example and result</summary>
```python
import sys
sys.path.append('../..')
from textgen import GptModel
model = GptModel("llama", "decapoda-research/llama-7b-hf", peft_name="ziqingyang/chinese-alpaca-lora-7b")
r = model.predict(["用一句话描述地球为什么是独一无二的。"])
print(r) # ['地球是唯一一颗拥有生命的行星。']
```
</details>
#### 训练 LLaMA 微调模型
1. 支持自定义训练数据集和训练参数，数据集格式参考[examples/data/sharegpt_zh_100_format.jsonl](https://github.com/shibing624/textgen/blob/main/examples/data/sharegpt_zh_100_format.jsonl)
2. 支持QLoRA、AdaLoRA、LoRA、P_Tuning、Prefix_Tuning等部分参数微调方法，也支持全参微调
3. 支持多卡训练，支持混合精度训练，使用方法同上（ChatGLM多卡训练）
4. 支持多卡推理
example: [examples/gpt/training_llama_demo.py](https://github.com/shibing624/textgen/blob/main/examples/gpt/training_llama_demo.py)
#### 基于微调(LoRA)模型继续训练
如果需要基于Lora模型继续训练，可以使用下面的脚本合并模型为新的base model，再微调训练即可。
执行以下命令：
```shell
python -m textgen/gpt/merge_peft_adapter \
--model_type llama \
--base_model_name_or_path path/to/llama/model \
--tokenizer_path path/to/llama/tokenizer \
--peft_model_path path/to/lora/model \
--output_dir merged
```
参数说明：
```
--model_type：模型类型，目前支持bloom,llama,baichuan和chatglm
--base_model_name_or_path：存放HF格式的底座模型权重和配置文件的目录
--tokenizer_path：存放HF格式的底座模型tokenizer文件的目录
--peft_model_path：中文LLaMA/Alpaca LoRA解压后文件所在目录，也可使用HF上的Lora模型名称，如`ziqingyang/chinese-alpaca-lora-7b`会自动下载对应模型
--output_dir：指定保存全量模型权重的目录，默认为./merged
```
#### 训练领域模型
Note: 为了全面的介绍训练医疗大模型的过程，把4阶段训练方法(Pretraining, Supervised Finetuning, Reward Modeling and Reinforcement Learning)单独新建了一个repo：[shibing624/MedicalGPT](https://github.com/shibing624/MedicalGPT)，请移步该repo查看训练方法。
### ConvSeq2Seq 模型
训练并预测ConvSeq2Seq模型：
example: [examples/seq2sesq/training_convseq2seq_model_demo.py](https://github.com/shibing624/textgen/blob/main/examples/seq2seq/training_convseq2seq_model_demo.py)
<details>
<summary>show code example and result</summary>
```python
import argparse
from loguru import logger
import sys
sys.path.append('../..')
from textgen.seq2seq.conv_seq2seq_model import ConvSeq2SeqModel
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--train_file', default='../data/zh_dialog.tsv', type=str, help='Training data file')
parser.add_argument('--do_train', action='store_true', help='Whether to run training.')
parser.add_argument('--do_predict', action='store_true', help='Whether to run predict.')
parser.add_argument('--output_dir', default='./outputs/convseq2seq_zh/', type=str, help='Model output directory')
parser.add_argument('--max_seq_length', default=50, type=int, help='Max sequence length')
parser.add_argument('--num_epochs', default=200, type=int, help='Number of training epochs')
parser.add_argument('--batch_size', default=32, type=int, help='Batch size')
args = parser.parse_args()
logger.info(args)
if args.do_train:
logger.info('Loading data...')
model = ConvSeq2SeqModel(epochs=args.num_epochs, batch_size=args.batch_size,
model_dir=args.output_dir, max_length=args.max_seq_length)
model.train_model(args.train_file)
print(model.eval_model(args.train_file))
if args.do_predict:
model = ConvSeq2SeqModel(epochs=args.num_epochs, batch_size=args.batch_size,
model_dir=args.output_dir, max_length=args.max_seq_length)
sentences = ["什么是ai", "你是什么类型的计算机", "你知道热力学吗"]
print("inputs:", sentences)
print('outputs:', model.predict(sentences))
if __name__ == '__main__':
main()
```
output:
```bash
inputs: ["什么是ai", "你是什么类型的计算机", "你知道热力学吗"]
outputs: ['人工智能是工程和科学的分支,致力于构建思维的机器。', '我的程序运行在python,所以我在任何运脑上工作！', '我不能错热是一个疯狂的人工智能"200年。']
```
</details>
### BART 模型
训练并预测BART模型：
example: [examples/seq2sesq/training_bartseq2seq_zh_demo.py](https://github.com/shibing624/textgen/blob/main/examples/seq2seq/training_bartseq2seq_zh_demo.py)
output:
```shell
inputs: ['什么是ai', '你是什么类型的计算机', '你知道热力学吗']
outputs: ['人工智能是工程和科学的分支,致力于构', '我的程序运行在python,所以我在任何电脑上', '什么是热力学吗？']
```
### T5 模型
example: [examples/t5/training_zh_t5_model_demo.py](https://github.com/shibing624/textgen/blob/main/examples/t5/training_zh_t5_model_demo.py)
<details>
<summary>show code example and result</summary>
```python
import argparse
from loguru import logger
import pandas as pd
import sys
sys.path.append('../..')
from textgen.t5 import T5Model
def load_data(file_path):
data = []
with open(file_path, 'r', encoding='utf-8') as f:
for line in f:
line = line.strip('\n')
terms = line.split('\t')
if len(terms) == 2:
data.append(['QA', terms[0], terms[1]])
else:
logger.warning(f'line error: {line}')
return data
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--train_file', default='../data/zh_dialog.tsv', type=str, help='Training data file')
parser.add_argument('--model_type', default='t5', type=str, help='Transformers model type')
parser.add_argument('--model_name', default='Langboat/mengzi-t5-base', type=str, help='Transformers model or path')
parser.add_argument('--do_train', action='store_true', help='Whether to run training.')
parser.add_argument('--do_predict', action='store_true', help='Whether to run predict.')
parser.add_argument('--output_dir', default='./outputs/mengzi_t5_zh/', type=str, help='Model output directory')
parser.add_argument('--max_seq_length', default=50, type=int, help='Max sequence length')
parser.add_argument('--num_epochs', default=10, type=int, help='Number of training epochs')
parser.add_argument('--batch_size', default=32, type=int, help='Batch size')
args = parser.parse_args()
logger.info(args)
if args.do_train:
logger.info('Loading data...')
# train_data: Pandas DataFrame containing the 3 columns - `prefix`, `input_text`, `target_text`.
# - `prefix`: A string indicating the task to perform. (E.g. `"question"`, `"stsb"`)
# - `input_text`: The input text. `prefix` is prepended to form the full input. (<prefix>: <input_text>)
# - `target_text`: The target sequence
train_data = load_data(args.train_file)
logger.debug('train_data: {}'.format(train_data[:10]))
train_df = pd.DataFrame(train_data, columns=["prefix", "input_text", "target_text"])
eval_data = load_data(args.train_file)[:10]
eval_df = pd.DataFrame(eval_data, columns=["prefix", "input_text", "target_text"])
model_args = {
"reprocess_input_data": True,
"overwrite_output_dir": True,
"max_seq_length": args.max_seq_length,
"train_batch_size": args.batch_size,
"num_train_epochs": args.num_epochs,
"save_eval_checkpoints": False,
"save_model_every_epoch": False,
"evaluate_generated_text": True,
"evaluate_during_training": True,
"evaluate_during_training_verbose": True,
"use_multiprocessing": True,
"save_best_model": True,
"output_dir": args.output_dir,
"use_early_stopping": True,
}
# model_type: t5 model_name: Langboat/mengzi-t5-base
model = T5Model(args.model_type, args.model_name, args=model_args)
def count_matches(labels, preds):
logger.debug(f"labels: {labels[:10]}")
logger.debug(f"preds: {preds[:10]}")
match = sum([1 if label == pred else 0 for label, pred in zip(labels, preds)])
logger.debug(f"match: {match}")
return match
model.train_model(train_df, eval_data=eval_df, matches=count_matches)
print(model.eval_model(eval_df, matches=count_matches))
if args.do_predict:
model = T5Model(args.model_type, args.output_dir)
sentences = ["什么是ai", "你是什么类型的计算机", "你知道热力学吗"]
print("inputs:", sentences)
print("outputs:", model.predict(sentences))
if __name__ == '__main__':
main()
```
output:
```shell
inputs: ['什么是ai', '你是什么类型的计算机', '你知道热力学吗']
outputs: ['人工智能有两个广义的定义,任何拟人的机械,如在卡雷尔capeks', '我的程序运行在Python,所以我在任何电脑上工作!', '什么是热力学']
```
</details>
### GPT2 模型
#### 中文GPT2 - 文章生成
使用中文数据集（段落格式，`\n`间隔），训练GPT2模型，可以用于诗歌生成、文章生成等任务。
example: [examples/gpt2/training_zh_gpt2_demo.py](https://github.com/shibing624/textgen/blob/main/examples/gpt2/training_zh_gpt2_demo.py)
#### 中文GPT2 - 对联生成
使用中文对联数据集（tsv格式，`\t`间隔），自定义数据集读取Dataset，训练GPT2模型，可以用于对联生成、对话生成等任务。
example: [examples/gpt2/training_couplet_gpt2_demo.py](https://github.com/shibing624/textgen/blob/main/examples/gpt2/training_couplet_gpt2_demo.py)
GPT2 vs T5：
1. 都是从Transformer改进来的，T5同时有编码器和解码器，GPT2只有解码器
2. T5的模型优势是处理给定输入，产出对应输出的任务，如翻译、对话、问答等
3. GPT2的模型优势是自由创作，如写一篇短文
4. T5的对联生成效果好于GPT2、GPT2的诗词生成效果好于T5
- [对联生成模型调研](https://github.com/shibing624/textgen/blob/main/docs/%E5%AF%B9%E8%81%94%E7%94%9F%E6%88%90%E6%A8%A1%E5%9E%8B%E5%AF%B9%E6%AF%94.md)
- [古诗生成模型调研](https://github.com/shibing624/textgen/blob/main/docs/%E5%8F%A4%E8%AF%97%E7%94%9F%E6%88%90%E6%A8%A1%E5%9E%8B%E5%AF%B9%E6%AF%94.md)
### SongNet 模型
格式控制的文本生成模型，paper见[SongNet: Rigid Formats Controlled Text Generation](https://arxiv.org/abs/2004.08022)，
适用于强韵律格式要求的诗歌、对联、歌词生成等任务。
example: [examples/songnet/training_zh_songnet_demo.py](https://github.com/shibing624/textgen/blob/main/examples/songnet/training_zh_songnet_demo.py)
### Keyword Text Augmentation(EDA/UDA)
example: [examples/text_augmentation/text_augmentation_demo.py](examples/text_augmentation/text_augmentation_demo.py)
<details>
<summary>show code example and result</summary>
```python
import sys
sys.path.append('..')
from textgen.augment import TextAugment
if __name__ == '__main__':
docs = ['主要研究机器学习、深度学习、计算机视觉、智能对话系统相关内容',
'晚上肚子好难受',
'你会武功吗，我不会',
'组装标题质量受限于广告主自提物料的片段质量，且表达丰富度有限',
]
m = TextAugment(sentence_list=docs)
a = docs[0]
print(a)
b = m.augment(a, aug_ops='random-0.2')
print('random-0.2:', b)
b = m.augment(a, aug_ops='insert-0.2')
print('insert-0.2:', b)
b = m.augment(a, aug_ops='delete-0.2')
print('delete-0.2:', b)
b = m.augment(a, aug_ops='tfidf-0.2')
print('tfidf-0.2:', b)
b = m.augment(a, aug_ops='mix-0.2')
print('mix-0.2:', b)
```
output:
```bash
主要研究机器学习、深度学习、计算机视觉、智能对话系统相关内容
random-0.2: ('主要陪陪机器学习、深度学习主要计算机视觉、智能对话系统受限于内容', [('研究', '陪陪', 2, 4), ('、', '主要', 13, 15), ('相关', '受限于', 27, 30)])
insert-0.2: ('主要研究机器机器学习学习、深度深度学习、计算机视觉、智能对话系统相关内容', [('机器', '机器机器', 4, 8), ('学习', '学习学习', 8, 12), ('深度', '深度深度', 13, 17)])
delete-0.2: ('主要研究机器学习、深度学习、计算机视觉、对话系统相关内容', [('智能', '', 20, 20)])
tfidf-0.2: ('一是研究机器学习、深度学习、计算机听觉、智能交谈系统密切相关内容', [('主要', '一是', 0, 2), ('视觉', '听觉', 17, 19), ('对话', '交谈', 22, 24), ('相关', '密切相关', 26, 30)])
mix-0.2: ('主要研究机器学习、深度学、计算机听觉、智能对话软件系统相关内容', [('学习', '学', 11, 12), ('视觉', '听觉', 16, 18), ('系统', '软件系统', 23, 27)])
```
</details>
### TGLS 模型（无监督相似文本生成模型）
无监督的中文电商评论生成：从**电商评论**中提取用户表达观点的短句并进行组合来生成仿真评论。
example: [examples/unsup_generation/unsup_generation_demo.py](examples/unsup_generation/unsup_generation_demo.py)
<details>
<summary>show code example and result</summary>
```python
import os
import sys
sys.path.append('..')
from textgen.unsup_generation import TglsModel, load_list
pwd_path = os.path.abspath(os.path.dirname(__file__))
samples = load_list(os.path.join(pwd_path, './data/ecommerce_comments.txt'))
docs_text = [
["挺好的，速度很快，也很实惠，不知效果如何",
"产品没得说，买了以后就降价，心情不美丽。",
"刚收到，包装很完整，不错",
"发货速度很快，物流也不错，同一时间买的两个东东，一个先到一个还在路上。这个水水很喜欢，不过盖子真的开了。盖不牢了现在。",
"包装的很好，是正品",
"被种草兰蔻粉水三百元一大瓶囤货，希望是正品好用，收到的时候用保鲜膜包裹得严严实实，只敢买考拉自营的护肤品",
],
['很温和，清洗的也很干净，不油腻，很不错，会考虑回购，第一次考拉买护肤品，满意',
'这款卸妆油我会无限回购的。即使我是油痘皮，也不会闷痘，同时在脸部按摩时，还能解决白头的脂肪粒的问题。用清水洗完脸后，非常的清爽。',
'自从用了fancl之后就不用其他卸妆了，卸的舒服又干净',
'买贵了，大润发才卖79。9。',
],
samples
]
m = TglsModel(docs_text)
r = m.generate(samples[:500])
print('size:', len(r))
for review in r:
print('\t' + review)
```
output:
[美迪惠尔 N.M.F针剂水库保湿面膜](https://goods.kaola.com/product/2227311.html)有如下的20句评论，其中有10句是真实用户评论，10句是生成的评论，能看出来么?😂
```
还不错还不错还不错还不错。
东西到了，不知道好不好用。试用过后再来评价。到时看网评都还可以。
哺乳期唯一使用的护肤品，每天都是素颜，脸面全靠面膜吊着😄补水💦不粘腻一如既往的支持，喜欢💕
搞活动时买的面膜，不知道这个面膜是真是假敷在脸上面膜纸都有小水泡鼓起来。
很不错，非常补水，用过的都知道，性价比之王，好用又不贵，正品，用着放心，物流也很快。
面膜非常好用哦。面膜薄薄的。好像是蚕丝面膜啊。精华很多呢。敷在脸上很舒服。感觉挺保湿的，味道也挺好闻的。就是里面只有单纯的面膜直接敷脸上有点不好弄，哈哈哈
还可以保湿效果不错水润润的每天贴一片脸也不干了用完了在买点，不错还会继续回购的。
快递很快，东西很赞！想要得点考拉豆不容易，还要三十个字。时间宝贵，废话不说！用过了就知道了
挺好用的，朋友推荐来的
挺好用的，淡淡的，虽然不是很浓精华的感觉，但是效果也蛮好的。划算
不得不说美迪惠尔的面膜是我用过的最好的面膜之一😎补水效果非常好，没想到这么便宜的价格竟真的能买到真品。
保湿效果挺好的，面膜很好用。
期待好的产品。
一打开包装里面的精华刚刚好，用了补水补水效果不错，物流非常快。
皮肤很光滑😇比上去速度快三天就到了。
前两天皮肤干燥连续敷了两个晚上感觉还不错😂补水效果明显！可想而知精华液又多充足😍敷上以后凉凉的很舒服。
补水效果一般吧～但是我用的韩国背回来的面膜纸不算薄，希望好用会回购的，敷上脸感觉比较清爽～价格还不便宜。
希望好用，面膜用过了很好用，皮肤水嫩光滑白皙，补水不错，价格也合适。
就是精华液太少了，保湿效果不错。
面膜的补水效果非常好，保湿效果确实很赞，这个面膜相对于胶原蛋白和美白的那两款的面膜纸要厚一些，看着价格合适。
```
前10句是真实用户评论，后10句是生成的。
</details>
## 📚 Dataset
#### SFT datasets
- 50万条中文ChatGPT指令Belle数据集：[BelleGroup/train_0.5M_CN](https://huggingface.co/datasets/BelleGroup/train_0.5M_CN)
- 100万条中文ChatGPT指令Belle数据集：[BelleGroup/train_1M_CN](https://huggingface.co/datasets/BelleGroup/train_1M_CN)
- 5万条英文ChatGPT指令Alpaca数据集：[50k English Stanford Alpaca dataset](https://github.com/tatsu-lab/stanford_alpaca#data-release)
- 2万条中文ChatGPT指令Alpaca数据集：[shibing624/alpaca-zh](https://huggingface.co/datasets/shibing624/alpaca-zh)
- 69万条中文指令Guanaco数据集(Belle50万条+Guanaco19万条)：[Chinese-Vicuna/guanaco_belle_merge_v1.0](https://huggingface.co/datasets/Chinese-Vicuna/guanaco_belle_merge_v1.0)
- 240万条中文医疗数据集(包括预训练数据和指令微调数据集)：[shibing624/medical](https://huggingface.co/datasets/shibing624/medical)
- 5万条英文ChatGPT多轮对话数据集：[RyokoAI/ShareGPT52K](https://huggingface.co/datasets/RyokoAI/ShareGPT52K)
- 80万条中文ChatGPT多轮对话数据集：[BelleGroup/multiturn_chat_0.8M](https://huggingface.co/datasets/BelleGroup/multiturn_chat_0.8M)
- 116万条中文ChatGPT多轮对话数据集：[fnlp/moss-002-sft-data](https://huggingface.co/datasets/fnlp/moss-002-sft-data)
#### Reward Model datasets
- 原版的oasst1数据集：[OpenAssistant/oasst1](https://huggingface.co/datasets/OpenAssistant/oasst1)
- 2万条多语言oasst1的reward数据集：[tasksource/oasst1_pairwise_rlhf_reward](https://huggingface.co/datasets/tasksource/oasst1_pairwise_rlhf_reward)
- 11万条英文hh-rlhf的reward数据集：[Dahoas/full-hh-rlhf](https://huggingface.co/datasets/Dahoas/full-hh-rlhf)
- 9万条英文reward数据集(来自Anthropic's Helpful Harmless dataset)：[Dahoas/static-hh](https://huggingface.co/datasets/Dahoas/static-hh)
- 7万条英文reward数据集（来源同上）：[Dahoas/rm-static](https://huggingface.co/datasets/Dahoas/rm-static)
- 7万条繁体中文的reward数据集（翻译自rm-static）[liswei/rm-static-m2m100-zh](https://huggingface.co/datasets/liswei/rm-static-m2m100-zh)
- 7万条英文Reward数据集：[yitingxie/rlhf-reward-datasets](https://huggingface.co/datasets/yitingxie/rlhf-reward-datasets)
- 3千条中文知乎问答偏好数据集：[liyucheng/zhihu_rlhf_3k](https://huggingface.co/datasets/liyucheng/zhihu_rlhf_3k)
## ✅ Todo
1. [x] add multiple rounds of dialogue data fine-tuning method
2. [x] add reward model finetuning, go to [shibing624/MeidcalGPT](https://github.com/shibing624/MedicalGPT)
3. [x] add rl finetuning, go to [shibing624/MeidcalGPT](https://github.com/shibing624/MedicalGPT)
4. [x] add medical reward dataset
5. [x] add llama in4 training, go to [shibing624/MeidcalGPT](https://github.com/shibing624/MedicalGPT)
6. [ ] add all training and predict demo in colab
## ☎️ Contact
- Issue(建议)
：[](https://github.com/shibing624/textgen/issues)
- 邮件我：xuming: xuming624@qq.com
- 微信我： 加我*微信号：xuming624, 备注：姓名-公司名-NLP* 进NLP交流群。
<img src="docs/wechat.jpeg" width="200" />
## 😇 Citation
如果你在研究中使用了textgen，请按如下格式引用：
```latex
@misc{textgen,
title={textgen: Text Generation Tool},
author={Ming Xu},
year={2021},
howpublished={\url{https://github.com/shibing624/textgen}},
}
```
## 🤗 License
This repository is licensed under [The Apache License 2.0](LICENSE).
Please follow the [Model Card](https://github.com/facebookresearch/llama/blob/main/MODEL_CARD.md) to use the LLaMA model.
Please follow the [RAIL License](https://huggingface.co/spaces/bigscience/license) to use the BLOOM & BLOOMZ model.
## 😍 Contribute
项目代码还很粗糙，如果大家对代码有所改进，欢迎提交回本项目，在提交之前，注意以下两点：
- 在`tests`添加相应的单元测试
- 使用`python -m pytest`来运行所有单元测试，确保所有单测都是通过的
之后即可提交PR。
## 💕 Acknowledgements
- [PaddlePaddle/ERNIE](https://github.com/PaddlePaddle/ERNIE)
- [minimaxir/textgenrnn](https://github.com/minimaxir/textgenrnn)
- [minimaxir/gpt-2-simple](https://github.com/minimaxir/gpt-2-simple)
- [asyml/texar](https://github.com/asyml/texar)
- [yangjianxin1/GPT2-chitchat](https://github.com/yangjianxin1/GPT2-chitchat)
- [williamSYSU/TextGAN-PyTorch](https://github.com/williamSYSU/TextGAN-PyTorch)
- [RUCAIBox/TextBox](https://github.com/RUCAIBox/TextBox)
- [Tiiiger/bert_score](https://github.com/Tiiiger/bert_score)
- [ThilinaRajapakse/simpletransformers](https://github.com/ThilinaRajapakse/simpletransformers)
- [1YCxZ/Fake-review-generation](https://github.com/1YCxZ/Fake-review-generation)
- [tloen/alpaca-lora](https://github.com/tloen/alpaca-lora/blob/main/finetune.py)
Thanks for their great work!
Raw data
{
"_id": null,
"home_page": "https://github.com/shibing624/textgen",
"name": "textgen",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "textgen,text-generation,Text Generation Tool,ernie-gen,chinese text generation",
"author": "XuMing",
"author_email": "xuming624@qq.com",
"download_url": "https://files.pythonhosted.org/packages/09/8c/9436002fa7bbf0596855782e34efe211940270be00bdeb3421c207fd1f66/textgen-1.1.1.tar.gz",
"platform": null,
"description": "[**\ud83c\udde8\ud83c\uddf3\u4e2d\u6587**](https://github.com/shibing624/textgen/blob/main/README.md) | [**\ud83c\udf10English**](https://github.com/shibing624/textgen/blob/main/README_EN.md) | [**\ud83d\udcd6\u6587\u6863/Docs**](https://github.com/shibing624/textgen/wiki) | [**\ud83e\udd16\u6a21\u578b/Models**](https://huggingface.co/shibing624) \n\n<div align=\"center\">\n <a href=\"https://github.com/shibing624/textgen\">\n <img src=\"https://github.com/shibing624/textgen/blob/main/docs/logo.svg\" alt=\"Logo\">\n </a>\n</div>\n\n-----------------\n\n# TextGen: Implementation of Text Generation models\n[](https://badge.fury.io/py/textgen)\n[](https://pepy.tech/project/textgen)\n[](CONTRIBUTING.md)\n[](LICENSE)\n[](requirements.txt)\n[](https://github.com/shibing624/textgen/issues)\n[](#Contact)\n\n## \ud83d\udcd6 Introduction\n\n**TextGen**\u5b9e\u73b0\u4e86\u591a\u79cd\u6587\u672c\u751f\u6210\u6a21\u578b\uff0c\u5305\u62ec\uff1aLLaMA\u3001ChatGLM\u3001UDA\u3001GPT2\u3001Seq2Seq\u3001BART\u3001T5\u3001SongNet\u7b49\u6a21\u578b\uff0c\u5f00\u7bb1\u5373\u7528\u3002\n\n## \ud83d\udd25 News\n\n[2023/09/05] v1.1.1\u7248\u672c: \u652f\u6301\u591a\u5361\u63a8\u7406\uff0c\u63a8\u7406\u901f\u5ea6\u52a0\u500d\uff0c\u8c03\u5e93textgen\u505abatch\u63a8\u7406\uff0c\u591a\u5361\u63a8\u7406\u66f4\u65b9\u4fbf\u3001\u5feb\u901f\u3002\u8be6\u89c1[Release-v1.1.1](https://github.com/shibing624/textgen/releases/tag/1.1.1)\n\n[2023/08/23] v1.1.0\u7248\u672c: \u53d1\u5e03\u57fa\u4e8eShareGPT4\u6570\u636e\u96c6\u5fae\u8c03\u7684\u4e2d\u82f1\u6587Vicuna-13B\u6a21\u578b[shibing624/vicuna-baichuan-13b-chat](https://huggingface.co/shibing624/vicuna-baichuan-13b-chat)\uff0c\u548c\u5bf9\u5e94\u7684LoRA\u6a21\u578b[shibing624/vicuna-baichuan-13b-chat-lora](https://huggingface.co/shibing624/vicuna-baichuan-13b-chat-lora)\uff0c\u652f\u6301\u591a\u8f6e\u5bf9\u8bdd\uff0c\u8bc4\u6d4b\u6548\u679c\u6709\u63d0\u5347\uff0c\u8be6\u89c1[Release-v1.1.0](https://github.com/shibing624/textgen/releases/tag/1.1.0)\n\n[2023/08/02] v1.0.2\u7248\u672c: \u65b0\u589e\u652f\u6301ChatGLM2\u548cLLaMA2\u6a21\u578b\u7684SFT\u5fae\u8c03\u8bad\u7ec3\uff0c\u8be6\u89c1[Release-v1.0.2](https://github.com/shibing624/textgen/releases/tag/1.0.2)\n\n[2023/06/15] v1.0.0\u7248\u672c: \u65b0\u589eChatGLM/LLaMA/Bloom\u6a21\u578b\u7684\u591a\u8f6e\u5bf9\u8bdd\u5fae\u8c03\u8bad\u7ec3\uff0c\u5e76\u53d1\u5e03\u533b\u7597\u95ee\u8bcaLoRA\u6a21\u578b[shibing624/ziya-llama-13b-medical-lora](https://huggingface.co/shibing624/ziya-llama-13b-medical-lora)\u3002\u8be6\u89c1[Release-v1.0.0](https://github.com/shibing624/textgen/releases/tag/1.0.0)\n\n[2023/06/02] v0.2.7\u7248\u672c: \u65b0\u589eChatGLM/LLaMA/Bloom\u6a21\u578b\u7684SFT\u5fae\u8c03\u8bad\u7ec3\uff0c\u5e76\u53d1\u5e03\u9002\u7528\u4e8e\u901a\u7528\u5bf9\u8bdd\u548c\u4e2d\u6587\u7ea0\u9519\u7684LoRA\u6a21\u578b\u3002\u8be6\u89c1[Release-v0.2.7](https://github.com/shibing624/textgen/releases/tag/0.2.7)\n\n\n## \ud83d\ude0a Feature\n\n- [GPT](textgen/gpt)\uff1a\u672c\u9879\u76ee\u57fa\u4e8ePyTorch\u5b9e\u73b0\u4e86ChatGLM-6B/Baichuan/LLaMA2/BLOOM\u7b49GPT\u6a21\u578bLoRA\u5fae\u8c03\u8bad\u7ec3\u548c\u9884\u6d4b\uff0c\u53ef\u4ee5\u7528\u4e8e\u5bf9\u8bdd\u751f\u6210\u4efb\u52a1\u548c\u9886\u57df\u5fae\u8c03\u8bad\u7ec3\n- [UDA/EDA](textgen/augment/word_level_augment.py)\uff1a\u672c\u9879\u76ee\u5b9e\u73b0\u4e86UDA(\u975e\u6838\u5fc3\u8bcd\u66ff\u6362)\u3001EDA\u548cBack Translation(\u56de\u8bd1)\u7b97\u6cd5\uff0c\u57fa\u4e8eTF-IDF\u5c06\u53e5\u5b50\u4e2d\u90e8\u5206\u4e0d\u91cd\u8981\u8bcd\u66ff\u6362\u4e3a\u540c\u4e49\u8bcd\uff0c\u968f\u673a\u8bcd\u63d2\u5165\u3001\u5220\u9664\u3001\u66ff\u6362\u7b49\u65b9\u6cd5\uff0c\u4ea7\u751f\u65b0\u7684\u6587\u672c\uff0c\u5b9e\u73b0\u4e86\u6587\u672c\u6269\u589e\n- [Seq2Seq](textgen/seq2seq)\uff1a\u672c\u9879\u76ee\u57fa\u4e8ePyTorch\u5b9e\u73b0\u4e86Seq2Seq\u3001ConvSeq2Seq\u3001BART\u6a21\u578b\u7684\u8bad\u7ec3\u548c\u9884\u6d4b\uff0c\u53ef\u4ee5\u7528\u4e8e\u6587\u672c\u7ffb\u8bd1\u3001\u5bf9\u8bdd\u751f\u6210\u3001\u6458\u8981\u751f\u6210\u7b49\u6587\u672c\u751f\u6210\u4efb\u52a1\n- [T5](textgen/t5)\uff1a\u672c\u9879\u76ee\u57fa\u4e8ePyTorch\u5b9e\u73b0\u4e86T5\u548cCopyT5\u6a21\u578b\u8bad\u7ec3\u548c\u9884\u6d4b\uff0c\u53ef\u4ee5\u7528\u4e8e\u6587\u672c\u7ffb\u8bd1\u3001\u5bf9\u8bdd\u751f\u6210\u3001\u5bf9\u8054\u751f\u6210\u3001\u6587\u6848\u64b0\u5199\u7b49\u6587\u672c\u751f\u6210\u4efb\u52a1\n- [GPT2](textgen/language_modeling)\uff1a\u672c\u9879\u76ee\u57fa\u4e8ePyTorch\u5b9e\u73b0\u4e86GTP2\u6a21\u578b\u8bad\u7ec3\u548c\u9884\u6d4b\uff0c\u53ef\u4ee5\u7528\u4e8e\u6587\u7ae0\u751f\u6210\u3001\u5bf9\u8054\u751f\u6210\u7b49\u6587\u672c\u751f\u6210\u4efb\u52a1\n- [SongNet](textgen/language_modeling/songnet_model.py)\uff1a\u672c\u9879\u76ee\u57fa\u4e8ePyTorch\u5b9e\u73b0\u4e86SongNet\u6a21\u578b\u8bad\u7ec3\u548c\u9884\u6d4b\uff0c\u53ef\u4ee5\u7528\u4e8e\u89c4\u8303\u683c\u5f0f\u7684\u8bd7\u8bcd\u3001\u6b4c\u8bcd\u7b49\u6587\u672c\u751f\u6210\u4efb\u52a1\n- [TGLS](textgen/unsup_generation)\uff1a\u672c\u9879\u76ee\u5b9e\u73b0\u4e86[TGLS](https://www.jiqizhixin.com/articles/2020-08-11-5)\u65e0\u76d1\u7763\u76f8\u4f3c\u6587\u672c\u751f\u6210\u6a21\u578b\uff0c\u662f\u4e00\u79cd\u201c\u5148\u641c\u7d22\u540e\u5b66\u4e60\u201d\u7684\u6587\u672c\u751f\u6210\u65b9\u6cd5\uff0c\u901a\u8fc7\u53cd\u590d\u8fed\u4ee3\u5b66\u4e60\u5019\u9009\u96c6\uff0c\u6700\u7ec8\u6a21\u578b\u80fd\u751f\u6210\u7c7b\u4f3c\u5019\u9009\u96c6\u7684\u9ad8\u8d28\u91cf\u76f8\u4f3c\u6587\u672c\n\n### Release Models\n\nrelease\u57fa\u4e8e`textgen`\u8bad\u7ec3\u7684\u4e2d\u6587\u6a21\u578b\uff0c\u6a21\u578b\u5df2\u7ecfrelease\u5230HuggingFace models\uff0c\u6307\u5b9a\u6a21\u578b\u540d\u79f0`textgen`\u4f1a\u81ea\u52a8\u4e0b\u8f7d\u6a21\u578b\uff0c\u53ef\u76f4\u63a5\u4f7f\u7528\u3002\n\n\n| Model | Arch | Introduction | Train Script | Predict Script | \n|:----------------------------------------------------------------------------------------------------------|:-------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------|\n| [shibing624/t5-chinese-couplet](https://huggingface.co/shibing624/t5-chinese-couplet) | T5 | fine-tuned\u4e2d\u6587\u5bf9\u8054\u540e\u7684\u6a21\u578b | [\u5bf9\u8054\u751f\u6210\u6a21\u578b\u8c03\u7814](https://github.com/shibing624/textgen/blob/main/docs/%E5%AF%B9%E8%81%94%E7%94%9F%E6%88%90%E6%A8%A1%E5%9E%8B%E5%AF%B9%E6%AF%94.md) | [predict script](https://github.com/shibing624/textgen/blob/main/examples/t5/t5_couplet_demo.py) |\n| [shibing624/songnet-base-chinese-songci](https://huggingface.co/shibing624/songnet-base-chinese-songci) | SongNet | fine-tuned\u5b8b\u8bcd\u540e\u7684\u6a21\u578b | [training script](https://github.com/shibing624/textgen/blob/main/examples/songnet/training_zh_songnet_demo.py) | [predict script](https://github.com/shibing624/textgen/blob/main/examples/songnet/songnet_songci_demo.py) |\n| [shibing624/songnet-base-chinese-couplet](https://huggingface.co/shibing624/songnet-base-chinese-couplet) | SongNet | fine-tuned\u5bf9\u8054\u540e\u7684\u6a21\u578b | [training script](https://github.com/shibing624/textgen/blob/main/examples/songnet/training_zh_songnet_demo.py) | [predict script](https://github.com/shibing624/textgen/blob/main/examples/songnet/songnet_couplet_demo.py) |\n| [shibing624/chatglm-6b-csc-zh-lora](https://huggingface.co/shibing624/chatglm-6b-csc-zh-lora) | ChatGLM-6B | \u572827\u4e07\u4e2d\u6587\u62fc\u5199\u7ea0\u9519\u6570\u636e[shibing624/CSC](https://huggingface.co/datasets/shibing624/CSC)\u4e0a\u5fae\u8c03\u4e86\u4e00\u7248ChatGLM-6B\uff0c\u7ea0\u9519\u6548\u679c\u6709\u63d0\u5347\uff0c\u53d1\u5e03\u5fae\u8c03\u540e\u7684LoRA\u6743\u91cd | [training script](https://github.com/shibing624/textgen/blob/main/examples/chatglm/training_chatglm_csc_demo.py) | [predict script](https://github.com/shibing624/textgen/blob/main/examples/chatglm/csc_demo.py) |\n| [shibing624/chatglm-6b-belle-zh-lora](https://huggingface.co/shibing624/chatglm-6b-belle-zh-lora) | ChatGLM-6B | \u5728100\u4e07\u6761\u4e2d\u6587ChatGPT\u6307\u4ee4Belle\u6570\u636e\u96c6[BelleGroup/train_1M_CN](https://huggingface.co/datasets/BelleGroup/train_1M_CN)\u4e0a\u5fae\u8c03\u4e86\u4e00\u7248ChatGLM-6B\uff0c\u95ee\u7b54\u6548\u679c\u6709\u63d0\u5347\uff0c\u53d1\u5e03\u5fae\u8c03\u540e\u7684LoRA\u6743\u91cd | [training script](https://github.com/shibing624/textgen/blob/main/examples/chatglm/training_chatglm_hfdataset_demo.py) | [predict script](https://github.com/shibing624/textgen/blob/main/examples/chatglm/training_chatglm_hfdataset_demo.py) |\n| [shibing624/llama-13b-belle-zh-lora](https://huggingface.co/shibing624/llama-13b-belle-zh-lora) | LLaMA-13B | \u5728100\u4e07\u6761\u4e2d\u6587ChatGPT\u6307\u4ee4Belle\u6570\u636e\u96c6[BelleGroup/train_1M_CN](https://huggingface.co/datasets/BelleGroup/train_1M_CN)\u4e0a\u5fae\u8c03\u4e86\u4e00\u7248Llama-13B\uff0c\u95ee\u7b54\u6548\u679c\u6709\u63d0\u5347\uff0c\u53d1\u5e03\u5fae\u8c03\u540e\u7684LoRA\u6743\u91cd | [training script](https://github.com/shibing624/textgen/blob/main/examples/llama/training_llama_hfdataset_demo.py) | [predict script](https://github.com/shibing624/textgen/blob/main/examples/llama/training_llama_hfdataset_demo.py) |\n| [shibing624/chinese-alpaca-plus-7b-hf](https://huggingface.co/shibing624/chinese-alpaca-plus-7b-hf) | LLaMA-7B | [\u4e2d\u6587LLaMA-Plus, Alpaca-Plus 7B\u7248\u672c](https://github.com/ymcui/Chinese-LLaMA-Alpaca/releases/tag/v3.0)\uff0c\u5728LLaMA-7B\u4e0a\u6269\u5145\u4e86\u4e2d\u6587\u8bcd\u8868\u5e76\u7ee7\u7eed\u9884\u8bad\u7ec3120G\u6587\u672c\uff08\u901a\u7528\u9886\u57df\uff09\uff0c\u57284M\u6307\u4ee4\u6570\u636e\u96c6\u4e0a\u5fae\u8c03\u540e\u5f97\u5230\u7684\u4e2d\u6587Alpaca-plus\u6a21\u578b | [training script](https://github.com/shibing624/textgen/blob/main/examples/gpt/training_llama_demo.py) | [predict script](https://github.com/shibing624/textgen/blob/main/examples/gpt/training_llama_demo.py) |\n| [shibing624/chinese-alpaca-plus-13b-hf](https://huggingface.co/shibing624/chinese-alpaca-plus-13b-hf) | LLaMA-13B | [\u4e2d\u6587LLaMA-Plus, Alpaca-Plus 13B\u7248\u672c](https://github.com/ymcui/Chinese-LLaMA-Alpaca/releases/tag/v3.1)\uff0c\u5728LLaMA-13B\u4e0a\u6269\u5145\u4e86\u4e2d\u6587\u8bcd\u8868\u5e76\u7ee7\u7eed\u9884\u8bad\u7ec3120G\u6587\u672c\uff08\u901a\u7528\u9886\u57df\uff09\uff0c\u57284.3M\u6307\u4ee4\u6570\u636e\u96c6\u4e0a\u5fae\u8c03\u540e\u5f97\u5230\u7684\u4e2d\u6587Alpaca-plus\u6a21\u578b | [training script](https://github.com/shibing624/textgen/blob/main/examples/gpt/training_llama_demo.py) | [predict script](https://github.com/shibing624/textgen/blob/main/examples/gpt/training_llama_demo.py) |\n| [shibing624/ziya-llama-13b-medical-lora](https://huggingface.co/shibing624/ziya-llama-13b-medical-lora) | LLaMA-13B | \u5728240\u4e07\u6761\u4e2d\u82f1\u6587\u533b\u7597\u6570\u636e\u96c6[shibing624/medical](https://huggingface.co/datasets/shibing624/medical)\u4e0a\u5fae\u8c03\u4e86\u4e00\u7248Ziya-LLaMA-13B\u6a21\u578b\uff0c\u533b\u7597\u95ee\u7b54\u6548\u679c\u6709\u63d0\u5347\uff0c\u53d1\u5e03\u5fae\u8c03\u540e\u7684LoRA\u6743\u91cd | [training script](https://github.com/shibing624/textgen/blob/main/examples/gpt/training_llama_demo.py) | [predict script](https://github.com/shibing624/textgen/blob/main/examples/gpt/training_llama_demo.py) |\n| [shibing624/vicuna-baichuan-13b-chat](https://huggingface.co/shibing624/vicuna-baichuan-13b-chat) | Baichuan-13B-Chat | \u572810\u4e07\u6761\u591a\u8bed\u8a00ShareGPT GPT4\u591a\u8f6e\u5bf9\u8bdd\u6570\u636e\u96c6[shibing624/sharegpt_gpt4](https://huggingface.co/datasets/shibing624/sharegpt_gpt4)\u4e0aSFT\u5fae\u8c03\u4e86\u4e00\u7248baichuan-13b-chat\u591a\u8f6e\u95ee\u7b54\u6a21\u578b\uff0c\u65e5\u5e38\u95ee\u7b54\u548c\u533b\u7597\u95ee\u7b54\u6548\u679c\u6709\u63d0\u5347\uff0c\u53d1\u5e03\u5fae\u8c03\u540e\u7684\u5b8c\u6574\u6a21\u578b\u6743\u91cd | [training script](https://github.com/shibing624/textgen/blob/main/examples/gpt/training_llama_demo.py) | [predict script](https://github.com/shibing624/textgen/blob/main/examples/gpt/training_llama_demo.py) |\n\n### Evaluation\n\n| Model | Arch | Introduction | Score |\n|:--------------------------------------------------------------------------------------------------------------------------------------------|:-----------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------|\n| [LLaMA-7B-Chinese-Alpaca](https://huggingface.co/ziqingyang/chinese-alpaca-lora-7b) | LLaMA-7B | \u590d\u7528[ymcui/Chinese-LLaMA-Alpaca](https://github.com/ymcui/Chinese-LLaMA-Alpaca/blob/main/examples/README.md)\u7684\u8bc4\u4f30case\u548c\u5f97\u5206 | 4.92 |\n| [LLaMA-13B-Chinese-Alpaca](https://huggingface.co/ziqingyang/chinese-alpaca-lora-13b) | LLaMA-13B | \u590d\u7528[ymcui/Chinese-LLaMA-Alpaca](https://github.com/ymcui/Chinese-LLaMA-Alpaca/blob/main/examples/README.md)\u7684\u8bc4\u4f30case\u548c\u5f97\u5206 | 7.05 |\n| [ChatGLM-6B](https://huggingface.co/THUDM/chatglm-6b) | ChatGLM-6B | \u57fa\u4e8e\u539f\u751f`THUDM/chatglm-6b`\u8bc4\u4f30\u6d4b\u8bd5\u96c6\u5f97\u5206 | 7.16 |\n| [ChatGLM-6B-v1.1](https://huggingface.co/THUDM/chatglm-6b) | ChatGLM-6B | \u57fa\u4e8e\u539f\u751f`THUDM/chatglm-6b`v1.1\u82f1\u6587\u4f18\u5316\u7248\u6a21\u578b\u8bc4\u4f30\u6d4b\u8bd5\u96c6\u5f97\u5206 | **7.18** |\n| [shibing624/chatglm-6b-belle-zh-lora](https://huggingface.co/shibing624/chatglm-6b-belle-zh-lora) | ChatGLM-6B | \u57fa\u4e8e`THUDM/chatglm-6b`\u52a0\u8f7d`shibing624/chatglm-6b-belle-zh-lora`LoRA\u6a21\u578b\u540e\u8bc4\u4f30\u6d4b\u8bd5\u96c6\u5f97\u5206 | 7.03 |\n| [facat/alpaca-lora-cn-13b](https://huggingface.co/facat/alpaca-lora-cn-13b)\t | LLaMA-13B | \u57fa\u4e8e`decapoda-research/llama-13b-hf`\u52a0\u8f7d`facat/alpaca-lora-cn-13b`LoRA\u6a21\u578b\u540e\u8bc4\u4f30\u6d4b\u8bd5\u96c6\u5e76\u6807\u6ce8\u5f97\u5206 | 4.13 | \n| [Chinese-Vicuna/Chinese-Vicuna-lora-13b-belle-and-guanaco](https://huggingface.co/Chinese-Vicuna/Chinese-Vicuna-lora-13b-belle-and-guanaco) | LLaMA-13B | \u57fa\u4e8e`decapoda-research/llama-13b-hf`\u52a0\u8f7d`Chinese-Vicuna/Chinese-Vicuna-lora-13b-belle-and-guanaco`LoRA\u6a21\u578b\u540e\u8bc4\u4f30\u6d4b\u8bd5\u96c6\u5e76\u6807\u6ce8\u5f97\u5206 | 3.98 |\n| [shibing624/chinese-alpaca-plus-7b-hf](https://huggingface.co/shibing624/chinese-alpaca-plus-7b-hf) | LLaMA-7B | \u4f7f\u7528[ymcui/Chinese-LLaMA-Alpaca \u5408\u5e76\u6a21\u578b\u65b9\u6cd5](https://github.com/ymcui/Chinese-LLaMA-Alpaca/wiki/%E6%89%8B%E5%8A%A8%E6%A8%A1%E5%9E%8B%E5%90%88%E5%B9%B6%E4%B8%8E%E8%BD%AC%E6%8D%A2#%E5%A4%9Alora%E6%9D%83%E9%87%8D%E5%90%88%E5%B9%B6%E9%80%82%E7%94%A8%E4%BA%8Echinese-alpaca-plus)\u5408\u5e76HF\u6743\u91cd\u540e\uff0c\u8bc4\u4f30\u6d4b\u8bd5\u96c6\u5e76\u6807\u6ce8\u5f97\u5206 | 6.93 |\n| [shibing624/chinese-alpaca-plus-13b-hf](https://huggingface.co/shibing624/chinese-alpaca-plus-13b-hf) | LLaMA-13B | \u4f7f\u7528[ymcui/Chinese-LLaMA-Alpaca \u5408\u5e76\u6a21\u578b\u65b9\u6cd5](https://github.com/ymcui/Chinese-LLaMA-Alpaca/wiki/%E6%89%8B%E5%8A%A8%E6%A8%A1%E5%9E%8B%E5%90%88%E5%B9%B6%E4%B8%8E%E8%BD%AC%E6%8D%A2#%E5%A4%9Alora%E6%9D%83%E9%87%8D%E5%90%88%E5%B9%B6%E9%80%82%E7%94%A8%E4%BA%8Echinese-alpaca-plus)\u5408\u5e76HF\u6743\u91cd\u540e\uff0c\u8bc4\u4f30\u6d4b\u8bd5\u96c6\u5e76\u6807\u6ce8\u5f97\u5206 | 7.07 |\n| [TheBloke/vicuna-13B-1.1-HF](https://huggingface.co/TheBloke/vicuna-13B-1.1-HF) | LLaMA-13B | \u4f7f\u7528\u539f\u751fvicuna-13B-1.1\u5408\u5e76\u540e\u7684\u6a21\u578b\uff0c\u8bc4\u4f30\u6d4b\u8bd5\u96c6\u5e76\u6807\u6ce8\u5f97\u5206 | 5.13 |\n| [IDEA-CCNL/Ziya-LLaMA-13B-v1](https://huggingface.co/IDEA-CCNL/Ziya-LLaMA-13B-v1) | LLaMA-13B | \u4f7f\u7528\u59dc\u5b50\u7259\u901a\u7528\u5927\u6a21\u578bV1\uff0c\u8bc4\u4f30\u6d4b\u8bd5\u96c6\u5e76\u6807\u6ce8\u5f97\u5206 | 6.63 |\n\n\u8bf4\u660e\uff1a\n- \u8bc4\u4f30case\uff0c\u8be6\u89c1\u5728\u7ebf\u6587\u6863\uff1a\u4e2d\u6587LLM-benchmark\u591a\u4efb\u52a1\u8bc4\u4f30\u96c6(\u817e\u8baf\u6587\u6863) https://docs.qq.com/sheet/DUUpsREtWbFBsUVJE?tab=r7io7g \u611f\u8c22\u97e9\u4fca\u660e\u3001[\u6768\u5bb6\u94ed](https://github.com/yangjiam)\u7b49\u540c\u5b66\u7684\u6807\u6ce8\n- \u8bc4\u4f30\u4efb\u52a1\u7c7b\u578b\u5305\u62ec\uff1a\u77e5\u8bc6\u95ee\u7b54\uff0c\u5f00\u653e\u5f0f\u95ee\u7b54\uff0c\u6570\u503c\u8ba1\u7b97\uff0c\u8bd7\u8bcd\u3001\u97f3\u4e50\u3001\u4f53\u80b2\uff0c\u5a31\u4e50\uff0c\u5199\u6587\u7ae0\uff0c\u6587\u672c\u7ffb\u8bd1\uff0c\u4ee3\u7801\u7f16\u7a0b\uff0c\u4f26\u7406\u3001\u62d2\u7b54\u7c7b\uff0c\u591a\u8f6e\u95ee\u7b54\uff0cScore \u8bc4\u5206\u662f\u524d100\u6761\uff0810\u5206\u5236\uff09\u7684\u5e73\u5747\u5206\u6570\uff0c\u4eba\u5de5\u6253\u5206\uff0c\u8d8a\u9ad8\u8d8a\u597d\n- \u8bc4\u4f30\u6570\u91cf\u5c11\uff0c\u4efb\u52a1\u7c7b\u578b\u4e0d\u591f\u5168\u9762\uff0c\u8bc4\u5206\u4e4b\u95f4\u7684\u5927\u5c0f\u5173\u7cfb\u6709\u4e00\u4e9b\u53c2\u8003\u4ef7\u503c\uff0c\u5206\u6570\u7684\u7edd\u5bf9\u503c\u6ca1\u592a\u5927\u53c2\u8003\u4ef7\u503c\n- \u8bc4\u4f30\u811a\u672c\uff1a[tests/test_benchmark.py](https://github.com/shibing624/textgen/blob/main/tests/test_benchmark.py) \uff0c\u4f7f\u7528fp16\u9884\u6d4b\uff0c\u65e0int\u91cf\u5316\u5904\u7406\uff0c\u8fd0\u884c\u811a\u672c\u53ef\u590d\u73b0\u8bc4\u4f30\u7ed3\u679c\uff0c\u4f46\u751f\u6210\u7ed3\u679c\u5177\u6709\u968f\u673a\u6027\uff0c\u53d7\u89e3\u7801\u8d85\u53c2\u3001\u968f\u673a\u79cd\u5b50\u7b49\u56e0\u7d20\u5f71\u54cd\u3002\u8bc4\u6d4b\u5e76\u975e\u7edd\u5bf9\u4e25\u8c28\uff0c\u6d4b\u8bd5\u7ed3\u679c\u4ec5\u4f9b\u667e\u6652\u53c2\u8003\n- \u7ed3\u8bba\uff1aChatGLM-6B\u3001LLaMA-13B\u7684\u4e2d\u6587\u884d\u751f\u6a21\u578b\uff08\u5305\u62ecalpaca-plus, vicuna, ziya\uff09\u7684\u8868\u73b0\u5c5e\u4e8e\u7b2c\u4e00\u68af\u961f\uff0c\u539f\u7248LLaMA-7B\u7684\u8868\u73b0\u6574\u4f53\u7a0d\u5dee\u4e9b\n- LLaMA-13B-Chinese-Alpaca\u662f\u5728\u539f\u7248LLaMA\u4e0a\u6269\u5145\u4e86\u4e2d\u6587\u8bcd\u8868\uff0c\u5e76\u878d\u5165\u4e86\u7ea620G\u7684\u901a\u7528\u4e2d\u6587\u8bed\u6599\u540e\u7684\u6307\u4ee4\u5fae\u8c03\u6a21\u578b\uff0c\u8868\u660e\u4e86LLaMA\u7684\u5e95\u5ea7\u4f18\u79c0\uff0c\u5177\u6709\u5f3a\u5927\u7684\u8bed\u8a00\u8fc1\u79fb\u80fd\u529b\n- ChatGLM\u8fd9\u79cd\u539f\u751f\u7684\u4e2d\u6587\u9884\u8bad\u7ec3\u6a21\u578b\u66f4\u7406\u89e3\u4e2d\u6587\u8bed\u4e49\uff0c\u4e14\u5728\u4e2d\u6587\u77e5\u8bc6\u95ee\u7b54\u3001\u5f00\u653e\u5f0f\u95ee\u7b54\u5f97\u5206\u9ad8\n- LLaMA\u7cfb\u5217\u6a21\u578b\u6570\u503c\u8ba1\u7b97\u3001\u4e2d\u82f1\u7ffb\u8bd1\u3001\u4ee3\u7801\u7f16\u7a0b\u7c7b\u5f97\u5206\u9ad8\n- \u7ecf\u8fc7\u4e2d\u6587\u9884\u8bad\u7ec3\u548cSFT\u5fae\u8c03\u540e\u7684Chinese-LLaMA\u6a21\u578b\u5728\u4e2d\u6587\u8bd7\u8bcd\u3001\u5a31\u4e50\u3001\u4f26\u7406\u7c7b\u5f97\u5206\u76f8\u8f83\u539f\u7248LLaMA\u6709\u63d0\u5347\n\n## \ud83d\ude80 Demo\n\nHuggingFace Demo: https://huggingface.co/spaces/shibing624/chinese-couplet-generate\n\n\n\nrun example: [examples/T5/gradio_demo.py](examples/T5/gradio_demo.py) to see the demo:\n\n```shell\npython examples/T5/gradio_demo.py\n```\n\nmodel trained by [examples/t5/T5_Finetune_Chinese_Couplet.ipynb](https://github.com/shibing624/textgen/blob/main/examples/t5/T5_Finetune_Chinese_Couplet.ipynb)\n\n## \ud83d\udcbe Install\n\n```shell\npip install -U textgen\n```\n\nor\n\ninstall develop version:\n```shell\npip install torch # conda install pytorch\ngit clone https://github.com/shibing624/textgen.git\ncd textgen\npython setup.py install\n```\n\n## \u25b6\ufe0f Usage\n\n### ChatGLM-6B \u6a21\u578b\n\n#### \u4f7f\u7528 ChatGLM-6B \u5fae\u8c03\u540e\u7684\u6a21\u578b\n\nexample: [examples/gpt/inference_demo.py](https://github.com/shibing624/textgen/blob/main/examples/gpt/inference_demo.py)\n\n```python\nfrom textgen import GptModel\n\nmodel = GptModel(\"chatglm\", \"THUDM/chatglm-6b\", peft_name=\"shibing624/chatglm-6b-csc-zh-lora\")\nr = model.predict([\"\u4ecb\u7ecd\u4e0b\u5317\u4eac\"])\nprint(r) # ['\u5317\u4eac\u662f\u4e2d\u56fd\u7684\u9996\u90fd...']\n```\n\n#### \u8bad\u7ec3 ChatGLM-6B \u5fae\u8c03\u6a21\u578b\n\n1. \u652f\u6301\u81ea\u5b9a\u4e49\u8bad\u7ec3\u6570\u636e\u96c6\u548c\u8bad\u7ec3\u53c2\u6570\uff0c\u6570\u636e\u96c6\u683c\u5f0f\u53c2\u8003[examples/data/sharegpt_zh_100_format.jsonl](https://github.com/shibing624/textgen/blob/main/examples/data/sharegpt_zh_100_format.jsonl)\n2. \u652f\u6301QLoRA\u3001AdaLoRA\u3001LoRA\u3001P_Tuning\u3001Prefix_Tuning\u7b49\u90e8\u5206\u53c2\u6570\u5fae\u8c03\u65b9\u6cd5\uff0c\u4e5f\u652f\u6301\u5168\u53c2\u5fae\u8c03\n3. \u652f\u6301\u591a\u5361\u8bad\u7ec3\uff0c\u652f\u6301\u6df7\u5408\u7cbe\u5ea6\u8bad\u7ec3\n4. \u652f\u6301\u591a\u5361\u63a8\u7406\n\nexample: [examples/gpt/training_chatglm_demo.py](https://github.com/shibing624/textgen/blob/main/examples/gpt/training_chatglm_demo.py)\n\n\u5355\u5361\u8bad\u7ec3\uff1a\n```shell\ncd examples/gpt\nCUDA_VISIBLE_DEVICES=0 python training_chatglm_demo.py --do_train --do_predict --num_epochs 1 --output_dir outputs_chatglm_v1\n```\n\n\u591a\u5361\u8bad\u7ec3\uff1a\n```shell\ncd examples/gpt\nCUDA_VISIBLE_DEVICES=0,1 torchrun --nproc_per_node 2 training_chatglm_demo.py --do_train --do_predict --num_epochs 20 --output_dir outputs_chatglm_v1\n```\n\n\u591a\u5361\u63a8\u7406\uff1a\n```shell\ncd examples/gpt\nCUDA_VISIBLE_DEVICES=0,1 torchrun --nproc_per_node 2 inference_multigpu_demo.py --model_type chatglm --base_model THUDM/chatglm-6b\n```\n\n\n### LLaMA \u6a21\u578b\n\n#### \u4f7f\u7528 LLaMA \u5fae\u8c03\u540e\u7684\u6a21\u578b\n\nexample: [examples/gpt/inference_demo.py](https://github.com/shibing624/textgen/blob/main/examples/gpt/inference_demo.py)\n\n<details>\n<summary>show code example and result</summary>\n\n```python\nimport sys\n\nsys.path.append('../..')\nfrom textgen import GptModel\n\nmodel = GptModel(\"llama\", \"decapoda-research/llama-7b-hf\", peft_name=\"ziqingyang/chinese-alpaca-lora-7b\")\nr = model.predict([\"\u7528\u4e00\u53e5\u8bdd\u63cf\u8ff0\u5730\u7403\u4e3a\u4ec0\u4e48\u662f\u72ec\u4e00\u65e0\u4e8c\u7684\u3002\"])\nprint(r) # ['\u5730\u7403\u662f\u552f\u4e00\u4e00\u9897\u62e5\u6709\u751f\u547d\u7684\u884c\u661f\u3002']\n```\n\n</details>\n\n#### \u8bad\u7ec3 LLaMA \u5fae\u8c03\u6a21\u578b\n1. \u652f\u6301\u81ea\u5b9a\u4e49\u8bad\u7ec3\u6570\u636e\u96c6\u548c\u8bad\u7ec3\u53c2\u6570\uff0c\u6570\u636e\u96c6\u683c\u5f0f\u53c2\u8003[examples/data/sharegpt_zh_100_format.jsonl](https://github.com/shibing624/textgen/blob/main/examples/data/sharegpt_zh_100_format.jsonl)\n2. \u652f\u6301QLoRA\u3001AdaLoRA\u3001LoRA\u3001P_Tuning\u3001Prefix_Tuning\u7b49\u90e8\u5206\u53c2\u6570\u5fae\u8c03\u65b9\u6cd5\uff0c\u4e5f\u652f\u6301\u5168\u53c2\u5fae\u8c03\n3. \u652f\u6301\u591a\u5361\u8bad\u7ec3\uff0c\u652f\u6301\u6df7\u5408\u7cbe\u5ea6\u8bad\u7ec3\uff0c\u4f7f\u7528\u65b9\u6cd5\u540c\u4e0a\uff08ChatGLM\u591a\u5361\u8bad\u7ec3\uff09\n4. \u652f\u6301\u591a\u5361\u63a8\u7406\n\nexample: [examples/gpt/training_llama_demo.py](https://github.com/shibing624/textgen/blob/main/examples/gpt/training_llama_demo.py)\n\n\n#### \u57fa\u4e8e\u5fae\u8c03(LoRA)\u6a21\u578b\u7ee7\u7eed\u8bad\u7ec3\n\u5982\u679c\u9700\u8981\u57fa\u4e8eLora\u6a21\u578b\u7ee7\u7eed\u8bad\u7ec3\uff0c\u53ef\u4ee5\u4f7f\u7528\u4e0b\u9762\u7684\u811a\u672c\u5408\u5e76\u6a21\u578b\u4e3a\u65b0\u7684base model\uff0c\u518d\u5fae\u8c03\u8bad\u7ec3\u5373\u53ef\u3002\n\n\u6267\u884c\u4ee5\u4e0b\u547d\u4ee4\uff1a\n```shell\npython -m textgen/gpt/merge_peft_adapter \\\n --model_type llama \\\n --base_model_name_or_path path/to/llama/model \\\n --tokenizer_path path/to/llama/tokenizer \\\n --peft_model_path path/to/lora/model \\\n --output_dir merged\n```\n\u53c2\u6570\u8bf4\u660e\uff1a\n```\n--model_type\uff1a\u6a21\u578b\u7c7b\u578b\uff0c\u76ee\u524d\u652f\u6301bloom,llama,baichuan\u548cchatglm\n--base_model_name_or_path\uff1a\u5b58\u653eHF\u683c\u5f0f\u7684\u5e95\u5ea7\u6a21\u578b\u6743\u91cd\u548c\u914d\u7f6e\u6587\u4ef6\u7684\u76ee\u5f55\n--tokenizer_path\uff1a\u5b58\u653eHF\u683c\u5f0f\u7684\u5e95\u5ea7\u6a21\u578btokenizer\u6587\u4ef6\u7684\u76ee\u5f55\n--peft_model_path\uff1a\u4e2d\u6587LLaMA/Alpaca LoRA\u89e3\u538b\u540e\u6587\u4ef6\u6240\u5728\u76ee\u5f55\uff0c\u4e5f\u53ef\u4f7f\u7528HF\u4e0a\u7684Lora\u6a21\u578b\u540d\u79f0\uff0c\u5982`ziqingyang/chinese-alpaca-lora-7b`\u4f1a\u81ea\u52a8\u4e0b\u8f7d\u5bf9\u5e94\u6a21\u578b\n--output_dir\uff1a\u6307\u5b9a\u4fdd\u5b58\u5168\u91cf\u6a21\u578b\u6743\u91cd\u7684\u76ee\u5f55\uff0c\u9ed8\u8ba4\u4e3a./merged\n```\n\n#### \u8bad\u7ec3\u9886\u57df\u6a21\u578b\nNote: \u4e3a\u4e86\u5168\u9762\u7684\u4ecb\u7ecd\u8bad\u7ec3\u533b\u7597\u5927\u6a21\u578b\u7684\u8fc7\u7a0b\uff0c\u628a4\u9636\u6bb5\u8bad\u7ec3\u65b9\u6cd5(Pretraining, Supervised Finetuning, Reward Modeling and Reinforcement Learning)\u5355\u72ec\u65b0\u5efa\u4e86\u4e00\u4e2arepo\uff1a[shibing624/MedicalGPT](https://github.com/shibing624/MedicalGPT)\uff0c\u8bf7\u79fb\u6b65\u8be5repo\u67e5\u770b\u8bad\u7ec3\u65b9\u6cd5\u3002\n\n### ConvSeq2Seq \u6a21\u578b\n\n\u8bad\u7ec3\u5e76\u9884\u6d4bConvSeq2Seq\u6a21\u578b\uff1a\n\nexample: [examples/seq2sesq/training_convseq2seq_model_demo.py](https://github.com/shibing624/textgen/blob/main/examples/seq2seq/training_convseq2seq_model_demo.py)\n\n<details>\n<summary>show code example and result</summary>\n\n```python\nimport argparse\nfrom loguru import logger\nimport sys\n\nsys.path.append('../..')\nfrom textgen.seq2seq.conv_seq2seq_model import ConvSeq2SeqModel\n\n\ndef main():\n parser = argparse.ArgumentParser()\n parser.add_argument('--train_file', default='../data/zh_dialog.tsv', type=str, help='Training data file')\n parser.add_argument('--do_train', action='store_true', help='Whether to run training.')\n parser.add_argument('--do_predict', action='store_true', help='Whether to run predict.')\n parser.add_argument('--output_dir', default='./outputs/convseq2seq_zh/', type=str, help='Model output directory')\n parser.add_argument('--max_seq_length', default=50, type=int, help='Max sequence length')\n parser.add_argument('--num_epochs', default=200, type=int, help='Number of training epochs')\n parser.add_argument('--batch_size', default=32, type=int, help='Batch size')\n args = parser.parse_args()\n logger.info(args)\n\n if args.do_train:\n logger.info('Loading data...')\n model = ConvSeq2SeqModel(epochs=args.num_epochs, batch_size=args.batch_size,\n model_dir=args.output_dir, max_length=args.max_seq_length)\n model.train_model(args.train_file)\n print(model.eval_model(args.train_file))\n\n if args.do_predict:\n model = ConvSeq2SeqModel(epochs=args.num_epochs, batch_size=args.batch_size,\n model_dir=args.output_dir, max_length=args.max_seq_length)\n sentences = [\"\u4ec0\u4e48\u662fai\", \"\u4f60\u662f\u4ec0\u4e48\u7c7b\u578b\u7684\u8ba1\u7b97\u673a\", \"\u4f60\u77e5\u9053\u70ed\u529b\u5b66\u5417\"]\n print(\"inputs:\", sentences)\n print('outputs:', model.predict(sentences))\n\n\nif __name__ == '__main__':\n main()\n```\n\noutput:\n\n```bash\ninputs: [\"\u4ec0\u4e48\u662fai\", \"\u4f60\u662f\u4ec0\u4e48\u7c7b\u578b\u7684\u8ba1\u7b97\u673a\", \"\u4f60\u77e5\u9053\u70ed\u529b\u5b66\u5417\"]\noutputs: ['\u4eba\u5de5\u667a\u80fd\u662f\u5de5\u7a0b\u548c\u79d1\u5b66\u7684\u5206\u652f,\u81f4\u529b\u4e8e\u6784\u5efa\u601d\u7ef4\u7684\u673a\u5668\u3002', '\u6211\u7684\u7a0b\u5e8f\u8fd0\u884c\u5728python,\u6240\u4ee5\u6211\u5728\u4efb\u4f55\u8fd0\u8111\u4e0a\u5de5\u4f5c\uff01', '\u6211\u4e0d\u80fd\u9519\u70ed\u662f\u4e00\u4e2a\u75af\u72c2\u7684\u4eba\u5de5\u667a\u80fd\"200\u5e74\u3002']\n```\n\n</details>\n\n### BART \u6a21\u578b\n\n\u8bad\u7ec3\u5e76\u9884\u6d4bBART\u6a21\u578b\uff1a\n\nexample: [examples/seq2sesq/training_bartseq2seq_zh_demo.py](https://github.com/shibing624/textgen/blob/main/examples/seq2seq/training_bartseq2seq_zh_demo.py)\n\noutput:\n\n```shell\ninputs: ['\u4ec0\u4e48\u662fai', '\u4f60\u662f\u4ec0\u4e48\u7c7b\u578b\u7684\u8ba1\u7b97\u673a', '\u4f60\u77e5\u9053\u70ed\u529b\u5b66\u5417']\noutputs: ['\u4eba\u5de5\u667a\u80fd\u662f\u5de5\u7a0b\u548c\u79d1\u5b66\u7684\u5206\u652f,\u81f4\u529b\u4e8e\u6784', '\u6211\u7684\u7a0b\u5e8f\u8fd0\u884c\u5728python,\u6240\u4ee5\u6211\u5728\u4efb\u4f55\u7535\u8111\u4e0a', '\u4ec0\u4e48\u662f\u70ed\u529b\u5b66\u5417\uff1f']\n```\n\n### T5 \u6a21\u578b\n\nexample: [examples/t5/training_zh_t5_model_demo.py](https://github.com/shibing624/textgen/blob/main/examples/t5/training_zh_t5_model_demo.py)\n\n<details>\n<summary>show code example and result</summary>\n\n```python\nimport argparse\nfrom loguru import logger\nimport pandas as pd\nimport sys\n\nsys.path.append('../..')\nfrom textgen.t5 import T5Model\n\n\ndef load_data(file_path):\n data = []\n with open(file_path, 'r', encoding='utf-8') as f:\n for line in f:\n line = line.strip('\\n')\n terms = line.split('\\t')\n if len(terms) == 2:\n data.append(['QA', terms[0], terms[1]])\n else:\n logger.warning(f'line error: {line}')\n return data\n\n\ndef main():\n parser = argparse.ArgumentParser()\n parser.add_argument('--train_file', default='../data/zh_dialog.tsv', type=str, help='Training data file')\n parser.add_argument('--model_type', default='t5', type=str, help='Transformers model type')\n parser.add_argument('--model_name', default='Langboat/mengzi-t5-base', type=str, help='Transformers model or path')\n parser.add_argument('--do_train', action='store_true', help='Whether to run training.')\n parser.add_argument('--do_predict', action='store_true', help='Whether to run predict.')\n parser.add_argument('--output_dir', default='./outputs/mengzi_t5_zh/', type=str, help='Model output directory')\n parser.add_argument('--max_seq_length', default=50, type=int, help='Max sequence length')\n parser.add_argument('--num_epochs', default=10, type=int, help='Number of training epochs')\n parser.add_argument('--batch_size', default=32, type=int, help='Batch size')\n args = parser.parse_args()\n logger.info(args)\n\n if args.do_train:\n logger.info('Loading data...')\n # train_data: Pandas DataFrame containing the 3 columns - `prefix`, `input_text`, `target_text`.\n # - `prefix`: A string indicating the task to perform. (E.g. `\"question\"`, `\"stsb\"`)\n # - `input_text`: The input text. `prefix` is prepended to form the full input. (<prefix>: <input_text>)\n # - `target_text`: The target sequence\n train_data = load_data(args.train_file)\n logger.debug('train_data: {}'.format(train_data[:10]))\n train_df = pd.DataFrame(train_data, columns=[\"prefix\", \"input_text\", \"target_text\"])\n\n eval_data = load_data(args.train_file)[:10]\n eval_df = pd.DataFrame(eval_data, columns=[\"prefix\", \"input_text\", \"target_text\"])\n\n model_args = {\n \"reprocess_input_data\": True,\n \"overwrite_output_dir\": True,\n \"max_seq_length\": args.max_seq_length,\n \"train_batch_size\": args.batch_size,\n \"num_train_epochs\": args.num_epochs,\n \"save_eval_checkpoints\": False,\n \"save_model_every_epoch\": False,\n \"evaluate_generated_text\": True,\n \"evaluate_during_training\": True,\n \"evaluate_during_training_verbose\": True,\n \"use_multiprocessing\": True,\n \"save_best_model\": True,\n \"output_dir\": args.output_dir,\n \"use_early_stopping\": True,\n }\n # model_type: t5 model_name: Langboat/mengzi-t5-base\n model = T5Model(args.model_type, args.model_name, args=model_args)\n\n def count_matches(labels, preds):\n logger.debug(f\"labels: {labels[:10]}\")\n logger.debug(f\"preds: {preds[:10]}\")\n match = sum([1 if label == pred else 0 for label, pred in zip(labels, preds)])\n logger.debug(f\"match: {match}\")\n return match\n\n model.train_model(train_df, eval_data=eval_df, matches=count_matches)\n print(model.eval_model(eval_df, matches=count_matches))\n\n if args.do_predict:\n model = T5Model(args.model_type, args.output_dir)\n sentences = [\"\u4ec0\u4e48\u662fai\", \"\u4f60\u662f\u4ec0\u4e48\u7c7b\u578b\u7684\u8ba1\u7b97\u673a\", \"\u4f60\u77e5\u9053\u70ed\u529b\u5b66\u5417\"]\n print(\"inputs:\", sentences)\n print(\"outputs:\", model.predict(sentences))\n\n\nif __name__ == '__main__':\n main()\n```\n\noutput:\n\n```shell\ninputs: ['\u4ec0\u4e48\u662fai', '\u4f60\u662f\u4ec0\u4e48\u7c7b\u578b\u7684\u8ba1\u7b97\u673a', '\u4f60\u77e5\u9053\u70ed\u529b\u5b66\u5417']\noutputs: ['\u4eba\u5de5\u667a\u80fd\u6709\u4e24\u4e2a\u5e7f\u4e49\u7684\u5b9a\u4e49,\u4efb\u4f55\u62df\u4eba\u7684\u673a\u68b0,\u5982\u5728\u5361\u96f7\u5c14capeks', '\u6211\u7684\u7a0b\u5e8f\u8fd0\u884c\u5728Python,\u6240\u4ee5\u6211\u5728\u4efb\u4f55\u7535\u8111\u4e0a\u5de5\u4f5c!', '\u4ec0\u4e48\u662f\u70ed\u529b\u5b66']\n```\n\n</details>\n\n### GPT2 \u6a21\u578b\n\n#### \u4e2d\u6587GPT2 - \u6587\u7ae0\u751f\u6210\n\n\u4f7f\u7528\u4e2d\u6587\u6570\u636e\u96c6\uff08\u6bb5\u843d\u683c\u5f0f\uff0c`\\n`\u95f4\u9694\uff09\uff0c\u8bad\u7ec3GPT2\u6a21\u578b\uff0c\u53ef\u4ee5\u7528\u4e8e\u8bd7\u6b4c\u751f\u6210\u3001\u6587\u7ae0\u751f\u6210\u7b49\u4efb\u52a1\u3002\n\nexample: [examples/gpt2/training_zh_gpt2_demo.py](https://github.com/shibing624/textgen/blob/main/examples/gpt2/training_zh_gpt2_demo.py)\n\n#### \u4e2d\u6587GPT2 - \u5bf9\u8054\u751f\u6210\n\n\u4f7f\u7528\u4e2d\u6587\u5bf9\u8054\u6570\u636e\u96c6\uff08tsv\u683c\u5f0f\uff0c`\\t`\u95f4\u9694\uff09\uff0c\u81ea\u5b9a\u4e49\u6570\u636e\u96c6\u8bfb\u53d6Dataset\uff0c\u8bad\u7ec3GPT2\u6a21\u578b\uff0c\u53ef\u4ee5\u7528\u4e8e\u5bf9\u8054\u751f\u6210\u3001\u5bf9\u8bdd\u751f\u6210\u7b49\u4efb\u52a1\u3002\n\nexample: [examples/gpt2/training_couplet_gpt2_demo.py](https://github.com/shibing624/textgen/blob/main/examples/gpt2/training_couplet_gpt2_demo.py)\n\nGPT2 vs T5\uff1a\n\n1. \u90fd\u662f\u4eceTransformer\u6539\u8fdb\u6765\u7684\uff0cT5\u540c\u65f6\u6709\u7f16\u7801\u5668\u548c\u89e3\u7801\u5668\uff0cGPT2\u53ea\u6709\u89e3\u7801\u5668\n2. T5\u7684\u6a21\u578b\u4f18\u52bf\u662f\u5904\u7406\u7ed9\u5b9a\u8f93\u5165\uff0c\u4ea7\u51fa\u5bf9\u5e94\u8f93\u51fa\u7684\u4efb\u52a1\uff0c\u5982\u7ffb\u8bd1\u3001\u5bf9\u8bdd\u3001\u95ee\u7b54\u7b49\n3. GPT2\u7684\u6a21\u578b\u4f18\u52bf\u662f\u81ea\u7531\u521b\u4f5c\uff0c\u5982\u5199\u4e00\u7bc7\u77ed\u6587\n4. T5\u7684\u5bf9\u8054\u751f\u6210\u6548\u679c\u597d\u4e8eGPT2\u3001GPT2\u7684\u8bd7\u8bcd\u751f\u6210\u6548\u679c\u597d\u4e8eT5\n\n- [\u5bf9\u8054\u751f\u6210\u6a21\u578b\u8c03\u7814](https://github.com/shibing624/textgen/blob/main/docs/%E5%AF%B9%E8%81%94%E7%94%9F%E6%88%90%E6%A8%A1%E5%9E%8B%E5%AF%B9%E6%AF%94.md)\n- [\u53e4\u8bd7\u751f\u6210\u6a21\u578b\u8c03\u7814](https://github.com/shibing624/textgen/blob/main/docs/%E5%8F%A4%E8%AF%97%E7%94%9F%E6%88%90%E6%A8%A1%E5%9E%8B%E5%AF%B9%E6%AF%94.md)\n\n### SongNet \u6a21\u578b\n\n\u683c\u5f0f\u63a7\u5236\u7684\u6587\u672c\u751f\u6210\u6a21\u578b\uff0cpaper\u89c1[SongNet: Rigid Formats Controlled Text Generation](https://arxiv.org/abs/2004.08022)\uff0c\n\u9002\u7528\u4e8e\u5f3a\u97f5\u5f8b\u683c\u5f0f\u8981\u6c42\u7684\u8bd7\u6b4c\u3001\u5bf9\u8054\u3001\u6b4c\u8bcd\u751f\u6210\u7b49\u4efb\u52a1\u3002\n\nexample: [examples/songnet/training_zh_songnet_demo.py](https://github.com/shibing624/textgen/blob/main/examples/songnet/training_zh_songnet_demo.py)\n\n### Keyword Text Augmentation(EDA/UDA)\n\nexample: [examples/text_augmentation/text_augmentation_demo.py](examples/text_augmentation/text_augmentation_demo.py)\n\n<details>\n<summary>show code example and result</summary>\n\n```python\nimport sys\n\nsys.path.append('..')\nfrom textgen.augment import TextAugment\n\nif __name__ == '__main__':\n docs = ['\u4e3b\u8981\u7814\u7a76\u673a\u5668\u5b66\u4e60\u3001\u6df1\u5ea6\u5b66\u4e60\u3001\u8ba1\u7b97\u673a\u89c6\u89c9\u3001\u667a\u80fd\u5bf9\u8bdd\u7cfb\u7edf\u76f8\u5173\u5185\u5bb9',\n '\u665a\u4e0a\u809a\u5b50\u597d\u96be\u53d7',\n '\u4f60\u4f1a\u6b66\u529f\u5417\uff0c\u6211\u4e0d\u4f1a',\n '\u7ec4\u88c5\u6807\u9898\u8d28\u91cf\u53d7\u9650\u4e8e\u5e7f\u544a\u4e3b\u81ea\u63d0\u7269\u6599\u7684\u7247\u6bb5\u8d28\u91cf\uff0c\u4e14\u8868\u8fbe\u4e30\u5bcc\u5ea6\u6709\u9650',\n ]\n m = TextAugment(sentence_list=docs)\n a = docs[0]\n print(a)\n\n b = m.augment(a, aug_ops='random-0.2')\n print('random-0.2:', b)\n\n b = m.augment(a, aug_ops='insert-0.2')\n print('insert-0.2:', b)\n\n b = m.augment(a, aug_ops='delete-0.2')\n print('delete-0.2:', b)\n\n b = m.augment(a, aug_ops='tfidf-0.2')\n print('tfidf-0.2:', b)\n\n b = m.augment(a, aug_ops='mix-0.2')\n print('mix-0.2:', b)\n```\n\noutput:\n\n```bash\n\u4e3b\u8981\u7814\u7a76\u673a\u5668\u5b66\u4e60\u3001\u6df1\u5ea6\u5b66\u4e60\u3001\u8ba1\u7b97\u673a\u89c6\u89c9\u3001\u667a\u80fd\u5bf9\u8bdd\u7cfb\u7edf\u76f8\u5173\u5185\u5bb9\nrandom-0.2: ('\u4e3b\u8981\u966a\u966a\u673a\u5668\u5b66\u4e60\u3001\u6df1\u5ea6\u5b66\u4e60\u4e3b\u8981\u8ba1\u7b97\u673a\u89c6\u89c9\u3001\u667a\u80fd\u5bf9\u8bdd\u7cfb\u7edf\u53d7\u9650\u4e8e\u5185\u5bb9', [('\u7814\u7a76', '\u966a\u966a', 2, 4), ('\u3001', '\u4e3b\u8981', 13, 15), ('\u76f8\u5173', '\u53d7\u9650\u4e8e', 27, 30)])\ninsert-0.2: ('\u4e3b\u8981\u7814\u7a76\u673a\u5668\u673a\u5668\u5b66\u4e60\u5b66\u4e60\u3001\u6df1\u5ea6\u6df1\u5ea6\u5b66\u4e60\u3001\u8ba1\u7b97\u673a\u89c6\u89c9\u3001\u667a\u80fd\u5bf9\u8bdd\u7cfb\u7edf\u76f8\u5173\u5185\u5bb9', [('\u673a\u5668', '\u673a\u5668\u673a\u5668', 4, 8), ('\u5b66\u4e60', '\u5b66\u4e60\u5b66\u4e60', 8, 12), ('\u6df1\u5ea6', '\u6df1\u5ea6\u6df1\u5ea6', 13, 17)])\ndelete-0.2: ('\u4e3b\u8981\u7814\u7a76\u673a\u5668\u5b66\u4e60\u3001\u6df1\u5ea6\u5b66\u4e60\u3001\u8ba1\u7b97\u673a\u89c6\u89c9\u3001\u5bf9\u8bdd\u7cfb\u7edf\u76f8\u5173\u5185\u5bb9', [('\u667a\u80fd', '', 20, 20)])\ntfidf-0.2: ('\u4e00\u662f\u7814\u7a76\u673a\u5668\u5b66\u4e60\u3001\u6df1\u5ea6\u5b66\u4e60\u3001\u8ba1\u7b97\u673a\u542c\u89c9\u3001\u667a\u80fd\u4ea4\u8c08\u7cfb\u7edf\u5bc6\u5207\u76f8\u5173\u5185\u5bb9', [('\u4e3b\u8981', '\u4e00\u662f', 0, 2), ('\u89c6\u89c9', '\u542c\u89c9', 17, 19), ('\u5bf9\u8bdd', '\u4ea4\u8c08', 22, 24), ('\u76f8\u5173', '\u5bc6\u5207\u76f8\u5173', 26, 30)])\nmix-0.2: ('\u4e3b\u8981\u7814\u7a76\u673a\u5668\u5b66\u4e60\u3001\u6df1\u5ea6\u5b66\u3001\u8ba1\u7b97\u673a\u542c\u89c9\u3001\u667a\u80fd\u5bf9\u8bdd\u8f6f\u4ef6\u7cfb\u7edf\u76f8\u5173\u5185\u5bb9', [('\u5b66\u4e60', '\u5b66', 11, 12), ('\u89c6\u89c9', '\u542c\u89c9', 16, 18), ('\u7cfb\u7edf', '\u8f6f\u4ef6\u7cfb\u7edf', 23, 27)])\n```\n</details>\n\n### TGLS \u6a21\u578b\uff08\u65e0\u76d1\u7763\u76f8\u4f3c\u6587\u672c\u751f\u6210\u6a21\u578b\uff09\n\n\u65e0\u76d1\u7763\u7684\u4e2d\u6587\u7535\u5546\u8bc4\u8bba\u751f\u6210\uff1a\u4ece**\u7535\u5546\u8bc4\u8bba**\u4e2d\u63d0\u53d6\u7528\u6237\u8868\u8fbe\u89c2\u70b9\u7684\u77ed\u53e5\u5e76\u8fdb\u884c\u7ec4\u5408\u6765\u751f\u6210\u4eff\u771f\u8bc4\u8bba\u3002\n\nexample: [examples/unsup_generation/unsup_generation_demo.py](examples/unsup_generation/unsup_generation_demo.py)\n\n<details>\n<summary>show code example and result</summary>\n\n```python\nimport os\nimport sys\n\nsys.path.append('..')\nfrom textgen.unsup_generation import TglsModel, load_list\n\npwd_path = os.path.abspath(os.path.dirname(__file__))\n\nsamples = load_list(os.path.join(pwd_path, './data/ecommerce_comments.txt'))\ndocs_text = [\n [\"\u633a\u597d\u7684\uff0c\u901f\u5ea6\u5f88\u5feb\uff0c\u4e5f\u5f88\u5b9e\u60e0\uff0c\u4e0d\u77e5\u6548\u679c\u5982\u4f55\",\n \"\u4ea7\u54c1\u6ca1\u5f97\u8bf4\uff0c\u4e70\u4e86\u4ee5\u540e\u5c31\u964d\u4ef7\uff0c\u5fc3\u60c5\u4e0d\u7f8e\u4e3d\u3002\",\n \"\u521a\u6536\u5230\uff0c\u5305\u88c5\u5f88\u5b8c\u6574\uff0c\u4e0d\u9519\",\n \"\u53d1\u8d27\u901f\u5ea6\u5f88\u5feb\uff0c\u7269\u6d41\u4e5f\u4e0d\u9519\uff0c\u540c\u4e00\u65f6\u95f4\u4e70\u7684\u4e24\u4e2a\u4e1c\u4e1c\uff0c\u4e00\u4e2a\u5148\u5230\u4e00\u4e2a\u8fd8\u5728\u8def\u4e0a\u3002\u8fd9\u4e2a\u6c34\u6c34\u5f88\u559c\u6b22\uff0c\u4e0d\u8fc7\u76d6\u5b50\u771f\u7684\u5f00\u4e86\u3002\u76d6\u4e0d\u7262\u4e86\u73b0\u5728\u3002\",\n \"\u5305\u88c5\u7684\u5f88\u597d\uff0c\u662f\u6b63\u54c1\",\n \"\u88ab\u79cd\u8349\u5170\u853b\u7c89\u6c34\u4e09\u767e\u5143\u4e00\u5927\u74f6\u56e4\u8d27\uff0c\u5e0c\u671b\u662f\u6b63\u54c1\u597d\u7528\uff0c\u6536\u5230\u7684\u65f6\u5019\u7528\u4fdd\u9c9c\u819c\u5305\u88f9\u5f97\u4e25\u4e25\u5b9e\u5b9e\uff0c\u53ea\u6562\u4e70\u8003\u62c9\u81ea\u8425\u7684\u62a4\u80a4\u54c1\",\n ],\n ['\u5f88\u6e29\u548c\uff0c\u6e05\u6d17\u7684\u4e5f\u5f88\u5e72\u51c0\uff0c\u4e0d\u6cb9\u817b\uff0c\u5f88\u4e0d\u9519\uff0c\u4f1a\u8003\u8651\u56de\u8d2d\uff0c\u7b2c\u4e00\u6b21\u8003\u62c9\u4e70\u62a4\u80a4\u54c1\uff0c\u6ee1\u610f',\n '\u8fd9\u6b3e\u5378\u5986\u6cb9\u6211\u4f1a\u65e0\u9650\u56de\u8d2d\u7684\u3002\u5373\u4f7f\u6211\u662f\u6cb9\u75d8\u76ae\uff0c\u4e5f\u4e0d\u4f1a\u95f7\u75d8\uff0c\u540c\u65f6\u5728\u8138\u90e8\u6309\u6469\u65f6\uff0c\u8fd8\u80fd\u89e3\u51b3\u767d\u5934\u7684\u8102\u80aa\u7c92\u7684\u95ee\u9898\u3002\u7528\u6e05\u6c34\u6d17\u5b8c\u8138\u540e\uff0c\u975e\u5e38\u7684\u6e05\u723d\u3002',\n '\u81ea\u4ece\u7528\u4e86fancl\u4e4b\u540e\u5c31\u4e0d\u7528\u5176\u4ed6\u5378\u5986\u4e86\uff0c\u5378\u7684\u8212\u670d\u53c8\u5e72\u51c0',\n '\u4e70\u8d35\u4e86\uff0c\u5927\u6da6\u53d1\u624d\u535679\u30029\u3002',\n ],\n samples\n]\nm = TglsModel(docs_text)\nr = m.generate(samples[:500])\nprint('size:', len(r))\nfor review in r:\n print('\\t' + review)\n```\n\noutput:\n\n[\u7f8e\u8fea\u60e0\u5c14 N.M.F\u9488\u5242\u6c34\u5e93\u4fdd\u6e7f\u9762\u819c](https://goods.kaola.com/product/2227311.html)\u6709\u5982\u4e0b\u768420\u53e5\u8bc4\u8bba\uff0c\u5176\u4e2d\u670910\u53e5\u662f\u771f\u5b9e\u7528\u6237\u8bc4\u8bba\uff0c10\u53e5\u662f\u751f\u6210\u7684\u8bc4\u8bba\uff0c\u80fd\u770b\u51fa\u6765\u4e48?\ud83d\ude02\n\n```\n\u8fd8\u4e0d\u9519\u8fd8\u4e0d\u9519\u8fd8\u4e0d\u9519\u8fd8\u4e0d\u9519\u3002\n\u4e1c\u897f\u5230\u4e86\uff0c\u4e0d\u77e5\u9053\u597d\u4e0d\u597d\u7528\u3002\u8bd5\u7528\u8fc7\u540e\u518d\u6765\u8bc4\u4ef7\u3002\u5230\u65f6\u770b\u7f51\u8bc4\u90fd\u8fd8\u53ef\u4ee5\u3002\n\u54fa\u4e73\u671f\u552f\u4e00\u4f7f\u7528\u7684\u62a4\u80a4\u54c1\uff0c\u6bcf\u5929\u90fd\u662f\u7d20\u989c\uff0c\u8138\u9762\u5168\u9760\u9762\u819c\u540a\u7740\ud83d\ude04\u8865\u6c34\ud83d\udca6\u4e0d\u7c98\u817b\u4e00\u5982\u65e2\u5f80\u7684\u652f\u6301\uff0c\u559c\u6b22\ud83d\udc95\n\u641e\u6d3b\u52a8\u65f6\u4e70\u7684\u9762\u819c\uff0c\u4e0d\u77e5\u9053\u8fd9\u4e2a\u9762\u819c\u662f\u771f\u662f\u5047\u6577\u5728\u8138\u4e0a\u9762\u819c\u7eb8\u90fd\u6709\u5c0f\u6c34\u6ce1\u9f13\u8d77\u6765\u3002\n\u5f88\u4e0d\u9519\uff0c\u975e\u5e38\u8865\u6c34\uff0c\u7528\u8fc7\u7684\u90fd\u77e5\u9053\uff0c\u6027\u4ef7\u6bd4\u4e4b\u738b\uff0c\u597d\u7528\u53c8\u4e0d\u8d35\uff0c\u6b63\u54c1\uff0c\u7528\u7740\u653e\u5fc3\uff0c\u7269\u6d41\u4e5f\u5f88\u5feb\u3002\n\u9762\u819c\u975e\u5e38\u597d\u7528\u54e6\u3002\u9762\u819c\u8584\u8584\u7684\u3002\u597d\u50cf\u662f\u8695\u4e1d\u9762\u819c\u554a\u3002\u7cbe\u534e\u5f88\u591a\u5462\u3002\u6577\u5728\u8138\u4e0a\u5f88\u8212\u670d\u3002\u611f\u89c9\u633a\u4fdd\u6e7f\u7684\uff0c\u5473\u9053\u4e5f\u633a\u597d\u95fb\u7684\u3002\u5c31\u662f\u91cc\u9762\u53ea\u6709\u5355\u7eaf\u7684\u9762\u819c\u76f4\u63a5\u6577\u8138\u4e0a\u6709\u70b9\u4e0d\u597d\u5f04\uff0c\u54c8\u54c8\u54c8\n\u8fd8\u53ef\u4ee5\u4fdd\u6e7f\u6548\u679c\u4e0d\u9519\u6c34\u6da6\u6da6\u7684\u6bcf\u5929\u8d34\u4e00\u7247\u8138\u4e5f\u4e0d\u5e72\u4e86\u7528\u5b8c\u4e86\u5728\u4e70\u70b9\uff0c\u4e0d\u9519\u8fd8\u4f1a\u7ee7\u7eed\u56de\u8d2d\u7684\u3002\n\u5feb\u9012\u5f88\u5feb\uff0c\u4e1c\u897f\u5f88\u8d5e\uff01\u60f3\u8981\u5f97\u70b9\u8003\u62c9\u8c46\u4e0d\u5bb9\u6613\uff0c\u8fd8\u8981\u4e09\u5341\u4e2a\u5b57\u3002\u65f6\u95f4\u5b9d\u8d35\uff0c\u5e9f\u8bdd\u4e0d\u8bf4\uff01\u7528\u8fc7\u4e86\u5c31\u77e5\u9053\u4e86\n\u633a\u597d\u7528\u7684\uff0c\u670b\u53cb\u63a8\u8350\u6765\u7684\n\u633a\u597d\u7528\u7684\uff0c\u6de1\u6de1\u7684\uff0c\u867d\u7136\u4e0d\u662f\u5f88\u6d53\u7cbe\u534e\u7684\u611f\u89c9\uff0c\u4f46\u662f\u6548\u679c\u4e5f\u86ee\u597d\u7684\u3002\u5212\u7b97\n\u4e0d\u5f97\u4e0d\u8bf4\u7f8e\u8fea\u60e0\u5c14\u7684\u9762\u819c\u662f\u6211\u7528\u8fc7\u7684\u6700\u597d\u7684\u9762\u819c\u4e4b\u4e00\ud83d\ude0e\u8865\u6c34\u6548\u679c\u975e\u5e38\u597d\uff0c\u6ca1\u60f3\u5230\u8fd9\u4e48\u4fbf\u5b9c\u7684\u4ef7\u683c\u7adf\u771f\u7684\u80fd\u4e70\u5230\u771f\u54c1\u3002\n\u4fdd\u6e7f\u6548\u679c\u633a\u597d\u7684\uff0c\u9762\u819c\u5f88\u597d\u7528\u3002\n\u671f\u5f85\u597d\u7684\u4ea7\u54c1\u3002\n\u4e00\u6253\u5f00\u5305\u88c5\u91cc\u9762\u7684\u7cbe\u534e\u521a\u521a\u597d\uff0c\u7528\u4e86\u8865\u6c34\u8865\u6c34\u6548\u679c\u4e0d\u9519\uff0c\u7269\u6d41\u975e\u5e38\u5feb\u3002\n\u76ae\u80a4\u5f88\u5149\u6ed1\ud83d\ude07\u6bd4\u4e0a\u53bb\u901f\u5ea6\u5feb\u4e09\u5929\u5c31\u5230\u4e86\u3002\n\u524d\u4e24\u5929\u76ae\u80a4\u5e72\u71e5\u8fde\u7eed\u6577\u4e86\u4e24\u4e2a\u665a\u4e0a\u611f\u89c9\u8fd8\u4e0d\u9519\ud83d\ude02\u8865\u6c34\u6548\u679c\u660e\u663e\uff01\u53ef\u60f3\u800c\u77e5\u7cbe\u534e\u6db2\u53c8\u591a\u5145\u8db3\ud83d\ude0d\u6577\u4e0a\u4ee5\u540e\u51c9\u51c9\u7684\u5f88\u8212\u670d\u3002\n\u8865\u6c34\u6548\u679c\u4e00\u822c\u5427\uff5e\u4f46\u662f\u6211\u7528\u7684\u97e9\u56fd\u80cc\u56de\u6765\u7684\u9762\u819c\u7eb8\u4e0d\u7b97\u8584\uff0c\u5e0c\u671b\u597d\u7528\u4f1a\u56de\u8d2d\u7684\uff0c\u6577\u4e0a\u8138\u611f\u89c9\u6bd4\u8f83\u6e05\u723d\uff5e\u4ef7\u683c\u8fd8\u4e0d\u4fbf\u5b9c\u3002\n\u5e0c\u671b\u597d\u7528\uff0c\u9762\u819c\u7528\u8fc7\u4e86\u5f88\u597d\u7528\uff0c\u76ae\u80a4\u6c34\u5ae9\u5149\u6ed1\u767d\u7699\uff0c\u8865\u6c34\u4e0d\u9519\uff0c\u4ef7\u683c\u4e5f\u5408\u9002\u3002\n\u5c31\u662f\u7cbe\u534e\u6db2\u592a\u5c11\u4e86\uff0c\u4fdd\u6e7f\u6548\u679c\u4e0d\u9519\u3002\n\u9762\u819c\u7684\u8865\u6c34\u6548\u679c\u975e\u5e38\u597d\uff0c\u4fdd\u6e7f\u6548\u679c\u786e\u5b9e\u5f88\u8d5e\uff0c\u8fd9\u4e2a\u9762\u819c\u76f8\u5bf9\u4e8e\u80f6\u539f\u86cb\u767d\u548c\u7f8e\u767d\u7684\u90a3\u4e24\u6b3e\u7684\u9762\u819c\u7eb8\u8981\u539a\u4e00\u4e9b\uff0c\u770b\u7740\u4ef7\u683c\u5408\u9002\u3002\n```\n\n\u524d10\u53e5\u662f\u771f\u5b9e\u7528\u6237\u8bc4\u8bba\uff0c\u540e10\u53e5\u662f\u751f\u6210\u7684\u3002\n\n</details>\n\n## \ud83d\udcda Dataset \n\n#### SFT datasets\n- 50\u4e07\u6761\u4e2d\u6587ChatGPT\u6307\u4ee4Belle\u6570\u636e\u96c6\uff1a[BelleGroup/train_0.5M_CN](https://huggingface.co/datasets/BelleGroup/train_0.5M_CN)\n- 100\u4e07\u6761\u4e2d\u6587ChatGPT\u6307\u4ee4Belle\u6570\u636e\u96c6\uff1a[BelleGroup/train_1M_CN](https://huggingface.co/datasets/BelleGroup/train_1M_CN)\n- 5\u4e07\u6761\u82f1\u6587ChatGPT\u6307\u4ee4Alpaca\u6570\u636e\u96c6\uff1a[50k English Stanford Alpaca dataset](https://github.com/tatsu-lab/stanford_alpaca#data-release)\n- 2\u4e07\u6761\u4e2d\u6587ChatGPT\u6307\u4ee4Alpaca\u6570\u636e\u96c6\uff1a[shibing624/alpaca-zh](https://huggingface.co/datasets/shibing624/alpaca-zh)\n- 69\u4e07\u6761\u4e2d\u6587\u6307\u4ee4Guanaco\u6570\u636e\u96c6(Belle50\u4e07\u6761+Guanaco19\u4e07\u6761)\uff1a[Chinese-Vicuna/guanaco_belle_merge_v1.0](https://huggingface.co/datasets/Chinese-Vicuna/guanaco_belle_merge_v1.0)\n- 240\u4e07\u6761\u4e2d\u6587\u533b\u7597\u6570\u636e\u96c6(\u5305\u62ec\u9884\u8bad\u7ec3\u6570\u636e\u548c\u6307\u4ee4\u5fae\u8c03\u6570\u636e\u96c6)\uff1a[shibing624/medical](https://huggingface.co/datasets/shibing624/medical)\n- 5\u4e07\u6761\u82f1\u6587ChatGPT\u591a\u8f6e\u5bf9\u8bdd\u6570\u636e\u96c6\uff1a[RyokoAI/ShareGPT52K](https://huggingface.co/datasets/RyokoAI/ShareGPT52K)\n- 80\u4e07\u6761\u4e2d\u6587ChatGPT\u591a\u8f6e\u5bf9\u8bdd\u6570\u636e\u96c6\uff1a[BelleGroup/multiturn_chat_0.8M](https://huggingface.co/datasets/BelleGroup/multiturn_chat_0.8M)\n- 116\u4e07\u6761\u4e2d\u6587ChatGPT\u591a\u8f6e\u5bf9\u8bdd\u6570\u636e\u96c6\uff1a[fnlp/moss-002-sft-data](https://huggingface.co/datasets/fnlp/moss-002-sft-data)\n\n#### Reward Model datasets\n- \u539f\u7248\u7684oasst1\u6570\u636e\u96c6\uff1a[OpenAssistant/oasst1](https://huggingface.co/datasets/OpenAssistant/oasst1)\n- 2\u4e07\u6761\u591a\u8bed\u8a00oasst1\u7684reward\u6570\u636e\u96c6\uff1a[tasksource/oasst1_pairwise_rlhf_reward](https://huggingface.co/datasets/tasksource/oasst1_pairwise_rlhf_reward)\n- 11\u4e07\u6761\u82f1\u6587hh-rlhf\u7684reward\u6570\u636e\u96c6\uff1a[Dahoas/full-hh-rlhf](https://huggingface.co/datasets/Dahoas/full-hh-rlhf)\n- 9\u4e07\u6761\u82f1\u6587reward\u6570\u636e\u96c6(\u6765\u81eaAnthropic's Helpful Harmless dataset)\uff1a[Dahoas/static-hh](https://huggingface.co/datasets/Dahoas/static-hh)\n- 7\u4e07\u6761\u82f1\u6587reward\u6570\u636e\u96c6\uff08\u6765\u6e90\u540c\u4e0a\uff09\uff1a[Dahoas/rm-static](https://huggingface.co/datasets/Dahoas/rm-static)\n- 7\u4e07\u6761\u7e41\u4f53\u4e2d\u6587\u7684reward\u6570\u636e\u96c6\uff08\u7ffb\u8bd1\u81earm-static\uff09[liswei/rm-static-m2m100-zh](https://huggingface.co/datasets/liswei/rm-static-m2m100-zh)\n- 7\u4e07\u6761\u82f1\u6587Reward\u6570\u636e\u96c6\uff1a[yitingxie/rlhf-reward-datasets](https://huggingface.co/datasets/yitingxie/rlhf-reward-datasets)\n- 3\u5343\u6761\u4e2d\u6587\u77e5\u4e4e\u95ee\u7b54\u504f\u597d\u6570\u636e\u96c6\uff1a[liyucheng/zhihu_rlhf_3k](https://huggingface.co/datasets/liyucheng/zhihu_rlhf_3k)\n\n## \u2705 Todo\n\n1. [x] add multiple rounds of dialogue data fine-tuning method\n2. [x] add reward model finetuning, go to [shibing624/MeidcalGPT](https://github.com/shibing624/MedicalGPT)\n3. [x] add rl finetuning, go to [shibing624/MeidcalGPT](https://github.com/shibing624/MedicalGPT)\n4. [x] add medical reward dataset\n5. [x] add llama in4 training, go to [shibing624/MeidcalGPT](https://github.com/shibing624/MedicalGPT)\n6. [ ] add all training and predict demo in colab\n\n## \u260e\ufe0f Contact\n\n- Issue(\u5efa\u8bae)\n \uff1a[](https://github.com/shibing624/textgen/issues)\n- \u90ae\u4ef6\u6211\uff1axuming: xuming624@qq.com\n- \u5fae\u4fe1\u6211\uff1a \u52a0\u6211*\u5fae\u4fe1\u53f7\uff1axuming624, \u5907\u6ce8\uff1a\u59d3\u540d-\u516c\u53f8\u540d-NLP* \u8fdbNLP\u4ea4\u6d41\u7fa4\u3002\n\n<img src=\"docs/wechat.jpeg\" width=\"200\" />\n\n## \ud83d\ude07 Citation\n\n\u5982\u679c\u4f60\u5728\u7814\u7a76\u4e2d\u4f7f\u7528\u4e86textgen\uff0c\u8bf7\u6309\u5982\u4e0b\u683c\u5f0f\u5f15\u7528\uff1a\n\n```latex\n@misc{textgen,\n title={textgen: Text Generation Tool},\n author={Ming Xu},\n year={2021},\n howpublished={\\url{https://github.com/shibing624/textgen}},\n}\n```\n\n## \ud83e\udd17 License\nThis repository is licensed under [The Apache License 2.0](LICENSE).\n\nPlease follow the [Model Card](https://github.com/facebookresearch/llama/blob/main/MODEL_CARD.md) to use the LLaMA model.\n\nPlease follow the [RAIL License](https://huggingface.co/spaces/bigscience/license) to use the BLOOM & BLOOMZ model.\n\n## \ud83d\ude0d Contribute\n\n\u9879\u76ee\u4ee3\u7801\u8fd8\u5f88\u7c97\u7cd9\uff0c\u5982\u679c\u5927\u5bb6\u5bf9\u4ee3\u7801\u6709\u6240\u6539\u8fdb\uff0c\u6b22\u8fce\u63d0\u4ea4\u56de\u672c\u9879\u76ee\uff0c\u5728\u63d0\u4ea4\u4e4b\u524d\uff0c\u6ce8\u610f\u4ee5\u4e0b\u4e24\u70b9\uff1a\n\n- \u5728`tests`\u6dfb\u52a0\u76f8\u5e94\u7684\u5355\u5143\u6d4b\u8bd5\n- \u4f7f\u7528`python -m pytest`\u6765\u8fd0\u884c\u6240\u6709\u5355\u5143\u6d4b\u8bd5\uff0c\u786e\u4fdd\u6240\u6709\u5355\u6d4b\u90fd\u662f\u901a\u8fc7\u7684\n\n\u4e4b\u540e\u5373\u53ef\u63d0\u4ea4PR\u3002\n\n## \ud83d\udc95 Acknowledgements \n\n- [PaddlePaddle/ERNIE](https://github.com/PaddlePaddle/ERNIE)\n- [minimaxir/textgenrnn](https://github.com/minimaxir/textgenrnn)\n- [minimaxir/gpt-2-simple](https://github.com/minimaxir/gpt-2-simple)\n- [asyml/texar](https://github.com/asyml/texar)\n- [yangjianxin1/GPT2-chitchat](https://github.com/yangjianxin1/GPT2-chitchat)\n- [williamSYSU/TextGAN-PyTorch](https://github.com/williamSYSU/TextGAN-PyTorch)\n- [RUCAIBox/TextBox](https://github.com/RUCAIBox/TextBox)\n- [Tiiiger/bert_score](https://github.com/Tiiiger/bert_score)\n- [ThilinaRajapakse/simpletransformers](https://github.com/ThilinaRajapakse/simpletransformers)\n- [1YCxZ/Fake-review-generation](https://github.com/1YCxZ/Fake-review-generation)\n- [tloen/alpaca-lora](https://github.com/tloen/alpaca-lora/blob/main/finetune.py)\n\nThanks for their great work!",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "Text Generation Model",
"version": "1.1.1",
"project_urls": {
"Homepage": "https://github.com/shibing624/textgen"
},
"split_keywords": [
"textgen",
"text-generation",
"text generation tool",
"ernie-gen",
"chinese text generation"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "098c9436002fa7bbf0596855782e34efe211940270be00bdeb3421c207fd1f66",
"md5": "d4c870a2ce20813b06ce66121f550fa1",
"sha256": "25af39651a84454a1d53badcd15c6293ddbd72be2e63d23c582bee3d69ca6679"
},
"downloads": -1,
"filename": "textgen-1.1.1.tar.gz",
"has_sig": false,
"md5_digest": "d4c870a2ce20813b06ce66121f550fa1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 185539,
"upload_time": "2023-09-08T11:20:55",
"upload_time_iso_8601": "2023-09-08T11:20:55.375825Z",
"url": "https://files.pythonhosted.org/packages/09/8c/9436002fa7bbf0596855782e34efe211940270be00bdeb3421c207fd1f66/textgen-1.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-09-08 11:20:55",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "shibing624",
"github_project": "textgen",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "textgen"
}
JSON
