  
<h1 align = "center">🔥ChatLLM 基于知识库🔥</h1>
<div align=center>
<img src="data/imgs/LLM.drawio.png"/>
</div>
## Install
```shell
pip install -U chatllm
```
## [Docs](https://yuanjie-ai.github.io/ChatLLM/)
## Usages
### [目前适配的LLMs](LLMS.md)
```python
from chatllm.applications import ChatBase
qa = ChatBase()
qa.load_llm(model_name_or_path="THUDM/chatglm-6b")
for i in qa(query='周杰伦是谁', knowledge_base='周杰伦是傻子'):
print(i, end='')
# 根据已知信息无法回答该问题，因为周杰伦是中国内地流行歌手、演员、音乐制作人、导演，
# 是具有一定的知名度和专业能力的人物，没有提供足够的信息无法判断他是傻子。
```
## OpenaiEcosystem
<details markdown="1">
<summary>Click to 无缝对接openai生态</summary>
```shell
# 服务端
pip install "chatllm[openai]" && chatllm-run openai <本地模型地址>
```
- SDK：`pip install openai`
```python
import openai
openai.api_base = 'http://0.0.0.0:8000/v1'
openai.api_key = 'chatllm'
prompt = "你好"
completion = openai.Completion.create(prompt=prompt, stream=True, model="text-davinci-003")
for c in completion:
print(c.choices[0].text, end='')
# 你好👋!我是人工智能助手 ChatGLM-6B,很高兴见到你，欢迎问我任何问题。
```
- 客户端：[点击下载chatbox](https://chatboxapp.xyz/)，也可接入客户端
### [openai_keys](./data/openai_keys.md): `不定期更新免费keys`
</details>
## ChatMind
<details markdown="1">
<summary>Click to ChatMind</summary>
```shell
pip install "chatllm" && chatllm-run webui --name chatmind
```
</details>
## ChatPDF
<details markdown="1">
<summary>Click to ChatPDF</summary>
```shell
pip install "chatllm[pdf]" && chatllm-run webui --name chatpdf
```
- python交互
```python
from chatllm.applications.chatpdf import ChatPDF
qa = ChatPDF(encode_model='nghuyong/ernie-3.0-nano-zh')
qa.load_llm(model_name_or_path="THUDM/chatglm-6b")
qa.create_index('财报.pdf') # 构建知识库
for i in qa(query='东北证券主营业务'):
print(i, end='')
# 根据已知信息，东北证券的主营业务为证券业务。公司作为证券公司，主要从事证券经纪、证券投资咨询、与证券交易、
# 证券投资活动有关的财务顾问、证券承销与保荐、证券自营、融资融券、证券投资基金代销和代销金融产品待业务。
```
- 支持召回结果查看
</details>
## Deploy
<details markdown="1">
<summary>Click to Deploy</summary>
- ChatGLM-6B 模型硬件需求
| **量化等级** | **最低 GPU 显存**（推理） | **最低 GPU 显存**（高效参数微调） |
| -------------- | ------------------------- | --------------------------------- |
| FP16（无量化） | 13 GB | 14 GB |
| INT8 | 8 GB | 9 GB |
| INT4 | 6 GB | 7 GB |
- 从本地加载模型
- [安装指南](docs/INSTALL.md)
- [ChatGLM-6B Mac 本地部署实操记录](https://www.yuque.com/arvinxx/llm/chatglm-6b-deployment-on-mac)
- [THUDM/ChatGLM-6B#从本地加载模型](https://github.com/THUDM/ChatGLM-6B#从本地加载模型)
</details>
## TODO
<details markdown="1">
<summary>Click to TODO</summary>
- [ ] ChatLLM 应用
- [x] 接入非结构化文档（已支持 md、pdf、docx、txt 文件格式）
- [ ] 搜索引擎与本地网页接入
- [ ] 结构化数据接入（如 csv、Excel、SQL 等）
- [ ] 知识图谱/图数据库接入
- [ ] 增加 ANN 后端，ES/RedisSearch【确保生产高可用】
- [ ] 增加多级缓存缓存
- [ ] 多路召回
- [ ] 问
- [ ] 标量匹配
- [x] 多种向量化，向量匹配
- [ ] 增加相似问，换几个问法
- [ ] 高置信度直接返回答案【匹配标准问】
- [ ] 答
- [ ] 高置信度篇章
- [ ] 增加上下文信息
- [ ] 增加夸篇章信息
- [ ] 增加召回信息的相似信息
- [ ] 提前生成标准问，匹配问
- [ ] 拒绝推断
- [ ] 增加更多 LLM 模型支持
- [x] [THUDM/chatglm-6b](https://huggingface.co/THUDM/chatglm-6b)
- [ ] [THUDM/chatglm-6b-int8](https://huggingface.co/THUDM/chatglm-6b-int8)
- [ ] [THUDM/chatglm-6b-int4](https://huggingface.co/THUDM/chatglm-6b-int4)
- [ ] [THUDM/chatglm-6b-int4-qe](https://huggingface.co/THUDM/chatglm-6b-int4-qe)
- [ ] [ClueAI/ChatYuan-large-v2](https://huggingface.co/ClueAI/ChatYuan-large-v2)
- [ ] 增加更多 Embedding 模型支持
- [x] [nghuyong/ernie-3.0-nano-zh](https://huggingface.co/nghuyong/ernie-3.0-nano-zh)
- [x] [nghuyong/ernie-3.0-base-zh](https://huggingface.co/nghuyong/ernie-3.0-base-zh)
- [x] [shibing624/text2vec-base-chinese](https://huggingface.co/shibing624/text2vec-base-chinese)
- [x] [GanymedeNil/text2vec-large-chinese](https://huggingface.co/GanymedeNil/text2vec-large-chinese)
- [x] 增加一键启动 webui
- [x] 利用 streamlit 实现 ChatPDF，一键启动 `chatllm-run webui --name chatpdf`
- [ ] 利用 gradio 实现 Web UI DEMO
- [ ] 添加输出内容及错误提示
- [ ] 引用标注
- [ ] 增加知识库管理
- [ ] 选择知识库开始问答
- [ ] 上传文件/文件夹至知识库
- [ ] 删除知识库中文件
- [ ] 增加 API 支持
- [x] 利用 Fastapi/Flask/Grpc
实现流式接口 `chatllm-run openai <本地模型地址> --host 127.0.0.1 --port 8000`
- [ ] 前后端分离，实现调用 API 的 Web UI Demo
</details>
## [交流群](http://wechat.yanyue.cloud/common/qun/redirect/?qid=449187)
<div align=center>
<img src="data/imgs/群.png" alt="群" width="200" height="200">
</div>
> 若二维码失效加微信拉群 313303303
Raw data
{
"_id": null,
"home_page": "https://github.com/yuanjie-ai/ChatLLM",
"name": "api2openai",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "chatllm",
"author": "yuanjie",
"author_email": "313303303@qq.com",
"download_url": "https://files.pythonhosted.org/packages/cb/86/e2952492f7779aecd71f76583c1a4fa9795e82d0e2b7a8179f98f8123aa1/api2openai-2023.7.19.18.51.0.tar.gz",
"platform": null,
"description": "  \n\n<h1 align = \"center\">\ud83d\udd25ChatLLM \u57fa\u4e8e\u77e5\u8bc6\u5e93\ud83d\udd25</h1>\n\n<div align=center>\n<img src=\"data/imgs/LLM.drawio.png\"/>\n</div>\n\n## Install\n\n```shell\npip install -U chatllm\n```\n\n## [Docs](https://yuanjie-ai.github.io/ChatLLM/)\n\n## Usages\n\n### [\u76ee\u524d\u9002\u914d\u7684LLMs](LLMS.md)\n\n```python\nfrom chatllm.applications import ChatBase\n\nqa = ChatBase()\nqa.load_llm(model_name_or_path=\"THUDM/chatglm-6b\")\nfor i in qa(query='\u5468\u6770\u4f26\u662f\u8c01', knowledge_base='\u5468\u6770\u4f26\u662f\u50bb\u5b50'):\n print(i, end='')\n# \u6839\u636e\u5df2\u77e5\u4fe1\u606f\u65e0\u6cd5\u56de\u7b54\u8be5\u95ee\u9898\uff0c\u56e0\u4e3a\u5468\u6770\u4f26\u662f\u4e2d\u56fd\u5185\u5730\u6d41\u884c\u6b4c\u624b\u3001\u6f14\u5458\u3001\u97f3\u4e50\u5236\u4f5c\u4eba\u3001\u5bfc\u6f14\uff0c\n# \u662f\u5177\u6709\u4e00\u5b9a\u7684\u77e5\u540d\u5ea6\u548c\u4e13\u4e1a\u80fd\u529b\u7684\u4eba\u7269\uff0c\u6ca1\u6709\u63d0\u4f9b\u8db3\u591f\u7684\u4fe1\u606f\u65e0\u6cd5\u5224\u65ad\u4ed6\u662f\u50bb\u5b50\u3002\n```\n\n## OpenaiEcosystem\n\n<details markdown=\"1\">\n <summary>Click to \u65e0\u7f1d\u5bf9\u63a5openai\u751f\u6001</summary>\n\n```shell\n# \u670d\u52a1\u7aef\npip install \"chatllm[openai]\" && chatllm-run openai <\u672c\u5730\u6a21\u578b\u5730\u5740>\n```\n\n- SDK\uff1a`pip install openai`\n\n```python\nimport openai\n\nopenai.api_base = 'http://0.0.0.0:8000/v1'\nopenai.api_key = 'chatllm'\nprompt = \"\u4f60\u597d\"\ncompletion = openai.Completion.create(prompt=prompt, stream=True, model=\"text-davinci-003\")\nfor c in completion:\n print(c.choices[0].text, end='')\n# \u4f60\u597d\ud83d\udc4b!\u6211\u662f\u4eba\u5de5\u667a\u80fd\u52a9\u624b ChatGLM-6B,\u5f88\u9ad8\u5174\u89c1\u5230\u4f60\uff0c\u6b22\u8fce\u95ee\u6211\u4efb\u4f55\u95ee\u9898\u3002\n```\n\n- \u5ba2\u6237\u7aef\uff1a[\u70b9\u51fb\u4e0b\u8f7dchatbox](https://chatboxapp.xyz/)\uff0c\u4e5f\u53ef\u63a5\u5165\u5ba2\u6237\u7aef\n \n\n### [openai_keys](./data/openai_keys.md): `\u4e0d\u5b9a\u671f\u66f4\u65b0\u514d\u8d39keys`\n\n</details>\n\n## ChatMind\n\n<details markdown=\"1\">\n <summary>Click to ChatMind</summary>\n\n```shell\npip install \"chatllm\" && chatllm-run webui --name chatmind\n```\n\n\n\n</details>\n\n## ChatPDF\n\n<details markdown=\"1\">\n <summary>Click to ChatPDF</summary>\n\n```shell\npip install \"chatllm[pdf]\" && chatllm-run webui --name chatpdf\n```\n\n- python\u4ea4\u4e92\n\n```python\nfrom chatllm.applications.chatpdf import ChatPDF\n\nqa = ChatPDF(encode_model='nghuyong/ernie-3.0-nano-zh')\nqa.load_llm(model_name_or_path=\"THUDM/chatglm-6b\")\nqa.create_index('\u8d22\u62a5.pdf') # \u6784\u5efa\u77e5\u8bc6\u5e93\n\nfor i in qa(query='\u4e1c\u5317\u8bc1\u5238\u4e3b\u8425\u4e1a\u52a1'):\n print(i, end='')\n# \u6839\u636e\u5df2\u77e5\u4fe1\u606f\uff0c\u4e1c\u5317\u8bc1\u5238\u7684\u4e3b\u8425\u4e1a\u52a1\u4e3a\u8bc1\u5238\u4e1a\u52a1\u3002\u516c\u53f8\u4f5c\u4e3a\u8bc1\u5238\u516c\u53f8\uff0c\u4e3b\u8981\u4ece\u4e8b\u8bc1\u5238\u7ecf\u7eaa\u3001\u8bc1\u5238\u6295\u8d44\u54a8\u8be2\u3001\u4e0e\u8bc1\u5238\u4ea4\u6613\u3001\n# \u8bc1\u5238\u6295\u8d44\u6d3b\u52a8\u6709\u5173\u7684\u8d22\u52a1\u987e\u95ee\u3001\u8bc1\u5238\u627f\u9500\u4e0e\u4fdd\u8350\u3001\u8bc1\u5238\u81ea\u8425\u3001\u878d\u8d44\u878d\u5238\u3001\u8bc1\u5238\u6295\u8d44\u57fa\u91d1\u4ee3\u9500\u548c\u4ee3\u9500\u91d1\u878d\u4ea7\u54c1\u5f85\u4e1a\u52a1\u3002\n```\n\n- \u652f\u6301\u53ec\u56de\u7ed3\u679c\u67e5\u770b\n \n\n</details>\n\n## Deploy\n\n<details markdown=\"1\">\n <summary>Click to Deploy</summary>\n\n- ChatGLM-6B \u6a21\u578b\u786c\u4ef6\u9700\u6c42\n\n | **\u91cf\u5316\u7b49\u7ea7** | **\u6700\u4f4e GPU \u663e\u5b58**\uff08\u63a8\u7406\uff09 | **\u6700\u4f4e GPU \u663e\u5b58**\uff08\u9ad8\u6548\u53c2\u6570\u5fae\u8c03\uff09 |\n | -------------- | ------------------------- | --------------------------------- |\n | FP16\uff08\u65e0\u91cf\u5316\uff09 | 13 GB | 14 GB |\n | INT8 | 8 GB | 9 GB |\n | INT4 | 6 GB | 7 GB |\n\n\n- \u4ece\u672c\u5730\u52a0\u8f7d\u6a21\u578b\n - [\u5b89\u88c5\u6307\u5357](docs/INSTALL.md)\n - [ChatGLM-6B Mac \u672c\u5730\u90e8\u7f72\u5b9e\u64cd\u8bb0\u5f55](https://www.yuque.com/arvinxx/llm/chatglm-6b-deployment-on-mac)\n - [THUDM/ChatGLM-6B#\u4ece\u672c\u5730\u52a0\u8f7d\u6a21\u578b](https://github.com/THUDM/ChatGLM-6B#\u4ece\u672c\u5730\u52a0\u8f7d\u6a21\u578b)\n\n</details>\n\n## TODO\n\n<details markdown=\"1\">\n <summary>Click to TODO</summary>\n\n- [ ] ChatLLM \u5e94\u7528\n - [x] \u63a5\u5165\u975e\u7ed3\u6784\u5316\u6587\u6863\uff08\u5df2\u652f\u6301 md\u3001pdf\u3001docx\u3001txt \u6587\u4ef6\u683c\u5f0f\uff09\n - [ ] \u641c\u7d22\u5f15\u64ce\u4e0e\u672c\u5730\u7f51\u9875\u63a5\u5165\n - [ ] \u7ed3\u6784\u5316\u6570\u636e\u63a5\u5165\uff08\u5982 csv\u3001Excel\u3001SQL \u7b49\uff09\n - [ ] \u77e5\u8bc6\u56fe\u8c31/\u56fe\u6570\u636e\u5e93\u63a5\u5165\n - [ ] \u589e\u52a0 ANN \u540e\u7aef\uff0cES/RedisSearch\u3010\u786e\u4fdd\u751f\u4ea7\u9ad8\u53ef\u7528\u3011\n - [ ] \u589e\u52a0\u591a\u7ea7\u7f13\u5b58\u7f13\u5b58\n\n- [ ] \u591a\u8def\u53ec\u56de\n - [ ] \u95ee\n - [ ] \u6807\u91cf\u5339\u914d\n - [x] \u591a\u79cd\u5411\u91cf\u5316\uff0c\u5411\u91cf\u5339\u914d\n - [ ] \u589e\u52a0\u76f8\u4f3c\u95ee\uff0c\u6362\u51e0\u4e2a\u95ee\u6cd5\n - [ ] \u9ad8\u7f6e\u4fe1\u5ea6\u76f4\u63a5\u8fd4\u56de\u7b54\u6848\u3010\u5339\u914d\u6807\u51c6\u95ee\u3011\n - [ ] \u7b54\n - [ ] \u9ad8\u7f6e\u4fe1\u5ea6\u7bc7\u7ae0\n - [ ] \u589e\u52a0\u4e0a\u4e0b\u6587\u4fe1\u606f\n - [ ] \u589e\u52a0\u5938\u7bc7\u7ae0\u4fe1\u606f\n - [ ] \u589e\u52a0\u53ec\u56de\u4fe1\u606f\u7684\u76f8\u4f3c\u4fe1\u606f\n - [ ] \u63d0\u524d\u751f\u6210\u6807\u51c6\u95ee\uff0c\u5339\u914d\u95ee\n - [ ] \u62d2\u7edd\u63a8\u65ad\n\n\n- [ ] \u589e\u52a0\u66f4\u591a LLM \u6a21\u578b\u652f\u6301\n - [x] [THUDM/chatglm-6b](https://huggingface.co/THUDM/chatglm-6b)\n - [ ] [THUDM/chatglm-6b-int8](https://huggingface.co/THUDM/chatglm-6b-int8)\n - [ ] [THUDM/chatglm-6b-int4](https://huggingface.co/THUDM/chatglm-6b-int4)\n - [ ] [THUDM/chatglm-6b-int4-qe](https://huggingface.co/THUDM/chatglm-6b-int4-qe)\n - [ ] [ClueAI/ChatYuan-large-v2](https://huggingface.co/ClueAI/ChatYuan-large-v2)\n- [ ] \u589e\u52a0\u66f4\u591a Embedding \u6a21\u578b\u652f\u6301\n - [x] [nghuyong/ernie-3.0-nano-zh](https://huggingface.co/nghuyong/ernie-3.0-nano-zh)\n - [x] [nghuyong/ernie-3.0-base-zh](https://huggingface.co/nghuyong/ernie-3.0-base-zh)\n - [x] [shibing624/text2vec-base-chinese](https://huggingface.co/shibing624/text2vec-base-chinese)\n - [x] [GanymedeNil/text2vec-large-chinese](https://huggingface.co/GanymedeNil/text2vec-large-chinese)\n- [x] \u589e\u52a0\u4e00\u952e\u542f\u52a8 webui\n - [x] \u5229\u7528 streamlit \u5b9e\u73b0 ChatPDF\uff0c\u4e00\u952e\u542f\u52a8 `chatllm-run webui --name chatpdf`\n - [ ] \u5229\u7528 gradio \u5b9e\u73b0 Web UI DEMO\n - [ ] \u6dfb\u52a0\u8f93\u51fa\u5185\u5bb9\u53ca\u9519\u8bef\u63d0\u793a\n - [ ] \u5f15\u7528\u6807\u6ce8\n - [ ] \u589e\u52a0\u77e5\u8bc6\u5e93\u7ba1\u7406\n - [ ] \u9009\u62e9\u77e5\u8bc6\u5e93\u5f00\u59cb\u95ee\u7b54\n - [ ] \u4e0a\u4f20\u6587\u4ef6/\u6587\u4ef6\u5939\u81f3\u77e5\u8bc6\u5e93\n - [ ] \u5220\u9664\u77e5\u8bc6\u5e93\u4e2d\u6587\u4ef6\n- [ ] \u589e\u52a0 API \u652f\u6301\n - [x] \u5229\u7528 Fastapi/Flask/Grpc\n \u5b9e\u73b0\u6d41\u5f0f\u63a5\u53e3 `chatllm-run openai <\u672c\u5730\u6a21\u578b\u5730\u5740> --host 127.0.0.1 --port 8000`\n - [ ] \u524d\u540e\u7aef\u5206\u79bb\uff0c\u5b9e\u73b0\u8c03\u7528 API \u7684 Web UI Demo\n\n</details>\n\n## [\u4ea4\u6d41\u7fa4](http://wechat.yanyue.cloud/common/qun/redirect/?qid=449187)\n\n<div align=center>\n<img src=\"data/imgs/\u7fa4.png\" alt=\"\u7fa4\" width=\"200\" height=\"200\">\n</div>\n\n> \u82e5\u4e8c\u7ef4\u7801\u5931\u6548\u52a0\u5fae\u4fe1\u62c9\u7fa4 313303303\n\n\n\n",
"bugtrack_url": null,
"license": "MIT license",
"summary": "Create a Python package.",
"version": "2023.7.19.18.51.0",
"project_urls": {
"Homepage": "https://github.com/yuanjie-ai/ChatLLM"
},
"split_keywords": [
"chatllm"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "05bcb5a600ab950a75e504f02d67e57b1a19f7d9eaa66083e08b79419d38244a",
"md5": "b6dc94a8d569e82371d9937378cdbf4a",
"sha256": "13cb3643912eeeee1b22de596daa728395a47067e35d6ad6263e8e0129da15c7"
},
"downloads": -1,
"filename": "api2openai-2023.7.19.18.51.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b6dc94a8d569e82371d9937378cdbf4a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 89360,
"upload_time": "2023-07-19T10:51:04",
"upload_time_iso_8601": "2023-07-19T10:51:04.962271Z",
"url": "https://files.pythonhosted.org/packages/05/bc/b5a600ab950a75e504f02d67e57b1a19f7d9eaa66083e08b79419d38244a/api2openai-2023.7.19.18.51.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "cb86e2952492f7779aecd71f76583c1a4fa9795e82d0e2b7a8179f98f8123aa1",
"md5": "351bc0011de3455eb690784935daf868",
"sha256": "1d7c7519650d48639c31f0e2ba15a74ac097b463039c738175ad206010cf6a8a"
},
"downloads": -1,
"filename": "api2openai-2023.7.19.18.51.0.tar.gz",
"has_sig": false,
"md5_digest": "351bc0011de3455eb690784935daf868",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 19796920,
"upload_time": "2023-07-19T10:51:35",
"upload_time_iso_8601": "2023-07-19T10:51:35.852561Z",
"url": "https://files.pythonhosted.org/packages/cb/86/e2952492f7779aecd71f76583c1a4fa9795e82d0e2b7a8179f98f8123aa1/api2openai-2023.7.19.18.51.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-19 10:51:35",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "yuanjie-ai",
"github_project": "ChatLLM",
"travis_ci": true,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "meutils",
"specs": []
},
{
"name": "transformers",
"specs": []
},
{
"name": "tokenizers",
"specs": []
},
{
"name": "sentence_transformers",
"specs": []
},
{
"name": "accelerate",
"specs": []
},
{
"name": "cpm_kernels",
"specs": []
}
],
"tox": true,
"lcname": "api2openai"
}
JSON
