**简体中文** | [**English**](https://github.com/KenyonY/openai-forward/blob/main/README_EN.md)
<h1 align="center">
<a href="https://github.com/KenyonY/openai-forward"> OpenAI Forward </a>
<br>
<br>
</h1>
<p align="center">
<a href="https://pypi.org/project/openai-forward/">
<img src="https://img.shields.io/pypi/v/openai-forward?color=brightgreen&style=flat-square" alt="PyPI version" >
</a>
<a href="https://github.com/KenyonY/openai-forward/blob/main/LICENSE">
<img alt="License" src="https://img.shields.io/github/license/KenyonY/openai-forward.svg?color=blue&style=flat-square">
</a>
<a href="https://hub.docker.com/r/beidongjiedeguang/openai-forward">
<img alt="docker pull" src="https://img.shields.io/docker/pulls/beidongjiedeguang/openai-forward?style=flat-square&label=docker pulls">
</a>
<a href="https://github.com/KenyonY/openai-forward/actions/workflows/ci.yml">
<img alt="tests" src="https://img.shields.io/github/actions/workflow/status/KenyonY/openai-forward/ci.yml?style=flat-square&label=tests">
</a>
<a href="https://pypistats.org/packages/openai-forward">
<img alt="pypi downloads" src="https://img.shields.io/pypi/dm/openai_forward?style=flat-square">
</a>
</p>
<div align="center">
[特点](#主要特性) |
[部署指南](deploy.md) |
[使用指南](#使用指南) |
[配置](#配置) |
[对话日志](#对话日志)
</div>
> [!IMPORTANT]
>
> 在v0.7.0以后在配置方面会有较大调整，并与之前版本不兼容。通过UI配置起来会更加方便，且提供了更强大的配置选项。
**OpenAI-Forward** 是为大型语言模型实现的高效转发服务。其核心功能包括
用户请求速率控制、Token速率限制、智能预测缓存、日志管理和API密钥管理等，旨在提供高效、便捷的模型转发服务。
无论是代理本地语言模型还是云端语言模型，如 [LocalAI](https://github.com/go-skynet/LocalAI)
或 [OpenAI](https://api.openai.com)，都可以由 OpenAI Forward 轻松实现。
得益于 [uvicorn](https://github.com/encode/uvicorn), [aiohttp](https://github.com/aio-libs/aiohttp),
和 [asyncio](https://docs.python.org/3/library/asyncio.html)
等库支持，OpenAI-Forward 实现了出色的异步性能。
### News
- 🎉🎉🎉 v0.7.0版本后支持通过WebUI进行配置管理
- gpt-1106版本已适配
- 缓存后端切换为高性能数据库后端：[🗲 FlaxKV](https://github.com/KenyonY/flaxkv)
<a>
<img src="https://raw.githubusercontent.com/KenyonY/openai-forward/main/.github/images/separators/aqua.png" height=8px width="100%">
</a>
## 主要特性
- **全能转发**：可转发几乎所有类型的请求
- **性能优先**：出色的异步性能
- **缓存AI预测**：对AI预测进行缓存，加速服务访问并节省费用
- **用户流量控制**：自定义请求速率与Token速率
- **实时响应日志**：提升LLMs可观察性
- **自定义秘钥**：替代原始API密钥
- **多目标路由**：转发多个服务地址至同一服务下的不同路由
- **黑白名单**：可对指定IP进行黑白名单限制
- **自动重试**：确保服务的稳定性，请求失败时将自动重试
- **快速部署**：支持通过pip和docker在本地或云端进行快速部署
**由本项目搭建的代理服务地址:**
- 原始OpenAI 服务地址
> https://api.openai-forward.com
> https://render.openai-forward.com
- 开启缓存的服务地址（用户请求结果将被保存一段时间）
> https://smart.openai-forward.com
<sub>
注：此处部署的代理服务仅供个人学习和研究目的使用，勿用于任何商业用途。
</sub>
## 部署指南
👉 [部署文档](deploy.md)
<a>
<img src="https://raw.githubusercontent.com/KenyonY/openai-forward/main/.github/images/separators/aqua.png" height=8px width="100%">
</a>
## 使用指南
### 快速入门
**安装**
```bash
pip install openai-forward
# 或安装webui版本：
pip install openai-forward[webui]
```
**启动服务**
```bash
aifd run
# 或启动带webui的服务
aifd run --webui
```
如果读入了根路径的`.env`的配置, 将会看到以下启动信息
```bash
❯ aifd run
╭────── 🤗 openai-forward is ready to serve! ───────╮
│ │
│ base url https://api.openai.com │
│ route prefix / │
│ api keys False │
│ forward keys False │
│ cache_backend MEMORY │
╰────────────────────────────────────────────────────╯
╭──────────── ⏱️ Rate Limit configuration ───────────╮
│ │
│ backend memory │
│ strategy moving-window │
│ global rate limit 100/minute (req) │
│ /v1/chat/completions 100/2minutes (req) │
│ /v1/completions 60/minute;600/hour (req) │
│ /v1/chat/completions 60/second (token) │
│ /v1/completions 60/second (token) │
╰────────────────────────────────────────────────────╯
INFO: Started server process [191471]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
```
### 代理OpenAI模型:
`aifd run`的默认选项便是代理`https://api.openai.com`
下面以搭建好的服务地址`https://api.openai-forward.com` 为例
**Python**
```diff
from openai import OpenAI # pip install openai>=1.0.0
client = OpenAI(
+ base_url="https://api.openai-forward.com/v1",
api_key="sk-******"
)
```
### 代理本地模型
- **适用场景：** 与 [LocalAI](https://github.com/go-skynet/LocalAI)，
[api-for-open-llm](https://github.com/xusenlinzy/api-for-open-llm)等项目一起使用
- **如何操作：**
以LocalAI为例，如果已在 http://localhost:8080 部署了LocalAI服务，仅需在环境变量或 .env
文件中设置 `FORWARD_CONFIG=[{"base_url":"http://localhost:8080","route":"/localai","type":"openai"}]`。
然后即可通过访问 http://localhost:8000/localai 使用LocalAI。
(更多)
### 代理任意云端模型
#### 代理[gemini pro](https://ai.google.dev/)
配置环境变量或 .env 文件如下：
```env
FORWARD_CONFIG=[{"base_url":"https://generativelanguage.googleapis.com","route":"/gemini","type":"general"}]
```
说明：`aidf run`启动后，即可通过访问 http://localhost:8000/gemini 使用gemini pro。
- **场景1:**
使用通用转发,可对任意来源服务进行转发，
可获得请求速率控制与token速率控制；但通用转发不支持自定义秘钥.
- **场景2：**
可通过 [LiteLLM](https://github.com/BerriAI/litellm) 可以将 众多云模型的 API 格式转换为 openai
的api格式，然后使用openai风格转发
(更多)
<a>
<img src="https://raw.githubusercontent.com/KenyonY/openai-forward/main/.github/images/separators/aqua.png" height=8px width="100%">
</a>
## 配置
执行 `aifd run --webui` 进入配置页面 (默认服务地址 http://localhost:8001)
你可以在项目的运行目录下创建 .env 文件来定制各项配置。参考配置可见根目录下的
[.env.example](.env.example)文件
### 智能缓存
开启缓存后，将会对指定路由的内容进行缓存，其中转发类型分别为`openai`与`general`两者行为略有不同，
使用`general`转发时，默认会将相同的请求一律使用缓存返回，
使用`openai`转发时，在开启缓存后，可以通过OpenAI 的`extra_body`参数来控制缓存的行为，如
**Python**
```diff
from openai import OpenAI
client = OpenAI(
+ base_url="https://smart.openai-forward.com/v1",
api_key="sk-******"
)
completion = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[
{"role": "user", "content": "Hello!"}
],
+ extra_body={"caching": True}
)
```
**Curl**
```bash
curl https://smart.openai.com/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer sk-******" \
-d '{
"model": "gpt-3.5-turbo",
"messages": [{"role": "user", "content": "Hello!"}],
"caching": true
}'
```
### 自定义秘钥
<details open>
<summary>Click for more details</summary>
见.env文件
**用例:**
```diff
import openai
+ openai.api_base = "https://api.openai-forward.com/v1"
- openai.api_key = "sk-******"
+ openai.api_key = "fk-******"
```
</details>
### 多目标服务转发
支持转发不同地址的服务至同一端口的不同路由下
用例见 `.env.example`
### 对话日志
<details open>
<summary>Click for more details</summary>
保存路径在当前目录下的`Log/openai/chat/chat.log`路径中。
记录格式为
```text
{'messages': [{'role': 'user', 'content': 'hi'}], 'model': 'gpt-3.5-turbo', 'stream': True, 'max_tokens': None, 'n': 1, 'temperature': 1, 'top_p': 1, 'logit_bias': None, 'frequency_penalty': 0, 'presence_penalty': 0, 'stop': None, 'user': None, 'ip': '127.0.0.1', 'uid': '2155fe1580e6aed626aa1ad74c1ce54e', 'datetime': '2023-10-17 15:27:12'}
{'assistant': 'Hello! How can I assist you today?', 'is_tool_calls': False, 'uid': '2155fe1580e6aed626aa1ad74c1ce54e'}
```
转换为`json`格式：
```bash
aifd convert
```
得到`chat_openai.json`：
```json
[
{
"datetime": "2023-10-17 15:27:12",
"ip": "127.0.0.1",
"model": "gpt-3.5-turbo",
"temperature": 1,
"messages": [
{
"user": "hi"
}
],
"tools": null,
"is_tool_calls": false,
"assistant": "Hello! How can I assist you today?"
}
]
```
</details>
## 贡献
欢迎通过提交拉取请求或在仓库中提出问题来为此项目做出贡献。
## 许可证
OpenAI-Forward 采用 [MIT](https://opensource.org/license/mit/) 许可证。
Raw data
{
"_id": null,
"home_page": null,
"name": "openai-forward",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "OpenAI API Forwarding, chatgpt, fastapi, openai, openai-api, openai-proxy, python, streaming-api",
"author": null,
"author_email": "kunyuan <beidongjiedeguang@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/11/10/0cf3585e277adac7f3be920a291b018e5bba24288becada6d2a86e649914/openai_forward-0.8.0.tar.gz",
"platform": null,
"description": "**\u7b80\u4f53\u4e2d\u6587** | [**English**](https://github.com/KenyonY/openai-forward/blob/main/README_EN.md)\n\n<h1 align=\"center\">\n <a href=\"https://github.com/KenyonY/openai-forward\"> OpenAI Forward </a>\n <br>\n <br>\n</h1>\n\n<p align=\"center\">\n <a href=\"https://pypi.org/project/openai-forward/\">\n <img src=\"https://img.shields.io/pypi/v/openai-forward?color=brightgreen&style=flat-square\" alt=\"PyPI version\" >\n </a>\n <a href=\"https://github.com/KenyonY/openai-forward/blob/main/LICENSE\">\n <img alt=\"License\" src=\"https://img.shields.io/github/license/KenyonY/openai-forward.svg?color=blue&style=flat-square\">\n </a>\n <a href=\"https://hub.docker.com/r/beidongjiedeguang/openai-forward\">\n <img alt=\"docker pull\" src=\"https://img.shields.io/docker/pulls/beidongjiedeguang/openai-forward?style=flat-square&label=docker pulls\">\n </a>\n <a href=\"https://github.com/KenyonY/openai-forward/actions/workflows/ci.yml\">\n <img alt=\"tests\" src=\"https://img.shields.io/github/actions/workflow/status/KenyonY/openai-forward/ci.yml?style=flat-square&label=tests\">\n </a>\n <a href=\"https://pypistats.org/packages/openai-forward\">\n <img alt=\"pypi downloads\" src=\"https://img.shields.io/pypi/dm/openai_forward?style=flat-square\">\n </a>\n</p>\n\n<div align=\"center\">\n\n[\u7279\u70b9](#\u4e3b\u8981\u7279\u6027) |\n[\u90e8\u7f72\u6307\u5357](deploy.md) |\n[\u4f7f\u7528\u6307\u5357](#\u4f7f\u7528\u6307\u5357) |\n[\u914d\u7f6e](#\u914d\u7f6e) |\n[\u5bf9\u8bdd\u65e5\u5fd7](#\u5bf9\u8bdd\u65e5\u5fd7)\n\n\n</div>\n\n\n> [!IMPORTANT]\n>\n> \u5728v0.7.0\u4ee5\u540e\u5728\u914d\u7f6e\u65b9\u9762\u4f1a\u6709\u8f83\u5927\u8c03\u6574\uff0c\u5e76\u4e0e\u4e4b\u524d\u7248\u672c\u4e0d\u517c\u5bb9\u3002\u901a\u8fc7UI\u914d\u7f6e\u8d77\u6765\u4f1a\u66f4\u52a0\u65b9\u4fbf\uff0c\u4e14\u63d0\u4f9b\u4e86\u66f4\u5f3a\u5927\u7684\u914d\u7f6e\u9009\u9879\u3002\n\n\n**OpenAI-Forward** \u662f\u4e3a\u5927\u578b\u8bed\u8a00\u6a21\u578b\u5b9e\u73b0\u7684\u9ad8\u6548\u8f6c\u53d1\u670d\u52a1\u3002\u5176\u6838\u5fc3\u529f\u80fd\u5305\u62ec\n\u7528\u6237\u8bf7\u6c42\u901f\u7387\u63a7\u5236\u3001Token\u901f\u7387\u9650\u5236\u3001\u667a\u80fd\u9884\u6d4b\u7f13\u5b58\u3001\u65e5\u5fd7\u7ba1\u7406\u548cAPI\u5bc6\u94a5\u7ba1\u7406\u7b49\uff0c\u65e8\u5728\u63d0\u4f9b\u9ad8\u6548\u3001\u4fbf\u6377\u7684\u6a21\u578b\u8f6c\u53d1\u670d\u52a1\u3002\n\u65e0\u8bba\u662f\u4ee3\u7406\u672c\u5730\u8bed\u8a00\u6a21\u578b\u8fd8\u662f\u4e91\u7aef\u8bed\u8a00\u6a21\u578b\uff0c\u5982 [LocalAI](https://github.com/go-skynet/LocalAI)\n\u6216 [OpenAI](https://api.openai.com)\uff0c\u90fd\u53ef\u4ee5\u7531 OpenAI Forward \u8f7b\u677e\u5b9e\u73b0\u3002\n\u5f97\u76ca\u4e8e [uvicorn](https://github.com/encode/uvicorn), [aiohttp](https://github.com/aio-libs/aiohttp),\n\u548c [asyncio](https://docs.python.org/3/library/asyncio.html)\n\u7b49\u5e93\u652f\u6301\uff0cOpenAI-Forward \u5b9e\u73b0\u4e86\u51fa\u8272\u7684\u5f02\u6b65\u6027\u80fd\u3002\n\n\n### News\n\n- \ud83c\udf89\ud83c\udf89\ud83c\udf89 v0.7.0\u7248\u672c\u540e\u652f\u6301\u901a\u8fc7WebUI\u8fdb\u884c\u914d\u7f6e\u7ba1\u7406\n- gpt-1106\u7248\u672c\u5df2\u9002\u914d\n- \u7f13\u5b58\u540e\u7aef\u5207\u6362\u4e3a\u9ad8\u6027\u80fd\u6570\u636e\u5e93\u540e\u7aef\uff1a[\ud83d\uddf2 FlaxKV](https://github.com/KenyonY/flaxkv)\n\n<a>\n <img src=\"https://raw.githubusercontent.com/KenyonY/openai-forward/main/.github/images/separators/aqua.png\" height=8px width=\"100%\">\n</a>\n\n## \u4e3b\u8981\u7279\u6027\n\n\n- **\u5168\u80fd\u8f6c\u53d1**\uff1a\u53ef\u8f6c\u53d1\u51e0\u4e4e\u6240\u6709\u7c7b\u578b\u7684\u8bf7\u6c42\n- **\u6027\u80fd\u4f18\u5148**\uff1a\u51fa\u8272\u7684\u5f02\u6b65\u6027\u80fd\n- **\u7f13\u5b58AI\u9884\u6d4b**\uff1a\u5bf9AI\u9884\u6d4b\u8fdb\u884c\u7f13\u5b58\uff0c\u52a0\u901f\u670d\u52a1\u8bbf\u95ee\u5e76\u8282\u7701\u8d39\u7528\n- **\u7528\u6237\u6d41\u91cf\u63a7\u5236**\uff1a\u81ea\u5b9a\u4e49\u8bf7\u6c42\u901f\u7387\u4e0eToken\u901f\u7387\n- **\u5b9e\u65f6\u54cd\u5e94\u65e5\u5fd7**\uff1a\u63d0\u5347LLMs\u53ef\u89c2\u5bdf\u6027\n- **\u81ea\u5b9a\u4e49\u79d8\u94a5**\uff1a\u66ff\u4ee3\u539f\u59cbAPI\u5bc6\u94a5\n- **\u591a\u76ee\u6807\u8def\u7531**\uff1a\u8f6c\u53d1\u591a\u4e2a\u670d\u52a1\u5730\u5740\u81f3\u540c\u4e00\u670d\u52a1\u4e0b\u7684\u4e0d\u540c\u8def\u7531\n- **\u9ed1\u767d\u540d\u5355**\uff1a\u53ef\u5bf9\u6307\u5b9aIP\u8fdb\u884c\u9ed1\u767d\u540d\u5355\u9650\u5236\n- **\u81ea\u52a8\u91cd\u8bd5**\uff1a\u786e\u4fdd\u670d\u52a1\u7684\u7a33\u5b9a\u6027\uff0c\u8bf7\u6c42\u5931\u8d25\u65f6\u5c06\u81ea\u52a8\u91cd\u8bd5\n- **\u5feb\u901f\u90e8\u7f72**\uff1a\u652f\u6301\u901a\u8fc7pip\u548cdocker\u5728\u672c\u5730\u6216\u4e91\u7aef\u8fdb\u884c\u5feb\u901f\u90e8\u7f72\n\n**\u7531\u672c\u9879\u76ee\u642d\u5efa\u7684\u4ee3\u7406\u670d\u52a1\u5730\u5740:**\n\n- \u539f\u59cbOpenAI \u670d\u52a1\u5730\u5740\n > https://api.openai-forward.com \n > https://render.openai-forward.com\n\n- \u5f00\u542f\u7f13\u5b58\u7684\u670d\u52a1\u5730\u5740\uff08\u7528\u6237\u8bf7\u6c42\u7ed3\u679c\u5c06\u88ab\u4fdd\u5b58\u4e00\u6bb5\u65f6\u95f4\uff09\n > https://smart.openai-forward.com\n\n<sub>\n\u6ce8\uff1a\u6b64\u5904\u90e8\u7f72\u7684\u4ee3\u7406\u670d\u52a1\u4ec5\u4f9b\u4e2a\u4eba\u5b66\u4e60\u548c\u7814\u7a76\u76ee\u7684\u4f7f\u7528\uff0c\u52ff\u7528\u4e8e\u4efb\u4f55\u5546\u4e1a\u7528\u9014\u3002\n</sub>\n\n## \u90e8\u7f72\u6307\u5357\n\n\ud83d\udc49 [\u90e8\u7f72\u6587\u6863](deploy.md)\n\n\n<a>\n <img src=\"https://raw.githubusercontent.com/KenyonY/openai-forward/main/.github/images/separators/aqua.png\" height=8px width=\"100%\">\n</a>\n\n## \u4f7f\u7528\u6307\u5357\n\n### \u5feb\u901f\u5165\u95e8\n\n**\u5b89\u88c5**\n\n```bash\npip install openai-forward \n\n# \u6216\u5b89\u88c5webui\u7248\u672c\uff1a\npip install openai-forward[webui]\n```\n\n**\u542f\u52a8\u670d\u52a1**\n\n```bash\naifd run\n# \u6216\u542f\u52a8\u5e26webui\u7684\u670d\u52a1\naifd run --webui\n```\n\n\u5982\u679c\u8bfb\u5165\u4e86\u6839\u8def\u5f84\u7684`.env`\u7684\u914d\u7f6e, \u5c06\u4f1a\u770b\u5230\u4ee5\u4e0b\u542f\u52a8\u4fe1\u606f\n\n```bash\n\u276f aifd run\n\u256d\u2500\u2500\u2500\u2500\u2500\u2500 \ud83e\udd17 openai-forward is ready to serve! \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256e\n\u2502 \u2502\n\u2502 base url https://api.openai.com \u2502\n\u2502 route prefix / \u2502\n\u2502 api keys False \u2502\n\u2502 forward keys False \u2502\n\u2502 cache_backend MEMORY \u2502\n\u2570\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256f\n\u256d\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500 \u23f1\ufe0f Rate Limit configuration \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256e\n\u2502 \u2502\n\u2502 backend memory \u2502\n\u2502 strategy moving-window \u2502\n\u2502 global rate limit 100/minute (req) \u2502\n\u2502 /v1/chat/completions 100/2minutes (req) \u2502\n\u2502 /v1/completions 60/minute;600/hour (req) \u2502\n\u2502 /v1/chat/completions 60/second (token) \u2502\n\u2502 /v1/completions 60/second (token) \u2502\n\u2570\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256f\nINFO: Started server process [191471]\nINFO: Waiting for application startup.\nINFO: Application startup complete.\nINFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)\n```\n\n### \u4ee3\u7406OpenAI\u6a21\u578b:\n\n`aifd run`\u7684\u9ed8\u8ba4\u9009\u9879\u4fbf\u662f\u4ee3\u7406`https://api.openai.com`\n\n\u4e0b\u9762\u4ee5\u642d\u5efa\u597d\u7684\u670d\u52a1\u5730\u5740`https://api.openai-forward.com` \u4e3a\u4f8b\n\n**Python**\n\n```diff\n from openai import OpenAI # pip install openai>=1.0.0\n client = OpenAI(\n+ base_url=\"https://api.openai-forward.com/v1\", \n api_key=\"sk-******\"\n )\n```\n\n\n### \u4ee3\u7406\u672c\u5730\u6a21\u578b\n\n- **\u9002\u7528\u573a\u666f\uff1a** \u4e0e [LocalAI](https://github.com/go-skynet/LocalAI)\uff0c\n [api-for-open-llm](https://github.com/xusenlinzy/api-for-open-llm)\u7b49\u9879\u76ee\u4e00\u8d77\u4f7f\u7528\n\n- **\u5982\u4f55\u64cd\u4f5c\uff1a**\n \u4ee5LocalAI\u4e3a\u4f8b\uff0c\u5982\u679c\u5df2\u5728 http://localhost:8080 \u90e8\u7f72\u4e86LocalAI\u670d\u52a1\uff0c\u4ec5\u9700\u5728\u73af\u5883\u53d8\u91cf\u6216 .env\n \u6587\u4ef6\u4e2d\u8bbe\u7f6e `FORWARD_CONFIG=[{\"base_url\":\"http://localhost:8080\",\"route\":\"/localai\",\"type\":\"openai\"}]`\u3002\n \u7136\u540e\u5373\u53ef\u901a\u8fc7\u8bbf\u95ee http://localhost:8000/localai \u4f7f\u7528LocalAI\u3002\n\n(\u66f4\u591a)\n\n### \u4ee3\u7406\u4efb\u610f\u4e91\u7aef\u6a21\u578b\n\n#### \u4ee3\u7406[gemini pro](https://ai.google.dev/)\n\u914d\u7f6e\u73af\u5883\u53d8\u91cf\u6216 .env \u6587\u4ef6\u5982\u4e0b\uff1a\n```env\nFORWARD_CONFIG=[{\"base_url\":\"https://generativelanguage.googleapis.com\",\"route\":\"/gemini\",\"type\":\"general\"}]\n```\n\u8bf4\u660e\uff1a`aidf run`\u542f\u52a8\u540e\uff0c\u5373\u53ef\u901a\u8fc7\u8bbf\u95ee http://localhost:8000/gemini \u4f7f\u7528gemini pro\u3002\n\n\n- **\u573a\u666f1:**\n \u4f7f\u7528\u901a\u7528\u8f6c\u53d1,\u53ef\u5bf9\u4efb\u610f\u6765\u6e90\u670d\u52a1\u8fdb\u884c\u8f6c\u53d1\uff0c\n \u53ef\u83b7\u5f97\u8bf7\u6c42\u901f\u7387\u63a7\u5236\u4e0etoken\u901f\u7387\u63a7\u5236\uff1b\u4f46\u901a\u7528\u8f6c\u53d1\u4e0d\u652f\u6301\u81ea\u5b9a\u4e49\u79d8\u94a5.\n\n- **\u573a\u666f2\uff1a**\n \u53ef\u901a\u8fc7 [LiteLLM](https://github.com/BerriAI/litellm) \u53ef\u4ee5\u5c06 \u4f17\u591a\u4e91\u6a21\u578b\u7684 API \u683c\u5f0f\u8f6c\u6362\u4e3a openai\n \u7684api\u683c\u5f0f\uff0c\u7136\u540e\u4f7f\u7528openai\u98ce\u683c\u8f6c\u53d1\n\n(\u66f4\u591a)\n\n\n<a>\n <img src=\"https://raw.githubusercontent.com/KenyonY/openai-forward/main/.github/images/separators/aqua.png\" height=8px width=\"100%\">\n</a>\n\n## \u914d\u7f6e\n\n\n\u6267\u884c `aifd run --webui` \u8fdb\u5165\u914d\u7f6e\u9875\u9762 (\u9ed8\u8ba4\u670d\u52a1\u5730\u5740 http://localhost:8001)\n\n\n\u4f60\u53ef\u4ee5\u5728\u9879\u76ee\u7684\u8fd0\u884c\u76ee\u5f55\u4e0b\u521b\u5efa .env \u6587\u4ef6\u6765\u5b9a\u5236\u5404\u9879\u914d\u7f6e\u3002\u53c2\u8003\u914d\u7f6e\u53ef\u89c1\u6839\u76ee\u5f55\u4e0b\u7684\n[.env.example](.env.example)\u6587\u4ef6\n\n\n### \u667a\u80fd\u7f13\u5b58\n\n\u5f00\u542f\u7f13\u5b58\u540e\uff0c\u5c06\u4f1a\u5bf9\u6307\u5b9a\u8def\u7531\u7684\u5185\u5bb9\u8fdb\u884c\u7f13\u5b58\uff0c\u5176\u4e2d\u8f6c\u53d1\u7c7b\u578b\u5206\u522b\u4e3a`openai`\u4e0e`general`\u4e24\u8005\u884c\u4e3a\u7565\u6709\u4e0d\u540c\uff0c\n\u4f7f\u7528`general`\u8f6c\u53d1\u65f6\uff0c\u9ed8\u8ba4\u4f1a\u5c06\u76f8\u540c\u7684\u8bf7\u6c42\u4e00\u5f8b\u4f7f\u7528\u7f13\u5b58\u8fd4\u56de\uff0c \n\u4f7f\u7528`openai`\u8f6c\u53d1\u65f6\uff0c\u5728\u5f00\u542f\u7f13\u5b58\u540e\uff0c\u53ef\u4ee5\u901a\u8fc7OpenAI \u7684`extra_body`\u53c2\u6570\u6765\u63a7\u5236\u7f13\u5b58\u7684\u884c\u4e3a\uff0c\u5982\n\n**Python**\n\n```diff\n from openai import OpenAI \n client = OpenAI(\n+ base_url=\"https://smart.openai-forward.com/v1\", \n api_key=\"sk-******\"\n )\n completion = client.chat.completions.create(\n model=\"gpt-3.5-turbo\",\n messages=[\n {\"role\": \"user\", \"content\": \"Hello!\"}\n ],\n+ extra_body={\"caching\": True}\n)\n```\n\n**Curl**\n\n```bash\ncurl https://smart.openai.com/v1/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -H \"Authorization: Bearer sk-******\" \\\n -d '{\n \"model\": \"gpt-3.5-turbo\",\n \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}],\n \"caching\": true\n }'\n\n```\n\n### \u81ea\u5b9a\u4e49\u79d8\u94a5\n\n<details open>\n <summary>Click for more details</summary>\n\n\u89c1.env\u6587\u4ef6\n\n**\u7528\u4f8b:**\n\n```diff\n import openai\n+ openai.api_base = \"https://api.openai-forward.com/v1\"\n- openai.api_key = \"sk-******\"\n+ openai.api_key = \"fk-******\"\n```\n\n</details>\n\n### \u591a\u76ee\u6807\u670d\u52a1\u8f6c\u53d1\n\n\u652f\u6301\u8f6c\u53d1\u4e0d\u540c\u5730\u5740\u7684\u670d\u52a1\u81f3\u540c\u4e00\u7aef\u53e3\u7684\u4e0d\u540c\u8def\u7531\u4e0b\n\u7528\u4f8b\u89c1 `.env.example`\n\n### \u5bf9\u8bdd\u65e5\u5fd7\n\n<details open>\n <summary>Click for more details</summary>\n\n\u4fdd\u5b58\u8def\u5f84\u5728\u5f53\u524d\u76ee\u5f55\u4e0b\u7684`Log/openai/chat/chat.log`\u8def\u5f84\u4e2d\u3002 \n\u8bb0\u5f55\u683c\u5f0f\u4e3a\n\n```text\n{'messages': [{'role': 'user', 'content': 'hi'}], 'model': 'gpt-3.5-turbo', 'stream': True, 'max_tokens': None, 'n': 1, 'temperature': 1, 'top_p': 1, 'logit_bias': None, 'frequency_penalty': 0, 'presence_penalty': 0, 'stop': None, 'user': None, 'ip': '127.0.0.1', 'uid': '2155fe1580e6aed626aa1ad74c1ce54e', 'datetime': '2023-10-17 15:27:12'}\n{'assistant': 'Hello! How can I assist you today?', 'is_tool_calls': False, 'uid': '2155fe1580e6aed626aa1ad74c1ce54e'}\n```\n\n\u8f6c\u6362\u4e3a`json`\u683c\u5f0f\uff1a\n\n```bash\naifd convert\n```\n\n\u5f97\u5230`chat_openai.json`\uff1a\n\n```json\n[\n {\n \"datetime\": \"2023-10-17 15:27:12\",\n \"ip\": \"127.0.0.1\",\n \"model\": \"gpt-3.5-turbo\",\n \"temperature\": 1,\n \"messages\": [\n {\n \"user\": \"hi\"\n }\n ],\n \"tools\": null,\n \"is_tool_calls\": false,\n \"assistant\": \"Hello! How can I assist you today?\"\n }\n]\n```\n\n</details>\n\n## \u8d21\u732e\n\n\u6b22\u8fce\u901a\u8fc7\u63d0\u4ea4\u62c9\u53d6\u8bf7\u6c42\u6216\u5728\u4ed3\u5e93\u4e2d\u63d0\u51fa\u95ee\u9898\u6765\u4e3a\u6b64\u9879\u76ee\u505a\u51fa\u8d21\u732e\u3002\n\n\n## \u8bb8\u53ef\u8bc1\n\nOpenAI-Forward \u91c7\u7528 [MIT](https://opensource.org/license/mit/) \u8bb8\u53ef\u8bc1\u3002\n",
"bugtrack_url": null,
"license": null,
"summary": "\ud83d\ude80 OpenAI-Forward \u662f\u4e00\u4e2a\u4e13\u4e3a\u5927\u578b\u8bed\u8a00\u6a21\u578b\u8bbe\u8ba1\u7684\u9ad8\u7ea7\u8f6c\u53d1\u4ee3\u7406\uff0c\u63d0\u4f9b\u5305\u62ec\u7528\u6237\u8bf7\u6c42\u901f\u7387\u63a7\u5236\u3001Token\u901f\u7387\u9650\u5236\u548c\u81ea\u5b9a\u4e49API\u5bc6\u94a5\u7b49\u589e\u5f3a\u529f\u80fd\u3002\u8be5\u670d\u52a1\u53ef\u7528\u4e8e\u4ee3\u7406\u672c\u5730\u6a21\u578b\u548c\u4e91\u7aef\u6a21\u578b\u3002OpenAI API Reverse Proxy",
"version": "0.8.0",
"project_urls": {
"Documentation": "https://github.com/KenyonY/openai-forward#openai-forward",
"Homepage": "https://github.com/KenyonY/openai-forward",
"Issues": "https://github.com/KenyonY/openai-forward/issues",
"Source": "https://github.com/KenyonY/openai-forward"
},
"split_keywords": [
"openai api forwarding",
" chatgpt",
" fastapi",
" openai",
" openai-api",
" openai-proxy",
" python",
" streaming-api"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "06f49ea7828c2f758aa74954b4c95a20e5f91963d55d9698afc88ed58e64ccf3",
"md5": "b2547fe1c3dc71efd270b771ad6dfc6b",
"sha256": "4f1ca8e43a7cd668c498e4a846a22e158e05fe1d37204d3cb6be72dc206c3ccc"
},
"downloads": -1,
"filename": "openai_forward-0.8.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b2547fe1c3dc71efd270b771ad6dfc6b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 48453,
"upload_time": "2024-05-15T16:52:56",
"upload_time_iso_8601": "2024-05-15T16:52:56.627202Z",
"url": "https://files.pythonhosted.org/packages/06/f4/9ea7828c2f758aa74954b4c95a20e5f91963d55d9698afc88ed58e64ccf3/openai_forward-0.8.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "11100cf3585e277adac7f3be920a291b018e5bba24288becada6d2a86e649914",
"md5": "6d0977c5efdf64e248f5e33d9b752cbe",
"sha256": "f38b6cd4cf5bb0d81047e88acb1850ad222fe266ce1d914e78a9c7ae206363f2"
},
"downloads": -1,
"filename": "openai_forward-0.8.0.tar.gz",
"has_sig": false,
"md5_digest": "6d0977c5efdf64e248f5e33d9b752cbe",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 39870,
"upload_time": "2024-05-15T16:52:59",
"upload_time_iso_8601": "2024-05-15T16:52:59.962473Z",
"url": "https://files.pythonhosted.org/packages/11/10/0cf3585e277adac7f3be920a291b018e5bba24288becada6d2a86e649914/openai_forward-0.8.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-15 16:52:59",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "KenyonY",
"github_project": "openai-forward#openai-forward",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "openai-forward"
}
JSON
