# Imitater
[](LICENSE)
[](https://pypi.org/project/imitater/)
A unified language model server built upon [vllm](https://github.com/vllm-project/vllm) and [infinity](https://github.com/michaelfeil/infinity).
## Usage
### Install
```bash
pip install -e .
```
### Launch Server
```bash
imitater -c config/example.yaml
```
<details><summary>Show configuration instruction.</summary>
#### Add an OpenAI model
```yaml
- name: OpenAI model name
- token: OpenAI token
```
#### Add a chat model
```yaml
- name: Display name
- path: Model name on hub or local model path
- device: Device IDs
- port: Port ID
- maxlen: Maximum model length (optional)
- agent_type: Agent type (optional) {react, aligned}
- template: Template jinja file (optional)
- gen_config: Generation config folder (optional)
```
#### Add an embedding model
```yaml
- name: Display name
- path: Model name on hub or local model path
- device: Device IDs (does not support multi-gpus)
- port: Port ID
- batch_size: Batch size (optional)
```
</details>
> [!NOTE]
> [Chat template](https://huggingface.co/docs/transformers/chat_templating) is required for the chat models.
>
> Use `export USE_MODELSCOPE_HUB=1` to download model from modelscope.
### Test Server
```bash
python tests/test_openai.py -c config/example.yaml
```
### Roadmap
- [ ] Response choices.
- [ ] Rerank model support.
Raw data
{
"_id": null,
"home_page": "https://github.com/the-seeds/imitater",
"name": "imitater",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9.0",
"maintainer_email": null,
"keywords": "LLM",
"author": "the-seeds",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/08/bb/e381e84631c56f2ad21609ec50995daf9796590eae6896c94e8521375843/imitater-0.2.4.tar.gz",
"platform": null,
"description": "# Imitater\n\n[](LICENSE)\n[](https://pypi.org/project/imitater/)\n\nA unified language model server built upon [vllm](https://github.com/vllm-project/vllm) and [infinity](https://github.com/michaelfeil/infinity).\n\n## Usage\n\n### Install\n\n```bash\npip install -e .\n```\n\n### Launch Server\n\n```bash\nimitater -c config/example.yaml\n```\n\n<details><summary>Show configuration instruction.</summary>\n\n#### Add an OpenAI model\n\n```yaml\n- name: OpenAI model name\n- token: OpenAI token\n```\n\n#### Add a chat model\n\n```yaml\n- name: Display name\n- path: Model name on hub or local model path\n- device: Device IDs\n- port: Port ID\n- maxlen: Maximum model length (optional)\n- agent_type: Agent type (optional) {react, aligned}\n- template: Template jinja file (optional)\n- gen_config: Generation config folder (optional)\n```\n\n#### Add an embedding model\n\n```yaml\n- name: Display name\n- path: Model name on hub or local model path\n- device: Device IDs (does not support multi-gpus)\n- port: Port ID\n- batch_size: Batch size (optional)\n```\n\n</details>\n\n> [!NOTE]\n> [Chat template](https://huggingface.co/docs/transformers/chat_templating) is required for the chat models.\n>\n> Use `export USE_MODELSCOPE_HUB=1` to download model from modelscope.\n\n### Test Server\n\n```bash\npython tests/test_openai.py -c config/example.yaml\n```\n\n### Roadmap\n\n- [ ] Response choices.\n- [ ] Rerank model support.\n",
"bugtrack_url": null,
"license": "Apache 2.0 License",
"summary": null,
"version": "0.2.4",
"project_urls": {
"Homepage": "https://github.com/the-seeds/imitater"
},
"split_keywords": [
"llm"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "fb9b492bb87c063dd65d27ebd2df0196023702a011c851d1db191131422b3c4d",
"md5": "5b30948267f0754af77df1c3a186366a",
"sha256": "34012bf16e724955bda91f78f7ec68303d7b6a236f3f0d9e81ed5cf7c9b64f91"
},
"downloads": -1,
"filename": "imitater-0.2.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "5b30948267f0754af77df1c3a186366a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9.0",
"size": 25247,
"upload_time": "2024-06-11T06:38:38",
"upload_time_iso_8601": "2024-06-11T06:38:38.002611Z",
"url": "https://files.pythonhosted.org/packages/fb/9b/492bb87c063dd65d27ebd2df0196023702a011c851d1db191131422b3c4d/imitater-0.2.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "08bbe381e84631c56f2ad21609ec50995daf9796590eae6896c94e8521375843",
"md5": "783295942eca2f2dc0435fed4df8c5f1",
"sha256": "1639829832dabe14196d6f653de4208c67767c7d2e29a4d21d05e8cec160f389"
},
"downloads": -1,
"filename": "imitater-0.2.4.tar.gz",
"has_sig": false,
"md5_digest": "783295942eca2f2dc0435fed4df8c5f1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9.0",
"size": 20484,
"upload_time": "2024-06-11T06:38:39",
"upload_time_iso_8601": "2024-06-11T06:38:39.575910Z",
"url": "https://files.pythonhosted.org/packages/08/bb/e381e84631c56f2ad21609ec50995daf9796590eae6896c94e8521375843/imitater-0.2.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-06-11 06:38:39",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "the-seeds",
"github_project": "imitater",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "vllm",
"specs": [
[
"==",
"0.4.3"
]
]
},
{
"name": "infinity-emb",
"specs": [
[
"==",
"0.0.40"
]
]
},
{
"name": "openai",
"specs": [
[
">=",
"1.5.0"
]
]
},
{
"name": "uvicorn",
"specs": []
},
{
"name": "pydantic",
"specs": []
},
{
"name": "fastapi",
"specs": []
},
{
"name": "sse-starlette",
"specs": []
}
],
"lcname": "imitater"
}
JSON
