# mistral.rs
`mistralrs` is a Python package which provides an easy to use API for `mistral.rs`.
## Example
More examples can be found [here](https://github.com/EricLBuehler/mistral.rs/tree/master/examples/python)!
```python
from mistralrs import Runner, Which, ChatCompletionRequest
runner = Runner(
which=Which.Plain(
model_id="microsoft/Phi-3.5-mini-instruct",
),
in_situ_quant="Q4K",
)
res = runner.send_chat_completion_request(
ChatCompletionRequest(
model="mistral",
messages=[
{"role": "user", "content": "Tell me a story about the Rust type system."}
],
max_tokens=256,
presence_penalty=1.0,
top_p=0.1,
temperature=0.1,
)
)
print(res.choices[0].message.content)
print(res.usage)
```
Please find [API docs here](API.md) and the type stubs [here](mistralrs.pyi), which are another great form of documentation.
We also provide [a cookbook here](../examples/python/cookbook.ipynb)!
Raw data
{
"_id": null,
"home_page": "https://github.com/EricLBuehler/mistral.rs",
"name": "mistralrs-metal",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "machine-learning",
"author": "Eric Buehler",
"author_email": null,
"download_url": null,
"platform": null,
"description": "# mistral.rs\n\n`mistralrs` is a Python package which provides an easy to use API for `mistral.rs`. \n\n## Example\nMore examples can be found [here](https://github.com/EricLBuehler/mistral.rs/tree/master/examples/python)!\n\n```python\nfrom mistralrs import Runner, Which, ChatCompletionRequest\n\nrunner = Runner(\n which=Which.Plain(\n model_id=\"microsoft/Phi-3.5-mini-instruct\",\n ),\n in_situ_quant=\"Q4K\",\n)\n\nres = runner.send_chat_completion_request(\n ChatCompletionRequest(\n model=\"mistral\",\n messages=[\n {\"role\": \"user\", \"content\": \"Tell me a story about the Rust type system.\"}\n ],\n max_tokens=256,\n presence_penalty=1.0,\n top_p=0.1,\n temperature=0.1,\n )\n)\nprint(res.choices[0].message.content)\nprint(res.usage)\n```\n\nPlease find [API docs here](API.md) and the type stubs [here](mistralrs.pyi), which are another great form of documentation.\n\nWe also provide [a cookbook here](../examples/python/cookbook.ipynb)!\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Fast and easy LLM serving.",
"version": "0.3.4",
"project_urls": {
"Homepage": "https://github.com/EricLBuehler/mistral.rs",
"Source Code": "https://github.com/EricLBuehler/mistral.rs"
},
"split_keywords": [
"machine-learning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7d704f09e68dc9aae0c62645e069b8baf2b55afc20c1030f0428d5a660972917",
"md5": "4937ac04fb9fe72f1a2a30c37f94a103",
"sha256": "08f40bcc2f5c08382cc7ae23c3c676a4693125f8df92f3dcd8455c08852339f6"
},
"downloads": -1,
"filename": "mistralrs_metal-0.3.4-cp310-cp310-macosx_11_0_arm64.whl",
"has_sig": false,
"md5_digest": "4937ac04fb9fe72f1a2a30c37f94a103",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": ">=3.10",
"size": 11338521,
"upload_time": "2024-11-28T18:47:27",
"upload_time_iso_8601": "2024-11-28T18:47:27.948121Z",
"url": "https://files.pythonhosted.org/packages/7d/70/4f09e68dc9aae0c62645e069b8baf2b55afc20c1030f0428d5a660972917/mistralrs_metal-0.3.4-cp310-cp310-macosx_11_0_arm64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5994770318ac0df5f183d3d5cbfea0d540baf268e6d85ee68441c073afff4f1d",
"md5": "eab45808933044c6cf1719a9fd515195",
"sha256": "6bfd550b55ee8c91c333ecfe06153ccae8bbdf667b867ae70ea3bec4ec422e81"
},
"downloads": -1,
"filename": "mistralrs_metal-0.3.4-cp311-cp311-macosx_11_0_arm64.whl",
"has_sig": false,
"md5_digest": "eab45808933044c6cf1719a9fd515195",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.10",
"size": 11335165,
"upload_time": "2024-11-28T18:47:34",
"upload_time_iso_8601": "2024-11-28T18:47:34.283586Z",
"url": "https://files.pythonhosted.org/packages/59/94/770318ac0df5f183d3d5cbfea0d540baf268e6d85ee68441c073afff4f1d/mistralrs_metal-0.3.4-cp311-cp311-macosx_11_0_arm64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f8c4e8f7f4adb4e75911f65755e3fea661f6b2d1b59c8e6b9f521ff79255ef3c",
"md5": "cc43bb323d46449fef7c0a33e60f4f2a",
"sha256": "acab6e0e15df20a19041b84850ed337d2197e9028b620bc1f85acc044e703218"
},
"downloads": -1,
"filename": "mistralrs_metal-0.3.4-cp312-cp312-macosx_11_0_arm64.whl",
"has_sig": false,
"md5_digest": "cc43bb323d46449fef7c0a33e60f4f2a",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": ">=3.10",
"size": 11339397,
"upload_time": "2024-11-28T18:47:40",
"upload_time_iso_8601": "2024-11-28T18:47:40.735001Z",
"url": "https://files.pythonhosted.org/packages/f8/c4/e8f7f4adb4e75911f65755e3fea661f6b2d1b59c8e6b9f521ff79255ef3c/mistralrs_metal-0.3.4-cp312-cp312-macosx_11_0_arm64.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-28 18:47:27",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "EricLBuehler",
"github_project": "mistral.rs",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "mistralrs-metal"
}
JSON
