# mistral.rs
`mistralrs` is a Python package which provides an easy to use API for `mistral.rs`.
## Example
More examples can be found [here](https://github.com/EricLBuehler/mistral.rs/tree/master/examples/python)!
```python
from mistralrs import Runner, Which, ChatCompletionRequest
runner = Runner(
which=Which.Plain(
model_id="microsoft/Phi-3.5-mini-instruct",
),
in_situ_quant="Q4K",
)
res = runner.send_chat_completion_request(
ChatCompletionRequest(
model="mistral",
messages=[
{"role": "user", "content": "Tell me a story about the Rust type system."}
],
max_tokens=256,
presence_penalty=1.0,
top_p=0.1,
temperature=0.1,
)
)
print(res.choices[0].message.content)
print(res.usage)
```
Please find [API docs here](API.md) and the type stubs [here](mistralrs.pyi), which are another great form of documentation.
We also provide [a cookbook here](../examples/python/cookbook.ipynb)!
Raw data
{
"_id": null,
"home_page": "https://github.com/EricLBuehler/mistral.rs",
"name": "mistralrs-mkl",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "machine-learning",
"author": "Eric Buehler",
"author_email": null,
"download_url": null,
"platform": null,
"description": "# mistral.rs\n\n`mistralrs` is a Python package which provides an easy to use API for `mistral.rs`. \n\n## Example\nMore examples can be found [here](https://github.com/EricLBuehler/mistral.rs/tree/master/examples/python)!\n\n```python\nfrom mistralrs import Runner, Which, ChatCompletionRequest\n\nrunner = Runner(\n which=Which.Plain(\n model_id=\"microsoft/Phi-3.5-mini-instruct\",\n ),\n in_situ_quant=\"Q4K\",\n)\n\nres = runner.send_chat_completion_request(\n ChatCompletionRequest(\n model=\"mistral\",\n messages=[\n {\"role\": \"user\", \"content\": \"Tell me a story about the Rust type system.\"}\n ],\n max_tokens=256,\n presence_penalty=1.0,\n top_p=0.1,\n temperature=0.1,\n )\n)\nprint(res.choices[0].message.content)\nprint(res.usage)\n```\n\nPlease find [API docs here](API.md) and the type stubs [here](mistralrs.pyi), which are another great form of documentation.\n\nWe also provide [a cookbook here](../examples/python/cookbook.ipynb)!\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Fast and easy LLM serving.",
"version": "0.3.4",
"project_urls": {
"Homepage": "https://github.com/EricLBuehler/mistral.rs",
"Source Code": "https://github.com/EricLBuehler/mistral.rs"
},
"split_keywords": [
"machine-learning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f5801b270a0b7d6d4355789d6ce10de22088c32a3a1383be3b8fb6a824498007",
"md5": "2978203ebb1cfac39a7d96e36fd8bd74",
"sha256": "721b8ecb222c6d5e04be39aa5e319a751f96c30e5e243b0db4e2ff2d66331c47"
},
"downloads": -1,
"filename": "mistralrs_mkl-0.3.4-cp310-cp310-manylinux_2_39_x86_64.whl",
"has_sig": false,
"md5_digest": "2978203ebb1cfac39a7d96e36fd8bd74",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": ">=3.10",
"size": 19019588,
"upload_time": "2024-11-28T19:16:47",
"upload_time_iso_8601": "2024-11-28T19:16:47.806771Z",
"url": "https://files.pythonhosted.org/packages/f5/80/1b270a0b7d6d4355789d6ce10de22088c32a3a1383be3b8fb6a824498007/mistralrs_mkl-0.3.4-cp310-cp310-manylinux_2_39_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c4aa11329b097c8f7da29f3bd777423ba21ecc5b308c265e6ad644b1bbb46faa",
"md5": "3953708916ae3861cc57f665729e78aa",
"sha256": "11af4a44b1468bdc32931f71b48af7a067eb59694c2057d3b6283560c69dde2a"
},
"downloads": -1,
"filename": "mistralrs_mkl-0.3.4-cp310-none-win_amd64.whl",
"has_sig": false,
"md5_digest": "3953708916ae3861cc57f665729e78aa",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": ">=3.10",
"size": 14546226,
"upload_time": "2024-11-28T19:16:55",
"upload_time_iso_8601": "2024-11-28T19:16:55.802295Z",
"url": "https://files.pythonhosted.org/packages/c4/aa/11329b097c8f7da29f3bd777423ba21ecc5b308c265e6ad644b1bbb46faa/mistralrs_mkl-0.3.4-cp310-none-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e50bcef34c68b46d5a31bf205594b74e9f12c733626a3755fff9819f53bdd715",
"md5": "bda7618a5110a10fc9643f706ed169f4",
"sha256": "b2cc838f167a87566e5e28247fbf16c84f3097736f026dab86df0129147c2dff"
},
"downloads": -1,
"filename": "mistralrs_mkl-0.3.4-cp311-cp311-manylinux_2_39_x86_64.whl",
"has_sig": false,
"md5_digest": "bda7618a5110a10fc9643f706ed169f4",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.10",
"size": 19016219,
"upload_time": "2024-11-28T19:17:04",
"upload_time_iso_8601": "2024-11-28T19:17:04.834334Z",
"url": "https://files.pythonhosted.org/packages/e5/0b/cef34c68b46d5a31bf205594b74e9f12c733626a3755fff9819f53bdd715/mistralrs_mkl-0.3.4-cp311-cp311-manylinux_2_39_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "383556f7cbe3677a556dc772efe87b4634a06ff7d593996306450efea420f9fc",
"md5": "1eb4bcd647b2988a6a0ec68192241e7f",
"sha256": "020cad3a7d68ecc398432d3c3100c66e3dcb6e4e65cc98461aaf55134bdce4fd"
},
"downloads": -1,
"filename": "mistralrs_mkl-0.3.4-cp311-none-win_amd64.whl",
"has_sig": false,
"md5_digest": "1eb4bcd647b2988a6a0ec68192241e7f",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.10",
"size": 14548716,
"upload_time": "2024-11-28T19:17:12",
"upload_time_iso_8601": "2024-11-28T19:17:12.155020Z",
"url": "https://files.pythonhosted.org/packages/38/35/56f7cbe3677a556dc772efe87b4634a06ff7d593996306450efea420f9fc/mistralrs_mkl-0.3.4-cp311-none-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f94f94ad3f4646063ddb7959a4d5902937c0e41e0063a53983b309a4d3d6112b",
"md5": "d36788025e2452e65eb2d7b10ea3cfe9",
"sha256": "a959d845d3a61e60801e85e8c40c799190cbeb3c48b0a3b0725f056780668e6c"
},
"downloads": -1,
"filename": "mistralrs_mkl-0.3.4-cp312-cp312-manylinux_2_39_x86_64.whl",
"has_sig": false,
"md5_digest": "d36788025e2452e65eb2d7b10ea3cfe9",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": ">=3.10",
"size": 19019571,
"upload_time": "2024-11-28T19:17:21",
"upload_time_iso_8601": "2024-11-28T19:17:21.257170Z",
"url": "https://files.pythonhosted.org/packages/f9/4f/94ad3f4646063ddb7959a4d5902937c0e41e0063a53983b309a4d3d6112b/mistralrs_mkl-0.3.4-cp312-cp312-manylinux_2_39_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "41dcd649cd7debcf8f57b0db72a75f5349423bb1dca4482683760a6b21a690b7",
"md5": "f358f6a83833b44b87c10f04580f5c63",
"sha256": "97c08f004f696abd4f1274e5d3087e9ff8eb408ee2e9ba24d387c2623afca086"
},
"downloads": -1,
"filename": "mistralrs_mkl-0.3.4-cp312-none-win_amd64.whl",
"has_sig": false,
"md5_digest": "f358f6a83833b44b87c10f04580f5c63",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": ">=3.10",
"size": 14546804,
"upload_time": "2024-11-28T19:17:28",
"upload_time_iso_8601": "2024-11-28T19:17:28.536179Z",
"url": "https://files.pythonhosted.org/packages/41/dc/d649cd7debcf8f57b0db72a75f5349423bb1dca4482683760a6b21a690b7/mistralrs_mkl-0.3.4-cp312-none-win_amd64.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-28 19:16:47",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "EricLBuehler",
"github_project": "mistral.rs",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "mistralrs-mkl"
}
JSON
