| Name | llm-mistral JSON |
| Version |
0.3.1
JSON |
| download |
| home_page | None |
| Summary | LLM plugin providing access to Mistral models busing the Mistral API |
| upload_time | 2024-04-18 03:07:30 |
| maintainer | None |
| docs_url | None |
| author | Simon Willison |
| requires_python | None |
| license | Apache-2.0 |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# llm-mistral
[](https://pypi.org/project/llm-mistral/)
[](https://github.com/simonw/llm-mistral/releases)
[](https://github.com/simonw/llm-mistral/actions?query=workflow%3ATest)
[](https://github.com/simonw/llm-mistral/blob/main/LICENSE)
[LLM](https://llm.datasette.io/) plugin providing access to [Mistral](https://mistral.ai) models using the Mistral API
## Installation
Install this plugin in the same environment as LLM:
```bash
llm install llm-mistral
```
## Usage
First, obtain an API key for [the Mistral API](https://console.mistral.ai/).
Configure the key using the `llm keys set mistral` command:
```bash
llm keys set mistral
```
```
<paste key here>
```
You can now access the Mistral hosted models. Run `llm models` for a list.
To run a prompt through `mistral-tiny`:
```bash
llm -m mistral-tiny 'A sassy name for a pet sasquatch'
```
To start an interactive chat session with `mistral-small`:
```bash
llm chat -m mistral-small
```
```
Chatting with mistral-small
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
> three proud names for a pet walrus
1. "Nanuq," the Inuit word for walrus, which symbolizes strength and resilience.
2. "Sir Tuskalot," a playful and regal name that highlights the walrus' distinctive tusks.
3. "Glacier," a name that reflects the walrus' icy Arctic habitat and majestic presence.
```
To use a system prompt with `mistral-medium` to explain some code:
```bash
cat example.py | llm -m mistral-medium -s 'explain this code'
```
## Model options
All three models accept the following options, using `-o name value` syntax:
- `-o temperature 0.7`: The sampling temperature, between 0 and 1. Higher increases randomness, lower values are more focused and deterministic.
- `-o top_p 0.1`: 0.1 means consider only tokens in the top 10% probability mass. Use this or temperature but not both.
- `-o max_tokens 20`: Maximum number of tokens to generate in the completion.
- `-o safe_mode 1`: Turns on [safe mode](https://docs.mistral.ai/platform/guardrailing/), which adds a system prompt to add guardrails to the model output.
- `-o random_seed 123`: Set an integer random seed to generate deterministic results.
## Embeddings
The Mistral [Embeddings API](https://docs.mistral.ai/platform/client#embeddings) can be used to generate 1,024 dimensional embeddings for any text.
To embed a single string:
```bash
llm embed -m mistral-embed -c 'this is text'
```
This will return a JSON array of 1,024 floating point numbers.
The [LLM documentation](https://llm.datasette.io/en/stable/embeddings/index.html) has more, including how to embed in bulk and store the results in a SQLite database.
See [LLM now provides tools for working with embeddings](https://simonwillison.net/2023/Sep/4/llm-embeddings/) and [Embeddings: What they are and why they matter](https://simonwillison.net/2023/Oct/23/embeddings/) for more about embeddings.
## Development
To set up this plugin locally, first checkout the code. Then create a new virtual environment:
```bash
cd llm-mistral
python3 -m venv venv
source venv/bin/activate
```
Now install the dependencies and test dependencies:
```bash
llm install -e '.[test]'
```
To run the tests:
```bash
pytest
```
Raw data
{
"_id": null,
"home_page": null,
"name": "llm-mistral",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "Simon Willison",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/fa/08/984534bc9b07a6242cbdb74b0b460382a344c3267ccb7323338df628a1d4/llm_mistral-0.3.1.tar.gz",
"platform": null,
"description": "# llm-mistral\n\n[](https://pypi.org/project/llm-mistral/)\n[](https://github.com/simonw/llm-mistral/releases)\n[](https://github.com/simonw/llm-mistral/actions?query=workflow%3ATest)\n[](https://github.com/simonw/llm-mistral/blob/main/LICENSE)\n\n[LLM](https://llm.datasette.io/) plugin providing access to [Mistral](https://mistral.ai) models using the Mistral API\n\n## Installation\n\nInstall this plugin in the same environment as LLM:\n```bash\nllm install llm-mistral\n```\n## Usage\n\nFirst, obtain an API key for [the Mistral API](https://console.mistral.ai/).\n\nConfigure the key using the `llm keys set mistral` command:\n```bash\nllm keys set mistral\n```\n```\n<paste key here>\n```\nYou can now access the Mistral hosted models. Run `llm models` for a list.\n\nTo run a prompt through `mistral-tiny`:\n\n```bash\nllm -m mistral-tiny 'A sassy name for a pet sasquatch'\n```\nTo start an interactive chat session with `mistral-small`:\n```bash\nllm chat -m mistral-small\n```\n```\nChatting with mistral-small\nType 'exit' or 'quit' to exit\nType '!multi' to enter multiple lines, then '!end' to finish\n> three proud names for a pet walrus\n1. \"Nanuq,\" the Inuit word for walrus, which symbolizes strength and resilience.\n2. \"Sir Tuskalot,\" a playful and regal name that highlights the walrus' distinctive tusks.\n3. \"Glacier,\" a name that reflects the walrus' icy Arctic habitat and majestic presence.\n```\nTo use a system prompt with `mistral-medium` to explain some code:\n```bash\ncat example.py | llm -m mistral-medium -s 'explain this code'\n```\n\n## Model options\n\nAll three models accept the following options, using `-o name value` syntax:\n\n- `-o temperature 0.7`: The sampling temperature, between 0 and 1. Higher increases randomness, lower values are more focused and deterministic.\n- `-o top_p 0.1`: 0.1 means consider only tokens in the top 10% probability mass. Use this or temperature but not both.\n- `-o max_tokens 20`: Maximum number of tokens to generate in the completion.\n- `-o safe_mode 1`: Turns on [safe mode](https://docs.mistral.ai/platform/guardrailing/), which adds a system prompt to add guardrails to the model output.\n- `-o random_seed 123`: Set an integer random seed to generate deterministic results.\n\n## Embeddings\n\nThe Mistral [Embeddings API](https://docs.mistral.ai/platform/client#embeddings) can be used to generate 1,024 dimensional embeddings for any text.\n\nTo embed a single string:\n\n```bash\nllm embed -m mistral-embed -c 'this is text'\n```\nThis will return a JSON array of 1,024 floating point numbers.\n\nThe [LLM documentation](https://llm.datasette.io/en/stable/embeddings/index.html) has more, including how to embed in bulk and store the results in a SQLite database.\n\nSee [LLM now provides tools for working with embeddings](https://simonwillison.net/2023/Sep/4/llm-embeddings/) and [Embeddings: What they are and why they matter](https://simonwillison.net/2023/Oct/23/embeddings/) for more about embeddings.\n\n## Development\n\nTo set up this plugin locally, first checkout the code. Then create a new virtual environment:\n```bash\ncd llm-mistral\npython3 -m venv venv\nsource venv/bin/activate\n```\nNow install the dependencies and test dependencies:\n```bash\nllm install -e '.[test]'\n```\nTo run the tests:\n```bash\npytest\n```\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "LLM plugin providing access to Mistral models busing the Mistral API",
"version": "0.3.1",
"project_urls": {
"CI": "https://github.com/simonw/llm-mistral/actions",
"Changelog": "https://github.com/simonw/llm-mistral/releases",
"Homepage": "https://github.com/simonw/llm-mistral",
"Issues": "https://github.com/simonw/llm-mistral/issues"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d43cbc9b5383087c631bd6b4b9b4d2c3fbff9d1ab903767dedefe370a89b9ebd",
"md5": "7b5bb228e911c79ea30040a1bd76639c",
"sha256": "75c6df741611e86e0d11a3197e9ef20d9a5ba91eca220317b804411544e83a6e"
},
"downloads": -1,
"filename": "llm_mistral-0.3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7b5bb228e911c79ea30040a1bd76639c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 9537,
"upload_time": "2024-04-18T03:07:29",
"upload_time_iso_8601": "2024-04-18T03:07:29.431343Z",
"url": "https://files.pythonhosted.org/packages/d4/3c/bc9b5383087c631bd6b4b9b4d2c3fbff9d1ab903767dedefe370a89b9ebd/llm_mistral-0.3.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "fa08984534bc9b07a6242cbdb74b0b460382a344c3267ccb7323338df628a1d4",
"md5": "0fbebc25ad18df98ea27c81476cc5db9",
"sha256": "03a4113829c07c596b99717673b80a075a27bf17a757abf68e96bcccd041bf10"
},
"downloads": -1,
"filename": "llm_mistral-0.3.1.tar.gz",
"has_sig": false,
"md5_digest": "0fbebc25ad18df98ea27c81476cc5db9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 10236,
"upload_time": "2024-04-18T03:07:30",
"upload_time_iso_8601": "2024-04-18T03:07:30.535731Z",
"url": "https://files.pythonhosted.org/packages/fa/08/984534bc9b07a6242cbdb74b0b460382a344c3267ccb7323338df628a1d4/llm_mistral-0.3.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-18 03:07:30",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "simonw",
"github_project": "llm-mistral",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "llm-mistral"
}
