llm-mistral


Namellm-mistral JSON
Version 0.8 PyPI version JSON
download
home_pageNone
SummaryLLM plugin providing access to Mistral models busing the Mistral API
upload_time2024-11-19 00:57:49
maintainerNone
docs_urlNone
authorSimon Willison
requires_pythonNone
licenseApache-2.0
keywords
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # llm-mistral

[![PyPI](https://img.shields.io/pypi/v/llm-mistral.svg)](https://pypi.org/project/llm-mistral/)
[![Changelog](https://img.shields.io/github/v/release/simonw/llm-mistral?include_prereleases&label=changelog)](https://github.com/simonw/llm-mistral/releases)
[![Tests](https://github.com/simonw/llm-mistral/workflows/Test/badge.svg)](https://github.com/simonw/llm-mistral/actions?query=workflow%3ATest)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/llm-mistral/blob/main/LICENSE)

[LLM](https://llm.datasette.io/) plugin providing access to [Mistral](https://mistral.ai) models using the Mistral API

## Installation

Install this plugin in the same environment as LLM:
```bash
llm install llm-mistral
```
## Usage

First, obtain an API key for [the Mistral API](https://console.mistral.ai/).

Configure the key using the `llm keys set mistral` command:
```bash
llm keys set mistral
```
```
<paste key here>
```
You can now access the Mistral hosted models. Run `llm models` for a list.

To run a prompt through `mistral-tiny`:

```bash
llm -m mistral-tiny 'A sassy name for a pet sasquatch'
```
To start an interactive chat session with `mistral-small`:
```bash
llm chat -m mistral-small
```
```
Chatting with mistral-small
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
> three proud names for a pet walrus
1. "Nanuq," the Inuit word for walrus, which symbolizes strength and resilience.
2. "Sir Tuskalot," a playful and regal name that highlights the walrus' distinctive tusks.
3. "Glacier," a name that reflects the walrus' icy Arctic habitat and majestic presence.
```
To use a system prompt with `mistral-medium` to explain some code:
```bash
cat example.py | llm -m mistral-medium -s 'explain this code'
```
## Vision

The `pixtral-12b` model is capable of interpreting images. You can use that like this:

```bash
llm -m pixtral-12b 'describe this image' -a https://static.simonwillison.net/static/2024/earth.jpg
```
You can also pass filenames instead of URLs.

## Model options

All three models accept the following options, using `-o name value` syntax:

- `-o temperature 0.7`: The sampling temperature, between 0 and 1. Higher increases randomness, lower values are more focused and deterministic.
- `-o top_p 0.1`: 0.1 means consider only tokens in the top 10% probability mass. Use this or temperature but not both.
- `-o max_tokens 20`: Maximum number of tokens to generate in the completion.
- `-o safe_mode 1`: Turns on [safe mode](https://docs.mistral.ai/platform/guardrailing/), which adds a system prompt to add guardrails to the model output.
- `-o random_seed 123`: Set an integer random seed to generate deterministic results.

## Available models

Run `llm models` for a full list of Mistral models. This plugin configures the following alias shortcuts:

<!-- [[[cog
import cog, json
from llm_mistral import DEFAULT_ALIASES
for model_id, alias in DEFAULT_ALIASES.items():
    cog.out(f"- `{alias}` for `{model_id}`\n")
]]] -->
- `mistral-tiny` for `mistral/mistral-tiny`
- `mistral-nemo` for `mistral/open-mistral-nemo`
- `mistral-small` for `mistral/mistral-small`
- `mistral-medium` for `mistral/mistral-medium`
- `mistral-large` for `mistral/mistral-large-latest`
- `codestral-mamba` for `mistral/codestral-mamba-latest`
- `codestral` for `mistral/codestral-latest`
- `ministral-3b` for `mistral/ministral-3b-latest`
- `ministral-8b` for `mistral/ministral-8b-latest`
- `pixtral-12b` for `mistral/pixtral-12b-latest`
- `pixtral-large` for `mistral/pixtral-large-latest`
<!-- [[[end]]] -->


## Refreshing the model list

Mistral sometimes release new models.

To make those models available to an existing installation of `llm-mistral` run this command:
```bash
llm mistral refresh
```
This will fetch and cache the latest list of available models. They should then become available in the output of the `llm models` command.

## Embeddings

The Mistral [Embeddings API](https://docs.mistral.ai/platform/client#embeddings) can be used to generate 1,024 dimensional embeddings for any text.

To embed a single string:

```bash
llm embed -m mistral-embed -c 'this is text'
```
This will return a JSON array of 1,024 floating point numbers.

The [LLM documentation](https://llm.datasette.io/en/stable/embeddings/index.html) has more, including how to embed in bulk and store the results in a SQLite database.

See [LLM now provides tools for working with embeddings](https://simonwillison.net/2023/Sep/4/llm-embeddings/) and [Embeddings: What they are and why they matter](https://simonwillison.net/2023/Oct/23/embeddings/) for more about embeddings.

## Development

To set up this plugin locally, first checkout the code. Then create a new virtual environment:
```bash
cd llm-mistral
python3 -m venv venv
source venv/bin/activate
```
Now install the dependencies and test dependencies:
```bash
llm install -e '.[test]'
```
To run the tests:
```bash
pytest
```

            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "llm-mistral",
    "maintainer": null,
    "docs_url": null,
    "requires_python": null,
    "maintainer_email": null,
    "keywords": null,
    "author": "Simon Willison",
    "author_email": null,
    "download_url": "https://files.pythonhosted.org/packages/5d/57/702033f837aff1a2b9f467f08eb91713f1eae1d4f6a6ab06446772172c72/llm_mistral-0.8.tar.gz",
    "platform": null,
    "description": "# llm-mistral\n\n[![PyPI](https://img.shields.io/pypi/v/llm-mistral.svg)](https://pypi.org/project/llm-mistral/)\n[![Changelog](https://img.shields.io/github/v/release/simonw/llm-mistral?include_prereleases&label=changelog)](https://github.com/simonw/llm-mistral/releases)\n[![Tests](https://github.com/simonw/llm-mistral/workflows/Test/badge.svg)](https://github.com/simonw/llm-mistral/actions?query=workflow%3ATest)\n[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/llm-mistral/blob/main/LICENSE)\n\n[LLM](https://llm.datasette.io/) plugin providing access to [Mistral](https://mistral.ai) models using the Mistral API\n\n## Installation\n\nInstall this plugin in the same environment as LLM:\n```bash\nllm install llm-mistral\n```\n## Usage\n\nFirst, obtain an API key for [the Mistral API](https://console.mistral.ai/).\n\nConfigure the key using the `llm keys set mistral` command:\n```bash\nllm keys set mistral\n```\n```\n<paste key here>\n```\nYou can now access the Mistral hosted models. Run `llm models` for a list.\n\nTo run a prompt through `mistral-tiny`:\n\n```bash\nllm -m mistral-tiny 'A sassy name for a pet sasquatch'\n```\nTo start an interactive chat session with `mistral-small`:\n```bash\nllm chat -m mistral-small\n```\n```\nChatting with mistral-small\nType 'exit' or 'quit' to exit\nType '!multi' to enter multiple lines, then '!end' to finish\n> three proud names for a pet walrus\n1. \"Nanuq,\" the Inuit word for walrus, which symbolizes strength and resilience.\n2. \"Sir Tuskalot,\" a playful and regal name that highlights the walrus' distinctive tusks.\n3. \"Glacier,\" a name that reflects the walrus' icy Arctic habitat and majestic presence.\n```\nTo use a system prompt with `mistral-medium` to explain some code:\n```bash\ncat example.py | llm -m mistral-medium -s 'explain this code'\n```\n## Vision\n\nThe `pixtral-12b` model is capable of interpreting images. You can use that like this:\n\n```bash\nllm -m pixtral-12b 'describe this image' -a https://static.simonwillison.net/static/2024/earth.jpg\n```\nYou can also pass filenames instead of URLs.\n\n## Model options\n\nAll three models accept the following options, using `-o name value` syntax:\n\n- `-o temperature 0.7`: The sampling temperature, between 0 and 1. Higher increases randomness, lower values are more focused and deterministic.\n- `-o top_p 0.1`: 0.1 means consider only tokens in the top 10% probability mass. Use this or temperature but not both.\n- `-o max_tokens 20`: Maximum number of tokens to generate in the completion.\n- `-o safe_mode 1`: Turns on [safe mode](https://docs.mistral.ai/platform/guardrailing/), which adds a system prompt to add guardrails to the model output.\n- `-o random_seed 123`: Set an integer random seed to generate deterministic results.\n\n## Available models\n\nRun `llm models` for a full list of Mistral models. This plugin configures the following alias shortcuts:\n\n<!-- [[[cog\nimport cog, json\nfrom llm_mistral import DEFAULT_ALIASES\nfor model_id, alias in DEFAULT_ALIASES.items():\n    cog.out(f\"- `{alias}` for `{model_id}`\\n\")\n]]] -->\n- `mistral-tiny` for `mistral/mistral-tiny`\n- `mistral-nemo` for `mistral/open-mistral-nemo`\n- `mistral-small` for `mistral/mistral-small`\n- `mistral-medium` for `mistral/mistral-medium`\n- `mistral-large` for `mistral/mistral-large-latest`\n- `codestral-mamba` for `mistral/codestral-mamba-latest`\n- `codestral` for `mistral/codestral-latest`\n- `ministral-3b` for `mistral/ministral-3b-latest`\n- `ministral-8b` for `mistral/ministral-8b-latest`\n- `pixtral-12b` for `mistral/pixtral-12b-latest`\n- `pixtral-large` for `mistral/pixtral-large-latest`\n<!-- [[[end]]] -->\n\n\n## Refreshing the model list\n\nMistral sometimes release new models.\n\nTo make those models available to an existing installation of `llm-mistral` run this command:\n```bash\nllm mistral refresh\n```\nThis will fetch and cache the latest list of available models. They should then become available in the output of the `llm models` command.\n\n## Embeddings\n\nThe Mistral [Embeddings API](https://docs.mistral.ai/platform/client#embeddings) can be used to generate 1,024 dimensional embeddings for any text.\n\nTo embed a single string:\n\n```bash\nllm embed -m mistral-embed -c 'this is text'\n```\nThis will return a JSON array of 1,024 floating point numbers.\n\nThe [LLM documentation](https://llm.datasette.io/en/stable/embeddings/index.html) has more, including how to embed in bulk and store the results in a SQLite database.\n\nSee [LLM now provides tools for working with embeddings](https://simonwillison.net/2023/Sep/4/llm-embeddings/) and [Embeddings: What they are and why they matter](https://simonwillison.net/2023/Oct/23/embeddings/) for more about embeddings.\n\n## Development\n\nTo set up this plugin locally, first checkout the code. Then create a new virtual environment:\n```bash\ncd llm-mistral\npython3 -m venv venv\nsource venv/bin/activate\n```\nNow install the dependencies and test dependencies:\n```bash\nllm install -e '.[test]'\n```\nTo run the tests:\n```bash\npytest\n```\n",
    "bugtrack_url": null,
    "license": "Apache-2.0",
    "summary": "LLM plugin providing access to Mistral models busing the Mistral API",
    "version": "0.8",
    "project_urls": {
        "CI": "https://github.com/simonw/llm-mistral/actions",
        "Changelog": "https://github.com/simonw/llm-mistral/releases",
        "Homepage": "https://github.com/simonw/llm-mistral",
        "Issues": "https://github.com/simonw/llm-mistral/issues"
    },
    "split_keywords": [],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "ae8fe0661dd3077b7343419ddb5ff840f6bcf571e8d11d8acb031e1b167de79c",
                "md5": "72879c3730fe48906f359706224b2c4c",
                "sha256": "ee9466b22ed732028464d61d4b7ae671af2ad0e628d74595dca2880fa920cd4d"
            },
            "downloads": -1,
            "filename": "llm_mistral-0.8-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "72879c3730fe48906f359706224b2c4c",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 10918,
            "upload_time": "2024-11-19T00:57:48",
            "upload_time_iso_8601": "2024-11-19T00:57:48.300269Z",
            "url": "https://files.pythonhosted.org/packages/ae/8f/e0661dd3077b7343419ddb5ff840f6bcf571e8d11d8acb031e1b167de79c/llm_mistral-0.8-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "5d57702033f837aff1a2b9f467f08eb91713f1eae1d4f6a6ab06446772172c72",
                "md5": "cdb0f32519a33c948f5fbcdcac3f459a",
                "sha256": "d2b3cd349a8a37864f088aa4396c3c00d78f76f602262c02106af28cb63480a6"
            },
            "downloads": -1,
            "filename": "llm_mistral-0.8.tar.gz",
            "has_sig": false,
            "md5_digest": "cdb0f32519a33c948f5fbcdcac3f459a",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 11764,
            "upload_time": "2024-11-19T00:57:49",
            "upload_time_iso_8601": "2024-11-19T00:57:49.439911Z",
            "url": "https://files.pythonhosted.org/packages/5d/57/702033f837aff1a2b9f467f08eb91713f1eae1d4f6a6ab06446772172c72/llm_mistral-0.8.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-11-19 00:57:49",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "simonw",
    "github_project": "llm-mistral",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "llm-mistral"
}
        
Elapsed time: 5.00582s