| Name | llm JSON |
| Version |
0.18
JSON |
| download |
| home_page | https://github.com/simonw/llm |
| Summary | A CLI utility and Python library for interacting with Large Language Models, including OpenAI, PaLM and local models installed on your own machine. |
| upload_time | 2024-11-17 20:35:57 |
| maintainer | None |
| docs_url | None |
| author | Simon Willison |
| requires_python | >=3.9 |
| license | Apache License, Version 2.0 |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# LLM
[](https://pypi.org/project/llm/)
[](https://llm.datasette.io/)
[](https://llm.datasette.io/en/stable/changelog.html)
[](https://github.com/simonw/llm/actions?query=workflow%3ATest)
[](https://github.com/simonw/llm/blob/main/LICENSE)
[](https://datasette.io/discord-llm)
[](https://formulae.brew.sh/formula/llm)
A CLI utility and Python library for interacting with Large Language Models, both via remote APIs and models that can be installed and run on your own machine.
[Run prompts from the command-line](https://llm.datasette.io/en/stable/usage.html#executing-a-prompt), [store the results in SQLite](https://llm.datasette.io/en/stable/logging.html), [generate embeddings](https://llm.datasette.io/en/stable/embeddings/index.html) and more.
Consult the **[LLM plugins directory](https://llm.datasette.io/en/stable/plugins/directory.html)** for plugins that provide access to remote and local models.
Full documentation: **[llm.datasette.io](https://llm.datasette.io/)**
Background on this project:
- [llm, ttok and strip-tags—CLI tools for working with ChatGPT and other LLMs](https://simonwillison.net/2023/May/18/cli-tools-for-llms/)
- [The LLM CLI tool now supports self-hosted language models via plugins](https://simonwillison.net/2023/Jul/12/llm/)
- [Accessing Llama 2 from the command-line with the llm-replicate plugin](https://simonwillison.net/2023/Jul/18/accessing-llama-2/)
- [Run Llama 2 on your own Mac using LLM and Homebrew](https://simonwillison.net/2023/Aug/1/llama-2-mac/)
- [Catching up on the weird world of LLMs](https://simonwillison.net/2023/Aug/3/weird-world-of-llms/)
- [LLM now provides tools for working with embeddings](https://simonwillison.net/2023/Sep/4/llm-embeddings/)
- [Build an image search engine with llm-clip, chat with models with llm chat](https://simonwillison.net/2023/Sep/12/llm-clip-and-chat/)
- [Many options for running Mistral models in your terminal using LLM](https://simonwillison.net/2023/Dec/18/mistral/)
## Installation
Install this tool using `pip`:
```bash
pip install llm
```
Or using [Homebrew](https://brew.sh/):
```bash
brew install llm
```
[Detailed installation instructions](https://llm.datasette.io/en/stable/setup.html).
## Getting started
If you have an [OpenAI API key](https://platform.openai.com/api-keys) you can get started using the OpenAI models right away.
As an alternative to OpenAI, you can [install plugins](https://llm.datasette.io/en/stable/plugins/installing-plugins.html) to access models by other providers, including models that can be installed and run on your own device.
Save your OpenAI API key like this:
```bash
llm keys set openai
```
This will prompt you for your key like so:
```
Enter key: <paste here>
```
Now that you've saved a key you can run a prompt like this:
```bash
llm "Five cute names for a pet penguin"
```
```
1. Waddles
2. Pebbles
3. Bubbles
4. Flappy
5. Chilly
```
Read the [usage instructions](https://llm.datasette.io/en/stable/usage.html) for more.
## Installing a model that runs on your own machine
[LLM plugins](https://llm.datasette.io/en/stable/plugins/index.html) can add support for alternative models, including models that run on your own machine.
To download and run Mistral 7B Instruct locally, you can install the [llm-gpt4all](https://github.com/simonw/llm-gpt4all) plugin:
```bash
llm install llm-gpt4all
```
Then run this command to see which models it makes available:
```bash
llm models
```
```
gpt4all: all-MiniLM-L6-v2-f16 - SBert, 43.76MB download, needs 1GB RAM
gpt4all: orca-mini-3b-gguf2-q4_0 - Mini Orca (Small), 1.84GB download, needs 4GB RAM
gpt4all: mistral-7b-instruct-v0 - Mistral Instruct, 3.83GB download, needs 8GB RAM
...
```
Each model file will be downloaded once the first time you use it. Try Mistral out like this:
```bash
llm -m mistral-7b-instruct-v0 'difference between a pelican and a walrus'
```
You can also start a chat session with the model using the `llm chat` command:
```bash
llm chat -m mistral-7b-instruct-v0
```
```
Chatting with mistral-7b-instruct-v0
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
>
```
## Using a system prompt
You can use the `-s/--system` option to set a system prompt, providing instructions for processing other input to the tool.
To describe how the code in a file works, try this:
```bash
cat mycode.py | llm -s "Explain this code"
```
## Help
For help, run:
llm --help
You can also use:
python -m llm --help
Raw data
{
"_id": null,
"home_page": "https://github.com/simonw/llm",
"name": "llm",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": null,
"author": "Simon Willison",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/8d/f1/0fb7b1fd468eb9d80ddb583cc1badcadbebf4549c3cdf4a7ff26f62803a6/llm-0.18.tar.gz",
"platform": null,
"description": "# LLM\n\n[](https://pypi.org/project/llm/)\n[](https://llm.datasette.io/)\n[](https://llm.datasette.io/en/stable/changelog.html)\n[](https://github.com/simonw/llm/actions?query=workflow%3ATest)\n[](https://github.com/simonw/llm/blob/main/LICENSE)\n[](https://datasette.io/discord-llm)\n[](https://formulae.brew.sh/formula/llm)\n\nA CLI utility and Python library for interacting with Large Language Models, both via remote APIs and models that can be installed and run on your own machine.\n\n[Run prompts from the command-line](https://llm.datasette.io/en/stable/usage.html#executing-a-prompt), [store the results in SQLite](https://llm.datasette.io/en/stable/logging.html), [generate embeddings](https://llm.datasette.io/en/stable/embeddings/index.html) and more.\n\nConsult the **[LLM plugins directory](https://llm.datasette.io/en/stable/plugins/directory.html)** for plugins that provide access to remote and local models.\n\nFull documentation: **[llm.datasette.io](https://llm.datasette.io/)**\n\nBackground on this project:\n- [llm, ttok and strip-tags\u2014CLI tools for working with ChatGPT and other LLMs](https://simonwillison.net/2023/May/18/cli-tools-for-llms/)\n- [The LLM CLI tool now supports self-hosted language models via plugins](https://simonwillison.net/2023/Jul/12/llm/)\n- [Accessing Llama 2 from the command-line with the llm-replicate plugin](https://simonwillison.net/2023/Jul/18/accessing-llama-2/)\n- [Run Llama 2 on your own Mac using LLM and Homebrew](https://simonwillison.net/2023/Aug/1/llama-2-mac/)\n- [Catching up on the weird world of LLMs](https://simonwillison.net/2023/Aug/3/weird-world-of-llms/)\n- [LLM now provides tools for working with embeddings](https://simonwillison.net/2023/Sep/4/llm-embeddings/)\n- [Build an image search engine with llm-clip, chat with models with llm chat](https://simonwillison.net/2023/Sep/12/llm-clip-and-chat/)\n- [Many options for running Mistral models in your terminal using LLM](https://simonwillison.net/2023/Dec/18/mistral/)\n\n## Installation\n\nInstall this tool using `pip`:\n```bash\npip install llm\n```\nOr using [Homebrew](https://brew.sh/):\n```bash\nbrew install llm\n```\n[Detailed installation instructions](https://llm.datasette.io/en/stable/setup.html).\n\n## Getting started\n\nIf you have an [OpenAI API key](https://platform.openai.com/api-keys) you can get started using the OpenAI models right away.\n\nAs an alternative to OpenAI, you can [install plugins](https://llm.datasette.io/en/stable/plugins/installing-plugins.html) to access models by other providers, including models that can be installed and run on your own device.\n\nSave your OpenAI API key like this:\n\n```bash\nllm keys set openai\n```\nThis will prompt you for your key like so:\n```\nEnter key: <paste here>\n```\nNow that you've saved a key you can run a prompt like this:\n```bash\nllm \"Five cute names for a pet penguin\"\n```\n```\n1. Waddles\n2. Pebbles\n3. Bubbles\n4. Flappy\n5. Chilly\n```\nRead the [usage instructions](https://llm.datasette.io/en/stable/usage.html) for more.\n\n## Installing a model that runs on your own machine\n\n[LLM plugins](https://llm.datasette.io/en/stable/plugins/index.html) can add support for alternative models, including models that run on your own machine.\n\nTo download and run Mistral 7B Instruct locally, you can install the [llm-gpt4all](https://github.com/simonw/llm-gpt4all) plugin:\n```bash\nllm install llm-gpt4all\n```\nThen run this command to see which models it makes available:\n```bash\nllm models\n```\n```\ngpt4all: all-MiniLM-L6-v2-f16 - SBert, 43.76MB download, needs 1GB RAM\ngpt4all: orca-mini-3b-gguf2-q4_0 - Mini Orca (Small), 1.84GB download, needs 4GB RAM\ngpt4all: mistral-7b-instruct-v0 - Mistral Instruct, 3.83GB download, needs 8GB RAM\n...\n```\nEach model file will be downloaded once the first time you use it. Try Mistral out like this:\n```bash\nllm -m mistral-7b-instruct-v0 'difference between a pelican and a walrus'\n```\nYou can also start a chat session with the model using the `llm chat` command:\n```bash\nllm chat -m mistral-7b-instruct-v0\n```\n```\nChatting with mistral-7b-instruct-v0\nType 'exit' or 'quit' to exit\nType '!multi' to enter multiple lines, then '!end' to finish\n> \n```\n\n## Using a system prompt\n\nYou can use the `-s/--system` option to set a system prompt, providing instructions for processing other input to the tool.\n\nTo describe how the code in a file works, try this:\n\n```bash\ncat mycode.py | llm -s \"Explain this code\"\n```\n\n## Help\n\nFor help, run:\n\n llm --help\n\nYou can also use:\n\n python -m llm --help\n",
"bugtrack_url": null,
"license": "Apache License, Version 2.0",
"summary": "A CLI utility and Python library for interacting with Large Language Models, including OpenAI, PaLM and local models installed on your own machine.",
"version": "0.18",
"project_urls": {
"CI": "https://github.com/simonw/llm/actions",
"Changelog": "https://github.com/simonw/llm/releases",
"Documentation": "https://llm.datasette.io/",
"Homepage": "https://github.com/simonw/llm",
"Issues": "https://github.com/simonw/llm/issues"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "962756aac0e5aa3b97cb4e06246beb8ad66c174d60ba0413f318c4fd15f50a5e",
"md5": "fe92afeefb552493faa405cbfdef0e7e",
"sha256": "bf1ad0b41425909a5aecd855ccb1ad2fb64d78fb2ac0d1ab07ddc2eba4b58c42"
},
"downloads": -1,
"filename": "llm-0.18-py3-none-any.whl",
"has_sig": false,
"md5_digest": "fe92afeefb552493faa405cbfdef0e7e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 43179,
"upload_time": "2024-11-17T20:35:56",
"upload_time_iso_8601": "2024-11-17T20:35:56.179388Z",
"url": "https://files.pythonhosted.org/packages/96/27/56aac0e5aa3b97cb4e06246beb8ad66c174d60ba0413f318c4fd15f50a5e/llm-0.18-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8df10fb7b1fd468eb9d80ddb583cc1badcadbebf4549c3cdf4a7ff26f62803a6",
"md5": "9a66cc455f1de4893c1f5ff2364940be",
"sha256": "1c4243dea5ec3e7ee50f4fd0b8ccd6475b8f5a6cca2b9aac8270c0e843c0e490"
},
"downloads": -1,
"filename": "llm-0.18.tar.gz",
"has_sig": false,
"md5_digest": "9a66cc455f1de4893c1f5ff2364940be",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 41597,
"upload_time": "2024-11-17T20:35:57",
"upload_time_iso_8601": "2024-11-17T20:35:57.877194Z",
"url": "https://files.pythonhosted.org/packages/8d/f1/0fb7b1fd468eb9d80ddb583cc1badcadbebf4549c3cdf4a7ff26f62803a6/llm-0.18.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-17 20:35:57",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "simonw",
"github_project": "llm",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "llm"
}
