| Name | llm-nosrun JSON |
| Version |
0.2
JSON |
| download |
| home_page | |
| Summary | LLM plugin for models hosted by NOS Cloud |
| upload_time | 2024-01-16 18:54:43 |
| maintainer | |
| docs_url | None |
| author | Sudeep Pillai |
| requires_python | |
| license | Apache-2.0 |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# llm-nosrun
[](https://pypi.org/project/llm-nosrun/)
[](https://github.com/autonomi-ai/llm-nosrun/releases)
[](https://github.com/autonomi-ai/llm-nosrun/actions?query=workflow%3ATest)
[](https://github.com/autonomi-ai/llm-nosrun/blob/main/LICENSE)
[LLM](https://llm.datasette.io/) plugin for models hosted by [NOS Cloud](https://app.nos.run/)
## Installation
First, [install the LLM command-line utility](https://llm.datasette.io/en/stable/setup.html).
Now install this plugin in the same environment as LLM.
```bash
llm install llm-nosrun
```
## Configuration
You will need an API key from NOS Cloud. You can [obtain one here](https://app.nos.run/).
You can set that as an environment variable called `LLM_NOSRUN_KEY`, or add it to the `llm` set of saved keys using:
```bash
llm keys set nosrun
```
```
Enter key: <paste key here>
```
## Usage
To list available models, run:
```bash
llm models list
```
You should see a list that looks something like this:
```
NOSRun: TinyLlama/TinyLlama-1.1B-Chat-v1.0
NOSRun: meta-llama/Llama-2-7b-chat-hf
NOSRun: meta-llama/Llama-2-13b-chat-hf
NOSRun: meta-llama/Llama-2-70b-chat-hf
NOSRun: HuggingFaceH4/zephyr-7b-beta
NOSRun: HuggingFaceH4/tiny-random-LlamaForCausalLM
NOSRun: NousResearch/Yarn-Mistral-7b-128k
NOSRun: mistralai/Mistral-7B-Instruct-v0.2
NOSRun: mistralai/Mixtral-8x7B-Instruct-v0.1
NOSRun: TheBloke/TinyLlama-1.1B-Chat-v1.0-AWQ
NOSRun: TheBloke/Mixtral-8x7B-Instruct-v0.1-AWQ
NOSRun: mlabonne/phixtral-2x2_8
NOSRun: mlabonne/phixtral-4x2_8
```
To run a prompt against a model, pass its full model ID to the `-m` option, like this:
```bash
llm -m TinyLlama/TinyLlama-1.1B-Chat-v1.0 \
'Five strident names for a pet walrus' \
--system 'You love coming up with creative names for pets'
```
You can set a shorter alias for a model using the `llm aliases` command like so:
```bash
llm aliases set tinyllama TinyLlama/TinyLlama-1.1B-Chat-v1.0
```
Now you can prompt Llama 2 70B using:
```bash
cat llm_nosrun.py | \
llm -m tinyllama -s 'explain this code'
```
## Development
To set up this plugin locally, first checkout the code. Then create a new virtual environment:
```bash
cd llm-nosrun
python3 -m venv venv
source venv/bin/activate
```
Now install the dependencies and test dependencies:
```bash
pip install -e '.[test]'
```
To run the tests:
```bash
pytest
```
Raw data
{
"_id": null,
"home_page": "",
"name": "llm-nosrun",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "Sudeep Pillai",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/13/d7/16b8635e612ab30d9f74e3613460c5cdbb54ef340052d5436a8381c47f46/llm-nosrun-0.2.tar.gz",
"platform": null,
"description": "# llm-nosrun\n\n[](https://pypi.org/project/llm-nosrun/)\n[](https://github.com/autonomi-ai/llm-nosrun/releases)\n[](https://github.com/autonomi-ai/llm-nosrun/actions?query=workflow%3ATest)\n[](https://github.com/autonomi-ai/llm-nosrun/blob/main/LICENSE)\n\n[LLM](https://llm.datasette.io/) plugin for models hosted by [NOS Cloud](https://app.nos.run/)\n\n## Installation\n\nFirst, [install the LLM command-line utility](https://llm.datasette.io/en/stable/setup.html).\n\nNow install this plugin in the same environment as LLM.\n```bash\nllm install llm-nosrun\n```\n## Configuration\n\nYou will need an API key from NOS Cloud. You can [obtain one here](https://app.nos.run/).\n\nYou can set that as an environment variable called `LLM_NOSRUN_KEY`, or add it to the `llm` set of saved keys using:\n\n```bash\nllm keys set nosrun\n```\n```\nEnter key: <paste key here>\n```\n\n## Usage\n\nTo list available models, run:\n```bash\nllm models list\n```\nYou should see a list that looks something like this:\n```\nNOSRun: TinyLlama/TinyLlama-1.1B-Chat-v1.0\nNOSRun: meta-llama/Llama-2-7b-chat-hf\nNOSRun: meta-llama/Llama-2-13b-chat-hf\nNOSRun: meta-llama/Llama-2-70b-chat-hf\nNOSRun: HuggingFaceH4/zephyr-7b-beta\nNOSRun: HuggingFaceH4/tiny-random-LlamaForCausalLM\nNOSRun: NousResearch/Yarn-Mistral-7b-128k\nNOSRun: mistralai/Mistral-7B-Instruct-v0.2\nNOSRun: mistralai/Mixtral-8x7B-Instruct-v0.1\nNOSRun: TheBloke/TinyLlama-1.1B-Chat-v1.0-AWQ\nNOSRun: TheBloke/Mixtral-8x7B-Instruct-v0.1-AWQ\nNOSRun: mlabonne/phixtral-2x2_8\nNOSRun: mlabonne/phixtral-4x2_8\n\n```\nTo run a prompt against a model, pass its full model ID to the `-m` option, like this:\n```bash\nllm -m TinyLlama/TinyLlama-1.1B-Chat-v1.0 \\\n 'Five strident names for a pet walrus' \\\n --system 'You love coming up with creative names for pets'\n```\nYou can set a shorter alias for a model using the `llm aliases` command like so:\n```bash\nllm aliases set tinyllama TinyLlama/TinyLlama-1.1B-Chat-v1.0\n```\nNow you can prompt Llama 2 70B using:\n```bash\ncat llm_nosrun.py | \\\n llm -m tinyllama -s 'explain this code'\n```\n## Development\n\nTo set up this plugin locally, first checkout the code. Then create a new virtual environment:\n```bash\ncd llm-nosrun\npython3 -m venv venv\nsource venv/bin/activate\n```\nNow install the dependencies and test dependencies:\n```bash\npip install -e '.[test]'\n```\nTo run the tests:\n```bash\npytest\n```\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "LLM plugin for models hosted by NOS Cloud",
"version": "0.2",
"project_urls": {
"CI": "https://github.com/autonomi-ai/llm-nosrun/actions",
"Changelog": "https://github.com/autonomi-ai/llm-nosrun/releases",
"Homepage": "https://github.com/autonomi-ai/llm-nosrun",
"Issues": "https://github.com/autonomi-ai/llm-nosrun/issues"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "42a0845737ed1c1d7e997aa6af6dd4084ab91fe214f84bb4fba85c72d6ae720b",
"md5": "e9df6717d90be77a271a29b0be6003fc",
"sha256": "e0522e94ecd68d61767e268348db4d25b40f47a40c5337443d995e1248bb0db4"
},
"downloads": -1,
"filename": "llm_nosrun-0.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e9df6717d90be77a271a29b0be6003fc",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 7330,
"upload_time": "2024-01-16T18:54:41",
"upload_time_iso_8601": "2024-01-16T18:54:41.905158Z",
"url": "https://files.pythonhosted.org/packages/42/a0/845737ed1c1d7e997aa6af6dd4084ab91fe214f84bb4fba85c72d6ae720b/llm_nosrun-0.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "13d716b8635e612ab30d9f74e3613460c5cdbb54ef340052d5436a8381c47f46",
"md5": "f48d3bee33e0bd6e0f30729efcd793f0",
"sha256": "a806308459e78b69d0443928057395a773572229f5a8d1e24f5345ca05dc15ec"
},
"downloads": -1,
"filename": "llm-nosrun-0.2.tar.gz",
"has_sig": false,
"md5_digest": "f48d3bee33e0bd6e0f30729efcd793f0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 7220,
"upload_time": "2024-01-16T18:54:43",
"upload_time_iso_8601": "2024-01-16T18:54:43.307463Z",
"url": "https://files.pythonhosted.org/packages/13/d7/16b8635e612ab30d9f74e3613460c5cdbb54ef340052d5436a8381c47f46/llm-nosrun-0.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-16 18:54:43",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "autonomi-ai",
"github_project": "llm-nosrun",
"github_not_found": true,
"lcname": "llm-nosrun"
}
