scale-llm-engine

Name	scale-llm-engine JSON
Version	0.0.0b45 JSON
	download
home_page	None
Summary	Scale LLM Engine Python client
upload_time	2025-01-14 21:55:06
maintainer	Phil Chen
docs_url	None
author	Phil Chen
requires_python	<4.0,>=3.8
license	Apache-2.0
keywords
VCS
bugtrack_url
requirements	No requirements were recorded.
Travis-CI	No Travis.
coveralls test coverage	No coveralls.

            # LLM Engine

The LLM Engine Python library provides a convenient way of interfacing with a
`llmengine` endpoint running on
[LLM Engine](https://scaleapi.github.io/llm-engine/) or on your own infrastructure.

## Get Started

### Install

```shell
pip install scale-llm-engine
```

### Usage

If you are using LLM Engine, you can get your API key from
[https://spellbook.scale.com/settings](https://spellbook.scale.com/settings).
Set the `SCALE_API_KEY` environment variable to your API key.

If you are using your own infrastructure, you can set the
`LLM_ENGINE_BASE_PATH` environment variable to the base URL of your
self-hosted `llmengine` endpoint.

```python
from llmengine import Completion

response = Completion.create(
    model="llama-2-7b",
    prompt="Hello, my name is",
    max_new_tokens=10,
    temperature=0.2,
)
print(response.outputs[0].text)
```

## Documentation

Documentation is available at
[https://scaleapi.github.io/llm-engine/](https://scaleapi.github.io/llm-engine/).


            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "scale-llm-engine",
    "maintainer": "Phil Chen",
    "docs_url": null,
    "requires_python": "<4.0,>=3.8",
    "maintainer_email": "phil.chen@scale.com",
    "keywords": null,
    "author": "Phil Chen",
    "author_email": "phil.chen@scale.com",
    "download_url": "https://files.pythonhosted.org/packages/e8/dc/901c7517c68c1de1f8a2b3b9584175f852048888ad6e7c6a07ae53aeb2fb/scale_llm_engine-0.0.0b45.tar.gz",
    "platform": null,
    "description": "# LLM Engine\n\nThe LLM Engine Python library provides a convenient way of interfacing with a\n`llmengine` endpoint running on\n[LLM Engine](https://scaleapi.github.io/llm-engine/) or on your own infrastructure.\n\n## Get Started\n\n### Install\n\n```shell\npip install scale-llm-engine\n```\n\n### Usage\n\nIf you are using LLM Engine, you can get your API key from\n[https://spellbook.scale.com/settings](https://spellbook.scale.com/settings).\nSet the `SCALE_API_KEY` environment variable to your API key.\n\nIf you are using your own infrastructure, you can set the\n`LLM_ENGINE_BASE_PATH` environment variable to the base URL of your\nself-hosted `llmengine` endpoint.\n\n```python\nfrom llmengine import Completion\n\nresponse = Completion.create(\n    model=\"llama-2-7b\",\n    prompt=\"Hello, my name is\",\n    max_new_tokens=10,\n    temperature=0.2,\n)\nprint(response.outputs[0].text)\n```\n\n## Documentation\n\nDocumentation is available at\n[https://scaleapi.github.io/llm-engine/](https://scaleapi.github.io/llm-engine/).\n\n",
    "bugtrack_url": null,
    "license": "Apache-2.0",
    "summary": "Scale LLM Engine Python client",
    "version": "0.0.0b45",
    "project_urls": {
        "Homepage": "https://scaleapi.github.io/llm-engine/",
        "Repository": "https://github.com/scaleapi/llm-engine"
    },
    "split_keywords": [],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "4786297013a4a3ae7a54d572d294c2cd47b27937dc5006ce6c1c4d11c6ba7f3e",
                "md5": "3c795f8cc87a0d02acac08e0c6c36d05",
                "sha256": "e3f0148e37dabd9665b49e15eab9a4798c0e3c7809fbc805c2fe1bfdc92319e9"
            },
            "downloads": -1,
            "filename": "scale_llm_engine-0.0.0b45-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "3c795f8cc87a0d02acac08e0c6c36d05",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": "<4.0,>=3.8",
            "size": 73873,
            "upload_time": "2025-01-14T21:55:04",
            "upload_time_iso_8601": "2025-01-14T21:55:04.790586Z",
            "url": "https://files.pythonhosted.org/packages/47/86/297013a4a3ae7a54d572d294c2cd47b27937dc5006ce6c1c4d11c6ba7f3e/scale_llm_engine-0.0.0b45-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "e8dc901c7517c68c1de1f8a2b3b9584175f852048888ad6e7c6a07ae53aeb2fb",
                "md5": "2f8cd24ee4fc20ad07753ccaf61e9558",
                "sha256": "c197de005152eef46c8867401c365c0a9c3221808e6fe048083c53b3ff065639"
            },
            "downloads": -1,
            "filename": "scale_llm_engine-0.0.0b45.tar.gz",
            "has_sig": false,
            "md5_digest": "2f8cd24ee4fc20ad07753ccaf61e9558",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": "<4.0,>=3.8",
            "size": 64359,
            "upload_time": "2025-01-14T21:55:06",
            "upload_time_iso_8601": "2025-01-14T21:55:06.004095Z",
            "url": "https://files.pythonhosted.org/packages/e8/dc/901c7517c68c1de1f8a2b3b9584175f852048888ad6e7c6a07ae53aeb2fb/scale_llm_engine-0.0.0b45.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-01-14 21:55:06",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "scaleapi",
    "github_project": "llm-engine",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "circle": true,
    "lcname": "scale-llm-engine"
}