| Name | lmnr JSON |
| Version |
0.4.23
JSON |
| download |
| home_page | None |
| Summary | Python SDK for Laminar AI |
| upload_time | 2024-10-21 06:00:16 |
| maintainer | None |
| docs_url | None |
| author | lmnr.ai |
| requires_python | <4,>=3.9 |
| license | Apache-2.0 |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Laminar Python
Python SDK for [Laminar](https://www.lmnr.ai).
[Laminar](https://www.lmnr.ai) is an open-source platform for engineering LLM products. Trace, evaluate, annotate, and analyze LLM data. Bring LLM applications to production with confidence.
Check our [open-source repo](https://github.com/lmnr-ai/lmnr) and don't forget to star it ⭐
<a href="https://pypi.org/project/lmnr/">  </a>
## Quickstart
First, install the package:
```sh
pip install lmnr
```
And then in the code
```python
from lmnr import Laminar as L
L.initialize(project_api_key="<PROJECT_API_KEY>")
```
This will automatically instrument most of the LLM, Vector DB, and related
calls with OpenTelemetry-compatible instrumentation.
Note that you need to only initialize Laminar once in your application.
## Instrumentation
### Manual instrumentation
To instrument any function in your code, we provide a simple `@observe()` decorator.
This can be useful if you want to trace a request handler or a function which combines multiple LLM calls.
```python
import os
from openai import OpenAI
from lmnr import Laminar as L, Instruments
L.initialize(project_api_key=os.environ["LMNR_PROJECT_API_KEY"])
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
def poem_writer(topic: str):
prompt = f"write a poem about {topic}"
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": prompt},
]
# OpenAI calls are still automatically instrumented
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
)
poem = response.choices[0].message.content
return poem
@observe()
def generate_poems():
poem1 = poem_writer(topic="laminar flow")
L.event("is_poem_generated", True)
poem2 = poem_writer(topic="turbulence")
L.event("is_poem_generated", True)
poems = f"{poem1}\n\n---\n\n{poem2}"
return poems
```
Also, you can use `Laminar.start_as_current_span` if you want to record a chunk of your code using `with` statement.
```python
def handle_user_request(topic: str):
with L.start_as_current_span(name="poem_writer", input=topic):
...
poem = poem_writer(topic=topic)
...
# while within the span, you can attach laminar events to it
L.event("is_poem_generated", True)
# Use set_span_output to record the output of the span
L.set_span_output(poem)
```
### Automatic instrumentation
Laminar allows you to automatically instrument majority of the most popular LLM, Vector DB, database, requests, and other libraries.
If you want to automatically instrument a default set of libraries, then simply do NOT pass `instruments` argument to `.initialize()`.
See the full list of available instrumentations in the [enum](/src/lmnr/traceloop_sdk/instruments.py).
If you want to automatically instrument only specific LLM, Vector DB, or other
calls with OpenTelemetry-compatible instrumentation, then pass the appropriate instruments to `.initialize()`.
For example, if you want to only instrument OpenAI and Anthropic, then do the following:
```python
from lmnr import Laminar as L, Instruments
L.initialize(project_api_key=os.environ["LMNR_PROJECT_API_KEY"], instruments={Instruments.OPENAI, Instruments.ANTHROPIC})
```
If you want to fully disable any kind of autoinstrumentation, pass an empty set as `instruments=set()` to `.initialize()`.
Autoinstrumentations are provided by Traceloop's [OpenLLMetry](https://github.com/traceloop/openllmetry).
## Sending events
You can send laminar events using `L.event(name, value)`.
Read our [docs](https://docs.lmnr.ai) to learn more about events and examples.
### Example
```python
from lmnr import Laminar as L
# ...
poem = response.choices[0].message.content
# this will register True or False value with Laminar
L.event("topic alignment", topic in poem)
```
## Evaluations
### Quickstart
Install the package:
```sh
pip install lmnr
```
Create a file named `my_first_eval.py` with the following code:
```python
from lmnr import evaluate
def write_poem(data):
return f"This is a good poem about {data['topic']}"
def contains_poem(output, target):
return 1 if output in target['poem'] else 0
# Evaluation data
data = [
{"data": {"topic": "flowers"}, "target": {"poem": "This is a good poem about flowers"}},
{"data": {"topic": "cars"}, "target": {"poem": "I like cars"}},
]
evaluate(
data=data,
executor=write_poem,
evaluators={
"containsPoem": contains_poem
},
group_id="my_first_feature"
)
```
Run the following commands:
```sh
export LMNR_PROJECT_API_KEY=<YOUR_PROJECT_API_KEY> # get from Laminar project settings
lmnr eval my_first_eval.py # run in the virtual environment where lmnr is installed
```
Visit the URL printed in the console to see the results.
### Overview
Bring rigor to the development of your LLM applications with evaluations.
You can run evaluations locally by providing executor (part of the logic used in your application) and evaluators (numeric scoring functions) to `evaluate` function.
`evaluate` takes in the following parameters:
- `data` – an array of `EvaluationDatapoint` objects, where each `EvaluationDatapoint` has two keys: `target` and `data`, each containing a key-value object. Alternatively, you can pass in dictionaries, and we will instantiate `EvaluationDatapoint`s with pydantic if possible
- `executor` – the logic you want to evaluate. This function must take `data` as the first argument, and produce any output. It can be both a function or an `async` function.
- `evaluators` – Dictionary which maps evaluator names to evaluators. Functions that take output of executor as the first argument, `target` as the second argument and produce a numeric scores. Each function can produce either a single number or `dict[str, int|float]` of scores. Each evaluator can be both a function or an `async` function.
- `name` – optional name for the evaluation. Automatically generated if not provided.
\* If you already have the outputs of executors you want to evaluate, you can specify the executor as an identity function, that takes in `data` and returns only needed value(s) from it.
Read the [docs](https://docs.lmnr.ai/evaluations/introduction) to learn more about evaluations.
## Laminar pipelines as prompt chain managers
You can create Laminar pipelines in the UI and manage chains of LLM calls there.
After you are ready to use your pipeline in your code, deploy it in Laminar by selecting the target version for the pipeline.
Once your pipeline target is set, you can call it from Python in just a few lines.
Example use:
```python
from lmnr import Laminar as L
L.initialize('<YOUR_PROJECT_API_KEY>', instruments=set())
result = l.run(
pipeline = 'my_pipeline_name',
inputs = {'input_node_name': 'some_value'},
# all environment variables
env = {'OPENAI_API_KEY': 'sk-some-key'},
)
```
Resulting in:
```python
>>> result
PipelineRunResponse(
outputs={'output': {'value': [ChatMessage(role='user', content='hello')]}},
# useful to locate your trace
run_id='53b012d5-5759-48a6-a9c5-0011610e3669'
)
```
Raw data
{
"_id": null,
"home_page": null,
"name": "lmnr",
"maintainer": null,
"docs_url": null,
"requires_python": "<4,>=3.9",
"maintainer_email": null,
"keywords": null,
"author": "lmnr.ai",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/dc/22/318183febc2246676ad21796cc66907999d28a255d7ac546df22da98576a/lmnr-0.4.23.tar.gz",
"platform": null,
"description": "# Laminar Python\n\nPython SDK for [Laminar](https://www.lmnr.ai).\n\n[Laminar](https://www.lmnr.ai) is an open-source platform for engineering LLM products. Trace, evaluate, annotate, and analyze LLM data. Bring LLM applications to production with confidence.\n\nCheck our [open-source repo](https://github.com/lmnr-ai/lmnr) and don't forget to star it \u2b50\n\n <a href=\"https://pypi.org/project/lmnr/\">  </a>\n\n\n\n\n## Quickstart\n\nFirst, install the package:\n\n```sh\npip install lmnr\n```\n\nAnd then in the code\n\n```python\nfrom lmnr import Laminar as L\n\nL.initialize(project_api_key=\"<PROJECT_API_KEY>\")\n```\n\nThis will automatically instrument most of the LLM, Vector DB, and related\ncalls with OpenTelemetry-compatible instrumentation.\n\nNote that you need to only initialize Laminar once in your application.\n\n## Instrumentation\n\n### Manual instrumentation\n\nTo instrument any function in your code, we provide a simple `@observe()` decorator.\nThis can be useful if you want to trace a request handler or a function which combines multiple LLM calls.\n\n```python\nimport os\nfrom openai import OpenAI\nfrom lmnr import Laminar as L, Instruments\n\nL.initialize(project_api_key=os.environ[\"LMNR_PROJECT_API_KEY\"])\n\nclient = OpenAI(api_key=os.environ[\"OPENAI_API_KEY\"])\n\ndef poem_writer(topic: str):\n prompt = f\"write a poem about {topic}\"\n messages = [\n {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n {\"role\": \"user\", \"content\": prompt},\n ]\n\n # OpenAI calls are still automatically instrumented\n response = client.chat.completions.create(\n model=\"gpt-4o\",\n messages=messages,\n )\n poem = response.choices[0].message.content\n\n return poem\n\n@observe()\ndef generate_poems():\n poem1 = poem_writer(topic=\"laminar flow\")\n L.event(\"is_poem_generated\", True)\n poem2 = poem_writer(topic=\"turbulence\")\n L.event(\"is_poem_generated\", True)\n poems = f\"{poem1}\\n\\n---\\n\\n{poem2}\"\n return poems\n```\n\nAlso, you can use `Laminar.start_as_current_span` if you want to record a chunk of your code using `with` statement.\n\n```python\ndef handle_user_request(topic: str):\n with L.start_as_current_span(name=\"poem_writer\", input=topic):\n ...\n\n poem = poem_writer(topic=topic)\n \n ...\n \n # while within the span, you can attach laminar events to it\n L.event(\"is_poem_generated\", True)\n\n # Use set_span_output to record the output of the span\n L.set_span_output(poem)\n```\n\n### Automatic instrumentation\n\nLaminar allows you to automatically instrument majority of the most popular LLM, Vector DB, database, requests, and other libraries.\n\nIf you want to automatically instrument a default set of libraries, then simply do NOT pass `instruments` argument to `.initialize()`.\nSee the full list of available instrumentations in the [enum](/src/lmnr/traceloop_sdk/instruments.py).\n\nIf you want to automatically instrument only specific LLM, Vector DB, or other\ncalls with OpenTelemetry-compatible instrumentation, then pass the appropriate instruments to `.initialize()`.\nFor example, if you want to only instrument OpenAI and Anthropic, then do the following:\n\n```python\nfrom lmnr import Laminar as L, Instruments\n\nL.initialize(project_api_key=os.environ[\"LMNR_PROJECT_API_KEY\"], instruments={Instruments.OPENAI, Instruments.ANTHROPIC})\n```\n\nIf you want to fully disable any kind of autoinstrumentation, pass an empty set as `instruments=set()` to `.initialize()`. \n\nAutoinstrumentations are provided by Traceloop's [OpenLLMetry](https://github.com/traceloop/openllmetry).\n\n## Sending events\n\nYou can send laminar events using `L.event(name, value)`.\n\nRead our [docs](https://docs.lmnr.ai) to learn more about events and examples.\n\n### Example\n\n```python\nfrom lmnr import Laminar as L\n# ...\npoem = response.choices[0].message.content\n\n# this will register True or False value with Laminar\nL.event(\"topic alignment\", topic in poem)\n\n```\n\n## Evaluations\n\n### Quickstart\n\nInstall the package:\n\n```sh\npip install lmnr\n```\n\nCreate a file named `my_first_eval.py` with the following code:\n\n```python\nfrom lmnr import evaluate\n\ndef write_poem(data):\n return f\"This is a good poem about {data['topic']}\"\n\ndef contains_poem(output, target):\n return 1 if output in target['poem'] else 0\n\n# Evaluation data\ndata = [\n {\"data\": {\"topic\": \"flowers\"}, \"target\": {\"poem\": \"This is a good poem about flowers\"}},\n {\"data\": {\"topic\": \"cars\"}, \"target\": {\"poem\": \"I like cars\"}},\n]\n\nevaluate(\n data=data,\n executor=write_poem,\n evaluators={\n \"containsPoem\": contains_poem\n },\n group_id=\"my_first_feature\"\n)\n```\n\nRun the following commands:\n\n```sh\nexport LMNR_PROJECT_API_KEY=<YOUR_PROJECT_API_KEY> # get from Laminar project settings\nlmnr eval my_first_eval.py # run in the virtual environment where lmnr is installed\n```\n\nVisit the URL printed in the console to see the results.\n\n### Overview\n\nBring rigor to the development of your LLM applications with evaluations.\n\nYou can run evaluations locally by providing executor (part of the logic used in your application) and evaluators (numeric scoring functions) to `evaluate` function.\n\n`evaluate` takes in the following parameters:\n- `data` \u2013 an array of `EvaluationDatapoint` objects, where each `EvaluationDatapoint` has two keys: `target` and `data`, each containing a key-value object. Alternatively, you can pass in dictionaries, and we will instantiate `EvaluationDatapoint`s with pydantic if possible\n- `executor` \u2013 the logic you want to evaluate. This function must take `data` as the first argument, and produce any output. It can be both a function or an `async` function.\n- `evaluators` \u2013 Dictionary which maps evaluator names to evaluators. Functions that take output of executor as the first argument, `target` as the second argument and produce a numeric scores. Each function can produce either a single number or `dict[str, int|float]` of scores. Each evaluator can be both a function or an `async` function.\n- `name` \u2013 optional name for the evaluation. Automatically generated if not provided.\n\n\\* If you already have the outputs of executors you want to evaluate, you can specify the executor as an identity function, that takes in `data` and returns only needed value(s) from it.\n\nRead the [docs](https://docs.lmnr.ai/evaluations/introduction) to learn more about evaluations.\n\n## Laminar pipelines as prompt chain managers\n\nYou can create Laminar pipelines in the UI and manage chains of LLM calls there.\n\nAfter you are ready to use your pipeline in your code, deploy it in Laminar by selecting the target version for the pipeline.\n\nOnce your pipeline target is set, you can call it from Python in just a few lines.\n\nExample use:\n\n```python\nfrom lmnr import Laminar as L\n\nL.initialize('<YOUR_PROJECT_API_KEY>', instruments=set())\n\nresult = l.run(\n pipeline = 'my_pipeline_name',\n inputs = {'input_node_name': 'some_value'},\n # all environment variables\n env = {'OPENAI_API_KEY': 'sk-some-key'},\n)\n```\n\nResulting in:\n\n```python\n>>> result\nPipelineRunResponse(\n outputs={'output': {'value': [ChatMessage(role='user', content='hello')]}},\n # useful to locate your trace\n run_id='53b012d5-5759-48a6-a9c5-0011610e3669'\n)\n```\n\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Python SDK for Laminar AI",
"version": "0.4.23",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b02dde2416519f8b805c4e7f12849fe2f94d1a65257f2f96e824afa626301c35",
"md5": "47861f184654d13861d2ac5a273d56a0",
"sha256": "6387f2d05a76149f7f355a42b9ea4ad7736a313a875e2a718271a703b1b1b446"
},
"downloads": -1,
"filename": "lmnr-0.4.23-py3-none-any.whl",
"has_sig": false,
"md5_digest": "47861f184654d13861d2ac5a273d56a0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4,>=3.9",
"size": 65822,
"upload_time": "2024-10-21T06:00:14",
"upload_time_iso_8601": "2024-10-21T06:00:14.234500Z",
"url": "https://files.pythonhosted.org/packages/b0/2d/de2416519f8b805c4e7f12849fe2f94d1a65257f2f96e824afa626301c35/lmnr-0.4.23-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "dc22318183febc2246676ad21796cc66907999d28a255d7ac546df22da98576a",
"md5": "1f51368e2669c81b925d5f7a357b04a6",
"sha256": "708114aaeae508ed365da414a4d86ec80c93ef24a81ce9158b7afac539f65f98"
},
"downloads": -1,
"filename": "lmnr-0.4.23.tar.gz",
"has_sig": false,
"md5_digest": "1f51368e2669c81b925d5f7a357b04a6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4,>=3.9",
"size": 47476,
"upload_time": "2024-10-21T06:00:16",
"upload_time_iso_8601": "2024-10-21T06:00:16.070758Z",
"url": "https://files.pythonhosted.org/packages/dc/22/318183febc2246676ad21796cc66907999d28a255d7ac546df22da98576a/lmnr-0.4.23.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-21 06:00:16",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "lmnr"
}
