| Name | llamatry JSON |
| Version |
0.1.8
JSON |
| download |
| home_page | https://github.com/jxnl/llamatry |
| Summary | opentelemetry instrumentation for openai's completions api |
| upload_time | 2023-05-09 21:46:33 |
| maintainer | |
| docs_url | None |
| author | Jason |
| requires_python | >=3.9,<4.0 |
| license | |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# [Llamatry](https://github.com/jxnl/llamatry)
Llamatry is a Python package that simplifies the process of instrumenting the OpenAI API using OpenTelemetry. It allows you to monitor and trace the interactions with the OpenAI API, providing insights into the performance and behavior of your code. By leveraging OpenTelemetry, Llamatry supports various output formats, making it easy to integrate with your existing observability stack.
## Why?
Observability is essential for complex applications using large language models (LLMs), as it provides transparency, performance insights, and control over your data and even costs. By integrating observability into your LLM tooling, you can better understand their inner workings, optimize resource usage, and streamline your workflow. Owning your data and leveraging observability empowers you to take control of your AI application's performance.
## Features
* OpenTelemetry instrumentation for OpenAI API
* Supports tracing and monitoring of OpenAI API interactions
* Compatible with a wide range of output formats through OpenTelemetry
* Easy-to-use and straightforward setup process
## Installation
Install Llamatry using pip:
```bash
pip install llamatry
```
## Usage
To use Llamatry with the OpenAI API, follow these steps:
Import the necessary packages:
```python
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor
from llamatry import OpenAIInstrumentor
import openai
import os
```
Set up open telemetry:
```python
trace.set_tracer_provider(TracerProvider())
console_exporter = ConsoleSpanExporter()
span_processor = SimpleSpanProcessor(console_exporter)
trace.get_tracer_provider().add_span_processor(span_processor)
```
Set up OpenAI API:
```python
openai.api_key = os.environ["OPENAI_API_KEY"]
```
Instrument the OpenAI API using Llamatry:
```python
OpenAIInstrumentor().instrument()
```
Make API calls to the OpenAI API:
```python
response = openai.ChatCompletion.create(
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Who won the world series in 2020?"},
],
max_tokens=50,
temperature=0.5,
)
```
Console Export:
```
{
"name": "ChatCompletion.create",
"context": {
"trace_id": "0x6026b10ff364a1954df343ac2e292fd7",
"span_id": "0x3f04991076717d88",
"trace_state": "[]"
},
"kind": "SpanKind.INTERNAL",
"parent_id": "0x14abcdaf2d49f177",
"start_time": "2023-05-09T20:32:10.674716Z",
"end_time": "2023-05-09T20:32:11.827791Z",
"status": {
"status_code": "UNSET"
},
"attributes": {
"openai.create.model": "gpt-3.5-turbo",
"openai.create.max_tokens": 500,
"openai.create.temperature": 0.5,
"openai.response.id": "chatcmpl-7EOIE4fVofq83WPl1HFNkDI6yBPZ8",
"openai.response.created": 1683664330,
"openai.response.model": "gpt-3.5-turbo-0301",
"openai.usage.completion_tokens": 13,
"openai.usage.prompt_tokens": 30,
"openai.usage.total_tokens": 43
},
"events": [],
"links": [],
"resource": {
"attributes": {
"telemetry.sdk.language": "python",
"telemetry.sdk.name": "opentelemetry",
"telemetry.sdk.version": "1.17.0",
"service.name": "llamatry"
},
"schema_url": ""
}
}
```
Traces and other information related to the OpenAI API calls will be output to the console. By using Llamatry, you can easily switch to other exporters supported by OpenTelemetry, such as Jaeger or Zipkin, to visualize and analyze the data in different ways.
## Decorator and Context Manager helpers
Llamatry provides a convenient tracing utility with both decorator and context manager support. This allows you to trace your functions and code blocks easily using the provided Trace class.
### Using the Trace decorator
To use the `Trace` class as a decorator, you can decorate your function using `@Trace.trace`. The function's name will be used as the span name by default. If you want to set a custom span name, you can provide it as an argument: `@Trace.trace("custom_span_name")`. By default if you decorate a function all arguments that are `(str, int, float, bool)` will be set as attributes.
```python
from llamatry import Trace
@Trace.trace
def your_function(a, b):
# Your function implementation
# a, b and automatically set as span attributes
pass
@Trace.trace("custom_span_name")
def another_function():
# Your function implementation
pass
with Trace.span("custom_span_name") as span:
# Your code block here
span.set_attribute("foo", "bar")
pass
```
### Using the Trace context manager
To use the Trace class as a context manager, use the `with` statement followed by `Trace.span("custom_span_name")`.
```python
from llamatry import Trace
with Trace.span("custom_span_name") as span:
# Your code block here
span.set_attribute("foo", "bar")
pass
```
Using the `Trace` class in this way allows you to easily trace your functions and code blocks, providing better observability and understanding of the performance and behavior of your code.
## Documentation
For more information about OpenTelemetry, visit the [official OpenTelemetry Python documentation](https://opentelemetry-python.readthedocs.io/en/stable/).
For more information about the OpenAI API, visit the [official OpenAI API documentation](https://beta.openai.com/docs/).
## License
Llamatry is released under the [MIT License](https://opensource.org/licenses/MIT).
Raw data
{
"_id": null,
"home_page": "https://github.com/jxnl/llamatry",
"name": "llamatry",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9,<4.0",
"maintainer_email": "",
"keywords": "",
"author": "Jason",
"author_email": "jason@jxnl.co",
"download_url": "https://files.pythonhosted.org/packages/7e/d8/1ff5e36bf6e41250206095003bcb7b00780f96b57a322423cee340f5881a/llamatry-0.1.8.tar.gz",
"platform": null,
"description": "# [Llamatry](https://github.com/jxnl/llamatry)\n\nLlamatry is a Python package that simplifies the process of instrumenting the OpenAI API using OpenTelemetry. It allows you to monitor and trace the interactions with the OpenAI API, providing insights into the performance and behavior of your code. By leveraging OpenTelemetry, Llamatry supports various output formats, making it easy to integrate with your existing observability stack.\n\n## Why?\n\nObservability is essential for complex applications using large language models (LLMs), as it provides transparency, performance insights, and control over your data and even costs. By integrating observability into your LLM tooling, you can better understand their inner workings, optimize resource usage, and streamline your workflow. Owning your data and leveraging observability empowers you to take control of your AI application's performance.\n\n## Features\n\n* OpenTelemetry instrumentation for OpenAI API\n* Supports tracing and monitoring of OpenAI API interactions\n* Compatible with a wide range of output formats through OpenTelemetry\n* Easy-to-use and straightforward setup process\n\n## Installation\n\nInstall Llamatry using pip:\n\n```bash\npip install llamatry\n```\n\n## Usage\n\nTo use Llamatry with the OpenAI API, follow these steps:\n\nImport the necessary packages:\n\n```python\nfrom opentelemetry import trace\nfrom opentelemetry.sdk.trace import TracerProvider\nfrom opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor\n\nfrom llamatry import OpenAIInstrumentor\n\nimport openai\nimport os\n```\n\nSet up open telemetry:\n\n```python\ntrace.set_tracer_provider(TracerProvider())\nconsole_exporter = ConsoleSpanExporter()\nspan_processor = SimpleSpanProcessor(console_exporter)\ntrace.get_tracer_provider().add_span_processor(span_processor)\n```\n\nSet up OpenAI API:\n\n```python\nopenai.api_key = os.environ[\"OPENAI_API_KEY\"]\n```\n\nInstrument the OpenAI API using Llamatry:\n\n```python\nOpenAIInstrumentor().instrument()\n```\n\nMake API calls to the OpenAI API:\n\n```python\nresponse = openai.ChatCompletion.create(\n messages=[\n {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n {\"role\": \"user\", \"content\": \"Who won the world series in 2020?\"},\n ],\n max_tokens=50,\n temperature=0.5,\n)\n```\n\nConsole Export:\n\n```\n{\n \"name\": \"ChatCompletion.create\",\n \"context\": {\n \"trace_id\": \"0x6026b10ff364a1954df343ac2e292fd7\",\n \"span_id\": \"0x3f04991076717d88\",\n \"trace_state\": \"[]\"\n },\n \"kind\": \"SpanKind.INTERNAL\",\n \"parent_id\": \"0x14abcdaf2d49f177\",\n \"start_time\": \"2023-05-09T20:32:10.674716Z\",\n \"end_time\": \"2023-05-09T20:32:11.827791Z\",\n \"status\": {\n \"status_code\": \"UNSET\"\n },\n \"attributes\": {\n \"openai.create.model\": \"gpt-3.5-turbo\",\n \"openai.create.max_tokens\": 500,\n \"openai.create.temperature\": 0.5,\n \"openai.response.id\": \"chatcmpl-7EOIE4fVofq83WPl1HFNkDI6yBPZ8\",\n \"openai.response.created\": 1683664330,\n \"openai.response.model\": \"gpt-3.5-turbo-0301\",\n \"openai.usage.completion_tokens\": 13,\n \"openai.usage.prompt_tokens\": 30,\n \"openai.usage.total_tokens\": 43\n },\n \"events\": [],\n \"links\": [],\n \"resource\": {\n \"attributes\": {\n \"telemetry.sdk.language\": \"python\",\n \"telemetry.sdk.name\": \"opentelemetry\",\n \"telemetry.sdk.version\": \"1.17.0\",\n \"service.name\": \"llamatry\"\n },\n \"schema_url\": \"\"\n }\n}\n```\n\nTraces and other information related to the OpenAI API calls will be output to the console. By using Llamatry, you can easily switch to other exporters supported by OpenTelemetry, such as Jaeger or Zipkin, to visualize and analyze the data in different ways.\n\n## Decorator and Context Manager helpers\n\nLlamatry provides a convenient tracing utility with both decorator and context manager support. This allows you to trace your functions and code blocks easily using the provided Trace class.\n\n### Using the Trace decorator\n\nTo use the `Trace` class as a decorator, you can decorate your function using `@Trace.trace`. The function's name will be used as the span name by default. If you want to set a custom span name, you can provide it as an argument: `@Trace.trace(\"custom_span_name\")`. By default if you decorate a function all arguments that are `(str, int, float, bool)` will be set as attributes.\n\n```python\nfrom llamatry import Trace\n\n@Trace.trace\ndef your_function(a, b):\n # Your function implementation\n # a, b and automatically set as span attributes\n pass\n\n@Trace.trace(\"custom_span_name\")\ndef another_function():\n # Your function implementation\n pass\n\nwith Trace.span(\"custom_span_name\") as span:\n # Your code block here\n span.set_attribute(\"foo\", \"bar\")\n pass\n```\n\n### Using the Trace context manager\n\nTo use the Trace class as a context manager, use the `with` statement followed by `Trace.span(\"custom_span_name\")`.\n\n```python\nfrom llamatry import Trace\n\nwith Trace.span(\"custom_span_name\") as span:\n # Your code block here\n span.set_attribute(\"foo\", \"bar\")\n pass\n```\n\nUsing the `Trace` class in this way allows you to easily trace your functions and code blocks, providing better observability and understanding of the performance and behavior of your code.\n\n## Documentation\n\nFor more information about OpenTelemetry, visit the [official OpenTelemetry Python documentation](https://opentelemetry-python.readthedocs.io/en/stable/).\n\nFor more information about the OpenAI API, visit the [official OpenAI API documentation](https://beta.openai.com/docs/).\n\n## License\n\nLlamatry is released under the [MIT License](https://opensource.org/licenses/MIT).\n",
"bugtrack_url": null,
"license": "",
"summary": "opentelemetry instrumentation for openai's completions api",
"version": "0.1.8",
"project_urls": {
"Homepage": "https://github.com/jxnl/llamatry",
"Repository": "https://github.com/jxnl/llamatry"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "69b195166b89d40d5a06542da29eba46efb05dbff86dfcf132c7b8773abb3c8b",
"md5": "bc28be256650b4bdd4e5bb6e29784109",
"sha256": "71fd6f61bc54c9c747b56c736995451d35bc7addd38f5d87b116d5feac73fb4a"
},
"downloads": -1,
"filename": "llamatry-0.1.8-py3-none-any.whl",
"has_sig": false,
"md5_digest": "bc28be256650b4bdd4e5bb6e29784109",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9,<4.0",
"size": 5762,
"upload_time": "2023-05-09T21:46:31",
"upload_time_iso_8601": "2023-05-09T21:46:31.399899Z",
"url": "https://files.pythonhosted.org/packages/69/b1/95166b89d40d5a06542da29eba46efb05dbff86dfcf132c7b8773abb3c8b/llamatry-0.1.8-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7ed81ff5e36bf6e41250206095003bcb7b00780f96b57a322423cee340f5881a",
"md5": "3298f521ccb9623b2ba8aaa8024c5c59",
"sha256": "748db25b5b57e8b2e3f609d08a785061be4d20e4254192ddd3e9eb16f5d71c4a"
},
"downloads": -1,
"filename": "llamatry-0.1.8.tar.gz",
"has_sig": false,
"md5_digest": "3298f521ccb9623b2ba8aaa8024c5c59",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9,<4.0",
"size": 5018,
"upload_time": "2023-05-09T21:46:33",
"upload_time_iso_8601": "2023-05-09T21:46:33.243278Z",
"url": "https://files.pythonhosted.org/packages/7e/d8/1ff5e36bf6e41250206095003bcb7b00780f96b57a322423cee340f5881a/llamatry-0.1.8.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-05-09 21:46:33",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "jxnl",
"github_project": "llamatry",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "llamatry"
}
