| Name | otlp-stdout-adapter JSON |
| Version |
0.1.0
JSON |
| download |
| home_page | None |
| Summary | A custom requests HTTPAdapter that serializes spans to stdout for OpenTelemetry OTLP exporters |
| upload_time | 2024-11-06 03:13:08 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.12 |
| license | MIT License Copyright (c) 2024 Alessandro Bologna Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
opentelemetry
otlp
stdout
lambda
aws
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# OTLP Stdout Adapter
The `otlp-stdout-adapter` library is designed to export OpenTelemetry data to stdout in a formatted JSON structure, suitable for serverless environments like AWS Lambda. It implements a custom HTTP adapter that can be used with OpenTelemetry OTLP exporters to send telemetry data to stdout.
By outputting telemetry data to stdout, this library enables seamless integration with log management systems in serverless environments. For instance, in AWS Lambda, CloudWatch Logs can capture this output, allowing tools like the [lambda-otlp-forwarder](https://github.com/dev7a/lambda-otlp-forwarder) to efficiently collect and forward the data to centralized OpenTelemetry collectors.
>[!IMPORTANT]
>This package is highly experimental and should not be used in production. Contributions are welcome.
## Features
- Implements a custom HTTP adapter for use in OpenTelemetry OTLP pipelines
- Exports OpenTelemetry data to stdout in a structured JSON format
- Designed for serverless environments, especially AWS Lambda
- Configurable through environment variables
- Support for GZIP compression
- Base64 encoding for binary payloads
## Installation
You can install the `otlp-stdout-adapter` using pip:
```bash
pip install otlp-stdout-adapter
```
## Usage
Here's a basic example of how to use the library with OpenTelemetry:
```python
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from otlp_stdout_adapter import StdoutAdapter, get_lambda_resource
# Initialize the StdoutAdapter
adapter = StdoutAdapter()
session = adapter.get_session()
# Create OTLP exporter with custom session
exporter = OTLPSpanExporter(
endpoint="http://your-collector:4318/v1/traces",
session=session
)
# Set up the trace provider
provider = TracerProvider(resource=get_lambda_resource())
processor = BatchSpanProcessor(exporter)
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)
# Get a tracer
tracer = trace.get_tracer(__name__)
# Create spans as usual
with tracer.start_as_current_span("example_span") as span:
span.set_attribute("example.attribute", "value")
```
## Environment Variables
The adapter can be configured using standard OpenTelemetry environment variables:
- `OTEL_SERVICE_NAME`: Sets the service name for the logs
- `AWS_LAMBDA_FUNCTION_NAME`: Used as fallback for service name if OTEL_SERVICE_NAME is not set
- `OTEL_EXPORTER_OTLP_ENDPOINT`: Sets the endpoint for the OTLP collector
- `OTEL_EXPORTER_OTLP_HEADERS`: Sets additional headers for the OTLP exporter
- `OTEL_EXPORTER_OTLP_COMPRESSION`: Specifies the compression algorithm (only gzip is supported)
- `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL`: Specifies the protocol (http/protobuf or http/json)*
[!IMPORTANT]: While the OpenTelemetry specification supports both JSON and Protobuf over HTTP,
the Python SDK currently only supports Protobuf (see [opentelemetry-python#1003](https://github.com/open-telemetry/opentelemetry-python/issues/1003)).
The environment variable is recognized but JSON format is not yet implemented.
All exports will use application/x-protobuf content-type.
## Output Format
Each log record includes:
- `__otel_otlp_stdout`: Record marker and version identifier for the adapter (format: package-name@version)
- `source`: The service name from configuration
- `endpoint`: The configured OTLP endpoint
- `method`: The HTTP method (always "POST")
- `content-type`: The content type (currently always "application/x-protobuf")
- `payload`: The OTLP data (base64 encoded if compressed)
- `base64`: Boolean indicating if the payload is base64 encoded
Example output:
```json
{
"__otel_otlp_stdout": "otlp-stdout-adapter@0.1.3",
"source": "my-lambda-function",
"headers": {
"content-type": "application/x-protobuf"
},
"endpoint": "http://collector:4318/v1/traces",
"method": "POST",
"content-type": "application/x-protobuf",
"payload": "base64-encoded-content",
"base64": true
}
```
## AWS Lambda Usage
Here's a complete example of using the adapter in an AWS Lambda function with distributed tracing:
```python
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator
from otlp_stdout_adapter import StdoutAdapter, get_lambda_resource
# Initialize once per Lambda container
adapter = StdoutAdapter()
session = adapter.get_session()
# Set up the trace provider with Lambda resource detection
provider = TracerProvider(resource=get_lambda_resource())
exporter = OTLPSpanExporter(
endpoint="http://your-collector:4318/v1/traces",
session=session
)
processor = BatchSpanProcessor(exporter)
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)
# Get a tracer
tracer = trace.get_tracer(__name__)
def process_event(event):
with tracer.start_as_current_span("process_event") as span:
# Your processing logic here
span.set_attribute("event.type", event.get("type"))
return {"processed": True}
def lambda_handler(event, context):
with tracer.start_as_current_span(
"lambda_handler",
kind=trace.SpanKind.SERVER
) as span:
try:
# Add event information to span
span.set_attribute("event.type", event.get("type"))
span.add_event("Processing Lambda event")
result = process_event(event)
return {
"statusCode": 200,
"body": json.dumps(result)
}
except Exception as e:
span.record_exception(e)
span.set_status(trace.Status(trace.StatusCode.ERROR))
raise
finally:
# Ensure spans are exported before Lambda freezes
provider.force_flush()
```
### Lambda Configuration Notes
- Initialize tracing components outside the handler for container reuse
- Always call `provider.force_flush()` before the handler completes
- Use environment variables for configuration:
```bash
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_EXPORTER_OTLP_COMPRESSION=gzip
```
## Best Practices
1. **Initialize Outside Handler**
- Create providers, exporters, and tracers outside the handler
- Reuse these components across invocations
2. **Resource Detection**
- Use `get_lambda_resource()` to automatically capture Lambda metadata
- Merge with custom resources as needed
3. **Span Processing**
- Use `BatchSpanProcessor` for efficient span processing
- Always flush spans before handler completion
- Configure appropriate batch size and timeout for Lambda
4. **Error Handling**
- Record exceptions using `span.record_exception()`
- Set appropriate span status
- Ensure spans are ended in finally blocks
5. **Context Propagation**
- Use TraceContext propagator for distributed tracing
- Propagate context in outgoing requests
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
Raw data
{
"_id": null,
"home_page": null,
"name": "otlp-stdout-adapter",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.12",
"maintainer_email": null,
"keywords": "opentelemetry, otlp, stdout, lambda, aws",
"author": null,
"author_email": "Alessandro Bologna <alessandro.bologna@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/cb/8f/45cf8d7fd11022fe07b6c14cccf2f707cb292178f523332e9f1b82965459/otlp_stdout_adapter-0.1.0.tar.gz",
"platform": null,
"description": "# OTLP Stdout Adapter\n\nThe `otlp-stdout-adapter` library is designed to export OpenTelemetry data to stdout in a formatted JSON structure, suitable for serverless environments like AWS Lambda. It implements a custom HTTP adapter that can be used with OpenTelemetry OTLP exporters to send telemetry data to stdout.\n\nBy outputting telemetry data to stdout, this library enables seamless integration with log management systems in serverless environments. For instance, in AWS Lambda, CloudWatch Logs can capture this output, allowing tools like the [lambda-otlp-forwarder](https://github.com/dev7a/lambda-otlp-forwarder) to efficiently collect and forward the data to centralized OpenTelemetry collectors.\n\n>[!IMPORTANT]\n>This package is highly experimental and should not be used in production. Contributions are welcome.\n\n## Features\n\n- Implements a custom HTTP adapter for use in OpenTelemetry OTLP pipelines\n- Exports OpenTelemetry data to stdout in a structured JSON format\n- Designed for serverless environments, especially AWS Lambda\n- Configurable through environment variables\n- Support for GZIP compression\n- Base64 encoding for binary payloads\n\n## Installation\n\nYou can install the `otlp-stdout-adapter` using pip:\n\n```bash\npip install otlp-stdout-adapter\n```\n\n## Usage\n\nHere's a basic example of how to use the library with OpenTelemetry:\n\n```python\nfrom opentelemetry import trace\nfrom opentelemetry.sdk.trace import TracerProvider\nfrom opentelemetry.sdk.trace.export import BatchSpanProcessor\nfrom opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter\nfrom otlp_stdout_adapter import StdoutAdapter, get_lambda_resource\n\n# Initialize the StdoutAdapter\nadapter = StdoutAdapter()\nsession = adapter.get_session()\n\n# Create OTLP exporter with custom session\nexporter = OTLPSpanExporter(\n endpoint=\"http://your-collector:4318/v1/traces\",\n session=session\n)\n\n# Set up the trace provider\nprovider = TracerProvider(resource=get_lambda_resource())\nprocessor = BatchSpanProcessor(exporter)\nprovider.add_span_processor(processor)\ntrace.set_tracer_provider(provider)\n\n# Get a tracer\ntracer = trace.get_tracer(__name__)\n\n# Create spans as usual\nwith tracer.start_as_current_span(\"example_span\") as span:\n span.set_attribute(\"example.attribute\", \"value\")\n```\n\n## Environment Variables\n\nThe adapter can be configured using standard OpenTelemetry environment variables:\n\n- `OTEL_SERVICE_NAME`: Sets the service name for the logs\n- `AWS_LAMBDA_FUNCTION_NAME`: Used as fallback for service name if OTEL_SERVICE_NAME is not set\n- `OTEL_EXPORTER_OTLP_ENDPOINT`: Sets the endpoint for the OTLP collector\n- `OTEL_EXPORTER_OTLP_HEADERS`: Sets additional headers for the OTLP exporter\n- `OTEL_EXPORTER_OTLP_COMPRESSION`: Specifies the compression algorithm (only gzip is supported)\n- `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL`: Specifies the protocol (http/protobuf or http/json)*\n\n[!IMPORTANT]: While the OpenTelemetry specification supports both JSON and Protobuf over HTTP,\nthe Python SDK currently only supports Protobuf (see [opentelemetry-python#1003](https://github.com/open-telemetry/opentelemetry-python/issues/1003)). \nThe environment variable is recognized but JSON format is not yet implemented. \nAll exports will use application/x-protobuf content-type.\n\n## Output Format\n\nEach log record includes:\n- `__otel_otlp_stdout`: Record marker and version identifier for the adapter (format: package-name@version)\n- `source`: The service name from configuration\n- `endpoint`: The configured OTLP endpoint\n- `method`: The HTTP method (always \"POST\")\n- `content-type`: The content type (currently always \"application/x-protobuf\")\n- `payload`: The OTLP data (base64 encoded if compressed)\n- `base64`: Boolean indicating if the payload is base64 encoded\n\nExample output:\n```json\n{\n \"__otel_otlp_stdout\": \"otlp-stdout-adapter@0.1.3\",\n \"source\": \"my-lambda-function\",\n \"headers\": {\n \"content-type\": \"application/x-protobuf\"\n },\n \"endpoint\": \"http://collector:4318/v1/traces\",\n \"method\": \"POST\",\n \"content-type\": \"application/x-protobuf\",\n \"payload\": \"base64-encoded-content\",\n \"base64\": true\n}\n```\n\n## AWS Lambda Usage\n\nHere's a complete example of using the adapter in an AWS Lambda function with distributed tracing:\n\n```python\nfrom opentelemetry import trace\nfrom opentelemetry.sdk.trace import TracerProvider\nfrom opentelemetry.sdk.trace.export import BatchSpanProcessor\nfrom opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter\nfrom opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator\nfrom otlp_stdout_adapter import StdoutAdapter, get_lambda_resource\n\n# Initialize once per Lambda container\nadapter = StdoutAdapter()\nsession = adapter.get_session()\n\n# Set up the trace provider with Lambda resource detection\nprovider = TracerProvider(resource=get_lambda_resource())\nexporter = OTLPSpanExporter(\n endpoint=\"http://your-collector:4318/v1/traces\",\n session=session\n)\nprocessor = BatchSpanProcessor(exporter)\nprovider.add_span_processor(processor)\ntrace.set_tracer_provider(provider)\n\n# Get a tracer\ntracer = trace.get_tracer(__name__)\n\ndef process_event(event):\n with tracer.start_as_current_span(\"process_event\") as span:\n # Your processing logic here\n span.set_attribute(\"event.type\", event.get(\"type\"))\n return {\"processed\": True}\n\ndef lambda_handler(event, context):\n with tracer.start_as_current_span(\n \"lambda_handler\",\n kind=trace.SpanKind.SERVER\n ) as span:\n try:\n # Add event information to span\n span.set_attribute(\"event.type\", event.get(\"type\"))\n span.add_event(\"Processing Lambda event\")\n \n result = process_event(event)\n \n return {\n \"statusCode\": 200,\n \"body\": json.dumps(result)\n }\n except Exception as e:\n span.record_exception(e)\n span.set_status(trace.Status(trace.StatusCode.ERROR))\n raise\n finally:\n # Ensure spans are exported before Lambda freezes\n provider.force_flush()\n```\n\n### Lambda Configuration Notes\n\n- Initialize tracing components outside the handler for container reuse\n- Always call `provider.force_flush()` before the handler completes\n- Use environment variables for configuration:\n ```bash\n OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf\n OTEL_EXPORTER_OTLP_COMPRESSION=gzip\n ```\n\n## Best Practices\n\n1. **Initialize Outside Handler**\n - Create providers, exporters, and tracers outside the handler\n - Reuse these components across invocations\n\n2. **Resource Detection**\n - Use `get_lambda_resource()` to automatically capture Lambda metadata\n - Merge with custom resources as needed\n\n3. **Span Processing**\n - Use `BatchSpanProcessor` for efficient span processing\n - Always flush spans before handler completion\n - Configure appropriate batch size and timeout for Lambda\n\n4. **Error Handling**\n - Record exceptions using `span.record_exception()`\n - Set appropriate span status\n - Ensure spans are ended in finally blocks\n\n5. **Context Propagation**\n - Use TraceContext propagator for distributed tracing\n - Propagate context in outgoing requests\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2024 Alessandro Bologna Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.",
"summary": "A custom requests HTTPAdapter that serializes spans to stdout for OpenTelemetry OTLP exporters",
"version": "0.1.0",
"project_urls": {
"Homepage": "https://github.com/dev7a/lambda-otlp-forwarder/tree/main/packages/python/adapter",
"Repository": "https://github.com/dev7a/lambda-otlp-forwarder/tree/main/packages/python/adapter"
},
"split_keywords": [
"opentelemetry",
" otlp",
" stdout",
" lambda",
" aws"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5c1203a98bf05e7213804c53c9785f70c0b4edb94689508f77779026c126ec0a",
"md5": "ffa1f6e850534d4a2b34302d0ebbd586",
"sha256": "48c70f5493597896b10162087c9cafbccef354ef0e4887f32aa4332f5e03013d"
},
"downloads": -1,
"filename": "otlp_stdout_adapter-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ffa1f6e850534d4a2b34302d0ebbd586",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.12",
"size": 8279,
"upload_time": "2024-11-06T03:13:06",
"upload_time_iso_8601": "2024-11-06T03:13:06.600617Z",
"url": "https://files.pythonhosted.org/packages/5c/12/03a98bf05e7213804c53c9785f70c0b4edb94689508f77779026c126ec0a/otlp_stdout_adapter-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "cb8f45cf8d7fd11022fe07b6c14cccf2f707cb292178f523332e9f1b82965459",
"md5": "4342bb03e965999d453527c181b997a6",
"sha256": "38991a35203420e988aaf9b620dfeca96f3532cd482d2d23e3797460c196d3ea"
},
"downloads": -1,
"filename": "otlp_stdout_adapter-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "4342bb03e965999d453527c181b997a6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.12",
"size": 10135,
"upload_time": "2024-11-06T03:13:08",
"upload_time_iso_8601": "2024-11-06T03:13:08.096565Z",
"url": "https://files.pythonhosted.org/packages/cb/8f/45cf8d7fd11022fe07b6c14cccf2f707cb292178f523332e9f1b82965459/otlp_stdout_adapter-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-06 03:13:08",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "dev7a",
"github_project": "lambda-otlp-forwarder",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "otlp-stdout-adapter"
}
