LocalStack Extension: Platform observability
===============================
LocalStack extension for providing insights into the LocalStack platform in the form of service-specific traces and metrics.
## Install
```bash
localstack extensions install localstack-extension-platform-observability
```
## Install local development version
To install the extension into localstack in developer mode, you will need Python 3.10, and create a virtual environment in the extensions project.
In the newly generated project, simply run
```bash
make install
```
Then, to enable the extension for LocalStack, run
```bash
localstack extensions dev enable .
```
You can then start LocalStack with `EXTENSION_DEV_MODE=1` to load all enabled extensions:
```bash
EXTENSION_DEV_MODE=1 localstack start
```
## Install from GitHub repository
To distribute your extension, simply upload it to your github account. Your extension can then be installed via:
```bash
localstack extensions install "git+https://github.com/localstack/localstack-extension-platform-observability/#egg=localstack-extension-platform-observability"
```
## Usage
### Metrics
Fetch all metrics
```bash
curl localhost:4566/_extension/observability/metrics
```
Fetch a specific instrument
```bash
curl localhost:4566/_extension/observability/metrics/<instrument>
```
The following instruments exist
* `system`: system metrics like number of threads
* `sns`: sns topic statistics
* `sqs`: sqs queue statistics
* `gateway`: HTTP gateway statistics on number of requests
Example:
```bash
curl -s "localhost:4566/_extension/observability/metrics" | jq .
```
```json
{
"system": [
{
"active_thread_count": 15,
"max_rss": 15
}
],
"gateway": [
{
"total": 14,
"sqs.SendMessage": 2,
"sqs.ReceiveMessage": 1,
"sns.Publish": 1,
"dynamodb.PutItem": 0,
"dynamodb.GetItem": 0,
"dynamodb.BatchWriteItem": 0,
"dynamodb.BatchGetItem": 0,
"lambda.Invoke": 0
}
],
"sqs": [
{
"queue": "arn:aws:sqs:us-east-1:000000000000:input-dead-letter-queue",
"visible": 0,
"invisible": 0,
"delayed": 0
},
{
"queue": "arn:aws:sqs:us-east-1:000000000000:input-queue",
"visible": 2,
"invisible": 0,
"delayed": 0
},
{
"queue": "arn:aws:sqs:us-east-1:000000000000:recovery-queue",
"visible": 0,
"invisible": 0,
"delayed": 0
}
],
"sns": [
{
"topic_arn": "arn:aws:sns:us-east-1:000000000000:localstack-topic",
"published": 1,
"delivered": 0,
"failed": 0
}
],
"timestamp": 1704986115.3762584
}
```
### Trace logs
#### Lambda
Find lambda traces in
```bash
/var/lib/localstack/cache/observability/traces-lambda-events/
```
Here is an example:
```json
{"timestamp": 1704984270.1660516, "event": "enqueued", "request_id": "ad2df0ed-c952-4f48-881c-8b944dad44c6", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-e0c504b2", "failure_cause": null}
{"timestamp": 1704984270.184178, "event": "enqueued", "request_id": "d5d2efb3-e781-411a-b718-e2345c118c39", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-e0c504b2", "failure_cause": null}
{"timestamp": 1704984270.3365452, "event": "submitted", "request_id": "ad2df0ed-c952-4f48-881c-8b944dad44c6", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-e0c504b2", "failure_cause": null}
{"timestamp": 1704984270.3368104, "event": "invoking", "request_id": "ad2df0ed-c952-4f48-881c-8b944dad44c6", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-e0c504b2", "failure_cause": null}
{"timestamp": 1704984270.4253993, "event": "submitted", "request_id": "d5d2efb3-e781-411a-b718-e2345c118c39", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-e0c504b2", "failure_cause": null}
```
#### Lambda SQS Event source listeners
Find traces that relate to lambda sqs event source listeners in
```bash
/var/lib/localstack/cache/observability/traces-lambda-sqs/
```
Here is an example of queueing two message to trigger a lambda.
```json
{"timestamp": 1705009138.683765, "event": "message_queued", "message_id": "94c3e579-dd40-48a6-bfaa-5d1d04c79044", "event_source_arn": "arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750", "lambda_arn": null, "request_id": null, "failure_cause": null}
{"timestamp": 1705009138.6840491, "event": "message_queued", "message_id": "00787a9f-1d70-452d-9fec-f25bf7064e32", "event_source_arn": "arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750", "lambda_arn": null, "request_id": null, "failure_cause": null}
{"timestamp": 1705009139.6799114, "event": "message_dequeued", "message_id": "94c3e579-dd40-48a6-bfaa-5d1d04c79044", "event_source_arn": "arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082", "request_id": null, "failure_cause": null}
{"timestamp": 1705009139.6799738, "event": "invoke_queued", "message_id": "94c3e579-dd40-48a6-bfaa-5d1d04c79044", "event_source_arn": "arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082", "request_id": "0d616a5e-2511-4c88-a7b2-de0f0a7161ed", "failure_cause": null}
{"timestamp": 1705009139.6801724, "event": "invoke", "message_id": "94c3e579-dd40-48a6-bfaa-5d1d04c79044", "event_source_arn": "arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082", "request_id": "0d616a5e-2511-4c88-a7b2-de0f0a7161ed", "failure_cause": null}
{"timestamp": 1705009140.0882578, "event": "invoke_success", "message_id": "94c3e579-dd40-48a6-bfaa-5d1d04c79044", "event_source_arn": "arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082", "request_id": "0d616a5e-2511-4c88-a7b2-de0f0a7161ed", "failure_cause": null}
{"timestamp": 1705009140.6862357, "event": "message_dequeued", "message_id": "00787a9f-1d70-452d-9fec-f25bf7064e32", "event_source_arn": "arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082", "request_id": null, "failure_cause": null}
{"timestamp": 1705009140.6865497, "event": "invoke_queued", "message_id": "00787a9f-1d70-452d-9fec-f25bf7064e32", "event_source_arn": "arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082", "request_id": "101a0017-172b-401b-9381-34aa1a4d3e7c", "failure_cause": null}
{"timestamp": 1705009140.6868262, "event": "invoke", "message_id": "00787a9f-1d70-452d-9fec-f25bf7064e32", "event_source_arn": "arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082", "request_id": "101a0017-172b-401b-9381-34aa1a4d3e7c", "failure_cause": null}
{"timestamp": 1705009140.6951976, "event": "invoke_success", "message_id": "00787a9f-1d70-452d-9fec-f25bf7064e32", "event_source_arn": "arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750", "lambda_arn": "arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082", "request_id": "101a0017-172b-401b-9381-34aa1a4d3e7c", "failure_cause": null}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/localstack/localstack-extension-platform-observability",
"name": "localstack-extension-platform-observability",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "LocalStack Contributors",
"author_email": "info@localstack.cloud",
"download_url": "https://files.pythonhosted.org/packages/9c/18/ec86af77a5b2ff57cf614fdbf45d1dcf6d7d311be036ce2d0bbf90db1176/localstack_extension_platform_observability-0.0.3.tar.gz",
"platform": null,
"description": "LocalStack Extension: Platform observability\n===============================\n\nLocalStack extension for providing insights into the LocalStack platform in the form of service-specific traces and metrics.\n\n## Install\n\n```bash\nlocalstack extensions install localstack-extension-platform-observability\n```\n\n## Install local development version\n\nTo install the extension into localstack in developer mode, you will need Python 3.10, and create a virtual environment in the extensions project.\n\nIn the newly generated project, simply run\n\n```bash\nmake install\n```\n\nThen, to enable the extension for LocalStack, run\n\n```bash\nlocalstack extensions dev enable .\n```\n\nYou can then start LocalStack with `EXTENSION_DEV_MODE=1` to load all enabled extensions:\n\n```bash\nEXTENSION_DEV_MODE=1 localstack start\n```\n\n## Install from GitHub repository\n\nTo distribute your extension, simply upload it to your github account. Your extension can then be installed via:\n\n```bash\nlocalstack extensions install \"git+https://github.com/localstack/localstack-extension-platform-observability/#egg=localstack-extension-platform-observability\"\n```\n\n## Usage\n\n### Metrics\n\nFetch all metrics\n\n```bash\ncurl localhost:4566/_extension/observability/metrics\n```\n\nFetch a specific instrument\n\n```bash\ncurl localhost:4566/_extension/observability/metrics/<instrument>\n```\n\nThe following instruments exist\n* `system`: system metrics like number of threads\n* `sns`: sns topic statistics\n* `sqs`: sqs queue statistics\n* `gateway`: HTTP gateway statistics on number of requests\n\nExample:\n\n```bash\ncurl -s \"localhost:4566/_extension/observability/metrics\" | jq .\n```\n```json\n{\n \"system\": [\n {\n \"active_thread_count\": 15,\n \"max_rss\": 15\n }\n ],\n \"gateway\": [\n {\n \"total\": 14,\n \"sqs.SendMessage\": 2,\n \"sqs.ReceiveMessage\": 1,\n \"sns.Publish\": 1,\n \"dynamodb.PutItem\": 0,\n \"dynamodb.GetItem\": 0,\n \"dynamodb.BatchWriteItem\": 0,\n \"dynamodb.BatchGetItem\": 0,\n \"lambda.Invoke\": 0\n }\n ],\n \"sqs\": [\n {\n \"queue\": \"arn:aws:sqs:us-east-1:000000000000:input-dead-letter-queue\",\n \"visible\": 0,\n \"invisible\": 0,\n \"delayed\": 0\n },\n {\n \"queue\": \"arn:aws:sqs:us-east-1:000000000000:input-queue\",\n \"visible\": 2,\n \"invisible\": 0,\n \"delayed\": 0\n },\n {\n \"queue\": \"arn:aws:sqs:us-east-1:000000000000:recovery-queue\",\n \"visible\": 0,\n \"invisible\": 0,\n \"delayed\": 0\n }\n ],\n \"sns\": [\n {\n \"topic_arn\": \"arn:aws:sns:us-east-1:000000000000:localstack-topic\",\n \"published\": 1,\n \"delivered\": 0,\n \"failed\": 0\n }\n ],\n \"timestamp\": 1704986115.3762584\n}\n```\n\n\n### Trace logs\n\n#### Lambda\n\nFind lambda traces in\n```bash\n/var/lib/localstack/cache/observability/traces-lambda-events/\n```\n\nHere is an example:\n\n```json\n{\"timestamp\": 1704984270.1660516, \"event\": \"enqueued\", \"request_id\": \"ad2df0ed-c952-4f48-881c-8b944dad44c6\", \"lambda_arn\": \"arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-e0c504b2\", \"failure_cause\": null}\n{\"timestamp\": 1704984270.184178, \"event\": \"enqueued\", \"request_id\": \"d5d2efb3-e781-411a-b718-e2345c118c39\", \"lambda_arn\": \"arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-e0c504b2\", \"failure_cause\": null}\n{\"timestamp\": 1704984270.3365452, \"event\": \"submitted\", \"request_id\": \"ad2df0ed-c952-4f48-881c-8b944dad44c6\", \"lambda_arn\": \"arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-e0c504b2\", \"failure_cause\": null}\n{\"timestamp\": 1704984270.3368104, \"event\": \"invoking\", \"request_id\": \"ad2df0ed-c952-4f48-881c-8b944dad44c6\", \"lambda_arn\": \"arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-e0c504b2\", \"failure_cause\": null}\n{\"timestamp\": 1704984270.4253993, \"event\": \"submitted\", \"request_id\": \"d5d2efb3-e781-411a-b718-e2345c118c39\", \"lambda_arn\": \"arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-e0c504b2\", \"failure_cause\": null}\n```\n\n\n#### Lambda SQS Event source listeners\n\nFind traces that relate to lambda sqs event source listeners in\n```bash\n/var/lib/localstack/cache/observability/traces-lambda-sqs/\n```\n\nHere is an example of queueing two message to trigger a lambda.\n\n```json\n{\"timestamp\": 1705009138.683765, \"event\": \"message_queued\", \"message_id\": \"94c3e579-dd40-48a6-bfaa-5d1d04c79044\", \"event_source_arn\": \"arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750\", \"lambda_arn\": null, \"request_id\": null, \"failure_cause\": null}\n{\"timestamp\": 1705009138.6840491, \"event\": \"message_queued\", \"message_id\": \"00787a9f-1d70-452d-9fec-f25bf7064e32\", \"event_source_arn\": \"arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750\", \"lambda_arn\": null, \"request_id\": null, \"failure_cause\": null}\n{\"timestamp\": 1705009139.6799114, \"event\": \"message_dequeued\", \"message_id\": \"94c3e579-dd40-48a6-bfaa-5d1d04c79044\", \"event_source_arn\": \"arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750\", \"lambda_arn\": \"arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082\", \"request_id\": null, \"failure_cause\": null}\n{\"timestamp\": 1705009139.6799738, \"event\": \"invoke_queued\", \"message_id\": \"94c3e579-dd40-48a6-bfaa-5d1d04c79044\", \"event_source_arn\": \"arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750\", \"lambda_arn\": \"arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082\", \"request_id\": \"0d616a5e-2511-4c88-a7b2-de0f0a7161ed\", \"failure_cause\": null}\n{\"timestamp\": 1705009139.6801724, \"event\": \"invoke\", \"message_id\": \"94c3e579-dd40-48a6-bfaa-5d1d04c79044\", \"event_source_arn\": \"arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750\", \"lambda_arn\": \"arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082\", \"request_id\": \"0d616a5e-2511-4c88-a7b2-de0f0a7161ed\", \"failure_cause\": null}\n{\"timestamp\": 1705009140.0882578, \"event\": \"invoke_success\", \"message_id\": \"94c3e579-dd40-48a6-bfaa-5d1d04c79044\", \"event_source_arn\": \"arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750\", \"lambda_arn\": \"arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082\", \"request_id\": \"0d616a5e-2511-4c88-a7b2-de0f0a7161ed\", \"failure_cause\": null}\n{\"timestamp\": 1705009140.6862357, \"event\": \"message_dequeued\", \"message_id\": \"00787a9f-1d70-452d-9fec-f25bf7064e32\", \"event_source_arn\": \"arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750\", \"lambda_arn\": \"arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082\", \"request_id\": null, \"failure_cause\": null}\n{\"timestamp\": 1705009140.6865497, \"event\": \"invoke_queued\", \"message_id\": \"00787a9f-1d70-452d-9fec-f25bf7064e32\", \"event_source_arn\": \"arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750\", \"lambda_arn\": \"arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082\", \"request_id\": \"101a0017-172b-401b-9381-34aa1a4d3e7c\", \"failure_cause\": null}\n{\"timestamp\": 1705009140.6868262, \"event\": \"invoke\", \"message_id\": \"00787a9f-1d70-452d-9fec-f25bf7064e32\", \"event_source_arn\": \"arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750\", \"lambda_arn\": \"arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082\", \"request_id\": \"101a0017-172b-401b-9381-34aa1a4d3e7c\", \"failure_cause\": null}\n{\"timestamp\": 1705009140.6951976, \"event\": \"invoke_success\", \"message_id\": \"00787a9f-1d70-452d-9fec-f25bf7064e32\", \"event_source_arn\": \"arn:aws:sqs:us-east-1:000000000000:test-queue-a5d98750\", \"lambda_arn\": \"arn:aws:lambda:us-east-1:000000000000:function:test-lambda-perf-33b02082\", \"request_id\": \"101a0017-172b-401b-9381-34aa1a4d3e7c\", \"failure_cause\": null}\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "LocalStack Extension: LocalStack Extension: Platform observability",
"version": "0.0.3",
"project_urls": {
"Homepage": "https://github.com/localstack/localstack-extension-platform-observability"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5c89c57909794904995dcabf24533b206c0730096bede3bbeb477e8d8987ce2e",
"md5": "91bc154020c00d351ea6e85ecca7bca0",
"sha256": "80b3a521776ac1c3c7765892d725e208d3079b0f861901973ab6ccc155b1367e"
},
"downloads": -1,
"filename": "localstack_extension_platform_observability-0.0.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "91bc154020c00d351ea6e85ecca7bca0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 13197,
"upload_time": "2024-07-16T10:03:37",
"upload_time_iso_8601": "2024-07-16T10:03:37.952404Z",
"url": "https://files.pythonhosted.org/packages/5c/89/c57909794904995dcabf24533b206c0730096bede3bbeb477e8d8987ce2e/localstack_extension_platform_observability-0.0.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9c18ec86af77a5b2ff57cf614fdbf45d1dcf6d7d311be036ce2d0bbf90db1176",
"md5": "3a3d808f848e74b5a8281676b4ac423d",
"sha256": "2ff9a85957e819c9ae73a4a6da437e91c8b2b88cfb3399adc46d33ba2bd443ca"
},
"downloads": -1,
"filename": "localstack_extension_platform_observability-0.0.3.tar.gz",
"has_sig": false,
"md5_digest": "3a3d808f848e74b5a8281676b4ac423d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 9600,
"upload_time": "2024-07-16T10:03:39",
"upload_time_iso_8601": "2024-07-16T10:03:39.134630Z",
"url": "https://files.pythonhosted.org/packages/9c/18/ec86af77a5b2ff57cf614fdbf45d1dcf6d7d311be036ce2d0bbf90db1176/localstack_extension_platform_observability-0.0.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-16 10:03:39",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "localstack",
"github_project": "localstack-extension-platform-observability",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "localstack-extension-platform-observability"
}
JSON
