| Name | seismos-package JSON |
| Version |
0.1.7
JSON |
| download |
| home_page | None |
| Summary | Seismos internal Python package for structured logging and utilities. |
| upload_time | 2025-01-27 12:48:55 |
| maintainer | None |
| docs_url | None |
| author | Seismos |
| requires_python | <3.13,>=3.8 |
| license | None |
| keywords |
logging
utilities
internal-package
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Seismos Python Package
### Structlog
Structlog is a powerful logging library for structured, context-aware logging.
More details can be found in the [structlog](https://www.structlog.org/en/stable/).
#### Example, basic structlog configuration
instead of `logger = logging.getLogger(__name__)` it is `logger = structlog.get_logger(__name__)`
```python
from seismos_package.logging import LoggingConfigurator
from seismos_package.config import SeismosConfig
import structlog
config = SeismosConfig()
LoggingConfigurator(
service_name=config.APP_NAME,
log_level='INFO',
setup_logging_dict=True
).configure_structlog(
formatter='plain_console',
formatter_std_lib='plain_console'
)
logger = structlog.get_logger(__name__)
logger.debug("This is a DEBUG log message", key_1="value_1", key_2="value_2", key_n="value_n")
logger.info("This is an INFO log message", key_1="value_1", key_2="value_2", key_n="value_n")
logger.warning("This is a WARNING log message", key_1="value_1", key_2="value_2", key_n="value_n")
logger.error("This is an ERROR log message", key_1="value_1", key_2="value_2", key_n="value_n")
logger.critical("This is a CRITICAL log message", key_1="value_1", key_2="value_2", key_n="value_n")
try:
1 / 0
except ZeroDivisionError:
logger.exception("An EXCEPTION log with stack trace occurred", key_1="value_1", key_2="value_2")
```
In production, you should aim for structured, machine-readable logs that can be easily ingested by log aggregation and monitoring tools like ELK (Elasticsearch, Logstash, Kibana), Datadog, or Prometheus:
```python
from seismos_package.logging import LoggingConfigurator
from seismos_package.config import SeismosConfig
import structlog
config = SeismosConfig()
LoggingConfigurator(
service_name=config.APP_NAME,
log_level='INFO',
setup_logging_dict=True
).configure_structlog(
formatter='json_formatter',
formatter_std_lib='json_formatter'
)
logger = structlog.get_logger(__name__)
logger.debug("This is a DEBUG log message", key_1="value_1", key_2="value_2", key_n="value_n")
logger.info("This is an INFO log message", key_1="value_1", key_2="value_2", key_n="value_n")
logger.warning("This is a WARNING log message", key_1="value_1", key_2="value_2", key_n="value_n")
logger.error("This is an ERROR log message", key_1="value_1", key_2="value_2", key_n="value_n")
logger.critical("This is a CRITICAL log message", key_1="value_1", key_2="value_2", key_n="value_n")
try:
1 / 0
except ZeroDivisionError:
logger.exception("An EXCEPTION log with stack trace occurred", key_1="value_1", key_2="value_2")
```
#### Using Middleware for Automatic Logging Context:
The middleware adds request_id, IP, and user_id to every log during a request/response cycle.
This middleware module provides logging context management for both Flask and FastAPI applications using structlog.
Flask Middleware (add_request_context_flask): Captures essential request data such as the request ID, method, and path, binding them to the structlog context for better traceability during the request lifecycle.
FastAPI Middleware (add_request_context_fastapi): Captures similar request metadata, ensuring a request ID is present, generating one if absent.
It binds the request context to structlog and clears it after the request completes.
Class-Based Middleware (FastAPIRequestContextMiddleware): A reusable FastAPI middleware class that integrates with the BaseHTTPMiddleware and delegates the logging setup to the add_request_context_fastapi function.
This setup ensures structured, consistent logging across both frameworks, improving traceability and debugging in distributed systems.
This guide explains how to set up and use structlog for structured logging in a Flask application. The goal is to have a consistent and centralized logging setup that can be reused across the application.
The logger is initialized once in the main application file (e.g., app.py).
```python
import sys
import uuid
from flask import Flask, request
from seismos_package.logging import LoggingConfigurator
from seismos_package.logging.middlewares import add_request_context_flask
from seismos_package.config import SeismosConfig
import structlog
config = SeismosConfig()
LoggingConfigurator(
service_name=config.APP_NAME,
log_level="INFO",
setup_logging_dict=True,
).configure_structlog(formatter='json_formatter', formatter_std_lib='json_formatter')
logger = structlog.get_logger(__name__)
app = Flask(__name__)
@app.before_request
def set_logging_context():
"""Bind context for each request using the middleware."""
add_request_context_flask()
logger.info("Context set for request")
with app.test_client() as client:
dynamic_request_id = str(uuid.uuid4())
client.get("/", headers={"X-User-Name": "John Doe", "X-Request-ID": dynamic_request_id})
logger.info("Test client request sent", request_id=dynamic_request_id)
```
You can use the same logger instance across different modules by importing structlog directly.
Example (services.py):
```python
import structlog
logger = structlog.get_logger(__name__)
logger.info("Processing data started", data_size=100)
```
Key Points:
- Centralized Configuration: The logger is initialized once in app.py.
- Consistent Usage: structlog.get_logger(__name__) is imported and used across all files.
- Context Management: Context is managed using structlog.contextvars.bind_contextvars().
- Structured Logging: The JSON formatter ensures logs are machine-readable.
FastAPI:
```python
import uuid
from fastapi import FastAPI, Request
from seismos_package.logging.middlewares import FastAPIRequestContextMiddleware
import structlog
config = SeismosConfig()
LoggingConfigurator(
service_name=config.APP_NAME,
log_level="INFO",
setup_logging_dict=True,
).configure_structlog(formatter='json_formatter', formatter_std_lib='json_formatter')
logger = structlog.get_logger(__name__)
app = FastAPI()
app.add_middleware(FastAPIRequestContextMiddleware)
```
Automatic injection of:
- user_id
- IP
- request_id
- request_method
This a console view, in prod it will be json (using python json logging to have standard logging and structlog logging as close as possible)
### Why Use a Structured Logger?
- Standard logging often outputs plain text logs, which can be challenging for log aggregation tools like EFK Stack or Grafana Loki to process effectively.
- Structured logging outputs data in a machine-readable format (e.g., JSON), making it easier for log analysis tools to filter and process logs efficiently.
- With structured logging, developers can filter logs by fields such as request_id, user_id, and transaction_id for better traceability across distributed systems.
- The primary goal is to simplify debugging, enable better error tracking, and improve observability with enhanced log analysis capabilities.
- Structured logs are designed to be consumed primarily by machines for monitoring and analytics, while still being readable for developers when needed.
- This package leverages structlog, a library that enhances Python's standard logging by providing better context management and a flexible structure for log messages.
# Development of this project
Please install [poetry](https://python-poetry.org/docs/#installation) as this is the tool we use for releasing and development.
poetry install && poetry run pytest -rs --cov=seismos_package -s
To run tests inside docker:
poetry install --with dev && poetry run pytest -rs --cov=seismos_package
To run pre-commit:
poetry run pre-commit run --all-files
Raw data
{
"_id": null,
"home_page": null,
"name": "seismos-package",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.13,>=3.8",
"maintainer_email": null,
"keywords": "logging, utilities, internal-package",
"author": "Seismos",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/68/33/a0e24143124f6808455ebf4801cc0871503103420141a5cd8f3e0c53a1f9/seismos_package-0.1.7.tar.gz",
"platform": null,
"description": "# Seismos Python Package\n\n### Structlog\nStructlog is a powerful logging library for structured, context-aware logging.\nMore details can be found in the [structlog](https://www.structlog.org/en/stable/).\n\n#### Example, basic structlog configuration\n\ninstead of `logger = logging.getLogger(__name__)` it is `logger = structlog.get_logger(__name__)`\n\n```python\n from seismos_package.logging import LoggingConfigurator\n from seismos_package.config import SeismosConfig\n import structlog\n\n config = SeismosConfig()\n\n LoggingConfigurator(\n service_name=config.APP_NAME,\n log_level='INFO',\n setup_logging_dict=True\n ).configure_structlog(\n formatter='plain_console',\n formatter_std_lib='plain_console'\n )\n\n logger = structlog.get_logger(__name__)\n logger.debug(\"This is a DEBUG log message\", key_1=\"value_1\", key_2=\"value_2\", key_n=\"value_n\")\n logger.info(\"This is an INFO log message\", key_1=\"value_1\", key_2=\"value_2\", key_n=\"value_n\")\n logger.warning(\"This is a WARNING log message\", key_1=\"value_1\", key_2=\"value_2\", key_n=\"value_n\")\n logger.error(\"This is an ERROR log message\", key_1=\"value_1\", key_2=\"value_2\", key_n=\"value_n\")\n logger.critical(\"This is a CRITICAL log message\", key_1=\"value_1\", key_2=\"value_2\", key_n=\"value_n\")\n\n try:\n 1 / 0\n except ZeroDivisionError:\n logger.exception(\"An EXCEPTION log with stack trace occurred\", key_1=\"value_1\", key_2=\"value_2\")\n\n\n```\n\n\n\nIn production, you should aim for structured, machine-readable logs that can be easily ingested by log aggregation and monitoring tools like ELK (Elasticsearch, Logstash, Kibana), Datadog, or Prometheus:\n\n```python\n from seismos_package.logging import LoggingConfigurator\n from seismos_package.config import SeismosConfig\n import structlog\n\n config = SeismosConfig()\n\n LoggingConfigurator(\n service_name=config.APP_NAME,\n log_level='INFO',\n setup_logging_dict=True\n ).configure_structlog(\n formatter='json_formatter',\n formatter_std_lib='json_formatter'\n )\n\n logger = structlog.get_logger(__name__)\n logger.debug(\"This is a DEBUG log message\", key_1=\"value_1\", key_2=\"value_2\", key_n=\"value_n\")\n logger.info(\"This is an INFO log message\", key_1=\"value_1\", key_2=\"value_2\", key_n=\"value_n\")\n logger.warning(\"This is a WARNING log message\", key_1=\"value_1\", key_2=\"value_2\", key_n=\"value_n\")\n logger.error(\"This is an ERROR log message\", key_1=\"value_1\", key_2=\"value_2\", key_n=\"value_n\")\n logger.critical(\"This is a CRITICAL log message\", key_1=\"value_1\", key_2=\"value_2\", key_n=\"value_n\")\n\n try:\n 1 / 0\n except ZeroDivisionError:\n logger.exception(\"An EXCEPTION log with stack trace occurred\", key_1=\"value_1\", key_2=\"value_2\")\n```\n\n\n\n\n#### Using Middleware for Automatic Logging Context:\n\nThe middleware adds request_id, IP, and user_id to every log during a request/response cycle.\nThis middleware module provides logging context management for both Flask and FastAPI applications using structlog.\n\nFlask Middleware (add_request_context_flask): Captures essential request data such as the request ID, method, and path, binding them to the structlog context for better traceability during the request lifecycle.\n\nFastAPI Middleware (add_request_context_fastapi): Captures similar request metadata, ensuring a request ID is present, generating one if absent.\nIt binds the request context to structlog and clears it after the request completes.\n\nClass-Based Middleware (FastAPIRequestContextMiddleware): A reusable FastAPI middleware class that integrates with the BaseHTTPMiddleware and delegates the logging setup to the add_request_context_fastapi function.\n\nThis setup ensures structured, consistent logging across both frameworks, improving traceability and debugging in distributed systems.\n\n\nThis guide explains how to set up and use structlog for structured logging in a Flask application. The goal is to have a consistent and centralized logging setup that can be reused across the application.\nThe logger is initialized once in the main application file (e.g., app.py).\n\n```python\n import sys\n import uuid\n from flask import Flask, request\n from seismos_package.logging import LoggingConfigurator\n from seismos_package.logging.middlewares import add_request_context_flask\n from seismos_package.config import SeismosConfig\n import structlog\n\n config = SeismosConfig()\n\n LoggingConfigurator(\n service_name=config.APP_NAME,\n log_level=\"INFO\",\n setup_logging_dict=True,\n ).configure_structlog(formatter='json_formatter', formatter_std_lib='json_formatter')\n\n logger = structlog.get_logger(__name__)\n\n app = Flask(__name__)\n\n @app.before_request\n def set_logging_context():\n \"\"\"Bind context for each request using the middleware.\"\"\"\n add_request_context_flask()\n logger.info(\"Context set for request\")\n\n with app.test_client() as client:\n dynamic_request_id = str(uuid.uuid4())\n client.get(\"/\", headers={\"X-User-Name\": \"John Doe\", \"X-Request-ID\": dynamic_request_id})\n logger.info(\"Test client request sent\", request_id=dynamic_request_id)\n\n```\n\n\n\nYou can use the same logger instance across different modules by importing structlog directly.\nExample (services.py):\n\n\n```python\n import structlog\n\n logger = structlog.get_logger(__name__)\n logger.info(\"Processing data started\", data_size=100)\n```\nKey Points:\n\n- Centralized Configuration: The logger is initialized once in app.py.\n- Consistent Usage: structlog.get_logger(__name__) is imported and used across all files.\n- Context Management: Context is managed using structlog.contextvars.bind_contextvars().\n- Structured Logging: The JSON formatter ensures logs are machine-readable.\n\nFastAPI:\n\n```python\n import uuid\n from fastapi import FastAPI, Request\n from seismos_package.logging.middlewares import FastAPIRequestContextMiddleware\n import structlog\n\n config = SeismosConfig()\n\n LoggingConfigurator(\n service_name=config.APP_NAME,\n log_level=\"INFO\",\n setup_logging_dict=True,\n ).configure_structlog(formatter='json_formatter', formatter_std_lib='json_formatter')\n\n logger = structlog.get_logger(__name__)\n app = FastAPI()\n app.add_middleware(FastAPIRequestContextMiddleware)\n\n```\n\n\n\nAutomatic injection of:\n- user_id\n- IP\n- request_id\n- request_method\n\n\nThis a console view, in prod it will be json (using python json logging to have standard logging and structlog logging as close as possible)\n\n\n### Why Use a Structured Logger?\n- Standard logging often outputs plain text logs, which can be challenging for log aggregation tools like EFK Stack or Grafana Loki to process effectively.\n- Structured logging outputs data in a machine-readable format (e.g., JSON), making it easier for log analysis tools to filter and process logs efficiently.\n- With structured logging, developers can filter logs by fields such as request_id, user_id, and transaction_id for better traceability across distributed systems.\n- The primary goal is to simplify debugging, enable better error tracking, and improve observability with enhanced log analysis capabilities.\n- Structured logs are designed to be consumed primarily by machines for monitoring and analytics, while still being readable for developers when needed.\n- This package leverages structlog, a library that enhances Python's standard logging by providing better context management and a flexible structure for log messages.\n\n\n# Development of this project\n\nPlease install [poetry](https://python-poetry.org/docs/#installation) as this is the tool we use for releasing and development.\n\n poetry install && poetry run pytest -rs --cov=seismos_package -s\n\nTo run tests inside docker:\n\n poetry install --with dev && poetry run pytest -rs --cov=seismos_package\n\nTo run pre-commit:\n poetry run pre-commit run --all-files\n",
"bugtrack_url": null,
"license": null,
"summary": "Seismos internal Python package for structured logging and utilities.",
"version": "0.1.7",
"project_urls": null,
"split_keywords": [
"logging",
" utilities",
" internal-package"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1c098f02a121d60f641c80800f0fe1d05bd06774ce90d2e8f332299b05b9ce2c",
"md5": "b51acb43cc480d1746c8c61adf1846e2",
"sha256": "cc7c3ac90b018d0a73930ff2352145556741da21cd17d3f226d09801f2bb0476"
},
"downloads": -1,
"filename": "seismos_package-0.1.7-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b51acb43cc480d1746c8c61adf1846e2",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.13,>=3.8",
"size": 7773,
"upload_time": "2025-01-27T12:48:53",
"upload_time_iso_8601": "2025-01-27T12:48:53.240121Z",
"url": "https://files.pythonhosted.org/packages/1c/09/8f02a121d60f641c80800f0fe1d05bd06774ce90d2e8f332299b05b9ce2c/seismos_package-0.1.7-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "6833a0e24143124f6808455ebf4801cc0871503103420141a5cd8f3e0c53a1f9",
"md5": "171c11250a676e79691e029840c89f0d",
"sha256": "0144a92536d6b32dbc7e417fe14841de7ed7dce106c49d03300ddc7b38380efd"
},
"downloads": -1,
"filename": "seismos_package-0.1.7.tar.gz",
"has_sig": false,
"md5_digest": "171c11250a676e79691e029840c89f0d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.13,>=3.8",
"size": 8728,
"upload_time": "2025-01-27T12:48:55",
"upload_time_iso_8601": "2025-01-27T12:48:55.085735Z",
"url": "https://files.pythonhosted.org/packages/68/33/a0e24143124f6808455ebf4801cc0871503103420141a5cd8f3e0c53a1f9/seismos_package-0.1.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-27 12:48:55",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "seismos-package"
}
