# Logging HTTP Client
[](https://github.com/u-ways/logging-http-client/actions/workflows/CICD.yml)
[](https://www.python.org/downloads/release/python-3110/)
[](https://python-poetry.org/)
[](https://flake8.pycqa.org/en/latest/)
[](https://github.com/psf/black)
[](https://pypi.org/project/logging-http-client/)
[](https://pypistats.org/packages/logging-http-client)
[](https://logging-http-client.readthedocs.io/en/latest/)
A logging library built on top of the [requests](https://pypi.org/project/requests/) library to provide a familiar
interface for sending HTTP requests with observability features out-of-the-box.
## Table of Contents
- [Logging HTTP Client](#logging-http-client)
- [Table of Contents](#table-of-contents)
- [Background](#background)
- [Usage](#usage)
- [1. Drop-in Replacement for requests](#1-drop-in-replacement-for-requests)
- [2. Using the HTTP Client with reusable Sessions](#2-using-the-http-client-with-reusable-sessions)
- [i. Disabling Reusable Sessions For The HTTP Client](#i-disabling-reusable-sessions-for-the-http-client)
- [ii. Adding Shared Headers to the HTTP Client](#ii-adding-shared-headers-to-the-http-client)
- [iii. Setting the client's `x-source`](#iii-setting-the-clients-x-source)
- [iii. `x-request-id` is automatically set](#iii-x-request-id-is-automatically-set)
- [iv. `x-correlation-id` can be automatically set](#iv-x-correlation-id-can-be-automatically-set)
- [3. Custom Logging Hooks](#3-custom-logging-hooks)
- [i. Request Logging Hook](#i-request-logging-hook)
- [ii. Response Logging Hook](#ii-response-logging-hook)
- [4. Default Logging Configurations](#4-default-logging-configurations)
- [i. Disabling Request or Response Logging](#i-disabling-request-or-response-logging)
- [ii. Enabling Request or Response Body Logging](#ii-enabling-request-or-response-body-logging)
- [iii. Customizing the logging level](#iii-customizing-the-logging-level)
- [5. Obscuring Sensitive Data](#5-obscuring-sensitive-data)
- [i. Request Log Record Obscurer](#i-request-log-record-obscurer)
- [ii. Response Log Record Obscurer](#ii-response-log-record-obscurer)
- [iii. Activating Obscurers In Your Own Logging Hooks](#iii-activating-obscurers-in-your-own-logging-hooks)
- [HTTP Log Record Structure](#http-log-record-structure)
- [Contributing](#contributing)
- [Prerequisites](#prerequisites)
- [Environment Setup](#environment-setup)
- [Code Quality](#code-quality)
- [Versioning Strategy](#versioning-strategy)
## Background
The [requests](https://pypi.org/project/requests/) library is a popular library for sending HTTP requests in Python.
However, it does not provide adequate observability features out of the box such as tracing and logging. This means
developers might be inclined to write their own custom traceability wrappers or logging utilities when writing code that
deals with requests. As such, this library serves as a drop-in replacement to the requests library that takes an
opinionated approach to provide you with common observability features out-of-the-box.
For example, by simply using this library for your requests, the following will be logged:
```python
import logging
import logging_http_client as requests
# Basic logging configuration for demonstration purposes
logging.basicConfig(
level=logging.INFO,
format='%(message)s - %(http)s'
)
requests.get(
url="https://www.python.org",
headers={"x-foo": "bar"},
)
# => Log records will include:
# message: REQUEST,
# http: {
# request_id: "6a09ec23-b318-43d2-81a1-8c1fcaf77d05",
# request_method: "GET",
# request_url: "https://www.python.org",
# request_headers: { "x-foo": "bar", "x-request-id": "6a09ec23-b318-43d2-81a1-8c1fcaf77d05", ... }
# }
#
# message: RESPONSE,
# http: {
# request_id: "6a09ec23-b318-43d2-81a1-8c1fcaf77d05",
# response_status: 200,
# response_headers: { "content-type": "text/html", ... },
# response_duration_ms: 30
# }
```
You have full control over the logging behaviour, and you can customise it to suit your needs. The library provides
hooks for custom logging, and you can disable or enable request or response logging as needed. You can also obscure
sensitive data in the log records, and set shared headers for the client instances that relies on reusable sessions
for better performance by default.
## Usage
The quickest way to get started is to install the package from PyPI:
```shell
pip install logging-http-client
```
For poetry users:
```shell
poetry add logging-http-client
```
### 1. Drop-in Replacement for requests
The library is designed to decorate requests library existing API.
Hence, you can use it in the same way you would use the [requests](https://pypi.org/project/requests/) library:
```python
import logging_http_client
response = logging_http_client.get('https://www.python.org')
print(response.status_code)
# => 200
```
Given it's built as a wrapper around the requests library, you can alias the
import to `requests` and use it as drop-in replacement for the requests' library.
```python
import logging_http_client as requests
response = requests.get('https://www.python.org')
print(response.status_code)
# => 200
```
The other HTTP methods are supported - see `requests.api`.
Full documentation is at: https://requests.readthedocs.io
### 2. Using the HTTP Client with reusable Sessions
The library provides a `LoggingHttpClient` class which is essentially a wrapper around the core component of the
requests library, the `Session` object, with additional features such as enabling reusable sessions or not.
```python
import logging_http_client
client = logging_http_client.create()
response = client.get('https://www.python.org')
print(response.status_code)
# => 200
```
#### i. Disabling Reusable Sessions For The HTTP Client
By default, the `LoggingHttpClient` class is created with a reusable session. If you want to disable this behaviour, you
can pass the `reusable_session=False` argument to the `create` method.
```python
import logging_http_client
client = logging_http_client.create(reusable_session=False)
response = client.get('https://www.python.org')
print(response.status_code)
# => 200
```
#### ii. Adding Shared Headers to the HTTP Client
You also have access to the session object headers within the `LoggingHttpClient` class, so you can add shared headers to the
session object [just like you would with the requests library](https://requests.readthedocs.io/en/latest/user/advanced/#session-objects).
```python
import logging_http_client
client = logging_http_client.create()
client.shared_headers = {"Authorization": "Bearer <token>"}
# To clear the headers, you can set it to None
client.shared_headers = None
# or delete the attribute
del client.shared_headers
```
#### iii. Setting the client's `x-source`
It's common to set a `x-source` header to identify the source of the request.
You can set this header on the client by passing the `source` argument to the
`create` method.
```python
import logging
import logging_http_client
root_logger = logging.getLogger()
root_logger.setLevel(level=logging.INFO)
client = logging_http_client.create(source="my-system-name", logger=root_logger)
response = client.get('https://www.python.org')
# => Log record will include:
# { http { request_source: "my-system-name", ... } }
```
#### iii. `x-request-id` is automatically set
The library automatically sets a `x-request-id` header on the request, and is logged within the response as well. The
`x-request-id` is a UUID that is generated for each request, and it's attached on both the request and the response
logs.
```python
import logging
import logging_http_client
root_logger = logging.getLogger()
root_logger.setLevel(level=logging.INFO)
client = logging_http_client.create(source="my-system-name", logger=root_logger)
response = client.get('https://www.python.org')
# => The client will append the `x-request-id` header to the request
#
# => Both request and response log records will include:
# { http { request_id: "<uuid>", ... } }
# => The reqeust log record will also attach it as a header:
# { http { request_headers: { "x-request-id": "<uuid>", ... }, ... } }
```
#### iv. `x-correlation-id` can be automatically set
It's common to set a `x-correlation-id` header to identify the correlation of the request within a distributed system.
Instead of having to set this header manually every single request you make, you can pass a correlation ID generator
function to the client, and it will automatically set the `x-correlation-id` header for each request.
> [!WARNING]
> Be aware that `x-request-id` is not the same as `x-correlation-id`.
>
> The `x-request-id` is unique to each request, while the `x-correlation-id` is used to correlate requests within a
> chain of events that can span multiple services, this is common in a microservice architecture. Please ensure you
> understand the difference between the two whilst using them with this library.
```python
import uuid
import logging_http_client
def correlation_id_provider() -> str:
return str(uuid.uuid4())
logging_http_client.set_correlation_id_provider(correlation_id_provider)
logging_http_client.create().get('https://www.python.org')
# => The client will append the `x-correlation-id` header to the request
#
# => The request log records will include:
# { http { request_headers: { "x-correlation-id": "<uuid>", ... }, ... } }
```
Do note we do NOT set the `x-correlation-id` header on the response, it's the responsibility of the server to set it
back on the response, if they don't, then you need to rely on your logging setup to append the `correlation_id` as an
extra log record attribute on the client side by other means.
### 3. Custom Logging Hooks
The library provides a way to attach custom logging hooks at the global level. They're intended to REPLACE the
default logging behaviour with your own logging logic. Here is how you can apply:
#### i. Request Logging Hook
The request logging hook is called **before** the request is sent. It gives you access to the client logger, and
the [prepared request](https://requests.readthedocs.io/en/latest/user/advanced/#prepared-requests) object. You can use
this hook to log the request before it's sent.
```python
import logging
from requests import PreparedRequest
import logging_http_client
def custom_request_logging_hook(logger: logging.Logger, request: PreparedRequest):
logger.debug("Custom request logging for %s", request.url)
logging_http_client.set_request_logging_hooks([custom_request_logging_hook])
logging_http_client.create().get('https://www.python.org')
# => Log record will include:
# { message { "Custom request logging for https://www.python.org" } }
```
It's worth noting that setting the logging hooks list applies a replace operation. As such, if you want to keep the
default logging hooks you will need to append them in your list, i.e.
```python
import logging
from requests import PreparedRequest
import logging_http_client
from logging_http_client import default_request_logging_hook
def custom_request_logging_hook(logger: logging.Logger, request: PreparedRequest):
logger.debug("Custom request logging for %s", request.url)
logging_http_client.set_request_logging_hooks(
[default_request_logging_hook, custom_request_logging_hook]
)
```
The hooks will be called in the order they are provided. Each hook will receive an IMMUTABLE request object.
#### ii. Response Logging Hook
The response logging hook is called **after** the response is received. It gives you access to the client logger, and
the [response object](https://requests.readthedocs.io/en/latest/api/#requests.Response). You can use this hook to log
the response after it's received.
```python
import logging
from requests import Response
import logging_http_client
def custom_response_logging_hook(logger: logging.Logger, response: Response):
logger.debug("Custom response logging for %s", response.url)
def custom_response_logging_hook_2(logger: logging.Logger, response: Response):
logger.debug("Another custom response logging for %s", response.url)
logging_http_client.set_response_logging_hooks(
[custom_response_logging_hook, custom_response_logging_hook_2]
)
logging_http_client.create().get('https://www.python.org')
# => Log records will include:
# { message { "Custom response logging for https://www.python.org" } }
# { message { "Another custom response logging for https://www.python.org" } }
```
The hooks will be called in the order they are provided. Each hook will receive an IMMUTABLE request object.
### 4. Default Logging Configurations
The default logging comes with a set of configurations that can be customised to suit your needs.
#### i. Disabling Request or Response Logging
You can disable request or response logging by calling the `disable_request_logging` or `disable_response_logging`
methods respectively. This will prevent the library from generating log records for requests or responses UNLESS you
have custom logging hooks set.
```python
import logging_http_client
logging_http_client.disable_request_logging()
logging_http_client.disable_response_logging()
logging_http_client.create().get('https://www.python.org')
# => No request log record will be generated
# => No response log record will be generated
```
#### ii. Enabling Request or Response Body Logging
By default, the library does not log the request or response body. You can enable this by calling the `enable_request_body_logging`
or `enable_response_body_logging` methods respectively. This will log the request or response body in the log record.
```python
import logging_http_client
logging_http_client.enable_request_body_logging()
logging_http_client.enable_response_body_logging()
logging_http_client.create().get('https://www.python.org')
# => Log record will include the request or response body (if present)
```
#### iii. Customizing the logging level
By default, the library provided logging hooks will log at the `INFO` level. To adjust this,
you can specify the desired level as an integer per Python log level setting conventions:
```python
import logging
import logging_http_client
"""
CRITICAL = 50
ERROR = 40
WARNING = 30
INFO = 20
DEBUG = 10
NOTSET = 0
"""
logging_http_client.set_default_hooks_logging_level(logging.DEBUG)
logging_http_client.create().get('https://www.python.org')
# => Logs will be recorded at the DEBUG level now.
```
### 5. Obscuring Sensitive Data
The library provides a way to obscure sensitive data in the request or response log records. This is useful when you
want to log the request or response body but want to obscure sensitive data such as passwords, tokens, etc.
#### i. Request Log Record Obscurer
You can set a request log record obscurer by calling the `set_request_log_record_obscurers` method. The obscurer
function should take a `HttpLogRecord` object and expects to return a modified `HttpLogRecord` object. The obscurer
function will be called JUST BEFORE the request is logged.
```python
import logging_http_client
from logging_http_client import HttpLogRecord
def request_log_record_obscurer(record: HttpLogRecord) -> HttpLogRecord:
record.request_method = "REDACTED"
if record.request_headers.get("Authorization") is not None:
record.request_headers["Authorization"] = "****"
return record
logging_http_client.set_request_log_record_obscurers([request_log_record_obscurer])
logging_http_client.create().get(
url='https://www.python.org',
headers={"Authorization": "Bearer SOME-SECRET-TOKEN"}
)
# => Log record will include:
# { http { request_headers: { "Authorization ": "****", ... }, ... } }
```
#### ii. Response Log Record Obscurer
Likewise, you can set a response log record obscurer by calling the `set_response_log_record_obscurers` method.
The obscurer function should take a `HttpLogRecord` object and expects to return a modified `HttpLogRecord` object.
```python
import logging_http_client
from logging_http_client import HttpLogRecord
def response_log_record_obscurer(record: HttpLogRecord) -> HttpLogRecord:
record.response_status = 999
if record.response_body is not None:
record.response_body = record.response_body.replace("SENSITIVE", "****")
return record
logging_http_client.set_response_log_record_obscurers([response_log_record_obscurer])
logging_http_client.enable_response_body_logging()
logging_http_client.create().get('https://www.python.org')
# Assume the response body contains "some response body with SENSITIVE information"
# => Log record will include:
# { http { response_status: 999, response_body: "some response body with **** information", ... } }
```
#### iii. Activating Obscurers In Your Own Logging Hooks
It's important to know that obscurers are applicable to hooks that utilize the `http_log_record.HttpLogRecord`
data structure, AND which call the `HttpLogRecord.from_request` method (or `HttpLogRecord.from_response` for
response obscurers) to generate the log record within the logging hook.
Also, much like the logging hooks, you can provide multiple obscurers. They will run in the order they are provided,
and they're accumulative. This means that the output of the first obscurer will be passed to the next obscurer,
and so on.
Here is a comprehensive example of a custom logging hook that correctly uses `HttpLogRecord.from_request` to apply
obscurers without having to manually run them yourself:
```python
import logging
import logging_http_client
from requests import PreparedRequest
from logging_http_client import HttpLogRecord
def custom_request_logging_hook(logger: logging.Logger, request: PreparedRequest):
logger.debug(
"Custom request logging for %s",
request.url,
# IMPORTANT: Usage of this static method will automatically apply the obscurers for you.
extra=HttpLogRecord.from_request(request)
)
def first_request_obscurer(record: HttpLogRecord) -> HttpLogRecord:
if record.request_headers.get("Authorization"):
record.request_headers["Authorization"] = "Bearer ****"
return record
def second_request_obscurer(record: HttpLogRecord) -> HttpLogRecord:
if record.request_body:
record.request_body = "OBSCURED_BODY"
return record
logging_http_client.enable_request_body_logging()
logging_http_client.set_request_logging_hooks([custom_request_logging_hook])
logging_http_client.set_request_log_record_obscurers([first_request_obscurer, second_request_obscurer])
root_logger = logging.getLogger()
root_logger.setLevel(level=logging.DEBUG)
client = logging_http_client.create(logger=root_logger)
client.post(
url="https://www.python.org",
headers={"accept": "application/json", "Authorization": "Bearer secret"},
json={"sensitive": "data"},
)
# => Log record will include:
# { http { 'request_headers': { 'Authorization': 'Bearer ****', ... }, 'request_body': 'OBSCURED_BODY', ... }
```
## HTTP Log Record Structure
The library logs HTTP requests and responses as structured log records. The log records are structured as JSON
object passed to the logger's `extra` keyword argument. The log records are structured as follows:
```json
{
"http": {
"request_id": "<uuid>",
"request_source": "<source>",
"request_method": "<method>",
"request_url": "<url>",
"request_query_params": "<query_params>",
"request_headers": "<headers>",
"request_body": "<body>",
"response_source": "<source>",
"response_status": "<status>",
"response_headers": "<headers>",
"response_duration_ms": "<duration>",
"response_body": "<body>"
}
}
```
The `request_id` is a UUID generated for each request, and it's attached to both the request and the response log
records. The `request_source` is collected form the request's `x-source` header, if it's not set, it will be `UNKNOWN`.
Likewise, the `response_source` is collected from the response's `x-source` header, but if it's not set, it will be
collected from the `request_url` host (and port) values instead.
If any of those top-level fields are `None`, `{}`, `[]`, `""`, `0`, or `0.0`,
they will be omitted from the log record for brevity purposes.
The actual data class used to represent the log record is `HttpLogRecord` and is available in the `logging_http_client`.
## Contributing
If you have any suggestions or improvements, feel free to open a PR or an issue. The build and development process has
been made to be as seamless as possible, so you can easily run and test your changes locally before submitting a PR.
### Prerequisites
- [Python](https://www.python.org/downloads/): The project is built with Python 3.12.
- [Poetry](https://python-poetry.org/docs/#installing-with-the-official-installer): The dependency management tool of
choice for this project.
- [Docker](https://docs.docker.com/engine/install/): For containerisation support, so it can be completely built and run
in an isolated environment.
- [Make](https://www.gnu.org/software/make/): For running common tasks such as installing dependencies, building the
project, running tests, etc.
### Environment Setup
Before opening the project in your IDE, I highly recommend running the following recipe:
```shell
make setup
```
This will create your Poetry's virtual environment, install the project's dependencies, set up the code quality
pre-commit hook, and configure your IDE (VSCode and PyCharm) as appropriate.
### Code Quality
We ask for adequate test coverage and adherence to the project's code quality standards. This includes running the
tests, formatter, and linter before submitting a PR. You can run the following command to ensure your changes are in
line with the project standards:
```bash
make check-code-quality
```
### Versioning Strategy
Since this project is tightly coupled with the requests library, we will follow the versioning strategy of the requests'
library. This means that the major, minor, and patch versions of this library will be the same as the requests' library
version it currently decorates. On top of that, an extra versioning suffix will be added to the end of the version to
indicate the iteration of this library.
So for example, if the requests library is at version `1.2.3`, then this library will be at version `1.2.3.X`, where `X`
is the iteration of this library, which will be numerical increments starting from `0`.
We have no intention to follow [Semantic Versioning](https://semver.org/) strategy to version this library, as I've made
a design decision to keep the features of this library's as small as possible, i.e.
_"Do few things, but do them well..."_
So for the most part, the maintenance of this library will be keeping it up-to-date with newer versions of the
the [requests](https://pypi.org/project/requests/) library, whilist ensuring **everything** still works as expeceted.
Therefore, maintaining our high test coverage is crucial for long-term useability.
___
Raw data
{
"_id": null,
"home_page": "https://github.com/u-ways/logging-http-client",
"name": "logging-http-client",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.12",
"maintainer_email": null,
"keywords": "logging, http, client, requests",
"author": "u-ways",
"author_email": "work@u-ways.info",
"download_url": "https://files.pythonhosted.org/packages/82/26/821b0a5ee49d093400b0ab7ebaffba9e8ae23f22b6a9921abb5e05eeabfd/logging_http_client-2.32.3.9.tar.gz",
"platform": null,
"description": "# Logging HTTP Client\n\n[](https://github.com/u-ways/logging-http-client/actions/workflows/CICD.yml)\n[](https://www.python.org/downloads/release/python-3110/)\n[](https://python-poetry.org/)\n[](https://flake8.pycqa.org/en/latest/)\n[](https://github.com/psf/black)\n[](https://pypi.org/project/logging-http-client/)\n[](https://pypistats.org/packages/logging-http-client)\n[](https://logging-http-client.readthedocs.io/en/latest/)\n\nA logging library built on top of the [requests](https://pypi.org/project/requests/) library to provide a familiar\ninterface for sending HTTP requests with observability features out-of-the-box.\n\n## Table of Contents\n\n- [Logging HTTP Client](#logging-http-client)\n - [Table of Contents](#table-of-contents)\n - [Background](#background)\n - [Usage](#usage)\n - [1. Drop-in Replacement for requests](#1-drop-in-replacement-for-requests)\n - [2. Using the HTTP Client with reusable Sessions](#2-using-the-http-client-with-reusable-sessions)\n - [i. Disabling Reusable Sessions For The HTTP Client](#i-disabling-reusable-sessions-for-the-http-client)\n - [ii. Adding Shared Headers to the HTTP Client](#ii-adding-shared-headers-to-the-http-client)\n - [iii. Setting the client's `x-source`](#iii-setting-the-clients-x-source)\n - [iii. `x-request-id` is automatically set](#iii-x-request-id-is-automatically-set)\n - [iv. `x-correlation-id` can be automatically set](#iv-x-correlation-id-can-be-automatically-set)\n - [3. Custom Logging Hooks](#3-custom-logging-hooks)\n - [i. Request Logging Hook](#i-request-logging-hook)\n - [ii. Response Logging Hook](#ii-response-logging-hook)\n - [4. Default Logging Configurations](#4-default-logging-configurations)\n - [i. Disabling Request or Response Logging](#i-disabling-request-or-response-logging)\n - [ii. Enabling Request or Response Body Logging](#ii-enabling-request-or-response-body-logging)\n - [iii. Customizing the logging level](#iii-customizing-the-logging-level)\n - [5. Obscuring Sensitive Data](#5-obscuring-sensitive-data)\n - [i. Request Log Record Obscurer](#i-request-log-record-obscurer)\n - [ii. Response Log Record Obscurer](#ii-response-log-record-obscurer)\n - [iii. Activating Obscurers In Your Own Logging Hooks](#iii-activating-obscurers-in-your-own-logging-hooks)\n - [HTTP Log Record Structure](#http-log-record-structure)\n - [Contributing](#contributing)\n - [Prerequisites](#prerequisites)\n - [Environment Setup](#environment-setup)\n - [Code Quality](#code-quality)\n - [Versioning Strategy](#versioning-strategy)\n\n## Background\n\nThe [requests](https://pypi.org/project/requests/) library is a popular library for sending HTTP requests in Python. \n\nHowever, it does not provide adequate observability features out of the box such as tracing and logging. This means \ndevelopers might be inclined to write their own custom traceability wrappers or logging utilities when writing code that \ndeals with requests. As such, this library serves as a drop-in replacement to the requests library that takes an \nopinionated approach to provide you with common observability features out-of-the-box.\n\nFor example, by simply using this library for your requests, the following will be logged:\n\n```python\nimport logging\nimport logging_http_client as requests\n\n# Basic logging configuration for demonstration purposes \nlogging.basicConfig(\n level=logging.INFO,\n format='%(message)s - %(http)s'\n)\n\nrequests.get(\n url=\"https://www.python.org\",\n headers={\"x-foo\": \"bar\"},\n)\n\n# => Log records will include:\n# message: REQUEST, \n# http: { \n# request_id: \"6a09ec23-b318-43d2-81a1-8c1fcaf77d05\", \n# request_method: \"GET\", \n# request_url: \"https://www.python.org\", \n# request_headers: { \"x-foo\": \"bar\", \"x-request-id\": \"6a09ec23-b318-43d2-81a1-8c1fcaf77d05\", ... } \n# }\n#\n# message: RESPONSE,\n# http: {\n# request_id: \"6a09ec23-b318-43d2-81a1-8c1fcaf77d05\",\n# response_status: 200,\n# response_headers: { \"content-type\": \"text/html\", ... },\n# response_duration_ms: 30\n# }\n```\n\nYou have full control over the logging behaviour, and you can customise it to suit your needs. The library provides\nhooks for custom logging, and you can disable or enable request or response logging as needed. You can also obscure\nsensitive data in the log records, and set shared headers for the client instances that relies on reusable sessions\nfor better performance by default.\n\n## Usage\n\nThe quickest way to get started is to install the package from PyPI:\n\n```shell\npip install logging-http-client\n```\n\nFor poetry users:\n\n```shell\npoetry add logging-http-client\n```\n\n### 1. Drop-in Replacement for requests\n\nThe library is designed to decorate requests library existing API.\nHence, you can use it in the same way you would use the [requests](https://pypi.org/project/requests/) library:\n\n```python\nimport logging_http_client\n\nresponse = logging_http_client.get('https://www.python.org')\nprint(response.status_code)\n# => 200\n```\n\nGiven it's built as a wrapper around the requests library, you can alias the\nimport to `requests` and use it as drop-in replacement for the requests' library.\n\n```python\nimport logging_http_client as requests\n\nresponse = requests.get('https://www.python.org')\nprint(response.status_code)\n# => 200\n```\n\nThe other HTTP methods are supported - see `requests.api`.\nFull documentation is at: https://requests.readthedocs.io\n\n### 2. Using the HTTP Client with reusable Sessions\n\nThe library provides a `LoggingHttpClient` class which is essentially a wrapper around the core component of the\nrequests library, the `Session` object, with additional features such as enabling reusable sessions or not.\n\n```python\nimport logging_http_client\n\nclient = logging_http_client.create()\n\nresponse = client.get('https://www.python.org')\nprint(response.status_code)\n# => 200\n```\n\n#### i. Disabling Reusable Sessions For The HTTP Client\n\nBy default, the `LoggingHttpClient` class is created with a reusable session. If you want to disable this behaviour, you\ncan pass the `reusable_session=False` argument to the `create` method.\n\n```python\nimport logging_http_client\n\nclient = logging_http_client.create(reusable_session=False)\n\nresponse = client.get('https://www.python.org')\nprint(response.status_code)\n# => 200\n```\n\n#### ii. Adding Shared Headers to the HTTP Client\n\nYou also have access to the session object headers within the `LoggingHttpClient` class, so you can add shared headers to the\nsession object [just like you would with the requests library](https://requests.readthedocs.io/en/latest/user/advanced/#session-objects).\n\n```python\nimport logging_http_client\n\nclient = logging_http_client.create()\n\nclient.shared_headers = {\"Authorization\": \"Bearer <token>\"}\n\n# To clear the headers, you can set it to None\nclient.shared_headers = None\n# or delete the attribute\ndel client.shared_headers\n```\n\n#### iii. Setting the client's `x-source`\n\nIt's common to set a `x-source` header to identify the source of the request.\nYou can set this header on the client by passing the `source` argument to the\n`create` method.\n\n```python\nimport logging\n\nimport logging_http_client\n\nroot_logger = logging.getLogger()\nroot_logger.setLevel(level=logging.INFO)\n\nclient = logging_http_client.create(source=\"my-system-name\", logger=root_logger)\n\nresponse = client.get('https://www.python.org')\n# => Log record will include: \n# { http { request_source: \"my-system-name\", ... } }\n```\n\n#### iii. `x-request-id` is automatically set\n\nThe library automatically sets a `x-request-id` header on the request, and is logged within the response as well. The\n`x-request-id` is a UUID that is generated for each request, and it's attached on both the request and the response\nlogs.\n\n```python\nimport logging\n\nimport logging_http_client\n\nroot_logger = logging.getLogger()\nroot_logger.setLevel(level=logging.INFO)\n\nclient = logging_http_client.create(source=\"my-system-name\", logger=root_logger)\n\nresponse = client.get('https://www.python.org')\n# => The client will append the `x-request-id` header to the request\n#\n# => Both request and response log records will include: \n# { http { request_id: \"<uuid>\", ... } }\n# => The reqeust log record will also attach it as a header: \n# { http { request_headers: { \"x-request-id\": \"<uuid>\", ... }, ... } }\n```\n\n#### iv. `x-correlation-id` can be automatically set\n\nIt's common to set a `x-correlation-id` header to identify the correlation of the request within a distributed system.\nInstead of having to set this header manually every single request you make, you can pass a correlation ID generator\nfunction to the client, and it will automatically set the `x-correlation-id` header for each request.\n\n> [!WARNING]\n> Be aware that `x-request-id` is not the same as `x-correlation-id`.\n> \n> The `x-request-id` is unique to each request, while the `x-correlation-id` is used to correlate requests within a\n> chain of events that can span multiple services, this is common in a microservice architecture. Please ensure you\n> understand the difference between the two whilst using them with this library.\n\n```python\nimport uuid\nimport logging_http_client \n\ndef correlation_id_provider() -> str:\n return str(uuid.uuid4())\n\nlogging_http_client.set_correlation_id_provider(correlation_id_provider)\n\nlogging_http_client.create().get('https://www.python.org')\n# => The client will append the `x-correlation-id` header to the request \n#\n# => The request log records will include:\n# { http { request_headers: { \"x-correlation-id\": \"<uuid>\", ... }, ... } }\n```\n\nDo note we do NOT set the `x-correlation-id` header on the response, it's the responsibility of the server to set it\nback on the response, if they don't, then you need to rely on your logging setup to append the `correlation_id` as an \nextra log record attribute on the client side by other means.\n\n### 3. Custom Logging Hooks\n\nThe library provides a way to attach custom logging hooks at the global level. They're intended to REPLACE the\ndefault logging behaviour with your own logging logic. Here is how you can apply:\n\n#### i. Request Logging Hook\n\nThe request logging hook is called **before** the request is sent. It gives you access to the client logger, and\nthe [prepared request](https://requests.readthedocs.io/en/latest/user/advanced/#prepared-requests) object. You can use \nthis hook to log the request before it's sent.\n\n```python\nimport logging\n\nfrom requests import PreparedRequest\n\nimport logging_http_client\n\n\ndef custom_request_logging_hook(logger: logging.Logger, request: PreparedRequest):\n logger.debug(\"Custom request logging for %s\", request.url)\n\n\nlogging_http_client.set_request_logging_hooks([custom_request_logging_hook])\n\nlogging_http_client.create().get('https://www.python.org')\n\n# => Log record will include:\n# { message { \"Custom request logging for https://www.python.org\" } }\n```\n\nIt's worth noting that setting the logging hooks list applies a replace operation. As such, if you want to keep the \ndefault logging hooks you will need to append them in your list, i.e.\n\n```python\nimport logging\n\nfrom requests import PreparedRequest\n\nimport logging_http_client\nfrom logging_http_client import default_request_logging_hook\n\n\ndef custom_request_logging_hook(logger: logging.Logger, request: PreparedRequest):\n logger.debug(\"Custom request logging for %s\", request.url)\n\n\nlogging_http_client.set_request_logging_hooks(\n [default_request_logging_hook, custom_request_logging_hook]\n)\n```\n\nThe hooks will be called in the order they are provided. Each hook will receive an IMMUTABLE request object.\n\n#### ii. Response Logging Hook\n\nThe response logging hook is called **after** the response is received. It gives you access to the client logger, and\nthe [response object](https://requests.readthedocs.io/en/latest/api/#requests.Response). You can use this hook to log \nthe response after it's received.\n\n```python\nimport logging\n\nfrom requests import Response\n\nimport logging_http_client\n\n\ndef custom_response_logging_hook(logger: logging.Logger, response: Response):\n logger.debug(\"Custom response logging for %s\", response.url)\n\n\ndef custom_response_logging_hook_2(logger: logging.Logger, response: Response):\n logger.debug(\"Another custom response logging for %s\", response.url)\n\n\nlogging_http_client.set_response_logging_hooks(\n [custom_response_logging_hook, custom_response_logging_hook_2]\n)\n\nlogging_http_client.create().get('https://www.python.org')\n\n# => Log records will include:\n# { message { \"Custom response logging for https://www.python.org\" } }\n# { message { \"Another custom response logging for https://www.python.org\" } }\n```\n\nThe hooks will be called in the order they are provided. Each hook will receive an IMMUTABLE request object.\n\n### 4. Default Logging Configurations\n\nThe default logging comes with a set of configurations that can be customised to suit your needs.\n\n#### i. Disabling Request or Response Logging\n\nYou can disable request or response logging by calling the `disable_request_logging` or `disable_response_logging`\nmethods respectively. This will prevent the library from generating log records for requests or responses UNLESS you\nhave custom logging hooks set.\n\n```python\nimport logging_http_client\n\nlogging_http_client.disable_request_logging()\nlogging_http_client.disable_response_logging()\n\nlogging_http_client.create().get('https://www.python.org')\n# => No request log record will be generated\n# => No response log record will be generated\n```\n\n#### ii. Enabling Request or Response Body Logging\n\nBy default, the library does not log the request or response body. You can enable this by calling the `enable_request_body_logging`\nor `enable_response_body_logging` methods respectively. This will log the request or response body in the log record.\n\n```python\nimport logging_http_client\n\nlogging_http_client.enable_request_body_logging()\nlogging_http_client.enable_response_body_logging()\n\nlogging_http_client.create().get('https://www.python.org')\n# => Log record will include the request or response body (if present)\n```\n\n#### iii. Customizing the logging level\n\nBy default, the library provided logging hooks will log at the `INFO` level. To adjust this, \nyou can specify the desired level as an integer per Python log level setting conventions:\n\n```python\nimport logging\n\nimport logging_http_client\n\n\"\"\"\nCRITICAL = 50\nERROR = 40\nWARNING = 30\nINFO = 20\nDEBUG = 10\nNOTSET = 0\n\"\"\"\nlogging_http_client.set_default_hooks_logging_level(logging.DEBUG)\n\nlogging_http_client.create().get('https://www.python.org')\n# => Logs will be recorded at the DEBUG level now.\n```\n\n### 5. Obscuring Sensitive Data\n\nThe library provides a way to obscure sensitive data in the request or response log records. This is useful when you\nwant to log the request or response body but want to obscure sensitive data such as passwords, tokens, etc.\n\n#### i. Request Log Record Obscurer\n\nYou can set a request log record obscurer by calling the `set_request_log_record_obscurers` method. The obscurer\nfunction should take a `HttpLogRecord` object and expects to return a modified `HttpLogRecord` object. The obscurer\nfunction will be called JUST BEFORE the request is logged.\n\n```python\nimport logging_http_client\nfrom logging_http_client import HttpLogRecord\n\n\ndef request_log_record_obscurer(record: HttpLogRecord) -> HttpLogRecord:\n record.request_method = \"REDACTED\"\n if record.request_headers.get(\"Authorization\") is not None:\n record.request_headers[\"Authorization\"] = \"****\"\n return record\n\n\nlogging_http_client.set_request_log_record_obscurers([request_log_record_obscurer])\n\nlogging_http_client.create().get(\n url='https://www.python.org',\n headers={\"Authorization\": \"Bearer SOME-SECRET-TOKEN\"}\n)\n\n# => Log record will include:\n# { http { request_headers: { \"Authorization \": \"****\", ... }, ... } }\n```\n\n#### ii. Response Log Record Obscurer\n\nLikewise, you can set a response log record obscurer by calling the `set_response_log_record_obscurers` method.\nThe obscurer function should take a `HttpLogRecord` object and expects to return a modified `HttpLogRecord` object.\n\n```python\nimport logging_http_client\nfrom logging_http_client import HttpLogRecord\n\n\ndef response_log_record_obscurer(record: HttpLogRecord) -> HttpLogRecord:\n record.response_status = 999\n if record.response_body is not None:\n record.response_body = record.response_body.replace(\"SENSITIVE\", \"****\")\n return record\n\n\nlogging_http_client.set_response_log_record_obscurers([response_log_record_obscurer])\nlogging_http_client.enable_response_body_logging()\n\nlogging_http_client.create().get('https://www.python.org')\n# Assume the response body contains \"some response body with SENSITIVE information\" \n\n# => Log record will include:\n# { http { response_status: 999, response_body: \"some response body with **** information\", ... } }\n```\n\n#### iii. Activating Obscurers In Your Own Logging Hooks\n\nIt's important to know that obscurers are applicable to hooks that utilize the `http_log_record.HttpLogRecord`\ndata structure, AND which call the `HttpLogRecord.from_request` method (or `HttpLogRecord.from_response` for \nresponse obscurers) to generate the log record within the logging hook. \n\nAlso, much like the logging hooks, you can provide multiple obscurers. They will run in the order they are provided, \nand they're accumulative. This means that the output of the first obscurer will be passed to the next obscurer, \nand so on.\n\nHere is a comprehensive example of a custom logging hook that correctly uses `HttpLogRecord.from_request` to apply \nobscurers without having to manually run them yourself:\n\n```python\nimport logging\nimport logging_http_client\n\nfrom requests import PreparedRequest\nfrom logging_http_client import HttpLogRecord\n\n\ndef custom_request_logging_hook(logger: logging.Logger, request: PreparedRequest):\n logger.debug(\n \"Custom request logging for %s\",\n request.url,\n # IMPORTANT: Usage of this static method will automatically apply the obscurers for you.\n extra=HttpLogRecord.from_request(request)\n )\n\n\ndef first_request_obscurer(record: HttpLogRecord) -> HttpLogRecord:\n if record.request_headers.get(\"Authorization\"):\n record.request_headers[\"Authorization\"] = \"Bearer ****\"\n return record\n\n\ndef second_request_obscurer(record: HttpLogRecord) -> HttpLogRecord:\n if record.request_body:\n record.request_body = \"OBSCURED_BODY\"\n return record\n\n\nlogging_http_client.enable_request_body_logging()\nlogging_http_client.set_request_logging_hooks([custom_request_logging_hook])\nlogging_http_client.set_request_log_record_obscurers([first_request_obscurer, second_request_obscurer])\n\nroot_logger = logging.getLogger()\nroot_logger.setLevel(level=logging.DEBUG)\n\nclient = logging_http_client.create(logger=root_logger)\nclient.post(\n url=\"https://www.python.org\",\n headers={\"accept\": \"application/json\", \"Authorization\": \"Bearer secret\"},\n json={\"sensitive\": \"data\"},\n)\n# => Log record will include:\n# { http { 'request_headers': { 'Authorization': 'Bearer ****', ... }, 'request_body': 'OBSCURED_BODY', ... }\n```\n\n## HTTP Log Record Structure\n\nThe library logs HTTP requests and responses as structured log records. The log records are structured as JSON\nobject passed to the logger's `extra` keyword argument. The log records are structured as follows:\n\n```json\n{\n \"http\": {\n \"request_id\": \"<uuid>\",\n \"request_source\": \"<source>\",\n \"request_method\": \"<method>\",\n \"request_url\": \"<url>\",\n \"request_query_params\": \"<query_params>\",\n \"request_headers\": \"<headers>\",\n \"request_body\": \"<body>\",\n \"response_source\": \"<source>\",\n \"response_status\": \"<status>\",\n \"response_headers\": \"<headers>\",\n \"response_duration_ms\": \"<duration>\",\n \"response_body\": \"<body>\"\n }\n}\n```\n\nThe `request_id` is a UUID generated for each request, and it's attached to both the request and the response log \nrecords. The `request_source` is collected form the request's `x-source` header, if it's not set, it will be `UNKNOWN`.\nLikewise, the `response_source` is collected from the response's `x-source` header, but if it's not set, it will be \ncollected from the `request_url` host (and port) values instead.\n\nIf any of those top-level fields are `None`, `{}`, `[]`, `\"\"`, `0`, or `0.0`,\nthey will be omitted from the log record for brevity purposes.\n\nThe actual data class used to represent the log record is `HttpLogRecord` and is available in the `logging_http_client`.\n\n## Contributing\n\nIf you have any suggestions or improvements, feel free to open a PR or an issue. The build and development process has\nbeen made to be as seamless as possible, so you can easily run and test your changes locally before submitting a PR.\n\n### Prerequisites\n\n- [Python](https://www.python.org/downloads/): The project is built with Python 3.12.\n- [Poetry](https://python-poetry.org/docs/#installing-with-the-official-installer): The dependency management tool of\n choice for this project.\n- [Docker](https://docs.docker.com/engine/install/): For containerisation support, so it can be completely built and run\n in an isolated environment.\n- [Make](https://www.gnu.org/software/make/): For running common tasks such as installing dependencies, building the\n project, running tests, etc.\n\n### Environment Setup\n\nBefore opening the project in your IDE, I highly recommend running the following recipe:\n\n```shell\nmake setup\n```\n\nThis will create your Poetry's virtual environment, install the project's dependencies, set up the code quality\npre-commit hook, and configure your IDE (VSCode and PyCharm) as appropriate.\n\n### Code Quality\n\nWe ask for adequate test coverage and adherence to the project's code quality standards. This includes running the\ntests, formatter, and linter before submitting a PR. You can run the following command to ensure your changes are in\nline with the project standards:\n\n```bash\nmake check-code-quality\n```\n\n### Versioning Strategy\n\nSince this project is tightly coupled with the requests library, we will follow the versioning strategy of the requests'\nlibrary. This means that the major, minor, and patch versions of this library will be the same as the requests' library\nversion it currently decorates. On top of that, an extra versioning suffix will be added to the end of the version to \nindicate the iteration of this library.\n\nSo for example, if the requests library is at version `1.2.3`, then this library will be at version `1.2.3.X`, where `X`\nis the iteration of this library, which will be numerical increments starting from `0`. \n\nWe have no intention to follow [Semantic Versioning](https://semver.org/) strategy to version this library, as I've made\na design decision to keep the features of this library's as small as possible, i.e. \n\n_\"Do few things, but do them well...\"_\n\nSo for the most part, the maintenance of this library will be keeping it up-to-date with newer versions of the\nthe [requests](https://pypi.org/project/requests/) library, whilist ensuring **everything** still works as expeceted.\nTherefore, maintaining our high test coverage is crucial for long-term useability.\n\n___\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A logging library built on top of the requests library to provide a familiar interface for sending HTTP requests with observability features out-of-the-box.",
"version": "2.32.3.9",
"project_urls": {
"Documentation": "https://github.com/u-ways/logging-http-client/blob/main/README.md",
"Homepage": "https://github.com/u-ways/logging-http-client",
"Repository": "https://github.com/u-ways/logging-http-client"
},
"split_keywords": [
"logging",
" http",
" client",
" requests"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "44728ea3604b4a88f577397fcc22b0347627fc8b3715c8315dec2ecc271bf6cf",
"md5": "f6b12c9ba792df11ffc05d0034dbbca5",
"sha256": "aa93dbde9652900f5a555e08a8d2b5663e720af1426e019b030f916fcd454dfc"
},
"downloads": -1,
"filename": "logging_http_client-2.32.3.9-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f6b12c9ba792df11ffc05d0034dbbca5",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.12",
"size": 17539,
"upload_time": "2024-12-17T09:57:52",
"upload_time_iso_8601": "2024-12-17T09:57:52.050703Z",
"url": "https://files.pythonhosted.org/packages/44/72/8ea3604b4a88f577397fcc22b0347627fc8b3715c8315dec2ecc271bf6cf/logging_http_client-2.32.3.9-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8226821b0a5ee49d093400b0ab7ebaffba9e8ae23f22b6a9921abb5e05eeabfd",
"md5": "89f91698fc729c7b1c057ea361127071",
"sha256": "87b6dcce6bf73178a8f34313a95838c572f649b92fbc8a1b6a13d12a91ca8e3a"
},
"downloads": -1,
"filename": "logging_http_client-2.32.3.9.tar.gz",
"has_sig": false,
"md5_digest": "89f91698fc729c7b1c057ea361127071",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.12",
"size": 20573,
"upload_time": "2024-12-17T09:57:54",
"upload_time_iso_8601": "2024-12-17T09:57:54.474378Z",
"url": "https://files.pythonhosted.org/packages/82/26/821b0a5ee49d093400b0ab7ebaffba9e8ae23f22b6a9921abb5e05eeabfd/logging_http_client-2.32.3.9.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-17 09:57:54",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "u-ways",
"github_project": "logging-http-client",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "logging-http-client"
}
JSON
