# SecureLogs Library with Sensitive Value Masking
The **SecureLogs Library** provides logging functionality with **Trace ID** support and the ability to **mask sensitive values** in logs (such as credit card numbers, SSNs, tokens, etc.). It is designed for easy integration with **FastAPI** applications.
This guide will walk you through how to install, configure, and use the library to log trace information while hiding sensitive values.
## Table of Contents
1. [Installation](#1-installationinstallation)
2. [Basic Setup](#2-basic-setupbasic-setup)
3. [Configuring Logger](#3-configuring-logger)
4. [Using Sensitive Value Masking](#4-using-sensitive-value-masking)
5. [FastAPI Integration](#5-fastapi-integration)
6. [Example Output](#6-example-output)
7. [Configuration Options](#7-configuration-options)
8. [Conclusion](#8-conclusion)
9. [Example Implementation](#9-example-implementation)
## 1. Installation
To install the SecureLogs Library, use `pip`:
```bash
pip install SecureLogs
```
## 2. Basic Setup
Once installed, you need to configure the logger and integrate it into your FastAPI project.
### a. Import Required Components
In your FastAPI application, import the necessary components from the library.
```python
from uuid import UUID
from fastapi import Depends, FastAPI
from secure_logs import configure_logging, get_logger
from secure_logs import get_trace_id
from secure_logs.middleware import TraceIDMiddleware
```
### b. Configure Logging
You need to explicitly configure logging by calling configure_logging() with your preferred logging level (e.g., debug, info, etc.).
```python
# Set up logging with the desired level
configure_logging(level="debug")
app = FastAPI()
app.add_middleware(TraceIDMiddleware)
```
## 3. Configuring Logger
The logger can be configured to mask sensitive values such as credit card numbers, tokens, and other confidential data. You can also define how many characters of the sensitive data should be shown after masking.
### a. Define Patterns for Sensitive Values
You can provide a list of regular expression patterns or strings that match sensitive data (e.g., credit card numbers, tokens).
Example patterns to mask:
Credit card numbers (\d{16})
SSNs ((?:\d{3}-\d{2}-\d{4}))
User-specific text (User)
Tokens ((?<=Bearer\s)[a-zA-Z0-9]+)
### b. Initialize Logger with Masking
Use the `get_logger()` function to initialize the logger and provide sensitive value patterns and the number of visible characters.
```python
# Define patterns for sensitive data
sensitive_patterns = [
r"\d{16}", # Example: credit card numbers
r"(?:\d{3}-\d{2}-\d{4})", # Example: SSNs
"User", # Example: any text like 'User'
r"(?<=Bearer\s)[a-zA-Z0-9]+" # Example: token pattern
]
# Get the logger instance with sensitive patterns and show the last 2 characters
logger = get_logger(__name__, sensitive_patterns=sensitive_patterns, show_last=2)
```
### c. Example Logger Usage
```python
logger.debug("Debug message with sensitive data: 1234567812345678.")
logger.info("User information: SSN 123-45-6789.")
logger.warning("Token authorization: Bearer abc123DEF456")
```
## 4. Using Sensitive Value Masking
The library allows you to redact sensitive values in log messages based on user-defined patterns.
### Example of Redacting Sensitive Values
When logging a message, the logger will automatically mask sensitive data in accordance with the defined patterns.
Example Input:
```python
logger.info("User credit card: 1234567812345678.")
logger.info("User SSN: 123-45-6789.")
logger.info("Token authorization: Bearer abc123DEF456")
```
Example Output:
```bash
2024-12-01 10:00:00 - __main__ - INFO - [trace_id: c0095715-d5bb-4991-9176-c5335368e481] [function: get_user_info] User credit card: **************78
2024-12-01 10:00:01 - __main__ - INFO - [trace_id: c0095715-d5bb-4991-9176-c5335368e481] [function: get_user_info] User SSN: ***-**-**89
2024-12-01 10:00:02 - __main__ - INFO - [trace_id: c0095715-d5bb-4991-9176-c5335368e481] [function: get_user_info] Token authorization: Bearer ********56
```
### Customizing the Masking Behavior
You can control how many characters should remain visible after masking the sensitive data by using the show_last option. By default, it will mask the entire sensitive value, but you can customize it like this:
```python
# Mask with 2 visible characters after the mask
logger = get_logger(__name__, sensitive_patterns=sensitive_patterns, show_last=2)
```
## 5. FastAPI Integration
The library works seamlessly with FastAPI. You can use TraceIDMiddleware to add trace IDs to every request and pass them along with the logs.
### a. FastAPI Middleware
The middleware captures the trace ID from the request headers or generates a new one and passes it to the logger.
```python
# Add TraceIDMiddleware to FastAPI
app.add_middleware(TraceIDMiddleware)
```
### b. Using Trace ID in FastAPI Endpoints
When defining your FastAPI endpoints, you can easily include the trace ID by using Depends(get_trace_id).
```python
@app.get("/")
def say_hello(name: str = "Dev", trace_id: UUID = Depends(get_trace_id)):
logger.debug("This is debug level log.")
logger.info("This is info level log.")
logger.error("This is error level log.")
logger.warning("This is warning level log.")
return {"Message": f"Hello {name}"}
```
## 6. Example Output
### Example Log Messages
When logging messages with sensitive data, the library will mask sensitive parts of the values based on the configured patterns.
```bash
2024-12-01 10:00:00 - __main__ - INFO - User credit card: ****************
2024-12-01 10:00:01 - __main__ - INFO - User SSN: ***-**-6789
2024-12-01 10:00:02 - __main__ - INFO - Token authorization: Bearer **********
```
## 7. Configuration Options
### a. Logging Level Configuration
The logging level can be configured using the configure_logging function.
```python
configure_logging(level="debug")
```
The available logging levels are:
* `DEBUG`
* `INFO`
* `WARNING`
* `ERROR`
* `CRITICAL`
### b. Sensitive Data Masking Patterns
You can provide a list of regular expressions or exact strings for sensitive data. Here are some examples:
* `r"\d{16}"`: Match credit card numbers.
* `r"(?:\d{3}-\d{2}-\d{4})"`: Match SSNs.
* `"User"`: Match the text "User".
* `r"(?<=Bearer\s)[a-zA-Z0-9]+"`: Match tokens (e.g., Bearer tokens).
### c. `show_last` Option
This option determines how many characters should remain visible after masking. The default is 0, which means the entire value is masked.
```python
logger = get_logger(__name__, sensitive_patterns=sensitive_patterns, show_last=2)
```
## 8. Conclusion
The SecureLogs Library simplifies logging with trace ID support and provides powerful features to mask sensitive data in logs. With easy integration into FastAPI, this library ensures that sensitive data like credit card numbers, SSNs, and tokens are securely hidden while providing useful trace information.
By using this library, you can ensure your application’s logs are secure, readable, and traceable.
## 9. Example Implementation
```python
from uuid import UUID
from fastapi import Depends, FastAPI
from secure_logs import configure_logging, get_logger
from secure_logs import get_trace_id
from secure_logs.middleware import TraceIDMiddleware
# You can explicitly configure logging
configure_logging(level="debug") # Optional
app = FastAPI()
app.add_middleware(TraceIDMiddleware)
# Keep this in a common file from where you can access through out the project
# logger = get_logger(__name__,) # General implementation
# logger = get_logger(__name__, sensitive_patterns=['This', 'log']) # Hide sensitive values with *
# logger = get_logger(__name__, sensitive_patterns=['This', 'log'],show_last=1) # Hide sensitive values with * by showing only last 1 item
# Configure sensitive value filter
sensitive_patterns = [
r"\d{16}", # Example: credit card numbers
r"(?:\d{3}-\d{2}-\d{4})", # Example: SSNs
"User", # Example: any text
"level",
"log",
r"(?<=Bearer\s)[a-zA-Z0-9]+", # Example: token
]
logger = get_logger(__name__, sensitive_patterns=sensitive_patterns, show_last=2)
@app.get("/")
def say_hello(name: str = "Dev", trace_id: UUID = Depends(get_trace_id)):
logger.debug("This is debug level log.")
logger.info("This is info level log.")
logger.error("This is error level log.")
logger.warning("This is warning level log.")
return {"Message": f"Hello {name}"}
@app.get("/userinfo")
def get_user_info(trace_id: UUID = Depends(get_trace_id)):
logger.info("User credit card: 1234567812345678.")
logger.info("User SSN: 123-45-6789.")
logger.info("Token authorization: Bearer abc123DEF456")
return {"user": "Dev"}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/Happy-Kumar-Sharma/trace-logger",
"name": "SecureLogs",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "secure logging, SecureLogs, python logging module, best logging library, mask sensitive values",
"author": "Happy Sharma",
"author_email": "happycse54@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/b3/a0/22d5e6be673895a86029bf42530fba4969ac8822659c851cdc10832adc2e/securelogs-0.0.2.tar.gz",
"platform": null,
"description": "# SecureLogs Library with Sensitive Value Masking\n\nThe **SecureLogs Library** provides logging functionality with **Trace ID** support and the ability to **mask sensitive values** in logs (such as credit card numbers, SSNs, tokens, etc.). It is designed for easy integration with **FastAPI** applications.\n\nThis guide will walk you through how to install, configure, and use the library to log trace information while hiding sensitive values.\n\n## Table of Contents\n\n1. [Installation](#1-installationinstallation)\n2. [Basic Setup](#2-basic-setupbasic-setup)\n3. [Configuring Logger](#3-configuring-logger)\n4. [Using Sensitive Value Masking](#4-using-sensitive-value-masking)\n5. [FastAPI Integration](#5-fastapi-integration)\n6. [Example Output](#6-example-output)\n7. [Configuration Options](#7-configuration-options)\n8. [Conclusion](#8-conclusion)\n9. [Example Implementation](#9-example-implementation)\n\n## 1. Installation\n\nTo install the SecureLogs Library, use `pip`:\n\n```bash\npip install SecureLogs\n```\n\n## 2. Basic Setup\nOnce installed, you need to configure the logger and integrate it into your FastAPI project.\n\n### a. Import Required Components\nIn your FastAPI application, import the necessary components from the library.\n```python\nfrom uuid import UUID\nfrom fastapi import Depends, FastAPI\nfrom secure_logs import configure_logging, get_logger\nfrom secure_logs import get_trace_id\nfrom secure_logs.middleware import TraceIDMiddleware\n```\n\n### b. Configure Logging\nYou need to explicitly configure logging by calling configure_logging() with your preferred logging level (e.g., debug, info, etc.).\n\n```python\n# Set up logging with the desired level\nconfigure_logging(level=\"debug\")\n\napp = FastAPI()\napp.add_middleware(TraceIDMiddleware)\n\n```\n## 3. Configuring Logger\nThe logger can be configured to mask sensitive values such as credit card numbers, tokens, and other confidential data. You can also define how many characters of the sensitive data should be shown after masking.\n\n### a. Define Patterns for Sensitive Values\nYou can provide a list of regular expression patterns or strings that match sensitive data (e.g., credit card numbers, tokens).\n\nExample patterns to mask:\n\n Credit card numbers (\\d{16})\n SSNs ((?:\\d{3}-\\d{2}-\\d{4}))\n User-specific text (User)\n Tokens ((?<=Bearer\\s)[a-zA-Z0-9]+)\n\n### b. Initialize Logger with Masking\nUse the `get_logger()` function to initialize the logger and provide sensitive value patterns and the number of visible characters.\n\n```python\n# Define patterns for sensitive data\nsensitive_patterns = [\n r\"\\d{16}\", # Example: credit card numbers\n r\"(?:\\d{3}-\\d{2}-\\d{4})\", # Example: SSNs\n \"User\", # Example: any text like 'User'\n r\"(?<=Bearer\\s)[a-zA-Z0-9]+\" # Example: token pattern\n]\n\n# Get the logger instance with sensitive patterns and show the last 2 characters\nlogger = get_logger(__name__, sensitive_patterns=sensitive_patterns, show_last=2)\n```\n\n### c. Example Logger Usage\n```python\nlogger.debug(\"Debug message with sensitive data: 1234567812345678.\")\nlogger.info(\"User information: SSN 123-45-6789.\")\nlogger.warning(\"Token authorization: Bearer abc123DEF456\")\n```\n\n## 4. Using Sensitive Value Masking\nThe library allows you to redact sensitive values in log messages based on user-defined patterns.\n\n### Example of Redacting Sensitive Values\n\nWhen logging a message, the logger will automatically mask sensitive data in accordance with the defined patterns.\n\nExample Input:\n\n```python\nlogger.info(\"User credit card: 1234567812345678.\")\nlogger.info(\"User SSN: 123-45-6789.\")\nlogger.info(\"Token authorization: Bearer abc123DEF456\")\n```\n\nExample Output:\n\n```bash\n2024-12-01 10:00:00 - __main__ - INFO - [trace_id: c0095715-d5bb-4991-9176-c5335368e481] [function: get_user_info] User credit card: **************78\n2024-12-01 10:00:01 - __main__ - INFO - [trace_id: c0095715-d5bb-4991-9176-c5335368e481] [function: get_user_info] User SSN: ***-**-**89\n2024-12-01 10:00:02 - __main__ - INFO - [trace_id: c0095715-d5bb-4991-9176-c5335368e481] [function: get_user_info] Token authorization: Bearer ********56\n```\n\n### Customizing the Masking Behavior\nYou can control how many characters should remain visible after masking the sensitive data by using the show_last option. By default, it will mask the entire sensitive value, but you can customize it like this:\n\n```python\n# Mask with 2 visible characters after the mask\nlogger = get_logger(__name__, sensitive_patterns=sensitive_patterns, show_last=2)\n```\n\n## 5. FastAPI Integration\nThe library works seamlessly with FastAPI. You can use TraceIDMiddleware to add trace IDs to every request and pass them along with the logs.\n\n### a. FastAPI Middleware\nThe middleware captures the trace ID from the request headers or generates a new one and passes it to the logger.\n\n```python\n# Add TraceIDMiddleware to FastAPI\napp.add_middleware(TraceIDMiddleware)\n```\n\n### b. Using Trace ID in FastAPI Endpoints\nWhen defining your FastAPI endpoints, you can easily include the trace ID by using Depends(get_trace_id).\n\n```python\n@app.get(\"/\")\ndef say_hello(name: str = \"Dev\", trace_id: UUID = Depends(get_trace_id)):\n logger.debug(\"This is debug level log.\")\n logger.info(\"This is info level log.\")\n logger.error(\"This is error level log.\")\n logger.warning(\"This is warning level log.\")\n return {\"Message\": f\"Hello {name}\"}\n```\n\n## 6. Example Output\n\n### Example Log Messages\nWhen logging messages with sensitive data, the library will mask sensitive parts of the values based on the configured patterns.\n\n```bash\n2024-12-01 10:00:00 - __main__ - INFO - User credit card: ****************\n2024-12-01 10:00:01 - __main__ - INFO - User SSN: ***-**-6789\n2024-12-01 10:00:02 - __main__ - INFO - Token authorization: Bearer **********\n```\n\n## 7. Configuration Options\n### a. Logging Level Configuration\nThe logging level can be configured using the configure_logging function.\n\n```python\nconfigure_logging(level=\"debug\")\n```\n\nThe available logging levels are:\n\n* `DEBUG`\n* `INFO`\n* `WARNING`\n* `ERROR`\n* `CRITICAL`\n\n### b. Sensitive Data Masking Patterns\nYou can provide a list of regular expressions or exact strings for sensitive data. Here are some examples:\n\n* `r\"\\d{16}\"`: Match credit card numbers.\n* `r\"(?:\\d{3}-\\d{2}-\\d{4})\"`: Match SSNs.\n* `\"User\"`: Match the text \"User\".\n* `r\"(?<=Bearer\\s)[a-zA-Z0-9]+\"`: Match tokens (e.g., Bearer tokens).\n\n### c. `show_last` Option\nThis option determines how many characters should remain visible after masking. The default is 0, which means the entire value is masked.\n\n```python\nlogger = get_logger(__name__, sensitive_patterns=sensitive_patterns, show_last=2)\n```\n\n## 8. Conclusion\nThe SecureLogs Library simplifies logging with trace ID support and provides powerful features to mask sensitive data in logs. With easy integration into FastAPI, this library ensures that sensitive data like credit card numbers, SSNs, and tokens are securely hidden while providing useful trace information.\n\nBy using this library, you can ensure your application\u2019s logs are secure, readable, and traceable.\n\n## 9. Example Implementation\n```python\nfrom uuid import UUID\n\nfrom fastapi import Depends, FastAPI\n\nfrom secure_logs import configure_logging, get_logger\nfrom secure_logs import get_trace_id\nfrom secure_logs.middleware import TraceIDMiddleware\n\n# You can explicitly configure logging\nconfigure_logging(level=\"debug\") # Optional\n\napp = FastAPI()\napp.add_middleware(TraceIDMiddleware)\n\n# Keep this in a common file from where you can access through out the project\n# logger = get_logger(__name__,) # General implementation\n# logger = get_logger(__name__, sensitive_patterns=['This', 'log']) # Hide sensitive values with *\n# logger = get_logger(__name__, sensitive_patterns=['This', 'log'],show_last=1) # Hide sensitive values with * by showing only last 1 item\n\n\n# Configure sensitive value filter\nsensitive_patterns = [\n r\"\\d{16}\", # Example: credit card numbers\n r\"(?:\\d{3}-\\d{2}-\\d{4})\", # Example: SSNs\n \"User\", # Example: any text\n \"level\",\n \"log\",\n r\"(?<=Bearer\\s)[a-zA-Z0-9]+\", # Example: token\n]\nlogger = get_logger(__name__, sensitive_patterns=sensitive_patterns, show_last=2)\n\n\n@app.get(\"/\")\ndef say_hello(name: str = \"Dev\", trace_id: UUID = Depends(get_trace_id)):\n logger.debug(\"This is debug level log.\")\n logger.info(\"This is info level log.\")\n logger.error(\"This is error level log.\")\n logger.warning(\"This is warning level log.\")\n return {\"Message\": f\"Hello {name}\"}\n\n\n@app.get(\"/userinfo\")\ndef get_user_info(trace_id: UUID = Depends(get_trace_id)):\n logger.info(\"User credit card: 1234567812345678.\")\n logger.info(\"User SSN: 123-45-6789.\")\n logger.info(\"Token authorization: Bearer abc123DEF456\")\n return {\"user\": \"Dev\"}\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "A logging library with trace ID support and FastAPI middleware.",
"version": "0.0.2",
"project_urls": {
"Homepage": "https://github.com/Happy-Kumar-Sharma/trace-logger"
},
"split_keywords": [
"secure logging",
" securelogs",
" python logging module",
" best logging library",
" mask sensitive values"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "cfd77f89374c97c297aa17d78de1777cca72e22bc249545647ce5d4a87ac516c",
"md5": "4e2526c3ef2610582bcdec9f99eb13fe",
"sha256": "f2cb76f6fe23afbc7b81b0332e524682ec3bffe1cbb388e11dae962ae35f5dcf"
},
"downloads": -1,
"filename": "SecureLogs-0.0.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4e2526c3ef2610582bcdec9f99eb13fe",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 9307,
"upload_time": "2024-12-03T18:06:11",
"upload_time_iso_8601": "2024-12-03T18:06:11.264801Z",
"url": "https://files.pythonhosted.org/packages/cf/d7/7f89374c97c297aa17d78de1777cca72e22bc249545647ce5d4a87ac516c/SecureLogs-0.0.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "b3a022d5e6be673895a86029bf42530fba4969ac8822659c851cdc10832adc2e",
"md5": "dcf2a399422b7149169d2c96915484ff",
"sha256": "26144b53dcbfc25139df1c6de733236674aaa3425d0391686fc179143d7b571e"
},
"downloads": -1,
"filename": "securelogs-0.0.2.tar.gz",
"has_sig": false,
"md5_digest": "dcf2a399422b7149169d2c96915484ff",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 7734,
"upload_time": "2024-12-03T18:06:13",
"upload_time_iso_8601": "2024-12-03T18:06:13.358569Z",
"url": "https://files.pythonhosted.org/packages/b3/a0/22d5e6be673895a86029bf42530fba4969ac8822659c851cdc10832adc2e/securelogs-0.0.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-03 18:06:13",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Happy-Kumar-Sharma",
"github_project": "trace-logger",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "securelogs"
}
JSON
