Collection of utility functions and classes designed to enhance Python projects.
The library is organized into several modules, including logging, cache, translation models,
client interactions, data manipulation with pandas, and general-purpose functions.
# Supported Python Versions
Python >= 3.10
# Unsupported Python Versions
Python < 3.10
## Installation
You can install `utils-b-infra` using pip:
```bash
pip install utils-b-infra
```
To include the translation utilities:
```bash
pip install utils-b-infra[translation]
````
## Structure
The library is organized into the following modules:
1. logging.py: Utilities for logging with SlackAPI and writing to a file.
2. cache.py: Utilities for caching data in memory, Redis or MongoDB.
3. ai.py: Utilities for working with AI models, such as token count, tokenization, and text generation.
4. translation.py: Utilities for working with translation APIs (Supported Google Translate and DeepL).
5. services.py: Services-related utilities, such as creating google service.
6. pandas.py: Utilities for working with pandas dataframes, (df cleaning, insertion into databases...).
7. generic.py: Miscellaneous utilities that don't fit into the other specific categories (retry, run in thread,
validate, etc.).
## Usage
Here are few examples, for more details, please refer to the docstrings in the source code.
Logging Utilities
```python
from utils_b_infra.logging import SlackLogger
logger = SlackLogger(project_name="your-project-name", slack_token="your-slack-token", slack_channel_id="channel-id")
logger.info("This is an info message")
logger.error(exc=Exception, header_message="Header message appears above the exception message in the Slack message")
```
Cache Utilities
```python
from time import sleep
from utils_b_infra.cache import Cache, CacheConfig
cache_config = CacheConfig(
cache_type="RedisCache",
redis_host="host",
redis_port=6379,
redis_password="password"
)
@cache.cached(60, namespace="test1", sliding_expiration=False)
def hello(arg1: int, arg2: str) -> dict:
sleep(5)
data = {
"orders": [
"668abd233909666c44033913",
"668ab5167a0b54248b044b14",
"668aad6f1cd076a89e0f4e87",
"668ac1ff28065eadb408a9b5",
"668ac23eb6bb7b781f069567"
],
"stats": {
"1": 10,
"2": 22
}
}
print(data)
return data
if __name__ == "__main__":
hello(arg1=1, arg2="test")
```
Services Utilities
```python
from utils_b_infra.services import get_google_service
google_sheet_service = get_google_service(google_token_path='common/google_token.json',
google_credentials_path='common/google_credentials.json',
service_name='sheets')
```
Pandas Utilities
```python
import pandas as pd
from utils_b_infra.pandas import clean_dataframe, insert_df_into_db_in_chunks
from connections import sqlalchemy_client # Your database connection client
df = pd.read_csv("data.csv")
clean_df = clean_dataframe(df)
with sqlalchemy_client.connect() as db_connection:
insert_df_into_db_in_chunks(
df=clean_df,
table_name="table_name",
conn=db_connection,
if_exists='append',
truncate_table=True,
index=False,
dtype=None,
chunk_size=20_000
)
```
Translation Utilities
To use the translation utilities, you need to install the translation extras and set up the necessary environment
variables for Google Translate:
```bash
pip install utils-b-infra[translation]
```
```python
import os
from utils_b_infra.translation import TextTranslator
# Set up Google Cloud credentials
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = 'path/to/google_service_account.json'
deepl_api_key = 'your-deepl-api-key'
languages = {
'ru': 'https://ru.example.com',
'ar': 'https://ar.example.com',
'de': 'https://de.example.com',
'es': 'https://es.example.com',
'fr': 'https://fr.example.com',
'uk': 'https://ua.example.com'
}
google_project_id = 'your-google-project-id'
translator = TextTranslator(deepl_api_key=deepl_api_key, languages=languages, google_project_id=google_project_id)
text_to_translate = "Hello, world!"
translations = translator.get_translations(
text=text_to_translate,
source_language="en",
target_langs=["ru", "ar", "de"],
engine="google"
)
for lang, translated_text in translations.items():
print(f"{lang}: {translated_text}")
```
Generic Utilities
```python
from utils_b_infra.generic import retry_with_timeout, validate_numeric_value, run_threaded, Timer
@retry_with_timeout(retries=3, timeout=5)
def fetch_data(arg1, arg2):
# function logic here
pass
with Timer() as t:
fetch_data("arg1", "arg2")
print(t.seconds_taken) # Output: Time taken to run fetch_data function (in seconds)
print(t.minutes_taken) # Output: Time taken to run fetch_data function (in minutes)
run_threaded(fetch_data, arg1="arg1", arg2="arg2")
is_valid = validate_numeric_value(123)
print(is_valid) # Output: True
```
## License
This project is licensed under the MIT License. See the LICENSE file for details.
# Changelog
[0.5.0] - 2024-07-30
### updated
- Switch cache mechanism from async to sync
[0.4.0] - 2024-07-20
### Added
- Caching modules
[0.3.0] - 2024-06-27
### Changed
Split the library into main and extra modules, including optional translation utilities.
## [0.2.0] - 2024-06-26:
### Added
- Support for `google-cloud-translate` V3 API.
- Support for OpenAI modules `gpt-4o` and `gpt-4o-2024-05-13` in `ai.calculate_openai_price`
### Fixed
- Issue with json parsing in `ai.TextGenerator.get_ai_response`.
### Changed
- Default openai model to `gpt-4o` in `ai.TextGenerator.get_ai_response`.
- Updated Readme file with more examples.
## [0.1.0] - 2024-06-25 initial release
### Added
- Initial release of the package.
Raw data
{
"_id": null,
"home_page": "https://github.com/Fahadukr/utils-b-infra",
"name": "utils-B-infra",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": null,
"author": "Fahad Mawlood",
"author_email": "fahadukr@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/70/64/08d0d62536146730ef0299dc9577b0184189cf03fe02dfe963382ea67848/utils-B-infra-0.5.13.tar.gz",
"platform": null,
"description": "Collection of utility functions and classes designed to enhance Python projects.\r\nThe library is organized into several modules, including logging, cache, translation models,\r\nclient interactions, data manipulation with pandas, and general-purpose functions.\r\n\r\n# Supported Python Versions\r\n\r\nPython >= 3.10\r\n\r\n# Unsupported Python Versions\r\n\r\nPython < 3.10\r\n\r\n## Installation\r\n\r\nYou can install `utils-b-infra` using pip:\r\n\r\n```bash\r\npip install utils-b-infra\r\n```\r\n\r\nTo include the translation utilities:\r\n\r\n```bash\r\npip install utils-b-infra[translation]\r\n````\r\n\r\n## Structure\r\n\r\nThe library is organized into the following modules:\r\n\r\n1. logging.py: Utilities for logging with SlackAPI and writing to a file.\r\n2. cache.py: Utilities for caching data in memory, Redis or MongoDB.\r\n3. ai.py: Utilities for working with AI models, such as token count, tokenization, and text generation.\r\n4. translation.py: Utilities for working with translation APIs (Supported Google Translate and DeepL).\r\n5. services.py: Services-related utilities, such as creating google service.\r\n6. pandas.py: Utilities for working with pandas dataframes, (df cleaning, insertion into databases...).\r\n7. generic.py: Miscellaneous utilities that don't fit into the other specific categories (retry, run in thread,\r\n validate, etc.).\r\n\r\n## Usage\r\n\r\nHere are few examples, for more details, please refer to the docstrings in the source code.\r\n\r\nLogging Utilities\r\n\r\n```python\r\nfrom utils_b_infra.logging import SlackLogger\r\n\r\nlogger = SlackLogger(project_name=\"your-project-name\", slack_token=\"your-slack-token\", slack_channel_id=\"channel-id\")\r\nlogger.info(\"This is an info message\")\r\nlogger.error(exc=Exception, header_message=\"Header message appears above the exception message in the Slack message\")\r\n```\r\n\r\nCache Utilities\r\n\r\n```python\r\nfrom time import sleep\r\nfrom utils_b_infra.cache import Cache, CacheConfig\r\n\r\ncache_config = CacheConfig(\r\n cache_type=\"RedisCache\",\r\n redis_host=\"host\",\r\n redis_port=6379,\r\n redis_password=\"password\"\r\n)\r\n\r\n\r\n@cache.cached(60, namespace=\"test1\", sliding_expiration=False)\r\ndef hello(arg1: int, arg2: str) -> dict:\r\n sleep(5)\r\n data = {\r\n \"orders\": [\r\n \"668abd233909666c44033913\",\r\n \"668ab5167a0b54248b044b14\",\r\n \"668aad6f1cd076a89e0f4e87\",\r\n \"668ac1ff28065eadb408a9b5\",\r\n \"668ac23eb6bb7b781f069567\"\r\n ],\r\n \"stats\": {\r\n \"1\": 10,\r\n \"2\": 22\r\n }\r\n }\r\n print(data)\r\n return data\r\n\r\n\r\nif __name__ == \"__main__\":\r\n hello(arg1=1, arg2=\"test\")\r\n```\r\n\r\nServices Utilities\r\n\r\n```python\r\nfrom utils_b_infra.services import get_google_service\r\n\r\ngoogle_sheet_service = get_google_service(google_token_path='common/google_token.json',\r\n google_credentials_path='common/google_credentials.json',\r\n service_name='sheets')\r\n```\r\n\r\nPandas Utilities\r\n\r\n```python\r\nimport pandas as pd\r\nfrom utils_b_infra.pandas import clean_dataframe, insert_df_into_db_in_chunks\r\n\r\nfrom connections import sqlalchemy_client # Your database connection client\r\n\r\ndf = pd.read_csv(\"data.csv\")\r\nclean_df = clean_dataframe(df)\r\nwith sqlalchemy_client.connect() as db_connection:\r\n insert_df_into_db_in_chunks(\r\n df=clean_df,\r\n table_name=\"table_name\",\r\n conn=db_connection,\r\n if_exists='append',\r\n truncate_table=True,\r\n index=False,\r\n dtype=None,\r\n chunk_size=20_000\r\n )\r\n```\r\n\r\nTranslation Utilities\r\nTo use the translation utilities, you need to install the translation extras and set up the necessary environment\r\nvariables for Google Translate:\r\n\r\n```bash\r\npip install utils-b-infra[translation]\r\n```\r\n\r\n```python\r\nimport os\r\nfrom utils_b_infra.translation import TextTranslator\r\n\r\n# Set up Google Cloud credentials\r\nos.environ['GOOGLE_APPLICATION_CREDENTIALS'] = 'path/to/google_service_account.json'\r\n\r\ndeepl_api_key = 'your-deepl-api-key'\r\nlanguages = {\r\n 'ru': 'https://ru.example.com',\r\n 'ar': 'https://ar.example.com',\r\n 'de': 'https://de.example.com',\r\n 'es': 'https://es.example.com',\r\n 'fr': 'https://fr.example.com',\r\n 'uk': 'https://ua.example.com'\r\n}\r\ngoogle_project_id = 'your-google-project-id'\r\n\r\ntranslator = TextTranslator(deepl_api_key=deepl_api_key, languages=languages, google_project_id=google_project_id)\r\n\r\ntext_to_translate = \"Hello, world!\"\r\ntranslations = translator.get_translations(\r\n text=text_to_translate,\r\n source_language=\"en\",\r\n target_langs=[\"ru\", \"ar\", \"de\"],\r\n engine=\"google\"\r\n)\r\n\r\nfor lang, translated_text in translations.items():\r\n print(f\"{lang}: {translated_text}\")\r\n```\r\n\r\nGeneric Utilities\r\n\r\n```python\r\nfrom utils_b_infra.generic import retry_with_timeout, validate_numeric_value, run_threaded, Timer\r\n\r\n\r\n@retry_with_timeout(retries=3, timeout=5)\r\ndef fetch_data(arg1, arg2):\r\n # function logic here\r\n pass\r\n\r\n\r\nwith Timer() as t:\r\n fetch_data(\"arg1\", \"arg2\")\r\nprint(t.seconds_taken) # Output: Time taken to run fetch_data function (in seconds)\r\nprint(t.minutes_taken) # Output: Time taken to run fetch_data function (in minutes)\r\n\r\nrun_threaded(fetch_data, arg1=\"arg1\", arg2=\"arg2\")\r\n\r\nis_valid = validate_numeric_value(123)\r\nprint(is_valid) # Output: True\r\n```\r\n\r\n## License\r\n\r\nThis project is licensed under the MIT License. See the LICENSE file for details.\r\n\r\n\r\n\r\n# Changelog\r\n\r\n[0.5.0] - 2024-07-30\r\n\r\n### updated\r\n\r\n- Switch cache mechanism from async to sync\r\n\r\n[0.4.0] - 2024-07-20\r\n\r\n### Added\r\n\r\n- Caching modules\r\n\r\n[0.3.0] - 2024-06-27\r\n\r\n### Changed\r\n\r\nSplit the library into main and extra modules, including optional translation utilities.\r\n\r\n## [0.2.0] - 2024-06-26:\r\n\r\n### Added\r\n\r\n- Support for `google-cloud-translate` V3 API.\r\n- Support for OpenAI modules `gpt-4o` and `gpt-4o-2024-05-13` in `ai.calculate_openai_price`\r\n\r\n### Fixed\r\n\r\n- Issue with json parsing in `ai.TextGenerator.get_ai_response`.\r\n\r\n### Changed\r\n\r\n- Default openai model to `gpt-4o` in `ai.TextGenerator.get_ai_response`.\r\n- Updated Readme file with more examples.\r\n\r\n## [0.1.0] - 2024-06-25 initial release\r\n\r\n### Added\r\n\r\n- Initial release of the package.\r\n",
"bugtrack_url": null,
"license": null,
"summary": "A collection of utility functions and classes for Python projects.",
"version": "0.5.13",
"project_urls": {
"Homepage": "https://github.com/Fahadukr/utils-b-infra"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d03d232e150eef85b8c9f968fc0d9500d14c0beb578266f20ba9b392ff12a8c0",
"md5": "b599cac2b19ff6b565abc06c5e44af30",
"sha256": "48ed04338c2a6798da89974b51bf2ce72feb3a52424d94522b0c7a8e4645daed"
},
"downloads": -1,
"filename": "utils_B_infra-0.5.13-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b599cac2b19ff6b565abc06c5e44af30",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 25700,
"upload_time": "2024-09-07T21:58:47",
"upload_time_iso_8601": "2024-09-07T21:58:47.877684Z",
"url": "https://files.pythonhosted.org/packages/d0/3d/232e150eef85b8c9f968fc0d9500d14c0beb578266f20ba9b392ff12a8c0/utils_B_infra-0.5.13-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "706408d0d62536146730ef0299dc9577b0184189cf03fe02dfe963382ea67848",
"md5": "1acffa3091758793beb0393895fbcbca",
"sha256": "513323f67bc6d7d47edc9ba37a34319b0188a116480a988cbae7c9c6401f38ec"
},
"downloads": -1,
"filename": "utils-B-infra-0.5.13.tar.gz",
"has_sig": false,
"md5_digest": "1acffa3091758793beb0393895fbcbca",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 21152,
"upload_time": "2024-09-07T21:58:52",
"upload_time_iso_8601": "2024-09-07T21:58:52.464556Z",
"url": "https://files.pythonhosted.org/packages/70/64/08d0d62536146730ef0299dc9577b0184189cf03fe02dfe963382ea67848/utils-B-infra-0.5.13.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-07 21:58:52",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Fahadukr",
"github_project": "utils-b-infra",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "utils-b-infra"
}
JSON
