| Name | elixirdb JSON |
| Version |
0.1.2
JSON |
| download |
| home_page | None |
| Summary | A python library for a streamlined database experience from a config file. |
| upload_time | 2025-02-11 02:52:29 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.12 |
| license | MIT |
| keywords |
database
sql
config
orm
streamlined
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# ElixirDB - Simplified SQLAlchemy Interaction
[](https://pypi.org/project/elixirdb)
[](https://pypi.org/project/elixirdb)
<!--start-->
ElixirDB simplifies interaction with SQLAlchemy, providing streamlined database operations, enhanced configuration management, and improved developer experience.
## Key Features
- **Automatic loading:** Define an \*\*elixir\*\*.yaml file in your project, and it will be automatically loaded into the ElixirDB instance.
- **Pydantic Integration:** Define and validate database configurations (including engine_options, session_options, and execution_options) using Pydantic models, ensuring type safety and robust settings management.
- **Multi-Engine Support:** Seamlessly manage multiple database engines through a central class object.
- **Multi-dialect Support:** Support for MySQL/MariaDB, postgresql, Oracle, and MSSQL.
- **Engine Types:** Allows for simple creation of `direct`, `session` and `scoped_session` engines with a single parameter.
- **Handler Framework:** A flexible handler framework empowers customized processing of parameters, result_objects, and central error control - mirroring middleware functionality.
- **Stored Procedure Support:** Execute stored procedures with ease, with automatically generated statements based on dialect.
- **Testing Suite:** A testing suite with a Docker Compose setup that provisions five testing databases, preloaded with the necessary tables and data for testing.
## Table of Contents
- [ElixirDB - Simplified SQLAlchemy Interaction](#elixirdb---simplified-sqlalchemy-interaction)
- [Key Features](#key-features)
- [Table of Contents](#table-of-contents)
- [Installation](#installation)
- [Using ElixirDB](#using-elixirdb)
- [Loading configuration and basic usage](#loading-configuration-and-basic-usage)
- [Manual Loading](#manual-loading)
- [Load file with different partial](#load-file-with-different-partial)
- [Pydantic Models for Configuration](#pydantic-models-for-configuration)
- [Multi-Engine Configuration](#multi-engine-configuration)
- [Loading with Models](#loading-with-models)
- [Loading an Invalid Configuration](#loading-an-invalid-configuration)
- [Engine Types](#engine-types)
- [Handler Framework](#handler-framework)
- [Parameter Handlers](#parameter-handlers)
- [Result Handlers](#result-handlers)
- [Error Handlers](#error-handlers)
- [Putting it all together](#putting-it-all-together)
- [Extras](#extras)
- [Attribute calls, Attribute Wrap, and Result Types](#attribute-calls-attribute-wrap-and-result-types)
- [Stored Procedures Mixin](#stored-procedures-mixin)
- [License](#license)
## Installation
ElixirDB is available on PyPI and can be installed using pip:
```bash
pip install elixirdb
```
Install with your dialect and default drivers.
```bash
pip install elixirdb[mysql]
```
You can also install the testing suite, which includes a complete pytest suite with html reports and coverage reports by cloning
the repo and installing the testing suite. The project was built using `uv` as the package manager.
```bash
# Clone repo
git clone https://github.com/hotnsoursoup/elixirdb.git
```
```bash
# Install uv
pip install uv
```
Change your directory to the cloned repo (and create a virtual environment if you wish)
```bash
# Run uv and create a virtual environment
uv venv
# Pick your extras/groups
uv sync --group dev --group test --extra mysql
# or
uv sync --all-extras --all-groups
```
If you have issues, try running --no-cache.
```bash
uv sync --extra mysql --no-cache
```
## Using ElixirDB
### Loading configuration and basic usage
You have various options in how you can load your configuration.
By default, ElixirDB will automatically load a yaml file with `elixir` in the name. (e.g. `myelixirdb.yaml`) by searching within your project structure, using pyrootutils.find_root(), and identifying a yaml or json file with the name `elixir` in the name.
```python
from elixirdb import ElixirDB
try:
connection = ElixirDB()
except FileNotFoundError:
print("No elixir.yaml file found.")
# Note - TextClause is automatically applied to the statement.
result = connection.execute("SELECT * FROM mytable")
for record in result:
print(record)
```
#### Manual Loading
Though the library does support automatic loading, you have the option to do it manually. One way is to pass in as a dictionary. The dictionary below uses the default drivername for the given dialect.
```python
myconfig = {
"dialect": "postgres",
"url": "postgresql//user:password@localhost:5432/mydatabase",
"url_params": {
"host": "localhost",
"port": 5432,
"database": "mydatabase",
"user": "user",
},
"engine_options": {
"pool_size": 10,
"pool_recycle": 3600,
"pool_pre_ping": True,
}
}
}
connection = ElixirDB(myconfig)
```
#### Load file with different partial
load_config can also be used to find a different file name, wherever it may be, using a `partial` string and a `file_type` (yaml/json).
```python
""" Will find any wildcard matching **dbfile**.json in the project structure."""
from elixirdb import load_config, ElixirDB
myconfig = load_config(partial="dbfile", file_type="json")
connection = ElixirDB(myconfig)
connection.execute("SELECT * FROM mytable")
results = connection.fetchall()
```
## Pydantic Models for Configuration
ElixirDB leverages Pydantic models for database configuration, providing type safety, validation, and a structured approach to managing connection parameters. This eliminates common configuration errors and improves code maintainability. The class definitions will have descriptions for each field. This is just an example.
```python
class EngineModel(BaseModel):
"""
Single sqlalchemy database engine configuration model.
"""
engine_key: str
dialect: Dialect
url: str | None = None
url_params: UrlParams | None = None
default: bool = False # Defines the default engine to use in a multi-enginee configuration
engine_options: EngineOptions | None = None
session_options: SessionOptions | None = None
execution_options: ExecutionOptions | None = None
# ... other fields
```
### Multi-Engine Configuration
Manage connections to multiple databases effortlessly with ElixirDB's multi-engine support. Define separate configurations for each database and switch between them seamlessly.
```python
class EngineManager(StrictModel):
"""
Configuration for the application and databases.
"""
default_engine_key: ClassVar[DatabaseKey | None] = None
defaults: Mapping[str, Any] | None = Field(
None,
description=(
"A mapping of EngineModel values. All fields become optional "
"and these values are inherited by all engines. Values within "
"each engine will take precedence over the defaults."
),
)
engines: Mapping[EngineKey, EngineModel]
# ... other fields
@model_validator(mode="after")
def get_and_set_default_engine_key(self) -> Self:
""" Get the and set default engine key if not provided. """
#... other validators
```
Sample yaml configuration for EngineManager
```yaml
app:
defaults: # All engines adopt these as a base.
engine_options:
echo: False
pool_size: 20
max_overflow: 10
pool_recycle: 3600
engines:
dbkey1:
dialect: mysql
url: mysql+pymysql://user:password@localhost:3306/db1
default: true # This engine will be used by default if an engine_key is not provided.
execution_options:
autocommit: True
isolation_level: READ_COMMITTED
preserve_rowcount: True
loggingdb:
dialect: postgres
url_params:
drivername: psycopg2
host: localhost
port: 5432
user: postgres
password: password
query:
schema: public
engine_options:
echo: True
pool_timeout: 30
hide_parameters: True
customerdb:
dialect: oracle
url: oracle+cx_oracle://user:password@localhost:1521/orcl
```
Loading `db1`.
**Note** - The instance will start in a disconnected state unless `auto_connect` is set to `True` in the config.
```python
from elixirdb import ElixirDB
# The configuration is automatically loaded from the elixir.yaml file
connection = ElixirDB(engine_key="db1")
with connection.connect() as conn:
# ... perform operations on db1
# Print the valdated pydantic model with only the values set by the user.
print(connection.db.model_dump(unset=True))
# Switch engines
connection.set_engine("customerdb")
```
### Loading with Models
The models provide a structured way to define and manage database configurations. The recommended approach is to view the model class, but if you want to dump it to a json file, you always have that option as well using `.model_json_schema()` on the class, but is much easier to read the pydantic model definition.
In this example, we load each model separately and reconstruct it to the EngineModel.
**Note** - Not all models are shown.
```python
from elixirdb import ElixirDB
from elixirdb import EngineModel
from elixirdb import EngineOptions
from elixirdb import ExecutionOptions
execution_options = ExecutionOptions(
preserve_rowcount=True,
isolation_level="READ_COMMITTED",
insertmanyvalues_page_size=1000,
no_parameters=True,
)
# Execution options are assigned to engine options
engine_options = EngineOptions(
echo=True,
pool_size=20,
nax_overflow=10,
execution_options=execution_options,
)
config = EngineModel(
dialect="mysql",
url="mysql+pymysql://user:password@localhost:3306/db1",
engine_options=engine_options,
)
connection = ElixirDB(config)
```
### Loading an Invalid Configuration
If your configuration is missing a required field or an invalid input is provided, you may see an error like this. In some cases, after resolving them, you may find more errors arise due to the way pydantic validates before and after validators.
```python
from pydantic import ValidationError
from elixirdb import EngineManager
from elixirdb import print_and_raise_validation_errors
invalid_config = {
"something": "something",
"engines": {
"engine1": {
"url": "postgresql+psycopg2://test_user:StrongPassword!123@localhost:5432/elixirdb",
"default": True,
},
"engine2": {
"name": "DB2",
"default": True,
"dialect": "mysql0",
"url": "wrong_url",
"engine_options": {"pool_size": 10},
},
},
}
try:
EngineManager(**invalid_config)
except ValidationError as e:
print_and_raise_validation_errors(e, raise_error=False)
```
Console Output:
```console
Configuration Errors:
- Type: extra_forbidden
Location: something
Message: Extra inputs are not permitted
Input: something
Engine Errors: engine1
- Type: missing
Location: dialect
Message: Field required
Engine Errors: engine2
- Type: literal_error
Location: dialect
Message: Input should be 'mysql', 'postgres', 'mariadb', 'sqlite', 'mssql' or 'oracle'
Input: mysql0
Expected: 'mysql', 'postgres', 'mariadb', 'sqlite', 'mssql' or 'oracle'
- Type: value_error
Location: url
Message: Invalid url format. Value provided: wrong_url
Input: wrong_url
Url: https://docs.sqlalchemy.org/en/20/core/engines.html#sqlalchemy.engine.make_url
```
## Engine Types
| Engine Type | Description | Documentation Link |
|-------------|-----------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `direct` | Utilizes the `create_engine` function to establish a direct connection to the database. | [SQLAlchemy Engine Configuration](https://docs.sqlalchemy.org/en/latest/core/engines.html) |
| `session` | Employs the `sessionmaker` function to create sessions for managing transactions and ORM operations.| [SQLAlchemy Session API](https://docs.sqlalchemy.org/en/latest/orm/session_api.html) |
| `scoped` | Uses `scoped_session` in conjunction with `sessionmaker` to provide thread-local sessions, ensuring each thread has its own session instance. | [SQLAlchemy Contextual/Thread-local Sessions](https://docs.sqlalchemy.org/en/latest/orm/contextual.html) |
Engine type is set during instantiation as a keyword argument. The default engine_type is `direct`. The `session` engine_type comes with a method to create a new session from the session_factory. This allows it to transfer the ElixirDB configuration to a new instance, including the session_factory.
```python
connection = ElixirDB(engine_type="session")
with connection.new_session() as session:
print(type(session))
<class 'elixirdb.db.ElixirDB'>
```
When the instance is created and a connection is made, ElixirDB will automatically assign
the `connection` for `direct` engines to the `connection` attribute. For sessions, it will assign it to the `session` attribute.
When executing any statements or queries, you can access the attribute to connection or attributes for session directly on the ElixirDB class itself. The `__getaattr__` definition will pass the command to the appropriate attribute. (e.g. connection.execute(), instead of connection.connection.execute() or connection.session.execute())
## Handler Framework
Implement custom logic for handling parameters before execution and processing results after fetching data.
### Parameter Handlers
```python
""" Parameter handlers """
from elixirdb import ElixirDB
# Create various parameter handlers.
def my_param_cleanser(params):
# ... modify params
def my_param_logger(params):
if self.engine_key == "loggingdb":
# ... log params
return params
param_handlers = [my_param_cleanser, my_param_logger]
```
### Result Handlers
**NOTE** - Result handlers may require more advanced configurations, see **configuring result_types**.
```python
""" Result handlers """
def convert_cursor_to_list(result):
if isinstance(result, CursorResult):
# Convert CursorResult to a list
else:
return result
def serialize_results(result):
# ... serialize result
def redact_results(result, **kwargs):
for row in result:
# ... redact results
result_handlers = [convert_cursor_to_list, serialize_results, redact_results]
```
### Error Handlers
```python
""" Error handlers """
def handle_operational_errors(error):
if isinstance(error, OperationalError):
# ... handle error
# raise error
return error
def log_specific_errors(error):
# ... do something else.
return error # so the error is passed to a different handler
error_handlers = [log_specific_errors, handle_operational_errors]
```
### Putting it all together
```python
handlers = {
"param_handlers": param_handlers,
"result_handlers": result_handlers,
"error_handlers": error_handlers,
}
connection = ElixirDB(handlers=handlers)
```
You can also append to existing handlers.
```python:
connection.result_handlers.append(serialize_results)
```
## Extras
### Attribute calls, Attribute Wrap, and Result Types
ElixirDB natively intercepts attribute and method lookups (by defining a `__getattr__`). Currently, this is still a work in progress, but the current implementation logic works as follows in cascading order:
1. **Checks Built-ins or Class Attributes**
- If the attribute is in the instance’s own `__slots__` or class-level attributes, as `falsy` attributes trigger **getattr** to recursively loop, so it must look at dict and class attributes first then return the falsy value.
2. **Handles Result 'type' Attributes**
- Checks to see if there is a result (self.result) and hasattr (e.g., `fetchone`, `fetchall`) and returns the corresponding method from the internal result object.
3. **Check connection/session attributes**
- Will check 'connection' for 'direct' engine_type and 'session' for 'session' and 'scopped' engine_type for the attributes.
4. **Checks attribute and wrap it**
- Ensures attribute is not None and callable(attribute). If it is, wrap it to process any args/kwargs (such as for param processing). I may separate the execute logic completely from this and assign a value to a lookup table to check states and trigger another getattr lookup with the lookup to determine the next action.
5. **Process attribute and return result**
- The `attribute(*args, **kwargs)` is called and is assigned to the result. If it is a result object in `self.result_types` - which is assigned by the user, then the result is processed with `self.result_handlers`. If it is a result type, it also assigned to `self.result`.
### Stored Procedures Mixin
The stored procedure mixin providess a convenient way to execute stored procedures in your database. It comes as a Mixin class, but also available through ElixirDBStatements.
```python
class ElixirDBStatements(StatementsMixin, ElixirDB):
"""Statement enabled connection class."""
```
Since it is a mixin class, you can use it the same as you would with ElixirDB. The mixin class automatically generates the stored procedure for the supported dialects (mysql, mariadb, postgres, oracle, mssql).
```python
from elixirdb import ElixirDBStatements
connection = ElixirDBStatements(engine_type="session", engine_key="my_db")
# Define the parameters
params = {
"param1": 123,
"param2": "example",
}
# Execute a stored procedure. The default behavior is for the class to create a connection.
result = connection.procedure("my_procedure", params)
print(connection.rowcount)
```
## License
ElixirDB is licensed under the MIT License - see the [LICENSE](https://github.com/hotnsoursoup/quik-db/blob/master/LICENSE) file for details
<!--end-->
Raw data
{
"_id": null,
"home_page": null,
"name": "elixirdb",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.12",
"maintainer_email": null,
"keywords": "database, SQL, config, ORM, streamlined",
"author": null,
"author_email": "Victor Nguyen <victor.win86@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/66/59/a91b6e8ce46c350846dcfe8024ac3a43a68335a0ae2f8472389fba790eb3/elixirdb-0.1.2.tar.gz",
"platform": null,
"description": "# ElixirDB - Simplified SQLAlchemy Interaction\r\n\r\n[](https://pypi.org/project/elixirdb)\r\n[](https://pypi.org/project/elixirdb)\r\n\r\n<!--start-->\r\n\r\nElixirDB simplifies interaction with SQLAlchemy, providing streamlined database operations, enhanced configuration management, and improved developer experience.\r\n\r\n## Key Features\r\n\r\n- **Automatic loading:** Define an \\*\\*elixir\\*\\*.yaml file in your project, and it will be automatically loaded into the ElixirDB instance.\r\n- **Pydantic Integration:** Define and validate database configurations (including engine_options, session_options, and execution_options) using Pydantic models, ensuring type safety and robust settings management.\r\n- **Multi-Engine Support:** Seamlessly manage multiple database engines through a central class object.\r\n- **Multi-dialect Support:** Support for MySQL/MariaDB, postgresql, Oracle, and MSSQL.\r\n- **Engine Types:** Allows for simple creation of `direct`, `session` and `scoped_session` engines with a single parameter.\r\n- **Handler Framework:** A flexible handler framework empowers customized processing of parameters, result_objects, and central error control - mirroring middleware functionality.\r\n- **Stored Procedure Support:** Execute stored procedures with ease, with automatically generated statements based on dialect.\r\n- **Testing Suite:** A testing suite with a Docker Compose setup that provisions five testing databases, preloaded with the necessary tables and data for testing.\r\n\r\n## Table of Contents\r\n\r\n- [ElixirDB - Simplified SQLAlchemy Interaction](#elixirdb---simplified-sqlalchemy-interaction)\r\n - [Key Features](#key-features)\r\n - [Table of Contents](#table-of-contents)\r\n - [Installation](#installation)\r\n - [Using ElixirDB](#using-elixirdb)\r\n - [Loading configuration and basic usage](#loading-configuration-and-basic-usage)\r\n - [Manual Loading](#manual-loading)\r\n - [Load file with different partial](#load-file-with-different-partial)\r\n - [Pydantic Models for Configuration](#pydantic-models-for-configuration)\r\n - [Multi-Engine Configuration](#multi-engine-configuration)\r\n - [Loading with Models](#loading-with-models)\r\n - [Loading an Invalid Configuration](#loading-an-invalid-configuration)\r\n - [Engine Types](#engine-types)\r\n - [Handler Framework](#handler-framework)\r\n - [Parameter Handlers](#parameter-handlers)\r\n - [Result Handlers](#result-handlers)\r\n - [Error Handlers](#error-handlers)\r\n - [Putting it all together](#putting-it-all-together)\r\n - [Extras](#extras)\r\n - [Attribute calls, Attribute Wrap, and Result Types](#attribute-calls-attribute-wrap-and-result-types)\r\n - [Stored Procedures Mixin](#stored-procedures-mixin)\r\n - [License](#license)\r\n\r\n## Installation\r\n\r\nElixirDB is available on PyPI and can be installed using pip:\r\n\r\n```bash\r\npip install elixirdb\r\n```\r\n\r\nInstall with your dialect and default drivers.\r\n\r\n```bash\r\npip install elixirdb[mysql]\r\n```\r\n\r\nYou can also install the testing suite, which includes a complete pytest suite with html reports and coverage reports by cloning\r\nthe repo and installing the testing suite. The project was built using `uv` as the package manager.\r\n\r\n```bash\r\n# Clone repo\r\ngit clone https://github.com/hotnsoursoup/elixirdb.git\r\n```\r\n\r\n```bash\r\n# Install uv\r\npip install uv\r\n```\r\n\r\nChange your directory to the cloned repo (and create a virtual environment if you wish)\r\n\r\n```bash\r\n# Run uv and create a virtual environment\r\nuv venv\r\n\r\n# Pick your extras/groups\r\nuv sync --group dev --group test --extra mysql\r\n\r\n# or\r\nuv sync --all-extras --all-groups\r\n```\r\n\r\nIf you have issues, try running --no-cache.\r\n\r\n```bash\r\nuv sync --extra mysql --no-cache\r\n```\r\n\r\n## Using ElixirDB\r\n\r\n### Loading configuration and basic usage\r\n\r\nYou have various options in how you can load your configuration.\r\nBy default, ElixirDB will automatically load a yaml file with `elixir` in the name. (e.g. `myelixirdb.yaml`) by searching within your project structure, using pyrootutils.find_root(), and identifying a yaml or json file with the name `elixir` in the name.\r\n\r\n```python\r\nfrom elixirdb import ElixirDB\r\n\r\ntry:\r\n connection = ElixirDB()\r\nexcept FileNotFoundError:\r\n print(\"No elixir.yaml file found.\")\r\n\r\n# Note - TextClause is automatically applied to the statement.\r\nresult = connection.execute(\"SELECT * FROM mytable\")\r\n\r\n\r\nfor record in result:\r\n print(record)\r\n```\r\n\r\n#### Manual Loading\r\n\r\nThough the library does support automatic loading, you have the option to do it manually. One way is to pass in as a dictionary. The dictionary below uses the default drivername for the given dialect.\r\n\r\n```python\r\nmyconfig = {\r\n \"dialect\": \"postgres\",\r\n \"url\": \"postgresql//user:password@localhost:5432/mydatabase\",\r\n \"url_params\": {\r\n \"host\": \"localhost\",\r\n \"port\": 5432,\r\n \"database\": \"mydatabase\",\r\n \"user\": \"user\",\r\n },\r\n \"engine_options\": {\r\n \"pool_size\": 10,\r\n \"pool_recycle\": 3600,\r\n \"pool_pre_ping\": True,\r\n }\r\n }\r\n}\r\n\r\nconnection = ElixirDB(myconfig)\r\n```\r\n\r\n#### Load file with different partial\r\n\r\nload_config can also be used to find a different file name, wherever it may be, using a `partial` string and a `file_type` (yaml/json).\r\n\r\n```python\r\n\"\"\" Will find any wildcard matching **dbfile**.json in the project structure.\"\"\"\r\n\r\nfrom elixirdb import load_config, ElixirDB\r\n\r\nmyconfig = load_config(partial=\"dbfile\", file_type=\"json\")\r\nconnection = ElixirDB(myconfig)\r\n\r\nconnection.execute(\"SELECT * FROM mytable\")\r\nresults = connection.fetchall()\r\n```\r\n\r\n## Pydantic Models for Configuration\r\n\r\nElixirDB leverages Pydantic models for database configuration, providing type safety, validation, and a structured approach to managing connection parameters. This eliminates common configuration errors and improves code maintainability. The class definitions will have descriptions for each field. This is just an example.\r\n\r\n```python\r\nclass EngineModel(BaseModel):\r\n \"\"\"\r\n Single sqlalchemy database engine configuration model.\r\n \"\"\"\r\n engine_key: str\r\n dialect: Dialect\r\n url: str | None = None\r\n url_params: UrlParams | None = None\r\n default: bool = False # Defines the default engine to use in a multi-enginee configuration\r\n engine_options: EngineOptions | None = None\r\n session_options: SessionOptions | None = None\r\n execution_options: ExecutionOptions | None = None\r\n # ... other fields\r\n```\r\n\r\n### Multi-Engine Configuration\r\n\r\nManage connections to multiple databases effortlessly with ElixirDB's multi-engine support. Define separate configurations for each database and switch between them seamlessly.\r\n\r\n```python\r\nclass EngineManager(StrictModel):\r\n \"\"\"\r\n Configuration for the application and databases.\r\n \"\"\"\r\n default_engine_key: ClassVar[DatabaseKey | None] = None\r\n\r\n defaults: Mapping[str, Any] | None = Field(\r\n None,\r\n description=(\r\n \"A mapping of EngineModel values. All fields become optional \"\r\n \"and these values are inherited by all engines. Values within \"\r\n \"each engine will take precedence over the defaults.\"\r\n ),\r\n )\r\n engines: Mapping[EngineKey, EngineModel]\r\n # ... other fields\r\n\r\n @model_validator(mode=\"after\")\r\n def get_and_set_default_engine_key(self) -> Self:\r\n \"\"\" Get the and set default engine key if not provided. \"\"\"\r\n #... other validators\r\n```\r\n\r\nSample yaml configuration for EngineManager\r\n\r\n```yaml\r\napp:\r\n defaults: # All engines adopt these as a base.\r\n engine_options:\r\n echo: False\r\n pool_size: 20\r\n max_overflow: 10\r\n pool_recycle: 3600\r\nengines:\r\n dbkey1:\r\n dialect: mysql\r\n url: mysql+pymysql://user:password@localhost:3306/db1\r\n default: true # This engine will be used by default if an engine_key is not provided.\r\n execution_options:\r\n autocommit: True\r\n isolation_level: READ_COMMITTED\r\n preserve_rowcount: True\r\n loggingdb:\r\n dialect: postgres\r\n url_params:\r\n drivername: psycopg2\r\n host: localhost\r\n port: 5432\r\n user: postgres\r\n password: password\r\n query:\r\n schema: public\r\n engine_options:\r\n echo: True\r\n pool_timeout: 30\r\n hide_parameters: True\r\n customerdb:\r\n dialect: oracle\r\n url: oracle+cx_oracle://user:password@localhost:1521/orcl\r\n```\r\n\r\nLoading `db1`.\r\n\r\n**Note** - The instance will start in a disconnected state unless `auto_connect` is set to `True` in the config.\r\n\r\n```python\r\nfrom elixirdb import ElixirDB\r\n\r\n# The configuration is automatically loaded from the elixir.yaml file\r\nconnection = ElixirDB(engine_key=\"db1\")\r\n\r\nwith connection.connect() as conn:\r\n # ... perform operations on db1\r\n\r\n# Print the valdated pydantic model with only the values set by the user.\r\nprint(connection.db.model_dump(unset=True))\r\n\r\n# Switch engines\r\nconnection.set_engine(\"customerdb\")\r\n\r\n```\r\n\r\n### Loading with Models\r\n\r\nThe models provide a structured way to define and manage database configurations. The recommended approach is to view the model class, but if you want to dump it to a json file, you always have that option as well using `.model_json_schema()` on the class, but is much easier to read the pydantic model definition.\r\n\r\nIn this example, we load each model separately and reconstruct it to the EngineModel.\r\n\r\n**Note** - Not all models are shown.\r\n\r\n```python\r\nfrom elixirdb import ElixirDB\r\nfrom elixirdb import EngineModel\r\nfrom elixirdb import EngineOptions\r\nfrom elixirdb import ExecutionOptions\r\n\r\n\r\nexecution_options = ExecutionOptions(\r\n preserve_rowcount=True,\r\n isolation_level=\"READ_COMMITTED\",\r\n insertmanyvalues_page_size=1000,\r\n no_parameters=True,\r\n)\r\n\r\n# Execution options are assigned to engine options\r\nengine_options = EngineOptions(\r\n echo=True,\r\n pool_size=20,\r\n nax_overflow=10,\r\n execution_options=execution_options,\r\n)\r\n\r\nconfig = EngineModel(\r\n dialect=\"mysql\",\r\n url=\"mysql+pymysql://user:password@localhost:3306/db1\",\r\n engine_options=engine_options,\r\n)\r\n\r\nconnection = ElixirDB(config)\r\n\r\n```\r\n\r\n### Loading an Invalid Configuration\r\n\r\nIf your configuration is missing a required field or an invalid input is provided, you may see an error like this. In some cases, after resolving them, you may find more errors arise due to the way pydantic validates before and after validators.\r\n\r\n```python\r\nfrom pydantic import ValidationError\r\nfrom elixirdb import EngineManager\r\nfrom elixirdb import print_and_raise_validation_errors\r\n\r\n\r\ninvalid_config = {\r\n \"something\": \"something\",\r\n \"engines\": {\r\n \"engine1\": {\r\n \"url\": \"postgresql+psycopg2://test_user:StrongPassword!123@localhost:5432/elixirdb\",\r\n \"default\": True,\r\n },\r\n \"engine2\": {\r\n \"name\": \"DB2\",\r\n \"default\": True,\r\n \"dialect\": \"mysql0\",\r\n \"url\": \"wrong_url\",\r\n \"engine_options\": {\"pool_size\": 10},\r\n },\r\n },\r\n}\r\n\r\ntry:\r\n EngineManager(**invalid_config)\r\nexcept ValidationError as e:\r\n print_and_raise_validation_errors(e, raise_error=False)\r\n```\r\n\r\nConsole Output:\r\n\r\n```console\r\nConfiguration Errors:\r\n - Type: extra_forbidden\r\n Location: something\r\n Message: Extra inputs are not permitted\r\n Input: something\r\nEngine Errors: engine1\r\n - Type: missing\r\n Location: dialect\r\n Message: Field required\r\nEngine Errors: engine2\r\n - Type: literal_error\r\n Location: dialect\r\n Message: Input should be 'mysql', 'postgres', 'mariadb', 'sqlite', 'mssql' or 'oracle'\r\n Input: mysql0\r\n Expected: 'mysql', 'postgres', 'mariadb', 'sqlite', 'mssql' or 'oracle'\r\n - Type: value_error\r\n Location: url\r\n Message: Invalid url format. Value provided: wrong_url\r\n Input: wrong_url\r\n Url: https://docs.sqlalchemy.org/en/20/core/engines.html#sqlalchemy.engine.make_url\r\n```\r\n\r\n## Engine Types\r\n\r\n| Engine Type | Description | Documentation Link |\r\n|-------------|-----------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------|\r\n| `direct` | Utilizes the `create_engine` function to establish a direct connection to the database. | [SQLAlchemy Engine Configuration](https://docs.sqlalchemy.org/en/latest/core/engines.html) |\r\n| `session` | Employs the `sessionmaker` function to create sessions for managing transactions and ORM operations.| [SQLAlchemy Session API](https://docs.sqlalchemy.org/en/latest/orm/session_api.html) |\r\n| `scoped` | Uses `scoped_session` in conjunction with `sessionmaker` to provide thread-local sessions, ensuring each thread has its own session instance. | [SQLAlchemy Contextual/Thread-local Sessions](https://docs.sqlalchemy.org/en/latest/orm/contextual.html) |\r\n\r\nEngine type is set during instantiation as a keyword argument. The default engine_type is `direct`. The `session` engine_type comes with a method to create a new session from the session_factory. This allows it to transfer the ElixirDB configuration to a new instance, including the session_factory.\r\n\r\n```python\r\nconnection = ElixirDB(engine_type=\"session\")\r\n\r\nwith connection.new_session() as session:\r\n print(type(session))\r\n\r\n<class 'elixirdb.db.ElixirDB'>\r\n```\r\n\r\nWhen the instance is created and a connection is made, ElixirDB will automatically assign\r\nthe `connection` for `direct` engines to the `connection` attribute. For sessions, it will assign it to the `session` attribute.\r\n\r\nWhen executing any statements or queries, you can access the attribute to connection or attributes for session directly on the ElixirDB class itself. The `__getaattr__` definition will pass the command to the appropriate attribute. (e.g. connection.execute(), instead of connection.connection.execute() or connection.session.execute())\r\n\r\n## Handler Framework\r\n\r\nImplement custom logic for handling parameters before execution and processing results after fetching data.\r\n\r\n### Parameter Handlers\r\n\r\n```python\r\n\"\"\" Parameter handlers \"\"\"\r\n\r\nfrom elixirdb import ElixirDB\r\n\r\n# Create various parameter handlers.\r\ndef my_param_cleanser(params):\r\n # ... modify params\r\n\r\ndef my_param_logger(params):\r\n if self.engine_key == \"loggingdb\":\r\n # ... log params\r\n return params\r\n\r\nparam_handlers = [my_param_cleanser, my_param_logger]\r\n\r\n```\r\n\r\n### Result Handlers\r\n\r\n**NOTE** - Result handlers may require more advanced configurations, see **configuring result_types**.\r\n\r\n```python\r\n\r\n\"\"\" Result handlers \"\"\"\r\n\r\ndef convert_cursor_to_list(result):\r\n if isinstance(result, CursorResult):\r\n # Convert CursorResult to a list\r\n else:\r\n return result\r\n\r\ndef serialize_results(result):\r\n # ... serialize result\r\n\r\ndef redact_results(result, **kwargs):\r\n for row in result:\r\n # ... redact results\r\n\r\nresult_handlers = [convert_cursor_to_list, serialize_results, redact_results]\r\n```\r\n\r\n### Error Handlers\r\n\r\n```python\r\n\"\"\" Error handlers \"\"\"\r\n\r\ndef handle_operational_errors(error):\r\n if isinstance(error, OperationalError):\r\n # ... handle error\r\n # raise error\r\n return error\r\n\r\ndef log_specific_errors(error):\r\n # ... do something else.\r\n return error # so the error is passed to a different handler\r\n\r\nerror_handlers = [log_specific_errors, handle_operational_errors]\r\n```\r\n\r\n### Putting it all together\r\n\r\n```python\r\nhandlers = {\r\n \"param_handlers\": param_handlers,\r\n \"result_handlers\": result_handlers,\r\n \"error_handlers\": error_handlers,\r\n}\r\n\r\nconnection = ElixirDB(handlers=handlers)\r\n```\r\n\r\nYou can also append to existing handlers.\r\n\r\n```python:\r\nconnection.result_handlers.append(serialize_results)\r\n```\r\n\r\n## Extras\r\n\r\n### Attribute calls, Attribute Wrap, and Result Types\r\n\r\nElixirDB natively intercepts attribute and method lookups (by defining a `__getattr__`). Currently, this is still a work in progress, but the current implementation logic works as follows in cascading order:\r\n\r\n1. **Checks Built-ins or Class Attributes**\r\n - If the attribute is in the instance\u2019s own `__slots__` or class-level attributes, as `falsy` attributes trigger **getattr** to recursively loop, so it must look at dict and class attributes first then return the falsy value.\r\n\r\n2. **Handles Result 'type' Attributes**\r\n - Checks to see if there is a result (self.result) and hasattr (e.g., `fetchone`, `fetchall`) and returns the corresponding method from the internal result object.\r\n\r\n3. **Check connection/session attributes**\r\n - Will check 'connection' for 'direct' engine_type and 'session' for 'session' and 'scopped' engine_type for the attributes.\r\n4. **Checks attribute and wrap it**\r\n - Ensures attribute is not None and callable(attribute). If it is, wrap it to process any args/kwargs (such as for param processing). I may separate the execute logic completely from this and assign a value to a lookup table to check states and trigger another getattr lookup with the lookup to determine the next action.\r\n5. **Process attribute and return result**\r\n - The `attribute(*args, **kwargs)` is called and is assigned to the result. If it is a result object in `self.result_types` - which is assigned by the user, then the result is processed with `self.result_handlers`. If it is a result type, it also assigned to `self.result`.\r\n\r\n### Stored Procedures Mixin\r\n\r\nThe stored procedure mixin providess a convenient way to execute stored procedures in your database. It comes as a Mixin class, but also available through ElixirDBStatements.\r\n\r\n```python\r\nclass ElixirDBStatements(StatementsMixin, ElixirDB):\r\n \"\"\"Statement enabled connection class.\"\"\"\r\n```\r\n\r\nSince it is a mixin class, you can use it the same as you would with ElixirDB. The mixin class automatically generates the stored procedure for the supported dialects (mysql, mariadb, postgres, oracle, mssql).\r\n\r\n```python\r\nfrom elixirdb import ElixirDBStatements\r\n\r\nconnection = ElixirDBStatements(engine_type=\"session\", engine_key=\"my_db\")\r\n\r\n# Define the parameters\r\n\r\nparams = {\r\n \"param1\": 123,\r\n \"param2\": \"example\",\r\n}\r\n\r\n# Execute a stored procedure. The default behavior is for the class to create a connection.\r\nresult = connection.procedure(\"my_procedure\", params)\r\n\r\nprint(connection.rowcount)\r\n```\r\n\r\n\r\n## License\r\n\r\nElixirDB is licensed under the MIT License - see the [LICENSE](https://github.com/hotnsoursoup/quik-db/blob/master/LICENSE) file for details\r\n\r\n<!--end-->\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A python library for a streamlined database experience from a config file.",
"version": "0.1.2",
"project_urls": {
"documentation": "https://github.com/hotnsoursoup/elixir-db/tree/master/docs",
"homepage": "https://github.com/hotnsoursoup/elixir-db",
"testing": "https://github.com/hotnsoursoup/elixir-db/blob/master/docs/tests.md"
},
"split_keywords": [
"database",
" sql",
" config",
" orm",
" streamlined"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "9512058c618ea66590a697dc3bc06046d149a76aa4b72c74da7124e8af1bbe69",
"md5": "75e090f099b5f9d08a0fb0b76fe5cbf1",
"sha256": "c0f74f6266cf20a4a5a317b5b7509edcfd8435ee2e058a00095ba60ba2ac4e17"
},
"downloads": -1,
"filename": "elixirdb-0.1.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "75e090f099b5f9d08a0fb0b76fe5cbf1",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.12",
"size": 45487,
"upload_time": "2025-02-11T02:52:28",
"upload_time_iso_8601": "2025-02-11T02:52:28.108952Z",
"url": "https://files.pythonhosted.org/packages/95/12/058c618ea66590a697dc3bc06046d149a76aa4b72c74da7124e8af1bbe69/elixirdb-0.1.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "6659a91b6e8ce46c350846dcfe8024ac3a43a68335a0ae2f8472389fba790eb3",
"md5": "9c750b4a160888905429b78198042578",
"sha256": "6e9a3e13e147451b986b8b25a40bfa71bfc3a22b35350273f1da084bcb86ea24"
},
"downloads": -1,
"filename": "elixirdb-0.1.2.tar.gz",
"has_sig": false,
"md5_digest": "9c750b4a160888905429b78198042578",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.12",
"size": 46830,
"upload_time": "2025-02-11T02:52:29",
"upload_time_iso_8601": "2025-02-11T02:52:29.730789Z",
"url": "https://files.pythonhosted.org/packages/66/59/a91b6e8ce46c350846dcfe8024ac3a43a68335a0ae2f8472389fba790eb3/elixirdb-0.1.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-11 02:52:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "hotnsoursoup",
"github_project": "elixir-db",
"github_not_found": true,
"lcname": "elixirdb"
}
