# DigitalArzEngine
**DigitalArzEngine** is a Python library designed to streamline raster data processing by extending the capabilities of the [`rasterio`](https://rasterio.readthedocs.io/) library. It provides a suite of tools for efficient geospatial transformations, mosaicing, and analysis, making it ideal for researchers, analysts, and developers working with geospatial raster data.
## 🚀 Features
- **Mosaicing:** Seamlessly merge multiple raster datasets into a unified output.
- **Summary Statistics:** Extract key metrics such as minimum, maximum, mean, and standard deviation from raster layers.
- **Reprojection & Resampling:** Transform raster datasets to different coordinate systems and resolutions with ease.
- **Clipping & Masking:** Apply geometric masks or clip rasters to specific regions of interest.
- **Pixel-wise Analysis:** Enable pixel-level operations for customized raster computations.
- **Efficient I/O Handling:** Support for reading, writing, and converting between various raster formats.
## 📦 Installation
To install the library using pip:
```bash
pip install digitalarzengine
```
---
## 📚 DataManager Utility Class
The `DataManager` class is a powerful and modular tool for handling geospatial data using SQLite databases. It supports structured storage of JSON records alongside geometric data (as WKB), with metadata tracking, querying, and integration with `GeoPandas`.
### ✅ Benefits and Usage
- **Structured Storage**: Stores geospatial records (geometry + attributes) in a portable `.db` format.
- **Metadata Management**: Tracks field names, geometry columns, and record counts.
- **Geometry Support**: Accepts and stores geometries as WKB with support for reprojection to EPSG:4326.
- **Integration with GeoPandas**: Easily convert the stored data into DataFrames and GeoDataFrames.
- **Custom Query Support**: Run filtered SQL queries and retrieve results as structured pandas objects.
- **Extendable Schema**: Dynamically add and update fields in your dataset.
### 🔧 Example Use-Cases
- Saving extracted geospatial features from remote sensing workflows.
- Iteratively storing geospatial model outputs with spatial context.
- Lightweight local spatial database for machine learning input.
### 🔄 Core Methods
- `add_record`, `update_record`, `delete_record`, `get_record`
- `get_data_as_df`, `get_data_as_gdf`, `get_gdf_list_under_aoi`
- `record_exists`, `change_key`, `add_column`, `update_column`
- Context manager support (`with DataManager(...) as dm:`)
This utility class is designed to complement the raster processing tools in `DigitalArzEngine`, making it easier to work with both raster and vector data consistently.
---
## 🗃️ DBManager Utility Class
The `DBManager` class provides a secure and flexible way to interact with PostgreSQL/PostGIS databases. It builds SQLAlchemy engines from settings and supports reading data directly into Pandas or GeoPandas.
### ✅ Benefits and Usage
- **Secure Configuration**: Pulls database credentials from a centralized encrypted config using `CryptoUtils`.
- **Flexible Access**: Supports multiple environments or databases through the `from_config` method.
- **SQLAlchemy Integration**: Simplifies connection management and avoids hardcoding sensitive details.
- **Read GeoData**: Loads spatial tables directly into `GeoDataFrame` using `read_postgis`.
- **Exclude Geometry When Needed**: Supports reading attribute-only tables as plain DataFrames.
### 🔧 Example Use-Cases
```python
import os
from digitalarzengine.settings import DATA_DIR
from digitalarzengine.adapters.data_manager import DataManager
output_dir = os.path.join(DATA_DIR, 'pak', 'snow_cover/stats')
data_manager = DataManager(output_dir, base_name="snow_cover_normal_data",
purpose="snow cover normal data from 2001 to 2024")
data_manager.add_record("key_value", {})
```
- Connecting securely to remote geospatial databases for querying.
- Reading and processing large PostGIS tables as GeoDataFrames.
- Integrating web dashboards or data pipelines with PostgreSQL/PostGIS backends.
### 🔄 Core Methods
- `from_config(db_key)`: Load credentials and settings from `DATABASES`
- `get_engine()`: Return SQLAlchemy engine object
- `read_as_geo_dataframe()`: Read spatial data into GeoDataFrame
- `read_as_dataframe()`: Read tabular data with option to exclude geometry
- `get_geometry_columns()`: Identify spatial fields in the schema
### ⚙️ Configuration Example
Below is a sample `DATABASES` configuration dictionary to be placed in `digitalarzengine/settings.py`:
```python
DATABASES = {
"drm": {
"ENGINE": "postgresql+psycopg2",
"NAME": "drm",
"USER": "dafast",
"PASSWORD": "***********************************",
"HOST": os.getenv("DB_HOST", "localhost"),
"PORT": "5432",
}
}
```
> ⚠️ Note: The password here is shown encrypted. It should be decrypted using `CryptoUtils` at runtime.
The `DBManager` is ideal for scenarios where spatial data needs to be read or processed securely from enterprise databases, complementing local `DataManager` workflows.
For more advanced usage patterns and custom queries, see the source or documentation site (coming soon).
## 👨💻 Developed by
**Ather Ashraf**
Geospatial Data Scientist and AI Specialist
* 📧 Email: [atherashraf@gmail.com](mailto:atherashraf@gmail.com)
* 🌐 LinkedIn: [https://sa.linkedin.com/in/ather-ashraf](https://sa.linkedin.com/in/ather-ashraf)
* 📜 Google Scholar: [View Profile](https://scholar.google.com.pk/citations?user=XbqhyrsAAAAJ&hl=en)
* 📱 WhatsApp: +966557252342 | +923224785104
---
Raw data
{
"_id": null,
"home_page": null,
"name": "digitalarzengine",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "raster, vector, digital earth engine",
"author": "Ather Ashraf",
"author_email": "atherashraf@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/4f/cd/4e57220eb59336ebb85f4122c06f4b259f77586a0fd1c36f33fa9cbbd13f/digitalarzengine-0.3.3.tar.gz",
"platform": null,
"description": "# DigitalArzEngine\n\n**DigitalArzEngine** is a Python library designed to streamline raster data processing by extending the capabilities of the [`rasterio`](https://rasterio.readthedocs.io/) library. It provides a suite of tools for efficient geospatial transformations, mosaicing, and analysis, making it ideal for researchers, analysts, and developers working with geospatial raster data.\n\n## \ud83d\ude80 Features\n\n- **Mosaicing:** Seamlessly merge multiple raster datasets into a unified output.\n- **Summary Statistics:** Extract key metrics such as minimum, maximum, mean, and standard deviation from raster layers.\n- **Reprojection & Resampling:** Transform raster datasets to different coordinate systems and resolutions with ease.\n- **Clipping & Masking:** Apply geometric masks or clip rasters to specific regions of interest.\n- **Pixel-wise Analysis:** Enable pixel-level operations for customized raster computations.\n- **Efficient I/O Handling:** Support for reading, writing, and converting between various raster formats.\n\n## \ud83d\udce6 Installation\n\nTo install the library using pip:\n\n```bash\npip install digitalarzengine\n```\n\n---\n\n## \ud83d\udcda DataManager Utility Class\n\nThe `DataManager` class is a powerful and modular tool for handling geospatial data using SQLite databases. It supports structured storage of JSON records alongside geometric data (as WKB), with metadata tracking, querying, and integration with `GeoPandas`.\n\n### \u2705 Benefits and Usage\n\n- **Structured Storage**: Stores geospatial records (geometry + attributes) in a portable `.db` format.\n- **Metadata Management**: Tracks field names, geometry columns, and record counts.\n- **Geometry Support**: Accepts and stores geometries as WKB with support for reprojection to EPSG:4326.\n- **Integration with GeoPandas**: Easily convert the stored data into DataFrames and GeoDataFrames.\n- **Custom Query Support**: Run filtered SQL queries and retrieve results as structured pandas objects.\n- **Extendable Schema**: Dynamically add and update fields in your dataset.\n\n### \ud83d\udd27 Example Use-Cases\n\n- Saving extracted geospatial features from remote sensing workflows.\n- Iteratively storing geospatial model outputs with spatial context.\n- Lightweight local spatial database for machine learning input.\n\n### \ud83d\udd04 Core Methods\n\n- `add_record`, `update_record`, `delete_record`, `get_record`\n- `get_data_as_df`, `get_data_as_gdf`, `get_gdf_list_under_aoi`\n- `record_exists`, `change_key`, `add_column`, `update_column`\n- Context manager support (`with DataManager(...) as dm:`)\n\nThis utility class is designed to complement the raster processing tools in `DigitalArzEngine`, making it easier to work with both raster and vector data consistently.\n\n---\n\n## \ud83d\uddc3\ufe0f DBManager Utility Class\n\nThe `DBManager` class provides a secure and flexible way to interact with PostgreSQL/PostGIS databases. It builds SQLAlchemy engines from settings and supports reading data directly into Pandas or GeoPandas.\n\n### \u2705 Benefits and Usage\n\n- **Secure Configuration**: Pulls database credentials from a centralized encrypted config using `CryptoUtils`.\n- **Flexible Access**: Supports multiple environments or databases through the `from_config` method.\n- **SQLAlchemy Integration**: Simplifies connection management and avoids hardcoding sensitive details.\n- **Read GeoData**: Loads spatial tables directly into `GeoDataFrame` using `read_postgis`.\n- **Exclude Geometry When Needed**: Supports reading attribute-only tables as plain DataFrames.\n\n### \ud83d\udd27 Example Use-Cases\n\n```python\n import os\nfrom digitalarzengine.settings import DATA_DIR\nfrom digitalarzengine.adapters.data_manager import DataManager\n\noutput_dir = os.path.join(DATA_DIR, 'pak', 'snow_cover/stats')\ndata_manager = DataManager(output_dir, base_name=\"snow_cover_normal_data\",\n purpose=\"snow cover normal data from 2001 to 2024\")\ndata_manager.add_record(\"key_value\", {})\n```\n- Connecting securely to remote geospatial databases for querying.\n- Reading and processing large PostGIS tables as GeoDataFrames.\n- Integrating web dashboards or data pipelines with PostgreSQL/PostGIS backends.\n\n### \ud83d\udd04 Core Methods\n\n- `from_config(db_key)`: Load credentials and settings from `DATABASES`\n- `get_engine()`: Return SQLAlchemy engine object\n- `read_as_geo_dataframe()`: Read spatial data into GeoDataFrame\n- `read_as_dataframe()`: Read tabular data with option to exclude geometry\n- `get_geometry_columns()`: Identify spatial fields in the schema\n\n### \u2699\ufe0f Configuration Example\n\nBelow is a sample `DATABASES` configuration dictionary to be placed in `digitalarzengine/settings.py`:\n\n```python\nDATABASES = {\n \"drm\": {\n \"ENGINE\": \"postgresql+psycopg2\",\n \"NAME\": \"drm\",\n \"USER\": \"dafast\",\n \"PASSWORD\": \"***********************************\",\n \"HOST\": os.getenv(\"DB_HOST\", \"localhost\"),\n \"PORT\": \"5432\",\n }\n}\n```\n\n> \u26a0\ufe0f Note: The password here is shown encrypted. It should be decrypted using `CryptoUtils` at runtime.\n\nThe `DBManager` is ideal for scenarios where spatial data needs to be read or processed securely from enterprise databases, complementing local `DataManager` workflows.\n\nFor more advanced usage patterns and custom queries, see the source or documentation site (coming soon).\n\n## \ud83d\udc68\u200d\ud83d\udcbb Developed by\n\n**Ather Ashraf**\nGeospatial Data Scientist and AI Specialist\n\n* \ud83d\udce7 Email: [atherashraf@gmail.com](mailto:atherashraf@gmail.com)\n* \ud83c\udf10 LinkedIn: [https://sa.linkedin.com/in/ather-ashraf](https://sa.linkedin.com/in/ather-ashraf)\n* \ud83d\udcdc Google Scholar: [View Profile](https://scholar.google.com.pk/citations?user=XbqhyrsAAAAJ&hl=en)\n* \ud83d\udcf1 WhatsApp: +966557252342 | +923224785104\n\n---\n\n\n",
"bugtrack_url": null,
"license": null,
"summary": "DigitalArzEngine for GEE, raster and vector data processing",
"version": "0.3.3",
"project_urls": null,
"split_keywords": [
"raster",
" vector",
" digital earth engine"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "6e358634141772972f2d274ced12ee8eb9defce1e43f1fcec3c4198557fc6824",
"md5": "bd2ffbc86e00028d0d7e64eb66968d15",
"sha256": "330166f867f47678dd8394e777545b4cc0f30acc415ecd38f073592d27f6026d"
},
"downloads": -1,
"filename": "digitalarzengine-0.3.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "bd2ffbc86e00028d0d7e64eb66968d15",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 56483,
"upload_time": "2025-07-25T22:40:37",
"upload_time_iso_8601": "2025-07-25T22:40:37.090010Z",
"url": "https://files.pythonhosted.org/packages/6e/35/8634141772972f2d274ced12ee8eb9defce1e43f1fcec3c4198557fc6824/digitalarzengine-0.3.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "4fcd4e57220eb59336ebb85f4122c06f4b259f77586a0fd1c36f33fa9cbbd13f",
"md5": "7c1b95b7033a4883c7917c01def1f8ed",
"sha256": "29bc3dc9024e866a8ac508e0dca9706577a987f2cc6eb1b1a61ff680030bd157"
},
"downloads": -1,
"filename": "digitalarzengine-0.3.3.tar.gz",
"has_sig": false,
"md5_digest": "7c1b95b7033a4883c7917c01def1f8ed",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 49074,
"upload_time": "2025-07-25T22:40:39",
"upload_time_iso_8601": "2025-07-25T22:40:39.053008Z",
"url": "https://files.pythonhosted.org/packages/4f/cd/4e57220eb59336ebb85f4122c06f4b259f77586a0fd1c36f33fa9cbbd13f/digitalarzengine-0.3.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-25 22:40:39",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "digitalarzengine"
}
JSON