pancham
# Pancham Data Pipelines
[](https://sonarcloud.io/summary/new_code?id=Loqui-Tech_pancham)
Pancham simplifies the process of reading and processing data with Pandas.
## What is it for
A common scenario in data migration projects is to need to take exports from one system, apply some transformation rules and write the data somewhere else. This could be files, a database or an API. All of this can be done with Pandas but we would need to write a lot of code, Pancham is here to make that process easier.
```mermaid
flowchart LR;
s[Source Files]-->t[Transformation];
t-->o[Output Files];
```
### Key features
- Load files
- Rename fields
- Validate data types and not-null columns
- Apply python functions to transform values
- Return data as a Pandas DataFrame
- Write data to a database
### Supported Source Files
- Excel
- SQL
- YAML
- CSV
- JSON
### Supported Output
- SQL
- CSV
Additional sources and output formats will be added with time.
## Usage
The most common approach is to use a mapping file.
```yaml
name: example
file_type: yaml
file_path: source/customer.yml
key: customer
output:
- table: customers
output_type: database
fields:
- name: id
source_name: customer_id
field_type: str
nullable: false
- name: status
source_name: account_status
field_type: str
nullable: false
```
This simple mapping file expects to be able to load a source file called `customer.yml` with the following content:
```yaml
customer:
- customer_id: 123
account_status: active
- customer_id: 456
account_status: on_hold
```
The pipeline wll read the input file and populate a database that you can then query.
The final part is the pancham configuration.
Raw data
{
"_id": null,
"home_page": null,
"name": "pancham",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.11",
"maintainer_email": null,
"keywords": null,
"author": null,
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/5b/cd/152844f67af854f9e67728d05876835c5c006db4a68587c348f61c66522b/pancham-0.23.0.tar.gz",
"platform": null,
"description": "# Pancham Data Pipelines\n\n\n[](https://sonarcloud.io/summary/new_code?id=Loqui-Tech_pancham)\n\nPancham simplifies the process of reading and processing data with Pandas. \n\n## What is it for\n\nA common scenario in data migration projects is to need to take exports from one system, apply some transformation rules and write the data somewhere else. This could be files, a database or an API. All of this can be done with Pandas but we would need to write a lot of code, Pancham is here to make that process easier.\n\n```mermaid\nflowchart LR;\n s[Source Files]-->t[Transformation];\n t-->o[Output Files];\n```\n### Key features\n\n- Load files\n- Rename fields\n- Validate data types and not-null columns\n- Apply python functions to transform values\n- Return data as a Pandas DataFrame\n- Write data to a database\n\n### Supported Source Files\n\n- Excel\n- SQL\n- YAML\n- CSV\n- JSON\n\n### Supported Output\n\n- SQL\n- CSV\n\nAdditional sources and output formats will be added with time.\n\n## Usage\n\nThe most common approach is to use a mapping file.\n\n```yaml\nname: example\nfile_type: yaml\nfile_path: source/customer.yml\nkey: customer\noutput:\n - table: customers\n output_type: database\nfields:\n - name: id\n source_name: customer_id\n field_type: str\n nullable: false\n - name: status\n source_name: account_status\n field_type: str\n nullable: false\n```\n\nThis simple mapping file expects to be able to load a source file called `customer.yml` with the following content:\n\n```yaml\ncustomer:\n - customer_id: 123\n account_status: active\n - customer_id: 456\n account_status: on_hold\n```\n\nThe pipeline wll read the input file and populate a database that you can then query.\n\nThe final part is the pancham configuration.\n",
"bugtrack_url": null,
"license": null,
"summary": null,
"version": "0.23.0",
"project_urls": {
"Homepage": "https://github.com/Loqui-Tech/pancham"
},
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "b86c6212058f429c315540e0c7e671047264eb04526ecd944ef20dd36c082756",
"md5": "311243ee695a47dc934c3093602797c7",
"sha256": "925d1e365ee63027e4a67677d514023bd7b44f6026d8cd7842221bf066b8565e"
},
"downloads": -1,
"filename": "pancham-0.23.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "311243ee695a47dc934c3093602797c7",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11",
"size": 99936,
"upload_time": "2025-07-25T08:44:55",
"upload_time_iso_8601": "2025-07-25T08:44:55.546377Z",
"url": "https://files.pythonhosted.org/packages/b8/6c/6212058f429c315540e0c7e671047264eb04526ecd944ef20dd36c082756/pancham-0.23.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "5bcd152844f67af854f9e67728d05876835c5c006db4a68587c348f61c66522b",
"md5": "69c87432b17f9448e5c87e5de3499534",
"sha256": "4f975706d95a4502bc16d31d1144cb35dda1f84b6eaad1eda7368f078344eace"
},
"downloads": -1,
"filename": "pancham-0.23.0.tar.gz",
"has_sig": false,
"md5_digest": "69c87432b17f9448e5c87e5de3499534",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11",
"size": 60996,
"upload_time": "2025-07-25T08:44:56",
"upload_time_iso_8601": "2025-07-25T08:44:56.977146Z",
"url": "https://files.pythonhosted.org/packages/5b/cd/152844f67af854f9e67728d05876835c5c006db4a68587c348f61c66522b/pancham-0.23.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-25 08:44:56",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Loqui-Tech",
"github_project": "pancham",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "annotated-types",
"specs": [
[
"==",
"0.7.0"
]
]
},
{
"name": "astroid",
"specs": [
[
"==",
"3.3.8"
]
]
},
{
"name": "async-typer",
"specs": [
[
"==",
"0.1.8"
]
]
},
{
"name": "attrs",
"specs": [
[
"==",
"25.3.0"
]
]
},
{
"name": "build",
"specs": [
[
"==",
"1.2.2.post1"
]
]
},
{
"name": "certifi",
"specs": [
[
"==",
"2024.12.14"
]
]
},
{
"name": "cffi",
"specs": [
[
"==",
"1.17.1"
]
]
},
{
"name": "charset-normalizer",
"specs": [
[
"==",
"3.4.1"
]
]
},
{
"name": "click",
"specs": [
[
"==",
"8.1.8"
]
]
},
{
"name": "colorama",
"specs": [
[
"==",
"0.4.6"
]
]
},
{
"name": "coverage",
"specs": [
[
"==",
"7.6.10"
]
]
},
{
"name": "cryptography",
"specs": [
[
"==",
"45.0.3"
]
]
},
{
"name": "dill",
"specs": [
[
"==",
"0.3.9"
]
]
},
{
"name": "docutils",
"specs": [
[
"==",
"0.21.2"
]
]
},
{
"name": "et_xmlfile",
"specs": [
[
"==",
"2.0.0"
]
]
},
{
"name": "halo",
"specs": [
[
"==",
"0.0.31"
]
]
},
{
"name": "id",
"specs": [
[
"==",
"1.5.0"
]
]
},
{
"name": "idna",
"specs": [
[
"==",
"3.10"
]
]
},
{
"name": "iniconfig",
"specs": [
[
"==",
"2.0.0"
]
]
},
{
"name": "isodate",
"specs": [
[
"==",
"0.7.2"
]
]
},
{
"name": "isort",
"specs": [
[
"==",
"6.0.0"
]
]
},
{
"name": "jaraco.classes",
"specs": [
[
"==",
"3.4.0"
]
]
},
{
"name": "jaraco.context",
"specs": [
[
"==",
"6.0.1"
]
]
},
{
"name": "jaraco.functools",
"specs": [
[
"==",
"4.1.0"
]
]
},
{
"name": "jsonstraw",
"specs": [
[
"==",
"0.1.2"
]
]
},
{
"name": "keyring",
"specs": [
[
"==",
"25.6.0"
]
]
},
{
"name": "log-symbols",
"specs": [
[
"==",
"0.0.14"
]
]
},
{
"name": "lxml",
"specs": [
[
"==",
"5.4.0"
]
]
},
{
"name": "markdown-it-py",
"specs": [
[
"==",
"3.0.0"
]
]
},
{
"name": "mccabe",
"specs": [
[
"==",
"0.7.0"
]
]
},
{
"name": "mdurl",
"specs": [
[
"==",
"0.1.2"
]
]
},
{
"name": "more-itertools",
"specs": [
[
"==",
"10.6.0"
]
]
},
{
"name": "mypy-extensions",
"specs": [
[
"==",
"1.0.0"
]
]
},
{
"name": "mysqlclient",
"specs": [
[
"==",
"2.2.7"
]
]
},
{
"name": "nh3",
"specs": [
[
"==",
"0.2.20"
]
]
},
{
"name": "numpy",
"specs": [
[
"==",
"2.2.2"
]
]
},
{
"name": "openpyxl",
"specs": [
[
"==",
"3.1.5"
]
]
},
{
"name": "packaging",
"specs": [
[
"==",
"24.2"
]
]
},
{
"name": "pandas",
"specs": [
[
"==",
"2.2.3"
]
]
},
{
"name": "pandera",
"specs": [
[
"==",
"0.22.1"
]
]
},
{
"name": "phonenumbers",
"specs": [
[
"==",
"9.0.1"
]
]
},
{
"name": "platformdirs",
"specs": [
[
"==",
"4.3.6"
]
]
},
{
"name": "pluggy",
"specs": [
[
"==",
"1.5.0"
]
]
},
{
"name": "psycopg2",
"specs": [
[
"==",
"2.9.10"
]
]
},
{
"name": "pycparser",
"specs": [
[
"==",
"2.22"
]
]
},
{
"name": "pydantic",
"specs": [
[
"==",
"2.10.5"
]
]
},
{
"name": "pydantic_core",
"specs": [
[
"==",
"2.27.2"
]
]
},
{
"name": "Pygments",
"specs": [
[
"==",
"2.19.1"
]
]
},
{
"name": "PyJWT",
"specs": [
[
"==",
"2.10.1"
]
]
},
{
"name": "pylint",
"specs": [
[
"==",
"3.3.4"
]
]
},
{
"name": "pyproject_hooks",
"specs": [
[
"==",
"1.2.0"
]
]
},
{
"name": "pytest",
"specs": [
[
"==",
"8.3.4"
]
]
},
{
"name": "pytest-asyncio",
"specs": [
[
"==",
"0.25.3"
]
]
},
{
"name": "pytest-cov",
"specs": [
[
"==",
"6.0.0"
]
]
},
{
"name": "pytest-mock",
"specs": [
[
"==",
"3.14.0"
]
]
},
{
"name": "python-benedict",
"specs": [
[
"==",
"0.34.1"
]
]
},
{
"name": "python-dateutil",
"specs": [
[
"==",
"2.9.0.post0"
]
]
},
{
"name": "python-fsutil",
"specs": [
[
"==",
"0.14.1"
]
]
},
{
"name": "python-slugify",
"specs": [
[
"==",
"8.0.4"
]
]
},
{
"name": "pytz",
"specs": [
[
"==",
"2024.2"
]
]
},
{
"name": "PyYAML",
"specs": [
[
"==",
"6.0.2"
]
]
},
{
"name": "readme_renderer",
"specs": [
[
"==",
"44.0"
]
]
},
{
"name": "requests",
"specs": [
[
"==",
"2.32.3"
]
]
},
{
"name": "requests-file",
"specs": [
[
"==",
"2.1.0"
]
]
},
{
"name": "requests-toolbelt",
"specs": [
[
"==",
"1.0.0"
]
]
},
{
"name": "rfc3986",
"specs": [
[
"==",
"2.0.0"
]
]
},
{
"name": "rich",
"specs": [
[
"==",
"13.9.4"
]
]
},
{
"name": "setuptools",
"specs": [
[
"==",
"78.1.1"
]
]
},
{
"name": "shellingham",
"specs": [
[
"==",
"1.5.4"
]
]
},
{
"name": "simple-salesforce",
"specs": [
[
"==",
"1.12.6"
]
]
},
{
"name": "six",
"specs": [
[
"==",
"1.17.0"
]
]
},
{
"name": "spinners",
"specs": [
[
"==",
"0.0.24"
]
]
},
{
"name": "SQLAlchemy",
"specs": [
[
"==",
"2.0.37"
]
]
},
{
"name": "termcolor",
"specs": [
[
"==",
"2.5.0"
]
]
},
{
"name": "text-unidecode",
"specs": [
[
"==",
"1.3"
]
]
},
{
"name": "tomlkit",
"specs": [
[
"==",
"0.13.2"
]
]
},
{
"name": "twine",
"specs": [
[
"==",
"6.1.0"
]
]
},
{
"name": "typeguard",
"specs": [
[
"==",
"4.4.1"
]
]
},
{
"name": "typer",
"specs": [
[
"==",
"0.9.4"
]
]
},
{
"name": "typing-inspect",
"specs": [
[
"==",
"0.9.0"
]
]
},
{
"name": "typing_extensions",
"specs": [
[
"==",
"4.12.2"
]
]
},
{
"name": "tzdata",
"specs": [
[
"==",
"2025.1"
]
]
},
{
"name": "urllib3",
"specs": [
[
"==",
"2.3.0"
]
]
},
{
"name": "wheel",
"specs": [
[
"==",
"0.45.1"
]
]
},
{
"name": "zeep",
"specs": [
[
"==",
"4.3.1"
]
]
}
],
"lcname": "pancham"
}
