# meltano-map-transformer
<img src="examples/mapperpic.png" width="800px"/>
[](https://results.pre-commit.ci/latest/github/MeltanoLabs/meltano-map-transform/main)
A map transformer to duplicate, filter, hash, mask and do many more small transformations in your extract & load pipelines. Implements the [`Stream Maps` capability](https://sdk.meltano.com/en/latest/stream_maps.html).
This mapper plugin is fully compliant with the Singer Spec and can be placed in between any Singer tap and target.
## meltano-map-transformer in action
For a collection of examples, take a look at [examples/README.md](examples/README.md).
## Settings
| Setting | Required | Default | Description |
|:--------------------|:--------:|:-------:|:------------|
| stream_maps | True | None | Stream maps |
| flattening_enabled | False | None | 'True' to enable schema flattening and automatically expand nested properties. |
| flattening_max_depth| False | None | The max depth to flatten schemas. |
| stream_map_config | False | None | User-defined config values to be used within map expressions. |
A full list of supported settings and capabilities is available by running: `meltano-map-transformer --about`
## Installation
We recommend using GitHub release tags when installing with `pip`. We also recommend using `pipx` or `meltano` instead of installing with `pip` directly.
You can see a full list of published releases [here](https://github.com/MeltanoLabs/meltano-map-transform/releases).
For example:
```shell
# Use `uv` so that virtual environments are managed automatically:
uv tool install meltano-map-transform==0.10.0
# Alternatively, use `pipx`:
pipx install meltano-map-transform==0.10.0
```
### Meltano installation instructions
1. Add this plugin to your Meltano project
```console
meltano add mapper meltano-map-transformer
```
1. Install the plugin
```console
meltano install
```
1. Configure the plugin
```yaml
# meltano.yml
plugins:
mappers:
- name: meltano-map-transformer
pip_url: meltano-map-transform
executable: meltano-map-transform
mappings:
- name: hash_email
config:
stream_maps:
customers:
email: null # drop the PII field from RECORD and SCHEMA messages
email_domain: email.split('@')[-1]
email_hash: md5(config['hash_seed'] + email)
__else__: null # drop all other fields
stream_map_config:
hash_seed: 01AWZh7A6DzGm6iJZZ2T
```
1. Use the plugin in your pipelines
```console
meltano run tap-csv hash_email target-sqlite
```
## Capabilities
* `stream-maps`
## Settings
| Setting | Required | Default | Description |
|:------------|:--------:|:-------:|:------------|
| stream_maps | True | None | Stream maps |
A full list of supported settings and capabilities is available by running: `meltano-map-transformer --about`
Raw data
{
"_id": null,
"home_page": null,
"name": "meltano-map-transform",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "ELT, singer.io, Inline Transformation",
"author": "Meltano Team and Contributors",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/42/86/9d0de23f7fd680efd19d0919ddc7b3576539ea28e07bbc9279b071b22496/meltano_map_transform-0.10.0.tar.gz",
"platform": null,
"description": "# meltano-map-transformer\n\n<img src=\"examples/mapperpic.png\" width=\"800px\"/>\n\n[](https://results.pre-commit.ci/latest/github/MeltanoLabs/meltano-map-transform/main)\n\nA map transformer to duplicate, filter, hash, mask and do many more small transformations in your extract & load pipelines. Implements the [`Stream Maps` capability](https://sdk.meltano.com/en/latest/stream_maps.html).\n\nThis mapper plugin is fully compliant with the Singer Spec and can be placed in between any Singer tap and target.\n\n## meltano-map-transformer in action\n\nFor a collection of examples, take a look at [examples/README.md](examples/README.md).\n\n## Settings\n\n| Setting | Required | Default | Description |\n|:--------------------|:--------:|:-------:|:------------|\n| stream_maps | True | None | Stream maps |\n| flattening_enabled | False | None | 'True' to enable schema flattening and automatically expand nested properties. |\n| flattening_max_depth| False | None | The max depth to flatten schemas. |\n| stream_map_config | False | None | User-defined config values to be used within map expressions. |\n\nA full list of supported settings and capabilities is available by running: `meltano-map-transformer --about`\n\n## Installation\n\nWe recommend using GitHub release tags when installing with `pip`. We also recommend using `pipx` or `meltano` instead of installing with `pip` directly.\n\nYou can see a full list of published releases [here](https://github.com/MeltanoLabs/meltano-map-transform/releases).\n\nFor example:\n\n```shell\n# Use `uv` so that virtual environments are managed automatically:\nuv tool install meltano-map-transform==0.10.0\n\n# Alternatively, use `pipx`:\npipx install meltano-map-transform==0.10.0\n```\n\n### Meltano installation instructions\n\n1. Add this plugin to your Meltano project\n\n ```console\n meltano add mapper meltano-map-transformer\n ```\n\n1. Install the plugin\n\n ```console\n meltano install\n ```\n\n1. Configure the plugin\n\n ```yaml\n # meltano.yml\n plugins:\n mappers:\n - name: meltano-map-transformer\n pip_url: meltano-map-transform\n executable: meltano-map-transform\n mappings:\n - name: hash_email\n config:\n stream_maps:\n customers:\n email: null # drop the PII field from RECORD and SCHEMA messages\n email_domain: email.split('@')[-1]\n email_hash: md5(config['hash_seed'] + email)\n __else__: null # drop all other fields\n stream_map_config:\n hash_seed: 01AWZh7A6DzGm6iJZZ2T\n ```\n\n1. Use the plugin in your pipelines\n\n ```console\n meltano run tap-csv hash_email target-sqlite\n ```\n\n## Capabilities\n\n* `stream-maps`\n\n## Settings\n\n| Setting | Required | Default | Description |\n|:------------|:--------:|:-------:|:------------|\n| stream_maps | True | None | Stream maps |\n\nA full list of supported settings and capabilities is available by running: `meltano-map-transformer --about`\n\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Singer-compatible inline map transformer, built with the Meltano SDK for Singer Taps.",
"version": "0.10.0",
"project_urls": {
"Documentation": "https://github.com/MeltanoLabs/meltano-map-transform#readme",
"Homepage": "https://github.com/MeltanoLabs/meltano-map-transform",
"Repository": "https://github.com/MeltanoLabs/meltano-map-transform"
},
"split_keywords": [
"elt",
" singer.io",
" inline transformation"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "0c8fa7ad4112c9dec041cc61f3f260d1bd59db9b5c5ccdb2517ed1b45c7d6c53",
"md5": "8343e7b3f205e636d43410d68d90406c",
"sha256": "7aed937bbc2be98e53e9510a9d43907d577908bb84ecceef652d991e9d2f445e"
},
"downloads": -1,
"filename": "meltano_map_transform-0.10.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8343e7b3f205e636d43410d68d90406c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 8861,
"upload_time": "2025-02-10T20:49:25",
"upload_time_iso_8601": "2025-02-10T20:49:25.800121Z",
"url": "https://files.pythonhosted.org/packages/0c/8f/a7ad4112c9dec041cc61f3f260d1bd59db9b5c5ccdb2517ed1b45c7d6c53/meltano_map_transform-0.10.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "42869d0de23f7fd680efd19d0919ddc7b3576539ea28e07bbc9279b071b22496",
"md5": "216c2b974c6c02e244b1b025afecb73a",
"sha256": "59f04800bffb542818dd8f4847fae60b8349b9717424a5c0e34bf43a2d1f843b"
},
"downloads": -1,
"filename": "meltano_map_transform-0.10.0.tar.gz",
"has_sig": false,
"md5_digest": "216c2b974c6c02e244b1b025afecb73a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 7930,
"upload_time": "2025-02-10T20:49:27",
"upload_time_iso_8601": "2025-02-10T20:49:27.325639Z",
"url": "https://files.pythonhosted.org/packages/42/86/9d0de23f7fd680efd19d0919ddc7b3576539ea28e07bbc9279b071b22496/meltano_map_transform-0.10.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-10 20:49:27",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "MeltanoLabs",
"github_project": "meltano-map-transform#readme",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "meltano-map-transform"
}
JSON
