meltano-map-transform


Namemeltano-map-transform JSON
Version 0.10.0 PyPI version JSON
download
home_pageNone
SummarySinger-compatible inline map transformer, built with the Meltano SDK for Singer Taps.
upload_time2025-02-10 20:49:27
maintainerNone
docs_urlNone
authorMeltano Team and Contributors
requires_python>=3.9
licenseApache-2.0
keywords elt singer.io inline transformation
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # meltano-map-transformer

<img src="examples/mapperpic.png" width="800px"/>

[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/MeltanoLabs/meltano-map-transform/main.svg)](https://results.pre-commit.ci/latest/github/MeltanoLabs/meltano-map-transform/main)

A map transformer to duplicate, filter, hash, mask and do many more small transformations in your extract & load pipelines. Implements the [`Stream Maps` capability](https://sdk.meltano.com/en/latest/stream_maps.html).

This mapper plugin is fully compliant with the Singer Spec and can be placed in between any Singer tap and target.

## meltano-map-transformer in action

For a collection of examples, take a look at [examples/README.md](examples/README.md).

## Settings

| Setting             | Required | Default | Description |
|:--------------------|:--------:|:-------:|:------------|
| stream_maps         | True     | None    | Stream maps |
| flattening_enabled  | False    | None    | 'True' to enable schema flattening and automatically expand nested properties. |
| flattening_max_depth| False    | None    | The max depth to flatten schemas. |
| stream_map_config   | False    | None    | User-defined config values to be used within map expressions. |

A full list of supported settings and capabilities is available by running: `meltano-map-transformer --about`

## Installation

We recommend using GitHub release tags when installing with `pip`. We also recommend using `pipx` or `meltano` instead of installing with `pip` directly.

You can see a full list of published releases [here](https://github.com/MeltanoLabs/meltano-map-transform/releases).

For example:

```shell
# Use `uv` so that virtual environments are managed automatically:
uv tool install meltano-map-transform==0.10.0

# Alternatively, use `pipx`:
pipx install meltano-map-transform==0.10.0
```

### Meltano installation instructions

1. Add this plugin to your Meltano project

   ```console
   meltano add mapper meltano-map-transformer
   ```

1. Install the plugin

   ```console
   meltano install
   ```

1. Configure the plugin

   ```yaml
   # meltano.yml
   plugins:
     mappers:
     - name: meltano-map-transformer
       pip_url: meltano-map-transform
       executable: meltano-map-transform
       mappings:
       - name: hash_email
         config:
           stream_maps:
             customers:
               email: null # drop the PII field from RECORD and SCHEMA messages
               email_domain: email.split('@')[-1]
               email_hash: md5(config['hash_seed'] + email)
               __else__: null # drop all other fields
           stream_map_config:
             hash_seed: 01AWZh7A6DzGm6iJZZ2T
    ```

1. Use the plugin in your pipelines

   ```console
   meltano run tap-csv hash_email target-sqlite
   ```

## Capabilities

* `stream-maps`

## Settings

| Setting     | Required | Default | Description |
|:------------|:--------:|:-------:|:------------|
| stream_maps | True     | None    | Stream maps |

A full list of supported settings and capabilities is available by running: `meltano-map-transformer --about`


            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "meltano-map-transform",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.9",
    "maintainer_email": null,
    "keywords": "ELT, singer.io, Inline Transformation",
    "author": "Meltano Team and Contributors",
    "author_email": null,
    "download_url": "https://files.pythonhosted.org/packages/42/86/9d0de23f7fd680efd19d0919ddc7b3576539ea28e07bbc9279b071b22496/meltano_map_transform-0.10.0.tar.gz",
    "platform": null,
    "description": "# meltano-map-transformer\n\n<img src=\"examples/mapperpic.png\" width=\"800px\"/>\n\n[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/MeltanoLabs/meltano-map-transform/main.svg)](https://results.pre-commit.ci/latest/github/MeltanoLabs/meltano-map-transform/main)\n\nA map transformer to duplicate, filter, hash, mask and do many more small transformations in your extract & load pipelines. Implements the [`Stream Maps` capability](https://sdk.meltano.com/en/latest/stream_maps.html).\n\nThis mapper plugin is fully compliant with the Singer Spec and can be placed in between any Singer tap and target.\n\n## meltano-map-transformer in action\n\nFor a collection of examples, take a look at [examples/README.md](examples/README.md).\n\n## Settings\n\n| Setting             | Required | Default | Description |\n|:--------------------|:--------:|:-------:|:------------|\n| stream_maps         | True     | None    | Stream maps |\n| flattening_enabled  | False    | None    | 'True' to enable schema flattening and automatically expand nested properties. |\n| flattening_max_depth| False    | None    | The max depth to flatten schemas. |\n| stream_map_config   | False    | None    | User-defined config values to be used within map expressions. |\n\nA full list of supported settings and capabilities is available by running: `meltano-map-transformer --about`\n\n## Installation\n\nWe recommend using GitHub release tags when installing with `pip`. We also recommend using `pipx` or `meltano` instead of installing with `pip` directly.\n\nYou can see a full list of published releases [here](https://github.com/MeltanoLabs/meltano-map-transform/releases).\n\nFor example:\n\n```shell\n# Use `uv` so that virtual environments are managed automatically:\nuv tool install meltano-map-transform==0.10.0\n\n# Alternatively, use `pipx`:\npipx install meltano-map-transform==0.10.0\n```\n\n### Meltano installation instructions\n\n1. Add this plugin to your Meltano project\n\n   ```console\n   meltano add mapper meltano-map-transformer\n   ```\n\n1. Install the plugin\n\n   ```console\n   meltano install\n   ```\n\n1. Configure the plugin\n\n   ```yaml\n   # meltano.yml\n   plugins:\n     mappers:\n     - name: meltano-map-transformer\n       pip_url: meltano-map-transform\n       executable: meltano-map-transform\n       mappings:\n       - name: hash_email\n         config:\n           stream_maps:\n             customers:\n               email: null # drop the PII field from RECORD and SCHEMA messages\n               email_domain: email.split('@')[-1]\n               email_hash: md5(config['hash_seed'] + email)\n               __else__: null # drop all other fields\n           stream_map_config:\n             hash_seed: 01AWZh7A6DzGm6iJZZ2T\n    ```\n\n1. Use the plugin in your pipelines\n\n   ```console\n   meltano run tap-csv hash_email target-sqlite\n   ```\n\n## Capabilities\n\n* `stream-maps`\n\n## Settings\n\n| Setting     | Required | Default | Description |\n|:------------|:--------:|:-------:|:------------|\n| stream_maps | True     | None    | Stream maps |\n\nA full list of supported settings and capabilities is available by running: `meltano-map-transformer --about`\n\n",
    "bugtrack_url": null,
    "license": "Apache-2.0",
    "summary": "Singer-compatible inline map transformer, built with the Meltano SDK for Singer Taps.",
    "version": "0.10.0",
    "project_urls": {
        "Documentation": "https://github.com/MeltanoLabs/meltano-map-transform#readme",
        "Homepage": "https://github.com/MeltanoLabs/meltano-map-transform",
        "Repository": "https://github.com/MeltanoLabs/meltano-map-transform"
    },
    "split_keywords": [
        "elt",
        " singer.io",
        " inline transformation"
    ],
    "urls": [
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "0c8fa7ad4112c9dec041cc61f3f260d1bd59db9b5c5ccdb2517ed1b45c7d6c53",
                "md5": "8343e7b3f205e636d43410d68d90406c",
                "sha256": "7aed937bbc2be98e53e9510a9d43907d577908bb84ecceef652d991e9d2f445e"
            },
            "downloads": -1,
            "filename": "meltano_map_transform-0.10.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "8343e7b3f205e636d43410d68d90406c",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.9",
            "size": 8861,
            "upload_time": "2025-02-10T20:49:25",
            "upload_time_iso_8601": "2025-02-10T20:49:25.800121Z",
            "url": "https://files.pythonhosted.org/packages/0c/8f/a7ad4112c9dec041cc61f3f260d1bd59db9b5c5ccdb2517ed1b45c7d6c53/meltano_map_transform-0.10.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "42869d0de23f7fd680efd19d0919ddc7b3576539ea28e07bbc9279b071b22496",
                "md5": "216c2b974c6c02e244b1b025afecb73a",
                "sha256": "59f04800bffb542818dd8f4847fae60b8349b9717424a5c0e34bf43a2d1f843b"
            },
            "downloads": -1,
            "filename": "meltano_map_transform-0.10.0.tar.gz",
            "has_sig": false,
            "md5_digest": "216c2b974c6c02e244b1b025afecb73a",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.9",
            "size": 7930,
            "upload_time": "2025-02-10T20:49:27",
            "upload_time_iso_8601": "2025-02-10T20:49:27.325639Z",
            "url": "https://files.pythonhosted.org/packages/42/86/9d0de23f7fd680efd19d0919ddc7b3576539ea28e07bbc9279b071b22496/meltano_map_transform-0.10.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-02-10 20:49:27",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "MeltanoLabs",
    "github_project": "meltano-map-transform#readme",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "meltano-map-transform"
}
        
Elapsed time: 0.76772s