| Name | meltanolabs-tap-snowflake JSON |
| Version |
0.0.25
JSON |
| download |
| home_page | None |
| Summary | `tap-snowflake` is a Singer tap for Snowflake, built with the Meltano SDK for Singer Taps. |
| upload_time | 2024-09-20 21:42:50 |
| maintainer | None |
| docs_url | None |
| author | Meltano Team |
| requires_python | <3.12,>=3.8 |
| license | Apache 2.0 |
| keywords |
elt
snowflake
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# tap-snowflake
`tap-snowflake` is a Singer tap for Snowflake.
Built with the [Meltano Tap SDK](https://sdk.meltano.com) for Singer Taps.
## Installation
```bash
pipx install git+https://github.com/MeltanoLabs/tap-snowflake.git
```
## Configuration
### Accepted Config Options
A full list of supported settings and capabilities for this
tap is available by running:
```bash
tap-snowflake --about
```
### `tables` configuration parameter.
An array of the table names that you want to sync. The table names should be fully qualified, including schema and table name.
NOTES:
* This limits discovery to the tables specified for performance reasons. Do not specify `tables` if you intend to discover the entire available catalog.
* Syntax to specify `tables` differs slightly from `select` (`schema_name.table_name` vs `schema_name-table_name.*`)
:bulb: When adding more elements to `select` ensure the table is specified in `tables` if using `tables`.
Example:
```yaml
...
tables:
- schema_name1.table1
- schema_name1.table2
```
### Configure using environment variables
This Singer tap will automatically import any environment variables within the working directory's
`.env` if the `--config=ENV` is provided, such that config values will be considered if a matching
environment variable is set either in the terminal context or in the `.env` file.
### Source Authentication and Authorization
This tap supports two authentication methods:
- Standard `username` and `password` auth
- [Key Pair Authentication](https://docs.snowflake.com/en/user-guide/key-pair-auth)
#### Key Pair Authentication
Key Pair authentication can be enabled by specifing either the `private_key` of the `private_key_path`, and optionally the `private_key_passphrase` if required.
### Enabling Batch Messaging
This tap is built using the Meltano SDK and therefore supports a `BATCH` [message type](https://sdk.meltano.com/en/latest/batch.html), in
addition to the `RECORD` messages of the Singer spec. This can be enabled either by adding the following to your `config.json`:
```json
{
// ...
"batch_config": {
"encoding": {
"format": "jsonl",
"compression": "gzip"
},
"storage": {
"root": "file://tests/core/resources",
"prefix": "test-batch"
}
}
}
```
or its equivalent to your `meltano.yml`
```yaml
config:
plugins:
extractors:
- name: tap-snowflake
config:
batch_config:
encoding:
format: jsonl
compression: gzip
storage:
root: "file://tests/core/resources"
prefix: test-batch
```
**Note:** This variant of `tap-snowflake` does not yet support the `INCREMENTAL` replication strategy in `BATCH` mode. Follow [here](https://github.com/meltano/sdk/issues/976#issuecomment-1257848119) for updates.
## Usage
You can easily run `tap-snowflake` by itself or in a pipeline using [Meltano](https://meltano.com/).
### Executing the Tap Directly
```bash
tap-snowflake --version
tap-snowflake --help
tap-snowflake --config CONFIG --discover > ./catalog.json
```
## Developer Resources
### Initialize your Development Environment
```bash
pipx install poetry
poetry install
```
### Create and Run Tests
Create tests within the `tap_snowflake/tests` subfolder and
then run:
```bash
poetry run pytest
```
You can also test the `tap-snowflake` CLI interface directly using `poetry run`:
```bash
poetry run tap-snowflake --help
```
### Testing with [Meltano](https://www.meltano.com)
_**Note:** This tap will work in any Singer environment and does not require Meltano.
Examples here are for convenience and to streamline end-to-end orchestration scenarios._
Your project comes with a custom `meltano.yml` project file already created. Open the `meltano.yml` and follow any _"TODO"_ items listed in
the file.
Next, install Meltano (if you haven't already) and any needed plugins:
```bash
# Install meltano
pipx install meltano
# Initialize meltano within this directory
cd tap-snowflake
meltano install
```
Now you can test and orchestrate using Meltano:
```bash
# Test invocation:
meltano invoke tap-snowflake --version
# OR run a test `elt` pipeline:
meltano elt tap-snowflake target-jsonl
```
### SDK Dev Guide
See the [dev guide](https://sdk.meltano.com/en/latest/dev_guide.html) for more instructions on how to use the SDK to
develop your own taps and targets.
Raw data
{
"_id": null,
"home_page": null,
"name": "meltanolabs-tap-snowflake",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.12,>=3.8",
"maintainer_email": null,
"keywords": "ELT, Snowflake",
"author": "Meltano Team",
"author_email": "hello@meltano.com",
"download_url": "https://files.pythonhosted.org/packages/c0/09/414f22a5b2d893a88e26272975087a246b22e575f2520fa9c636085e8446/meltanolabs_tap_snowflake-0.0.25.tar.gz",
"platform": null,
"description": "# tap-snowflake\n\n`tap-snowflake` is a Singer tap for Snowflake.\n\nBuilt with the [Meltano Tap SDK](https://sdk.meltano.com) for Singer Taps.\n\n## Installation\n\n```bash\npipx install git+https://github.com/MeltanoLabs/tap-snowflake.git\n```\n\n## Configuration\n\n### Accepted Config Options\n\nA full list of supported settings and capabilities for this\ntap is available by running:\n\n```bash\ntap-snowflake --about\n```\n\n### `tables` configuration parameter.\n\nAn array of the table names that you want to sync. The table names should be fully qualified, including schema and table name.\n\nNOTES: \n* This limits discovery to the tables specified for performance reasons. Do not specify `tables` if you intend to discover the entire available catalog.\n* Syntax to specify `tables` differs slightly from `select` (`schema_name.table_name` vs `schema_name-table_name.*`)\n\n:bulb: When adding more elements to `select` ensure the table is specified in `tables` if using `tables`. \n\nExample:\n```yaml\n ...\n tables:\n - schema_name1.table1\n - schema_name1.table2\n```\n\n### Configure using environment variables\n\nThis Singer tap will automatically import any environment variables within the working directory's\n`.env` if the `--config=ENV` is provided, such that config values will be considered if a matching\nenvironment variable is set either in the terminal context or in the `.env` file.\n\n### Source Authentication and Authorization\n\nThis tap supports two authentication methods:\n- Standard `username` and `password` auth\n- [Key Pair Authentication](https://docs.snowflake.com/en/user-guide/key-pair-auth)\n\n#### Key Pair Authentication\n\nKey Pair authentication can be enabled by specifing either the `private_key` of the `private_key_path`, and optionally the `private_key_passphrase` if required.\n\n### Enabling Batch Messaging\n\nThis tap is built using the Meltano SDK and therefore supports a `BATCH` [message type](https://sdk.meltano.com/en/latest/batch.html), in\naddition to the `RECORD` messages of the Singer spec. This can be enabled either by adding the following to your `config.json`:\n\n```json\n{\n // ...\n \"batch_config\": {\n \"encoding\": {\n \"format\": \"jsonl\",\n \"compression\": \"gzip\"\n },\n \"storage\": {\n \"root\": \"file://tests/core/resources\",\n \"prefix\": \"test-batch\"\n }\n }\n}\n```\n\nor its equivalent to your `meltano.yml`\n\n```yaml\nconfig:\n plugins:\n extractors:\n - name: tap-snowflake\n config:\n batch_config:\n encoding:\n format: jsonl\n compression: gzip\n storage:\n root: \"file://tests/core/resources\"\n prefix: test-batch\n```\n\n**Note:** This variant of `tap-snowflake` does not yet support the `INCREMENTAL` replication strategy in `BATCH` mode. Follow [here](https://github.com/meltano/sdk/issues/976#issuecomment-1257848119) for updates.\n\n## Usage\n\nYou can easily run `tap-snowflake` by itself or in a pipeline using [Meltano](https://meltano.com/).\n\n### Executing the Tap Directly\n\n```bash\ntap-snowflake --version\ntap-snowflake --help\ntap-snowflake --config CONFIG --discover > ./catalog.json\n```\n\n## Developer Resources\n\n### Initialize your Development Environment\n\n```bash\npipx install poetry\npoetry install\n```\n\n### Create and Run Tests\n\nCreate tests within the `tap_snowflake/tests` subfolder and\nthen run:\n\n```bash\npoetry run pytest\n```\n\nYou can also test the `tap-snowflake` CLI interface directly using `poetry run`:\n\n```bash\npoetry run tap-snowflake --help\n```\n\n### Testing with [Meltano](https://www.meltano.com)\n\n_**Note:** This tap will work in any Singer environment and does not require Meltano.\nExamples here are for convenience and to streamline end-to-end orchestration scenarios._\n\nYour project comes with a custom `meltano.yml` project file already created. Open the `meltano.yml` and follow any _\"TODO\"_ items listed in\nthe file.\n\nNext, install Meltano (if you haven't already) and any needed plugins:\n\n```bash\n# Install meltano\npipx install meltano\n# Initialize meltano within this directory\ncd tap-snowflake\nmeltano install\n```\n\nNow you can test and orchestrate using Meltano:\n\n```bash\n# Test invocation:\nmeltano invoke tap-snowflake --version\n# OR run a test `elt` pipeline:\nmeltano elt tap-snowflake target-jsonl\n```\n\n### SDK Dev Guide\n\nSee the [dev guide](https://sdk.meltano.com/en/latest/dev_guide.html) for more instructions on how to use the SDK to\ndevelop your own taps and targets.\n\n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "`tap-snowflake` is a Singer tap for Snowflake, built with the Meltano SDK for Singer Taps.",
"version": "0.0.25",
"project_urls": null,
"split_keywords": [
"elt",
" snowflake"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6985f11e429ada04bfde68e45a12fe6aca400112908b3bb6e71427a789af092d",
"md5": "57db760e8fc4c1e5301f11f967d0e6e6",
"sha256": "c269282bd8ae587632c275945181232fe4a131a71ac7db1120898e222ff3a2e4"
},
"downloads": -1,
"filename": "meltanolabs_tap_snowflake-0.0.25-py3-none-any.whl",
"has_sig": false,
"md5_digest": "57db760e8fc4c1e5301f11f967d0e6e6",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.12,>=3.8",
"size": 12613,
"upload_time": "2024-09-20T21:42:49",
"upload_time_iso_8601": "2024-09-20T21:42:49.372483Z",
"url": "https://files.pythonhosted.org/packages/69/85/f11e429ada04bfde68e45a12fe6aca400112908b3bb6e71427a789af092d/meltanolabs_tap_snowflake-0.0.25-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c009414f22a5b2d893a88e26272975087a246b22e575f2520fa9c636085e8446",
"md5": "4517404ec60bd63fc23888bbb0717807",
"sha256": "55801a7abf9fe029ed5f96afd4eea538b680327c0a07723b428ebacc474c8e18"
},
"downloads": -1,
"filename": "meltanolabs_tap_snowflake-0.0.25.tar.gz",
"has_sig": false,
"md5_digest": "4517404ec60bd63fc23888bbb0717807",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.12,>=3.8",
"size": 12796,
"upload_time": "2024-09-20T21:42:50",
"upload_time_iso_8601": "2024-09-20T21:42:50.934934Z",
"url": "https://files.pythonhosted.org/packages/c0/09/414f22a5b2d893a88e26272975087a246b22e575f2520fa9c636085e8446/meltanolabs_tap_snowflake-0.0.25.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-20 21:42:50",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "meltanolabs-tap-snowflake"
}
