realit-singer-python
===================
[](https://badge.fury.io/py/realit-singer-python)
[](https://pypi.org/project/realit-singer-python/)
[](https://opensource.org/licenses/Apache-2.0)
Writes the Singer format from Python.
This is a fork of [Singer's singer-python](https://github.com/singer-io/singer-python) made for [PipelineWise](https://transferwise.github.io/pipelinewise) and [Meltano](https://meltano.com/)
Usage
---
### Setup environment
This library depends on python3. We recommend using a `virtualenv` like this:
```bash
python3 -m venv ~/.virtualenvs/singer-python
```
### Installation
Next, install this library:
```bash
source ~/.virtualenvs/singer-python/bin/activate
git clone http://github.com/singer-io/singer-python
cd singer-python
pip install
```
### Usage example
Now, from python code within the same `virtualenv`, you can use the library:
```python
import singer
singer.write_schema('my_table',
{'properties':{'id': {'type': 'string', 'key': True}}},
['id'])
singer.write_records('my_table',
[{'id': 'b'}, {'id':'d'}])
singer.write_state({'my_table': 'd'})
```
### Logging configuration
**realit-singer-python** by default doesn't use any predefined logging configuration, it's up to the calling
library to define it. However, if the environment variable `LOGGING_CONF_FILE` is found and set then the **realit-singer-python**
would use the path provided in the env variable as the logging configuration for the logger.
### Singer Decimal
Enabling the use_singer_decimal = True in a tap will output **decimal** and **floats** as a string
rather than their numeric representation.
**Optional Setting**:
A boolean setting: when enabled `true` in the config will outputs decimal and floating point numbers as strings to avoid loss of precision and scale.
For supporting taps, there are hints in the schema message, format = "singer.decimal", and additionalProperties scale_precision dictionary providing precision and scale. For decimal data, the target can use this
information to correctly replicate decimal data without loss. For the Floats and Number data type without precision and scale it is recommended that post processing formats the datatype based on an inspection of the data because the true data size is unknown / dynamic.
```json
{
"use_singer_decimal": true,
}
```
## Developer Resources
### Initialize your Development Environment
```bash
pip install poetry
poetry install
```
### Create and Run Tests
Create tests within the `tests/` directory and
then run:
```bash
poetry run pytest
```
or
```bash
poetry run coverage run --parallel -m pytest
```
### Continuous Integration
Run through the full suite of tests and linters by running
```bash
poetry run tox
```
These must pass in order for PR's to be merged.
License
-------
Distributed under the Apache License Version 2.0
Raw data
{
"_id": null,
"home_page": "https://github.com/s7clarke10/realit-singer-python",
"name": "realit-singer-python",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.13,>=3.8",
"maintainer_email": null,
"keywords": "singer, meltano, pipelinewise, framework",
"author": null,
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/20/1a/e637e0ab4c6c08f32d503707ffccca759ef53f61d30bb4583e6e6fee21fa/realit_singer_python-5.0.3.tar.gz",
"platform": null,
"description": "realit-singer-python\n===================\n[](https://badge.fury.io/py/realit-singer-python)\n[](https://pypi.org/project/realit-singer-python/)\n[](https://opensource.org/licenses/Apache-2.0)\n\nWrites the Singer format from Python.\n\nThis is a fork of [Singer's singer-python](https://github.com/singer-io/singer-python) made for [PipelineWise](https://transferwise.github.io/pipelinewise) and [Meltano](https://meltano.com/)\n\nUsage\n---\n\n### Setup environment\nThis library depends on python3. We recommend using a `virtualenv` like this:\n\n```bash\npython3 -m venv ~/.virtualenvs/singer-python\n```\n\n### Installation\nNext, install this library:\n\n```bash\nsource ~/.virtualenvs/singer-python/bin/activate\ngit clone http://github.com/singer-io/singer-python\ncd singer-python\npip install\n```\n\n### Usage example\nNow, from python code within the same `virtualenv`, you can use the library:\n\n```python\nimport singer\n\nsinger.write_schema('my_table',\n\t {'properties':{'id': {'type': 'string', 'key': True}}},\n\t\t ['id'])\nsinger.write_records('my_table',\n [{'id': 'b'}, {'id':'d'}])\nsinger.write_state({'my_table': 'd'})\n```\n\n### Logging configuration\n\n**realit-singer-python** by default doesn't use any predefined logging configuration, it's up to the calling \nlibrary to define it. However, if the environment variable `LOGGING_CONF_FILE` is found and set then the **realit-singer-python** \nwould use the path provided in the env variable as the logging configuration for the logger. \n\n### Singer Decimal\n\nEnabling the use_singer_decimal = True in a tap will output **decimal** and **floats** as a string\nrather than their numeric representation.\n\n**Optional Setting**:\n\nA boolean setting: when enabled `true` in the config will outputs decimal and floating point numbers as strings to avoid loss of precision and scale.\nFor supporting taps, there are hints in the schema message, format = \"singer.decimal\", and additionalProperties scale_precision dictionary providing precision and scale. For decimal data, the target can use this \ninformation to correctly replicate decimal data without loss. For the Floats and Number data type without precision and scale it is recommended that post processing formats the datatype based on an inspection of the data because the true data size is unknown / dynamic.\n\n```json\n{\n \"use_singer_decimal\": true,\n}\n```\n\n## Developer Resources\n\n### Initialize your Development Environment\n\n```bash\npip install poetry\npoetry install\n```\n\n### Create and Run Tests\n\nCreate tests within the `tests/` directory and\nthen run:\n\n```bash\npoetry run pytest\n```\n\nor \n\n```bash\npoetry run coverage run --parallel -m pytest\n```\n\n### Continuous Integration\nRun through the full suite of tests and linters by running\n\n```bash\npoetry run tox\n```\n\nThese must pass in order for PR's to be merged.\n\n\nLicense\n-------\n\nDistributed under the Apache License Version 2.0\n\n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "Singer.io utility library - PipelineWise and Meltano compatible",
"version": "5.0.3",
"project_urls": {
"Homepage": "https://github.com/s7clarke10/realit-singer-python",
"Repository": "https://github.com/s7clarke10/realit-singer-python"
},
"split_keywords": [
"singer",
" meltano",
" pipelinewise",
" framework"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "62c33076cc8bfd709256a082a93e752c121df901b59f8b8e8ab421a6bb03de41",
"md5": "3d3a6b9ffaef0f036896ce01cfa9c453",
"sha256": "1d6d3a236b67c750d3df1294cad2e817c848f5d7df8cf57936b8297bae62fb00"
},
"downloads": -1,
"filename": "realit_singer_python-5.0.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "3d3a6b9ffaef0f036896ce01cfa9c453",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.13,>=3.8",
"size": 27034,
"upload_time": "2024-08-18T23:11:41",
"upload_time_iso_8601": "2024-08-18T23:11:41.382371Z",
"url": "https://files.pythonhosted.org/packages/62/c3/3076cc8bfd709256a082a93e752c121df901b59f8b8e8ab421a6bb03de41/realit_singer_python-5.0.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "201ae637e0ab4c6c08f32d503707ffccca759ef53f61d30bb4583e6e6fee21fa",
"md5": "8a06db4bcbf543e55e82bc8fc5aa08bd",
"sha256": "d6bb036a9d9beb6d417bf1d3c607409fd577ae0f76ee853ed950b5574011d181"
},
"downloads": -1,
"filename": "realit_singer_python-5.0.3.tar.gz",
"has_sig": false,
"md5_digest": "8a06db4bcbf543e55e82bc8fc5aa08bd",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.13,>=3.8",
"size": 24428,
"upload_time": "2024-08-18T23:11:42",
"upload_time_iso_8601": "2024-08-18T23:11:42.624151Z",
"url": "https://files.pythonhosted.org/packages/20/1a/e637e0ab4c6c08f32d503707ffccca759ef53f61d30bb4583e6e6fee21fa/realit_singer_python-5.0.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-18 23:11:42",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "s7clarke10",
"github_project": "realit-singer-python",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "realit-singer-python"
}
JSON
