dbt-schema-builder
=============================
|pypi-badge| |ci-badge| |codecov-badge| |pyversions-badge|
|license-badge|
The Schema Builder tool is used to create dbt schema files, sql models, and
default PII / non-PII views for tables in the given Snowflake schemas.
For each specified application schema, the script will generate dbt models for
a ``<SCHEMA>`` and ``<SCHEMA>_PII`` schema. We refer to these schemas as a
"trifecta".
* ``<SCHEMA>_<RAW_SUFFIX>`` contains the original source tables.
* ``<SCHEMA>_PII`` contains views on the _RAW tables that have un-redacted PII.
* ``<SCHEMA>`` contains views on the _RAW tables sensitive data redacted.
Application schemas can be sourced from multiple raw schemas. This allows you
to specify which tables should be pulled from which raw schema to construct the
"trifecta".
Schema Builder ensures that all three schemas provide the same interface to the
data (number and order of columns match what is present in the _RAW schema).
Once the script is successfully run, you can execute a `dbt run` to create or
update the views in ``<SCHEMA>`` and ``<SCHEMA>_PII``. If your source data in
the ``<SCHEMA>_<RAW_SUFFIX>`` schema changes you should run Schema Builder frequently
to keep up with changes in the tables and columns stored there.
Schema Builder will also automatically create sources in one or more other dbt
projects so that they can use the results of these models as sources.
See `the docs <https://dbt-schema-builder.readthedocs.io/en/latest/>`_ for more
information.
License
-------
The code in this repository is licensed under the AGPL 3.0 unless
otherwise noted.
Please see `LICENSE.txt <LICENSE.txt>`_ for details.
How To Contribute
-----------------
Contributions are very welcome.
Please read `The Contribution Guide <https://github.com/edx/edx-platform/blob/master/CONTRIBUTING.rst>`_ for details.
Even though they were written with ``edx-platform`` in mind, the guidelines
should be followed for all Open edX projects.
The pull request description template should be automatically applied if you are creating a pull request from GitHub. Otherwise you
can find it at `PULL_REQUEST_TEMPLATE.md <.github/PULL_REQUEST_TEMPLATE.md>`_.
The issue report template should be automatically applied if you are creating an issue on GitHub as well. Otherwise you
can find it at `ISSUE_TEMPLATE.md <.github/ISSUE_TEMPLATE.md>`_.
Reporting Security Issues
-------------------------
Please do not report security issues in public. Please email security@edx.org.
Getting Help
------------
If you're having trouble, we have discussion forums at
https://discuss.openedx.org where you can connect with others in the community.
Our real-time conversations are on Slack. You can request a `Slack
invitation`_, then join our `community Slack team`_.
For more information about these options, see the `getting assistance`_ page.
.. _Slack invitation: https://openedx-slack-invite.herokuapp.com/
.. _community Slack team: https://openedx.slack.com/
.. _getting assistance: https://openedx.org/getting-help
.. |pypi-badge| image:: https://img.shields.io/pypi/v/dbt-schema-builder.svg
:target: https://pypi.python.org/pypi/dbt-schema-builder/
:alt: PyPI
.. |ci-badge| image:: https://github.com/edx/dbt-schema-builder/workflows/Python%20CI/badge.svg?branch=master
:target: https://github.com/edx/dbt-schema-builder/actions?query=workflow%3A%22Python+CI%22
:alt: GitHub CI
.. |codecov-badge| image:: https://codecov.io/github/edx/dbt-schema-builder/coverage.svg?branch=main
:target: https://codecov.io/github/edx/dbt-schema-builder?branch=main
:alt: Codecov
.. |pyversions-badge| image:: https://img.shields.io/pypi/pyversions/dbt-schema-builder.svg
:target: https://pypi.python.org/pypi/dbt-schema-builder/
:alt: Supported Python versions
.. |license-badge| image:: https://img.shields.io/github/license/edx/dbt-schema-builder.svg
:target: https://github.com/edx/dbt-schema-builder/blob/main/LICENSE.txt
:alt: License
Raw data
{
"_id": null,
"home_page": "https://github.com/edx/dbt-schema-builder",
"name": "dbt-schema-builder",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "Python edx dbt schema builder",
"author": "edX",
"author_email": "oscm@edx.org",
"download_url": "https://files.pythonhosted.org/packages/85/01/1840b49874f810b7350b10fb773a9360028470dfcd8a6dee01e0867e2d2f/dbt-schema-builder-0.5.0.tar.gz",
"platform": null,
"description": "dbt-schema-builder\n=============================\n\n|pypi-badge| |ci-badge| |codecov-badge| |pyversions-badge|\n|license-badge|\n\nThe Schema Builder tool is used to create dbt schema files, sql models, and\ndefault PII / non-PII views for tables in the given Snowflake schemas.\n\nFor each specified application schema, the script will generate dbt models for\na ``<SCHEMA>`` and ``<SCHEMA>_PII`` schema. We refer to these schemas as a\n\"trifecta\".\n\n* ``<SCHEMA>_<RAW_SUFFIX>`` contains the original source tables.\n* ``<SCHEMA>_PII`` contains views on the _RAW tables that have un-redacted PII.\n* ``<SCHEMA>`` contains views on the _RAW tables sensitive data redacted.\n\nApplication schemas can be sourced from multiple raw schemas. This allows you\nto specify which tables should be pulled from which raw schema to construct the\n\"trifecta\".\n\nSchema Builder ensures that all three schemas provide the same interface to the\ndata (number and order of columns match what is present in the _RAW schema).\n\nOnce the script is successfully run, you can execute a `dbt run` to create or\nupdate the views in ``<SCHEMA>`` and ``<SCHEMA>_PII``. If your source data in\nthe ``<SCHEMA>_<RAW_SUFFIX>`` schema changes you should run Schema Builder frequently\nto keep up with changes in the tables and columns stored there.\n\nSchema Builder will also automatically create sources in one or more other dbt\nprojects so that they can use the results of these models as sources.\n\nSee `the docs <https://dbt-schema-builder.readthedocs.io/en/latest/>`_ for more\ninformation.\n\n\nLicense\n-------\n\nThe code in this repository is licensed under the AGPL 3.0 unless\notherwise noted.\n\nPlease see `LICENSE.txt <LICENSE.txt>`_ for details.\n\n\nHow To Contribute\n-----------------\n\nContributions are very welcome.\nPlease read `The Contribution Guide <https://github.com/edx/edx-platform/blob/master/CONTRIBUTING.rst>`_ for details.\nEven though they were written with ``edx-platform`` in mind, the guidelines\nshould be followed for all Open edX projects.\n\nThe pull request description template should be automatically applied if you are creating a pull request from GitHub. Otherwise you\ncan find it at `PULL_REQUEST_TEMPLATE.md <.github/PULL_REQUEST_TEMPLATE.md>`_.\n\nThe issue report template should be automatically applied if you are creating an issue on GitHub as well. Otherwise you\ncan find it at `ISSUE_TEMPLATE.md <.github/ISSUE_TEMPLATE.md>`_.\n\n\nReporting Security Issues\n-------------------------\n\nPlease do not report security issues in public. Please email security@edx.org.\n\n\nGetting Help\n------------\n\nIf you're having trouble, we have discussion forums at\nhttps://discuss.openedx.org where you can connect with others in the community.\n\nOur real-time conversations are on Slack. You can request a `Slack\ninvitation`_, then join our `community Slack team`_.\n\nFor more information about these options, see the `getting assistance`_ page.\n\n.. _Slack invitation: https://openedx-slack-invite.herokuapp.com/\n.. _community Slack team: https://openedx.slack.com/\n.. _getting assistance: https://openedx.org/getting-help\n\n.. |pypi-badge| image:: https://img.shields.io/pypi/v/dbt-schema-builder.svg\n :target: https://pypi.python.org/pypi/dbt-schema-builder/\n :alt: PyPI\n\n.. |ci-badge| image:: https://github.com/edx/dbt-schema-builder/workflows/Python%20CI/badge.svg?branch=master\n :target: https://github.com/edx/dbt-schema-builder/actions?query=workflow%3A%22Python+CI%22\n :alt: GitHub CI\n\n.. |codecov-badge| image:: https://codecov.io/github/edx/dbt-schema-builder/coverage.svg?branch=main\n :target: https://codecov.io/github/edx/dbt-schema-builder?branch=main\n :alt: Codecov\n\n.. |pyversions-badge| image:: https://img.shields.io/pypi/pyversions/dbt-schema-builder.svg\n :target: https://pypi.python.org/pypi/dbt-schema-builder/\n :alt: Supported Python versions\n\n.. |license-badge| image:: https://img.shields.io/github/license/edx/dbt-schema-builder.svg\n :target: https://github.com/edx/dbt-schema-builder/blob/main/LICENSE.txt\n :alt: License\n",
"bugtrack_url": null,
"license": "AGPL 3.0",
"summary": "Automate management of PII redacted schemas for dbt projects.",
"version": "0.5.0",
"project_urls": {
"Homepage": "https://github.com/edx/dbt-schema-builder"
},
"split_keywords": [
"python",
"edx",
"dbt",
"schema",
"builder"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ee219c60425fbfb1c5b7566bf8935a04d2f9e7550e6e096cfe8f3329496ce935",
"md5": "a566992c4c3d509c416ef10216974e60",
"sha256": "1ca866499b36be1cad7d9119b20851816428a9e6611dfb57891855fe8d3a362e"
},
"downloads": -1,
"filename": "dbt_schema_builder-0.5.0-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "a566992c4c3d509c416ef10216974e60",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.8",
"size": 32094,
"upload_time": "2024-03-28T12:52:27",
"upload_time_iso_8601": "2024-03-28T12:52:27.269247Z",
"url": "https://files.pythonhosted.org/packages/ee/21/9c60425fbfb1c5b7566bf8935a04d2f9e7550e6e096cfe8f3329496ce935/dbt_schema_builder-0.5.0-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "85011840b49874f810b7350b10fb773a9360028470dfcd8a6dee01e0867e2d2f",
"md5": "6e60e50161172225f1772765a08e4089",
"sha256": "6b3893c81c7cef6da28bda0bf4cd499878a15e024a8220cd5afcd0707cdd781a"
},
"downloads": -1,
"filename": "dbt-schema-builder-0.5.0.tar.gz",
"has_sig": false,
"md5_digest": "6e60e50161172225f1772765a08e4089",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 36850,
"upload_time": "2024-03-28T12:52:29",
"upload_time_iso_8601": "2024-03-28T12:52:29.056007Z",
"url": "https://files.pythonhosted.org/packages/85/01/1840b49874f810b7350b10fb773a9360028470dfcd8a6dee01e0867e2d2f/dbt-schema-builder-0.5.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-28 12:52:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "edx",
"github_project": "dbt-schema-builder",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"tox": true,
"lcname": "dbt-schema-builder"
}
JSON
