# Feast Teradata Connector
[](https://github.com/Teradata/feast-teradata/actions/workflows/ci-integeration-tests.yml)
## Overview
We recommend you familiarize yourself with the terminology and concepts of feast by reading the official [feast documentation](https://docs.feast.dev/).
The `feast-teradata` library adds support for Teradata as
- OfflineStore
- OnlineStore
Additional, using Teradata as the registry (catalog) is already supported via the `registry_type: sql` and included in our examples. This means that everything is located in Teradata. However, depending on the requirements, installation, etc, this can be mixed and matched with other systems as appropriate.
## Getting Started
To get started, install the `feast-teradata` library
```bash
pip install feast-teradata
```
Let's create a simple feast setup with Teradata using the standard drivers dataset. Note that you cannot use `feast init` as this command only works for templates which are part of the core feast library. We intend on getting this library merged into feast core eventually but for now, you will need to use the following cli command for this specific task. All other `feast` cli commands work as expected.
```bash
feast-td init-repo
```
This will then prompt you for the required information for the Teradata system and upload the example dataset. Let's assume you used the repo name `demo` when running the above command. You can find the repository files along with a file called `test_workflow.py`. Running this `test_workflow.py` will execute a complete workflow for feast with Teradata as the Registry, OfflineStore and OnlineStore.
```
demo/
feature_repo/
driver_repo.py
feature_store.yml
test_workflow.py
```
From within the `demo/feature_repo` directory, execute the following feast command to apply (import/update) the repo definition into the registry. You will be able to see the registry metadata tables in the teradata database after running this command.
```bash
feast apply
```
To see the registry information in the feast ui, run the following command. Note the --registry_ttl_sec is important as by default it polls every 5 seconds.
```bash
feast ui --registry_ttl_sec=120
```
## Example Usage
Now, lets batch read some features for training, using only entities (population) for which we have seen an event for in the last `60` days. The predicates (filter) used can be on anything that is relevant for the entity (population) selection for the given training dataset. The `event_timestamp` is only for example purposes.
```python
from feast import FeatureStore
store = FeatureStore(repo_path="feature_repo")
training_df = store.get_historical_features(
entity_df=f"""
SELECT
driver_id,
event_timestamp
FROM demo_feast_driver_hourly_stats
WHERE event_timestamp BETWEEN (CURRENT_TIMESTAMP - INTERVAL '60' DAY) AND CURRENT_TIMESTAMP
""",
features=[
"driver_hourly_stats:conv_rate",
"driver_hourly_stats:acc_rate",
"driver_hourly_stats:avg_daily_trips"
],
).to_df()
print(training_df.head())
```
The `feast-teradata` library allows you to use the complete set of feast APIs and functionality. Please refer to the official [feast quickstart](https://docs.feast.dev/getting-started/quickstart) for more details on the various things you can do.
Additionally, if you want to see a complete (but not real-world), end-to-end example workflow example, see the `demo/test_workflow.py` script. This is used for testing the complete feast functionality.
## Repo Configuration
A feast repository is configured via the `feature_store.yaml`. There are 3 sections in this that can be configured to use Teradata
- Registry
- OfflineStore
- OnlineStore
To configure Teradata as the `OnlineStore`, use the following configuration
```yaml
online_store:
type: feast_teradata.online.teradata.TeradataOnlineStore
host: <host>
database: <db>
user: <user>
password: <password>
log_mech: <TDNEGO|LDAP|etc>
```
To configure Teradata as the `OfflineStore`, use the following configuration
```yaml
offline_store:
type: feast_teradata.offline.teradata.TeradataOfflineStore
host: <host>
database: <db>
user: <user>
password: <password>
log_mech: <TDNEGO|LDAP|etc>
```
To configure Teradata as the `Registry`, configure the `registry_type` as `sql` and the path as the sqlalchemy url for teradata as follows
```yaml
registry:
registry_type: sql
path: teradatasql://<user>:<password>@<host>/?database=<database>&LOGMECH=<TDNEGO|LDAP|etc>
cache_ttl_seconds: 120
```
## Release Notes
### 1.0.4
- Update: bump Feast dependency to 0.31.1
### 1.0.3
- Fix: Added string mapping for columns.
### 1.0.2
- Doc: Improve README with details on repo configuration
- Fix: Fix Github Release on CI Release
- Fix: Updated path variable to become OS independent.
### 1.0.1
- Doc: Improve README with better getting started information.
- Fix: Remove pytest from requirements.txt
- Fix: Set minimum python version to 3.8 due to feast dependency on pandas>=1.4.3
- Fix: Updated feast-td types conversion
### 1.0.0
- Feature: Initial implementation of feast-teradata library
Raw data
{
"_id": null,
"home_page": "https://github.com/Teradata/feast-teradata",
"name": "feast-teradata",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<3.11",
"maintainer_email": "",
"keywords": "",
"author": "Teradata Corporation",
"author_email": "developers@teradata.com",
"download_url": "https://files.pythonhosted.org/packages/e2/9b/80354c79909b8770f71f9c2671cff2d39dd61ed44d36bded26ea7553e901/feast-teradata-1.0.4.tar.gz",
"platform": null,
"description": "# Feast Teradata Connector\n[](https://github.com/Teradata/feast-teradata/actions/workflows/ci-integeration-tests.yml)\n\n## Overview\n\nWe recommend you familiarize yourself with the terminology and concepts of feast by reading the official [feast documentation](https://docs.feast.dev/). \n\nThe `feast-teradata` library adds support for Teradata as \n- OfflineStore \n- OnlineStore\n\nAdditional, using Teradata as the registry (catalog) is already supported via the `registry_type: sql` and included in our examples. This means that everything is located in Teradata. However, depending on the requirements, installation, etc, this can be mixed and matched with other systems as appropriate. \n\n## Getting Started\n\nTo get started, install the `feast-teradata` library\n\n```bash\npip install feast-teradata\n```\n\nLet's create a simple feast setup with Teradata using the standard drivers dataset. Note that you cannot use `feast init` as this command only works for templates which are part of the core feast library. We intend on getting this library merged into feast core eventually but for now, you will need to use the following cli command for this specific task. All other `feast` cli commands work as expected. \n\n```bash\nfeast-td init-repo\n```\n\nThis will then prompt you for the required information for the Teradata system and upload the example dataset. Let's assume you used the repo name `demo` when running the above command. You can find the repository files along with a file called `test_workflow.py`. Running this `test_workflow.py` will execute a complete workflow for feast with Teradata as the Registry, OfflineStore and OnlineStore. \n\n\n```\ndemo/\n feature_repo/\n driver_repo.py\n feature_store.yml\n test_workflow.py\n```\n\n\nFrom within the `demo/feature_repo` directory, execute the following feast command to apply (import/update) the repo definition into the registry. You will be able to see the registry metadata tables in the teradata database after running this command.\n\n\n```bash\nfeast apply\n```\n\n\nTo see the registry information in the feast ui, run the following command. Note the --registry_ttl_sec is important as by default it polls every 5 seconds. \n\n```bash\nfeast ui --registry_ttl_sec=120\n```\n\n\n## Example Usage\n\nNow, lets batch read some features for training, using only entities (population) for which we have seen an event for in the last `60` days. The predicates (filter) used can be on anything that is relevant for the entity (population) selection for the given training dataset. The `event_timestamp` is only for example purposes.\n\n\n```python\nfrom feast import FeatureStore\n\n\nstore = FeatureStore(repo_path=\"feature_repo\")\n\ntraining_df = store.get_historical_features(\n entity_df=f\"\"\"\n SELECT\n driver_id,\n event_timestamp\n FROM demo_feast_driver_hourly_stats\n WHERE event_timestamp BETWEEN (CURRENT_TIMESTAMP - INTERVAL '60' DAY) AND CURRENT_TIMESTAMP\n \"\"\",\n features=[\n \"driver_hourly_stats:conv_rate\",\n \"driver_hourly_stats:acc_rate\",\n \"driver_hourly_stats:avg_daily_trips\"\n ],\n).to_df()\nprint(training_df.head())\n```\n\n\nThe `feast-teradata` library allows you to use the complete set of feast APIs and functionality. Please refer to the official [feast quickstart](https://docs.feast.dev/getting-started/quickstart) for more details on the various things you can do. \n\nAdditionally, if you want to see a complete (but not real-world), end-to-end example workflow example, see the `demo/test_workflow.py` script. This is used for testing the complete feast functionality.\n\n## Repo Configuration\n\nA feast repository is configured via the `feature_store.yaml`. There are 3 sections in this that can be configured to use Teradata \n\n- Registry\n- OfflineStore\n- OnlineStore\n\nTo configure Teradata as the `OnlineStore`, use the following configuration\n```yaml\nonline_store:\n type: feast_teradata.online.teradata.TeradataOnlineStore\n host: <host>\n database: <db>\n user: <user>\n password: <password>\n log_mech: <TDNEGO|LDAP|etc>\n```\n\nTo configure Teradata as the `OfflineStore`, use the following configuration\n```yaml\noffline_store:\n type: feast_teradata.offline.teradata.TeradataOfflineStore\n host: <host>\n database: <db>\n user: <user>\n password: <password>\n log_mech: <TDNEGO|LDAP|etc>\n```\n\nTo configure Teradata as the `Registry`, configure the `registry_type` as `sql` and the path as the sqlalchemy url for teradata as follows\n```yaml\nregistry:\n registry_type: sql\n path: teradatasql://<user>:<password>@<host>/?database=<database>&LOGMECH=<TDNEGO|LDAP|etc>\n cache_ttl_seconds: 120\n```\n\n## Release Notes\n\n### 1.0.4\n\n- Update: bump Feast dependency to 0.31.1 \n\n### 1.0.3\n\n- Fix: Added string mapping for columns. \n\n### 1.0.2\n\n- Doc: Improve README with details on repo configuration\n- Fix: Fix Github Release on CI Release\n- Fix: Updated path variable to become OS independent.\n\n### 1.0.1\n\n- Doc: Improve README with better getting started information. \n- Fix: Remove pytest from requirements.txt\n- Fix: Set minimum python version to 3.8 due to feast dependency on pandas>=1.4.3\n- Fix: Updated feast-td types conversion\n\n\n### 1.0.0\n\n- Feature: Initial implementation of feast-teradata library\n\n\n",
"bugtrack_url": null,
"license": "",
"summary": "",
"version": "1.0.4",
"project_urls": {
"Homepage": "https://github.com/Teradata/feast-teradata"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "932880067e87716e6739af37f7205266d816309637f4adcf1aaaf3f2638e06d8",
"md5": "a66af6f150904562bf8b95c64091fb9f",
"sha256": "c7e5d409c7b105f16f3bd767b66d0313f1a7b5f9524301bd0e86e921da1fd85d"
},
"downloads": -1,
"filename": "feast_teradata-1.0.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "a66af6f150904562bf8b95c64091fb9f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<3.11",
"size": 27648,
"upload_time": "2023-05-29T21:22:50",
"upload_time_iso_8601": "2023-05-29T21:22:50.518992Z",
"url": "https://files.pythonhosted.org/packages/93/28/80067e87716e6739af37f7205266d816309637f4adcf1aaaf3f2638e06d8/feast_teradata-1.0.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e29b80354c79909b8770f71f9c2671cff2d39dd61ed44d36bded26ea7553e901",
"md5": "93a6ebc84f8ba1de4958015a8881dbf9",
"sha256": "f5586cd43408feecf16721b726451a7953aef945cb35a00a7e5a1551e5853715"
},
"downloads": -1,
"filename": "feast-teradata-1.0.4.tar.gz",
"has_sig": false,
"md5_digest": "93a6ebc84f8ba1de4958015a8881dbf9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<3.11",
"size": 24894,
"upload_time": "2023-05-29T21:22:51",
"upload_time_iso_8601": "2023-05-29T21:22:51.886295Z",
"url": "https://files.pythonhosted.org/packages/e2/9b/80354c79909b8770f71f9c2671cff2d39dd61ed44d36bded26ea7553e901/feast-teradata-1.0.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-05-29 21:22:51",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Teradata",
"github_project": "feast-teradata",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "feast-teradata"
}
JSON
