## Teradata Python package for Advanced Analytics.
teradataml makes available to Python users a collection of analytic functions that reside on Teradata Vantage. This allows users to perform analytics on Teradata Vantage with no SQL coding. In addition, the teradataml library provides functions for scaling data manipulation and transformation, data filtering and sub-setting, and can be used in conjunction with other open-source python libraries.
For community support, please visit the [Teradata Community](https://support.teradata.com/community?id=community_forum&sys_id=14fe131e1bf7f304682ca8233a4bcb1d).
For Teradata customer support, please visit [Teradata Support](https://support.teradata.com/csm).
Copyright 2024, Teradata. All Rights Reserved.
### Table of Contents
* [Release Notes](#release-notes)
* [Installation and Requirements](#installation-and-requirements)
* [Using the Teradata Python Package](#using-the-teradata-python-package)
* [Documentation](#documentation)
* [License](#license)
## Release Notes:
#### teradataml 20.00.00.03
* teradataml no longer supports setting the `auth_token` using `set_config_params()`. Users should use `set_auth_token()` to set the token.
* ##### New Features/Functionality
* ###### teradataml: DataFrame
* New Function
* `alias()` - Creates a DataFrame with alias name.
* New Properties
* `db_object_name` - Get the underlying database object name, on which DataFrame is created.
* ###### teradataml: GeoDataFrame
* New Function
* `alias()` - Creates a GeoDataFrame with alias name.
* ###### teradataml: DataFrameColumn a.k.a. ColumnExpression
* _Arithmetic Functions_
* `DataFrameColumn.isnan()` - Function evaluates expression to determine if the floating-point
argument is a NaN (Not-a-Number) value.
* `DataFrameColumn.isinf()` - Function evaluates expression to determine if the floating-point
argument is an infinite number.
* `DataFrameColumn.isfinite()` - Function evaluates expression to determine if it is a finite
floating value.
* ###### FeatureStore - handles feature management within the Vantage environment
* FeatureStore Components
* Feature - Represents a feature which is used in ML Modeling.
* Entity - Represents the columns which serves as uniqueness for the data used in ML Modeling.
* DataSource - Represents the source of Data.
* FeatureGroup - Collection of Feature, Entity and DataSource.
* Methods
* `apply()` - Adds Feature, Entity, DataSource to a FeatureGroup.
* `from_DataFrame()` - Creates a FeatureGroup from teradataml DataFrame.
* `from_query()` - Creates a FeatureGroup using a SQL query.
* `remove()` - Removes Feature, Entity, or DataSource from a FeatureGroup.
* `reset_labels()` - Removes the labels assigned to the FeatureGroup, that are set using `set_labels()`.
* `set_labels()` - Sets the Features as labels for a FeatureGroup.
* Properties
* `features` - Get the features of a FeatureGroup.
* `labels` - Get the labels of FeatureGroup.
* FeatureStore
* Methods
* `apply()` - Adds Feature, Entity, DataSource, FeatureGroup to FeatureStore.
* `archive_data_source()` - Archives a specified DataSource from a FeatureStore.
* `archive_entity()` - Archives a specified Entity from a FeatureStore.
* `archive_feature()` - Archives a specified Feature from a FeatureStore.
* `archive_feature_group()` - Archives a specified FeatureGroup from a FeatureStore. Method archives underlying Feature, Entity, DataSource also.
* `delete_data_source()` - Deletes an archived DataSource.
* `delete_entity()` - Deletes an archived Entity.
* `delete_feature()` - Deletes an archived Feature.
* `delete_feature_group()` - Deletes an archived FeatureGroup.
* `get_data_source()` - Get the DataSources associated with FeatureStore.
* `get_dataset()` - Get the teradataml DataFrame based on Features, Entities and DataSource from FeatureGroup.
* `get_entity()` - Get the Entity associated with FeatureStore.
* `get_feature()` - Get the Feature associated with FeatureStore.
* `get_feature_group()` - Get the FeatureGroup associated with FeatureStore.
* `list_data_sources()` - List DataSources.
* `list_entities()` - List Entities.
* `list_feature_groups()` - List FeatureGroups.
* `list_features()` - List Features.
* `list_repos()` - List available repos which are configured for FeatureStore.
* `repair()` - Repairs the underlying FeatureStore schema on database.
* `set_features_active()` - Marks the Features as active.
* `set_features_inactive()` - Marks the Features as inactive.
* `setup()` - Setup the FeatureStore for a repo.
* Property
* `repo` - Property for FeatureStore repo.
* `grant` - Property to Grant access on FeatureStore to user.
* `revoke` - Property to Revoke access on FeatureStore from user.
* ###### teradataml: Table Operator Functions
* `Image2Matrix()` - Converts an image into a matrix.
* ###### teradataml: SQLE Engine Analytic Functions
* New Analytics Database Analytic Functions:
* `CFilter()`
* `NaiveBayes()`
* `TDNaiveBayesPredict()`
* `Shap()`
* `SMOTE()`
* ###### teradataml: Unbounded Array Framework (UAF) Functions
* New Unbounded Array Framework(UAF) Functions:
* `CopyArt()`
* ###### General functions
* Vantage File Management Functions
* `list_files()` - List the installed files in Database.
* ###### OpensourceML: LightGBM
* teradataml adds support for lightGBM package through `OpensourceML` (`OpenML`) feature.
The following functionality is added in the current release:
* `td_lightgbm` - Interface object to run lightgbm functions and classes through Teradata Vantage.
Example usage below:
```
from teradataml import td_lightgbm, DataFrame
df_train = DataFrame("multi_model_classification")
feature_columns = ["col1", "col2", "col3", "col4"]
label_columns = ["label"]
part_columns = ["partition_column_1", "partition_column_2"]
df_x = df_train.select(feature_columns)
df_y = df_train.select(label_columns)
# Dataset creation.
# Single model case.
obj_s = td_lightgbm.Dataset(df_x, df_y, silent=True, free_raw_data=False)
# Multi model case.
obj_m = td_lightgbm.Dataset(df_x, df_y, free_raw_data=False, partition_columns=part_columns)
obj_m_v = td_lightgbm.Dataset(df_x, df_y, free_raw_data=False, partition_columns=part_columns)
## Model training.
# Single model case.
opt = td_lightgbm.train(params={}, train_set = obj_s, num_boost_round=30)
opt.predict(data=df_x, num_iteration=20, pred_contrib=True)
# Multi model case.
opt = td_lightgbm.train(params={}, train_set = obj_m, num_boost_round=30,
callbacks=[td_lightgbm.record_evaluation(rec)],
valid_sets=[obj_m_v, obj_m_v])
# Passing `label` argument to get it returned in output DataFrame.
opt.predict(data=df_x, label=df_y, num_iteration=20)
```
* Added support for accessing scikit-learn APIs using exposed inteface object `td_lightgbm`.
Refer Teradata Python Package User Guide for more details of this feature, arguments, usage, examples and supportability in Vantage.
* ###### teradataml: Functions
* `register()` - Registers a user defined function (UDF).
* `call_udf()` - Calls a registered user defined function (UDF) and returns ColumnExpression.
* `list_udfs()` - List all the UDFs registered using 'register()' function.
* `deregister()` - Deregisters a user defined function (UDF).
* ###### teradataml: Options
* Configuration Options
* `table_operator` - Specifies the name of table operator.
* ##### Updates
* ###### General functions
* `set_auth_token()` - Added `base_url` parameter which accepts the CCP url.
'ues_url' will be deprecated in future and users
will need to specify 'base_url' instead.
* ###### teradataml: DataFrame function
* `join()`
* Now supports compound ColumExpression having more than one binary operator in `on` argument.
* Now supports ColumExpression containing FunctionExpression(s) in `on` argument.
* self-join now expects aliased DataFrame in `other` argument.
* ###### teradataml: GeoDataFrame function
* `join()`
* Now supports compound ColumExpression having more than one binary operator in `on` argument.
* Now supports ColumExpression containing FunctionExpression(s) in `on` argument.
* self-join now expects aliased DataFrame in `other` argument.
* ###### teradataml: Unbounded Array Framework (UAF) Functions
* `SAX()` - Default value added for `window_size` and `output_frequency`.
* `DickeyFuller()`
* Supports TDAnalyticResult as input.
* Default value added for `max_lags`.
* Removed parameter `drift_trend_formula`.
* Updated permitted values for `algorithm`.
* ##### teradataml: AutoML
* `AutoML`, `AutoRegressor` and `AutoClassifier`
* Now supports DECIMAL datatype as input.
* ##### teradataml: SQLE Engine Analytic Functions
* `TextParser()`
* Argument name `covert_to_lowercase` changed to `convert_to_lowercase`.
* ##### Bug Fixes
* `db_list_tables()` now returns correct results when '%' is used.
#### teradataml 20.00.00.02
* teradataml will no longer be supported with SQLAlchemy < 2.0.
* teradataml no longer shows the warnings from Vantage by default.
* Users should set `display.suppress_vantage_runtime_warnings` to `False` to display warnings.
* ##### New Features/Functionality
* ##### teradataml: SQLE Engine Analytic Functions
* New Analytics Database Analytic Functions:
* `TFIDF()`
* `Pivoting()`
* `UnPivoting()`
* New Unbounded Array Framework(UAF) Functions:
* `AutoArima()`
* `DWT()`
* `DWT2D()`
* `FilterFactory1d()`
* `IDWT()`
* `IDWT2D()`
* `IQR()`
* `Matrix2Image()`
* `SAX()`
* `WindowDFFT()`
* ###### teradataml: Functions
* `udf()` - Creates a user defined function (UDF) and returns ColumnExpression.
* `set_session_param()` is added to set the database session parameters.
* `unset_session_param()` is added to unset database session parameters.
* ###### teradataml: DataFrame
* `materialize()` - Persists DataFrame into database for current session.
* `create_temp_view()` - Creates a temporary view for session on the DataFrame.
* ###### teradataml DataFrameColumn a.k.a. ColumnExpression
* _Date Time Functions_
* `DataFrameColumn.to_timestamp()` - Converts string or integer value to a TIMESTAMP data type or TIMESTAMP WITH TIME ZONE data type.
* `DataFrameColumn.extract()` - Extracts date component to a numeric value.
* `DataFrameColumn.to_interval()` - Converts a numeric value or string value into an INTERVAL_DAY_TO_SECOND or INTERVAL_YEAR_TO_MONTH value.
* _String Functions_
* `DataFrameColumn.parse_url()` - Extracts a part from a URL.
* _Arithmetic Functions_
* `DataFrameColumn.log` - Returns the logarithm value of the column with respect to 'base'.
* ##### teradataml: AutoML
* New methods added for `AutoML()`, `AutoRegressor()` and `AutoClassifier()`:
* `evaluate()` - Performs evaluation on the data using the best model or the model of users choice
from the leaderboard.
* `load()`: Loads the saved model from database.
* `deploy()`: Saves the trained model inside database.
* `remove_saved_model()`: Removes the saved model in database.
* `model_hyperparameters()`: Returns the hyperparameter of fitted or loaded models.
* ##### Updates
* ##### teradataml: AutoML
* `AutoML()`, `AutoRegressor()`
* New performance metrics added for task type regression i.e., "MAPE", "MPE", "ME", "EV", "MPD" and "MGD".
* `AutoML()`, `AutoRegressor()` and `AutoClassifier`
* New arguments added: `volatile`, `persist`.
* `predict()` - Data input is now mandatory for generating predictions. Default model
evaluation is now removed.
* `DataFrameColumn.cast()`: Accepts 2 new arguments `format` and `timezone`.
* `DataFrame.assign()`: Accepts ColumnExpressions returned by `udf()`.
* ##### teradataml: Options
* `set_config_params()`
* Following arguments will be deprecated in the future:
* `ues_url`
* `auth_token`
* #### teradata DataFrame
* `to_pandas()` - Function returns the pandas dataframe with Decimal columns types as float instead of object.
If user want datatype to be object, set argument `coerce_float` to False.
* ###### Database Utility
* `list_td_reserved_keywords()` - Accepts a list of strings as argument.
* ##### Updates to existing UAF Functions:
* `ACF()` - `round_results` parameter removed as it was used for internal testing.
* `BreuschGodfrey()` - Added default_value 0.05 for parameter `significance_level`.
* `GoldfeldQuandt()` -
* Removed parameters `weights` and `formula`.
Replaced parameter `orig_regr_paramcnt` with `const_term`.
Changed description for parameter `algorithm`. Please refer document for more details.
* Note: This will break backward compatibility.
* `HoltWintersForecaster()` - Default value of parameter `seasonal_periods` removed.
* `IDFFT2()` - Removed parameter `output_fmt_row_major` as it is used for internal testing.
* `Resample()` - Added parameter `output_fmt_index_style`.
* ##### Bug Fixes
* KNN `predict()` function can now predict on test data which does not contain target column.
* Metrics functions are supported on the Lake system.
* The following OpensourceML functions from different sklearn modules in single model case are fixed.
* `sklearn.ensemble`:
* ExtraTreesClassifier - `apply()`
* ExtraTreesRegressor - `apply()`
* RandomForestClassifier - `apply()`
* RandomForestRegressor - `apply()`
* `sklearn.impute`:
* SimpleImputer - `transform()`, `fit_transform()`, `inverse_transform()`
* MissingIndicator - `transform()`, `fit_transform()`
* `sklearn.kernel_approximations`:
* Nystroem - `transform()`, `fit_transform()`
* PolynomialCountSketch - `transform()`, `fit_transform()`
* RBFSampler - `transform()`, `fit_transform()`
* `sklearn.neighbors`:
* KNeighborsTransformer - `transform()`, `fit_transform()`
* RadiusNeighborsTransformer - `transform()`, `fit_transform()`
* `sklearn.preprocessing`:
* KernelCenterer - `transform()`
* OneHotEncoder - `transform()`, `inverse_transform()`
* The following OpensourceML functions from different sklearn modules in multi model case are fixed.
* `sklearn.feature_selection`:
* SelectFpr - `transform()`, `fit_transform()`, `inverse_transform()`
* SelectFdr - `transform()`, `fit_transform()`, `inverse_transform()`
* SelectFromModel - `transform()`, `fit_transform()`, `inverse_transform()`
* SelectFwe - `transform()`, `fit_transform()`, `inverse_transform()`
* RFECV - `transform()`, `fit_transform()`, `inverse_transform()`
* `sklearn.clustering`:
* Birch - `transform()`, `fit_transform()`
* OpensourceML returns teradataml objects for model attributes and functions instead of sklearn
objects so that the user can perform further operations like `score()`, `predict()` etc on top
of the returned objects.
* AutoML `predict()` function now generates correct ROC-AUC value for positive class.
* `deploy()` method of `Script` and `Apply` classes retries model deployment if there is any
intermittent network issues.
#### teradataml 20.00.00.01
* teradataml no longer supports Python versions less than 3.8.
* ##### New Features/Functionality
* ##### Personal Access Token (PAT) support in teradataml
* `set_auth_token()` - teradataml now supports authentication via PAT in addition to
OAuth 2.0 Device Authorization Grant (formerly known as the Device Flow).
* It accepts UES URL, Personal AccessToken (PAT) and Private Key file generated from VantageCloud Lake Console
and optional argument `username` and `expiration_time` in seconds.
* ##### Updates
* ##### teradataml: SQLE Engine Analytic Functions
* `ANOVA()`
* New arguments added: `group_name_column`, `group_value_name`, `group_names`, `num_groups` for data containing group values and group names.
* `FTest()`
* New arguments added: `sample_name_column`, `sample_name_value`, `first_sample_name`, `second_sample_name`.
* `GLM()`
* Supports stepwise regression and accept new arguments `stepwise_direction`, `max_steps_num` and `initial_stepwise_columns`.
* New arguments added: `attribute_data`, `parameter_data`, `iteration_mode` and `partition_column`.
* `GetFutileColumns()`
* Arguments `category_summary_column` and `threshold_value` are now optional.
* `KMeans()`
* New argument added: `initialcentroids_method`.
* `NonLinearCombineFit()`
* Argument `result_column` is now optional.
* `ROC()`
* Argument `positive_class` is now optional.
* `SVMPredict()`
* New argument added: `model_type`.
* `ScaleFit()`
* New arguments added: `ignoreinvalid_locationscale`, `unused_attributes`, `attribute_name_column`, `attribute_value_column`.
* Arguments `attribute_name_column`, `attribute_value_column` and `target_attributes` are supported for sparse input.
* Arguments `attribute_data`, `parameter_data` and `partition_column` are supported for partitioning.
* `ScaleTransform()`
* New arguments added: `attribute_name_column` and `attribute_value_column` support for sparse input.
* `TDGLMPredict()`
* New arguments added: `family` and `partition_column`.
* `XGBoost()`
* New argument `base_score` is added for initial prediction value for all data points.
* `XGBoostPredict()`
* New argument `detailed` is added for detailed information of each prediction.
* `ZTest()`
* New arguments added: `sample_name_column`, `sample_value_column`, `first_sample_name` and `second_sample_name`.
* ##### teradataml: AutoML
* `AutoML()`, `AutoRegressor()` and `AutoClassifier()`
* New argument `max_models` is added as an early stopping criterion to limit the maximum number of models to be trained.
* ##### teradataml: DataFrame functions
* `DataFrame.agg()`
* Accepts ColumnExpressions and list of ColumnExpressions as arguments.
* ##### teradataml: General Functions
* Data Transfer Utility
* `fastload()` - Improved error and warning table handling with below-mentioned new arguments.
* `err_staging_db`
* `err_tbl_name`
* `warn_tbl_name`
* `err_tbl_1_suffix`
* `err_tbl_2_suffix`
* `fastload()` - Change in behaviour of `save_errors` argument.
When `save_errors` is set to `True`, error information will be available in two persistent tables `ERR_1` and `ERR_2`.
When `save_errors` is set to `False`, error information will be available in single pandas dataframe.
* Garbage collector location is now configurable.
User can set configure.local_storage to a desired location.
* ##### Bug Fixes
* UAF functions now work if the database name has special characters.
* OpensourceML can now read and process NULL/nan values.
* Boolean values output will now be returned as VARBYTE column with 0 or 1 values in OpensourceML.
* Fixed bug for `Apply`'s `deploy()`.
* Issue with volatile table creation is fixed where it is created in the right database, i.e., user's spool space, regardless of the temp database specified.
* `ColumnTransformer` function now processes its arguments in the order they are passed.
#### teradataml 20.00.00.00
* ##### New Features/Functionality
* ###### teradataml OpenML: Run Opensource packages through Teradata Vantage
`OpenML` dynamically exposes opensource packages through Teradata Vantage. `OpenML` provides an
interface object through which exposed classes and functions of opensource packages can be accessed
with the same syntax and arguments.
The following functionality is added in the current release:
* `td_sklearn` - Interface object to run scikit-learn functions and classes through Teradata Vantage.
Example usage below:
```
from teradataml import td_sklearn, DataFrame
df_train = DataFrame("multi_model_classification")
feature_columns = ["col1", "col2", "col3", "col4"]
label_columns = ["label"]
part_columns = ["partition_column_1", "partition_column_2"]
linear_svc = td_sklearn.LinearSVC()
```
* `OpenML` is supported in both Teradata Vantage Enterprise and Teradata Vantage Lake.
* Argument Support:
* `Use of X and y arguments` - Scikit-learn users are familiar with using `X` and `y` as argument names
which take data as pandas DataFrames, numpy arrays or lists etc. However, in OpenML, we pass
teradataml DataFrames for arguments `X` and `y`.
```
df_x = df_train.select(feature_columns)
df_y = df_train.select(label_columns)
linear_svc = linear_svc.fit(X=df_x, y=df_y)
```
* `Additional support for data, feature_columns, label_columns and group_columns arguments` -
Apart from traditional arguments, OpenML supports additional arguments - `data`,
`feature_columns`, `label_columns` and `group_columns`. These are used as alternatives to `X`, `y`
and `groups`.
```
linear_svc = linear_svc.fit(data=df_train, feature_columns=feature_columns, label_colums=label_columns)
```
* `Support for classification and regression metrics` - Metrics functions for classification and
regression in `sklearn.metrics` module are supported. Other metrics functions' support will be added
in future releases.
* `Distributed Modeling and partition_columns argument support` - Existing scikit-learn supports
only single model generation. However, OpenML supports both single model use case and distributed
(multi) model use case. For this, user has to additionally pass `partition_columns` argument to
existing `fit()`, `predict()` or any other function to be run. This will generate multiple models
for multiple partitions, using the data in corresponding partition.
```
df_x_1 = df_train.select(feature_columns + part_columns)
linear_svc = linear_svc.fit(X=df_x_1, y=df_y, partition_columns=part_columns)
```
* `Support for load and deploy models` - OpenML provides additional support for saving (deploying) the
trained models. These models can be loaded later to perform operations like prediction, score etc. The
following functions are provided by OpenML:
* `<obj>.deploy()` - Used to deploy/save the model created and/or trained by OpenML.
* `td_sklearn.deploy()` - Used to deploy/save the model created and/or trained outside teradataml.
* `td_sklearn.load()` - Used to load the saved models.
<br>Refer Teradata Python Package User Guide for more details of this feature, arguments, usage, examples and supportability in both VantageCloud Enterprise and VantageCloud Lake.
* ###### teradataml: AutoML - Automated end to end Machine Learning flow.
AutoML is an approach to automate the process of building, training, and validating machine learning models.
It involves automation of various aspects of the machine learning workflow, such as feature exploration,
feature engineering, data preparation, model training and evaluation for given dataset.
teradataml AutoML feature offers best model identification, model leaderboard generation, parallel execution,
early stopping feature, model evaluation, model prediction, live logging, customization on default process.
* `AutoML`
AutoML is a generic algorithm that supports all three tasks, i.e. 'Regression',
'Binary Classification' and 'Multiclass Classification'.
* Methods of AutoML
* `__init__()` - Instantiate an object of AutoML with given parameters.
* `fit()` - Perform fit on specified data and target column.
* `leaderboard()` - Get the leaderboard for the AutoML. Presents diverse models, feature
selection method, and performance metrics.
* `leader()` - Show best performing model and its details such as feature
selection method, and performance metrics.
* `predict()` - Perform prediction on the data using the best model or the model of users
choice from the leaderboard.
* `generate_custom_config()` - Generate custom config JSON file required for customized
run of AutoML.
* `AutoRegressor`
AutoRegressor is a special purpose AutoML feature to run regression specific tasks.
* Methods of AutoRegressor
* `__init__()` - Instantiate an object of AutoRegressor with given parameters.
* `fit()` - Perform fit on specified data and target column.
* `leaderboard()` - Get the leaderboard for the AutoRegressor. Presents diverse models, feature
selection method, and performance metrics.
* `leader()` - Show best performing model and its details such as feature
selection method, and performance metrics.
* `predict()` - Perform prediction on the data using the best model or the model of users
choice from the leaderboard.
* `generate_custom_config()` - Generate custom config JSON file required for customized
run of AutoRegressor.
* `AutoClassifier`
AutoClassifier is a special purpose AutoML feature to run classification specific tasks.
* Methods of AutoClassifier
* `__init__()` - Instantiate an object of AutoClassifier with given parameters.
* `fit()` - Perform fit on specified data and target column.
* `leaderboard()` - Get the leaderboard for the AutoClassifier. Presents diverse models, feature
selection method, and performance metrics.
* `leader()` - Show best performing model and its details such as feature
selection method, and performance metrics.
* `predict()` - Perform prediction on the data using the best model or the model of users
choice from the leaderboard.
* `generate_custom_config()` - Generate custom config JSON file required for customized
run of AutoClassifier.
* ###### teradataml: DataFrame
* `fillna` - Replace the null values in a column with the value specified.
* Data Manipulation
* `cube()`- Analyzes data by grouping it into multiple dimensions.
* `rollup()` - Analyzes a set of data across a single dimension with more than one level of detail.
* `replace()` - Replaces the values for columns.
* ###### teradataml: Script and Apply
* `deploy()` - Function deploys the model, generated after `execute_script()`, in database or user
environment in lake. The function is available in both Script and Apply.
* ###### teradataml: DataFrameColumn
* `fillna` - Replaces every occurrence of null value in column with the value specified.
* ###### teradataml DataFrameColumn a.k.a. ColumnExpression
* _Date Time Functions_
* `DataFrameColumn.week_start()` - Returns the first date or timestamp of the week that begins immediately before the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.week_begin()` - It is an alias for `DataFrameColumn.week_start()` function.
* `DataFrameColumn.week_end()` - Returns the last date or timestamp of the week that ends immediately after the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.month_start()` - Returns the first date or timestamp of the month that begins immediately before the specified date or timestamp value in a column or as a literal.
* `DataFrameColumn.month_begin()` - It is an alias for `DataFrameColumn.month_start()` function.
* `DataFrameColumn.month_end()` - Returns the last date or timestamp of the month that ends immediately after the specified date or timestamp value in a column or as a literal.
* `DataFrameColumn.year_start()` - Returns the first date or timestamp of the year that begins immediately before the specified date or timestamp value in a column or as a literal.
* `DataFrameColumn.year_begin()` - It is an alias for `DataFrameColumn.year_start()` function.
* `DataFrameColumn.year_end()` - Returns the last date or timestamp of the year that ends immediately after the specified date or timestamp value in a column or as a literal.
* `DataFrameColumn.quarter_start()` - Returns the first date or timestamp of the quarter that begins immediately before the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.quarter_begin()` - It is an alias for `DataFrameColumn.quarter_start()` function.
* `DataFrameColumn.quarter_end()` - Returns the last date or timestamp of the quarter that ends immediately after the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.last_sunday()` - Returns the date or timestamp of Sunday that falls immediately before the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.last_monday()` - Returns the date or timestamp of Monday that falls immediately before the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.last_tuesday()` - Returns the date or timestamp of Tuesday that falls immediately before the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.last_wednesday()` - Returns the date or timestamp of Wednesday that falls immediately before specified date or timestamp value in a column as a literal.
* `DataFrameColumn.last_thursday()`- Returns the date or timestamp of Thursday that falls immediately before specified date or timestamp value in a column as a literal.
* `DataFrameColumn.last_friday()` - Returns the date or timestamp of Friday that falls immediately before specified date or timestamp value in a column as a literal.
* `DataFrameColumn.last_saturday()` - Returns the date or timestamp of Saturday that falls immediately before specified date or timestamp value in a column as a literal.
* `DataFrameColumn.day_of_week()` - Returns the number of days from the beginning of the week to the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.day_of_month()` - Returns the number of days from the beginning of the month to the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.day_of_year()` - Returns the number of days from the beginning of the year to the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.day_of_calendar()` - Returns the number of days from the beginning of the business calendar to the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.week_of_month()` - Returns the number of weeks from the beginning of the month to the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.week_of_quarter()` - Returns the number of weeks from the beginning of the quarter to the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.week_of_year()` - Returns the number of weeks from the beginning of the year to the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.week_of_calendar()` - Returns the number of weeks from the beginning of the calendar to the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.month_of_year()` - Returns the number of months from the beginning of the year to the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.month_of_calendar()` - Returns the number of months from the beginning of the calendar to the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.month_of_quarter()` - Returns the number of months from the beginning of the quarter to the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.quarter_of_year()` - Returns the number of quarters from the beginning of the year to the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.quarter_of_calendar()` - Returns the number of quarters from the beginning of the calendar to the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.year_of_calendar()` - Returns the year of the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.day_occurrence_of_month()` - Returns the nth occurrence of the weekday in the month for the date to the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.year()` - Returns the integer value for year in the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.month()` - Returns the integer value for month in the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.hour()` - Returns the integer value for hour in the specified timestamp value in a column as a literal.
* `DataFrameColumn.minute()` - Returns the integer value for minute in the specified timestamp value in a column as a literal.
* `DataFrameColumn.second()` - Returns the integer value for seconds in the specified timestamp value in a column as a literal.
* `DataFrameColumn.week()` - Returns the number of weeks from the beginning of the year to the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.next_day()` - Returns the date of the first weekday specified as 'day_value' that is later than the specified date or timestamp value in a column as a literal.
* `DataFrameColumn.months_between()` - Returns the number of months between value in specified date or timestamp value in a column as a literal and date or timestamp value in argument.
* `DataFrameColumn.add_months()` - Adds an integer number of months to specified date or timestamp value in a column as a literal.
* `DataFrameColumn.oadd_months()` - Adds an integer number of months, date or timestamp value in specified date or timestamp value in a column as a literal.
* `DataFrameColumn.to_date()` - Function converts a string-like representation of a DATE or PERIOD type to Date type.
* _String Functions_
* `DataFrameColumn.concat()` - Function to concatenate the columns with a separator.
* `DataFrameColumn.like()` - Function to match the string pattern. String match is case sensitive.
* `DataFrameColumn.ilike()` - Function to match the string pattern. String match is not case sensitive.
* `DataFrameColumn.substr()` - Returns the substring from a string column.
* `DataFrameColumn.startswith()` - Function to check if the column value starts with the specified value or not.
* `DataFrameColumn.endswith()` - Function to check if the column value ends with the specified value or not.
* `DataFrameColumn.format()` - Function to format the values in column based on formatter.
* `DataFrameColumn.to_char()` - Function converts numeric type or datetype to character type.
* `DataFrameColumn.trim()` - Function trims the string values in the column.
* _Regular Arithmetic Functions_
* `DataFrameColumn.cbrt()` - Computes the cube root of values in the column.
* `DataFrameColumn.hex()` - Computes the Hexadecimal from decimal for the values in the column.
* `DataframeColumn.hypot()` - Computes the decimal from Hexadecimal for the values in the column.
* `DataFrameColumn.unhex()` - computes the hypotenuse for the values between two columns.
* _Bit Byte Manipulation Functions_
* `DataFrameColumn.from_byte()` - Encodes a sequence of bits into a sequence of characters.
* _Comparison Functions_
* `DataFrameColumn.greatest()` - Returns the greatest values from columns.
* `DataFrameColumn.least()` - Returns the least values from columns.
* Behaviour of `DataFrameColumn.replace()` is changed.
* Behaviour of `DataFrameColumn.to_byte()` is changed. It now decodes a sequence of characters in a given encoding into a sequence of bits.
* Behaviour of `DataFrameColumn.trunc()` is changed. It now accepts Date type columns.
* ##### Bug Fixes
* Argument `url_encode` is no longer used in `create_context()` and is deprecated.
* **Important notes**
* Users do not need to encode password even if password contain special characters.
* Pass the password to the `create_context()` function argument `password` as it is without changing special characters.
* `fillna()` in VAL transformation allows to replace NULL values with empty string.
* ##### Updates
* Support for following deprecated functionality is removed:
* ML Engine functions
* STO and APPLY sandbox feature support for testing the script.
* sandbox_container_utils is removed. Following methods can no longer be used:
* `setup_sandbox_env()`
* `copy_files_from_container()`
* `cleanup_sandbox_env()`
* Model Cataloging APIs can no longer be used:
* `describe_model()`
* `delete_model()`
* `list_models()`
* `publish_model()`
* `retrieve_model()`
* `save_model()`
* `DataFrame.join()`
* Arguments `lsuffix` and `rsuffix` now add suffixes to new column names for join operation.
* `DataFrame.describe()`
* New argument `columns` is added to generate statistics on only those columns instead of all applicable columns.
* `DataFrame.groupby()`
* Supports `CUBE` and `ROLLUP` with additional optional argument `option`.
* `DataFrame.column.window()`
* Supports ColumnExpressions for `partition_columns` and `order_columns` arguments.
* `DataFrame.column.contains()` allows ColumnExpressions for `pattern` argument.
* `DataFrame.window()`
* Supports ColumnExpressions for `partition_columns` and `order_columns` arguments.
#### teradataml 17.20.00.07
* ##### New Features/Functionality
* ###### Open Analytics Framework (OpenAF) APIs:
* Manage all user environments.
* `create_env()`:
* new argument `conda_env` is added to create a conda environment.
* `list_user_envs()`:
* User can list conda environment(s) by using filter with new argument `conda_env`.
* Conda environment(s) can be managed using APIs for installing , updating, removing files/libraries.
* ##### Bug Fixes
* `columns` argument for `FillNa` function is made optional.
#### teradataml 17.20.00.06
* ##### New Features/Functionality
* ###### teradataml DataFrameColumn a.k.a. ColumnExpression
* `ColumnExpression.nulls_first()` - Displays NULL values at first.
* `ColumnExpression.nulls_last()` - Displays NULL values at last.
* _Bit Byte Manipulation Functions_
* `DataFrameColumn.bit_and()` - Returns the logical AND operation on the bits from
the column and corresponding bits from the argument.
* `DataFrameColumn.bit_get()` - Returns the bit specified by input argument from the column and
returns either 0 or 1 to indicate the value of that bit.
* `DataFrameColumn.bit_or()` - Returns the logical OR operation on the bits from the column and
corresponding bits from the argument.
* `DataFrameColumn.bit_xor()` - Returns the bitwise XOR operation on the binary representation of the
column and corresponding bits from the argument.
* `DataFrameColumn.bitand()` - It is an alias for `DataFrameColumn.bit_and()` function.
* `DataFrameColumn.bitnot()` - Returns a bitwise complement on the binary representation of the column.
* `DataFrameColumn.bitor()` - It is an alias for `DataFrameColumn.bit_or()` function.
* `DataFrameColumn.bitwise_not()` - It is an alias for `DataFrameColumn.bitnot()` function.
* `DataFrameColumn.bitwiseNOT()` - It is an alias for `DataFrameColumn.bitnot()` function.
* `DataFrameColumn.bitxor()` - It is an alias for `DataFrameColumn.bit_xor()` function.
* `DataFrameColumn.countset()` - Returns the count of the binary bits within the column that are either set to 1
or set to 0, depending on the input argument value.
* `DataFrameColumn.getbit()` - It is an alias for `DataFrameColumn.bit_get()` function.
* `DataFrameColumn.rotateleft()` - Returns an expression rotated to the left by the specified number of bits,
with the most significant bits wrapping around to the right.
* `DataFrameColumn.rotateright()` - Returns an expression rotated to the right by the specified number of bits,
with the least significant bits wrapping around to the left.
* `DataFrameColumn.setbit()` - Sets the value of the bit specified by input argument to the value
of column.
* `DataFrameColumn.shiftleft()` - Returns the expression when value in column is shifted by the specified
number of bits to the left.
* `DataFrameColumn.shiftright()` - Returns the expression when column expression is shifted by the specified
number of bits to the right.
* `DataFrameColumn.subbitstr()` - Extracts a bit substring from the column expression based on the specified
bit position.
* `DataFrameColumn.to_byte()` - Converts a numeric data type to the Vantage byte representation
(byte value) of the column expression value.
* _Regular Expression Functions_
* `DataFrameColumn.regexp_instr()` - Searches string value in column for a match to value specified in argument.
* `DataFrameColumn.regexp_replace()` - Replaces the portions of string value in a column that matches the value
specified regex string and replaces with the replace string.
* `DataFrameColumn.regexp_similar()` - Compares value in column to value in argument and returns integer value.
* `DataFrameColumn.regexp_substr()` - Extracts a substring from column that matches a regular expression
specified in the input argument.
* ###### Open Analytics Framework (OpenAF) APIs:
* Manage all user environments.
* `create_env()`:
* User can create one or more user environments using newly added argument `template` by providing specifications in template json file. New feature allows user to create complete user environment, including file and library installation, in just single function call.
* UserEnv Class – Manage individual user environment.
* Properties:
* `models` - Supports listing of models in user environment.
* Methods:
* `install_model()` - Install a model in user environment.
* `uninstall_model()` - Uninstall a model from user environment.
* `snapshot()`- Take the snapshot of the user environment.
* ###### teradataml: Bring Your Own Model
* _New Functions_
* `DataRobotPredict()` - Score the data in Vantage using the model trained externally in datarobot and stored
in Vantage.
* ##### Updates
* `DataFrame.describe()`
* Method now accepts an argument `statistics`, which specifies the aggregate operation to be performed.
* `DataFrame.sort()`
* Method now accepts ColumnExpressions as well.
* Enables sorting using NULLS FIRST and NULLS LAST.
* `view_log()` downloads the Apply query logs based on query id.
* Arguments which accepts floating numbers will accept integers also for `Analytics Database Analytic Functions`.
* Argument `ignore_nulls` added to `DataFrame.plot()` to ignore the null values while plotting the data.
* `Dataframe.sample()`
* Method supports column stratification.
* ##### Bug Fixes
* `DataFrameColumn.cast()` accepts all teradatasqlalchemy types.
* Minor bug fix related to `DataFrame.merge()`.
#### teradataml 17.20.00.05
* ##### New Features/Functionality
* ###### teradataml: Hyperparameter-Tuning - Technique to identify best model parameters.
Hyperparameter tuning is an optimization method to determine the optimal set of
hyperparameters for the given dataset and learning model. teradataml hyperparameter tuning feature
offers best model identification, parallel execution, early stopping feature, best data identification,
model evaluation, model prediction, live logging, input data hyper-parameterization, input data sampling,
numerous scoring functions, hyper-parameterization for non-model trainer functions.
* `GridSearch`
GridSearch is an exhaustive search algorithm that covers all possible
parameter values to identify optimal hyperparameters.
* Methods of GridSearch
* `__init__()` - Instantiate an object of GridSearch for given model function and parameters.
* `evaluate()` - Function to perform evaluation on the given teradataml DataFrame using default model.
* `fit()` - Function to perform hyperparameter-tuning for given hyperparameters and model on teradataml DataFrame.
* `get_error_log()` - Useful to get the error log if model execution failed, using the model identifier.
* `get_input_data()` - Useful to get the input data using the data identifier, when input data is also parameterized.
* `get_model()` - Returns the trained model for the given model identifier.
* `get_parameter_grid()` - Returns the hyperparameter space used for hyperparameter optimization.
* `is_running()` - Returns the execution status of hyperaparameter tuning.
* `predict()` - Function to perform prediction on the given teradataml DataFrame using default model.
* `set_model()` - Function to update the default model.
* Properties of GridSearch
* `best_data_id` - Returns the best data identifier used for model training.
* `best_model` - Returns the best trained model.
* `best_model_id` - Returns the identifier for best model.
* `best_params_` - Returns the best set of hyperparameter.
* `best_sampled_data_` - Returns the best sampled data used to train the best model.
* `best_score_` - Returns the best trained model score.
* `model_stats` - Returns the model evaluation reports.
* `models` - Returns the metadata of all the models.
* `RandomSearch`
RandomSearch algorithm performs random sampling on hyperparameter
space to identify optimal hyperparameters.
* Methods of RandomSearch
* `__init__()` - Instantiate an object of RandomSearch for given model function and parameters.
* `evaluate()` - Function to perform evaluation on the given teradataml DataFrame using default model.
* `fit()` - Function to perform hyperparameter-tuning for given hyperparameters and model on teradataml DataFrame.
* `get_error_log()` - Useful to get the error log if model execution failed, using the model identifier.
* `get_input_data()` - Useful to get the input data using the data identifier, when input data is also parameterized.
* `get_model()` - Returns the trained model for the given model identifier.
* `get_parameter_grid()` - Returns the hyperparameter space used for hyperparameter optimization.
* `is_running()` - Returns the execution status of hyperaparameter tuning.
* `predict()` - Function to perform prediction on the given teradataml DataFrame using default model.
* `set_model()` - Function to update the default model.
* Properties of GridSearch
* `best_data_id` - Returns the best data identifier used for model training.
* `best_model` - Returns the best trained model.
* `best_model_id` - Returns the identifier for best model.
* `best_params_` - Returns the best set of hyperparameter.
* `best_sampled_data_` - Returns the best sampled data used to train the best model.
* `best_score_` - Returns the best trained model score.
* `model_stats` - Returns the model evaluation reports.
* `models` - Returns the metadata of all the models.
* ###### teradataml: Analytic Functions
teradataml currently has different functions to generate a model, predict, transform and evaluate. All these functions are needed to be invoked individually, i.e., predict(), evaluate(), transform() cannot be invoked using the model trainer function output. Enhancement done to this feature now enables user to invoke these functions as methods of the model trainer function. Below is the list of functions, updated with this enhancement:
* Analytics Database Analytic Functions
* `BincodeFit()` - Supports `transform()` method.
* `DecisionForest()` - Supports `predict()`, `evaluate()` methods.
* `Fit()` - Supports `transform()` method.
* `GLM()` - Supports `predict()`, `evaluate()` methods.
* `GLMPerSegment()` - Supports `predict()`, `evaluate()` methods.
* `KMeans()` - Supports `predict()` method.
* `KNN()` - Supports `predict()`, `evaluate()` methods.
* `NaiveBayesTextClassifierTrainer()` - Supports `predict()`, `evaluate()` methods.
* `NonLinearCombineFit()` - Supports `transform()` method.
* `OneClassSVM()` - Supports `predict()` method.
* `OneHotEncodingFit()` - Supports `transform()` method.
* `OrdinalEncodingFit()` - Supports `transform()` method.
* `OutlierFilterFit()` - Supports `transform()` method.
* `PolynomialFeaturesFit()` - Supports `transform()` method.
* `RandomProjectionFit()` - Supports `transform()` method.
* `RowNormalizeFit()` - Supports `transform()` method.
* `ScaleFit()` - Supports `transform()` method.
* `SimpleImputeFit()` - Supports `transform()` method.
* `SVM()` - Supports `predict()`, `evaluate()` methods.
* `TargetEncodingFit()` - Supports `transform()` method.
* `XGBoost()` - Supports `predict()`, `evaluate()` methods.
* Time Series Analytic (UAF) Functions
* `ArimaEstimate()` - Supports `forecast()`, `validate()` methods.
* `DFFT()` - Supports `convolve()`, `inverse()` methods.
* `IDFFT()` - Supports `inverse()` method.
* `DFFT2()` - Supports `convolve()`, `inverse()` methods.
* `IDFFT2()` - Supports `inverse()` method.
* `DIFF()` - Supports `inverse()` method.
* `UNDIFF()` - Supports `inverse()` method.
* `SeasonalNormalize()` - Supports `inverse()` method.
* ###### teradataml: DataFrame
* New Functions
* `DataFrame.plot()` - Generates the below type of plots on teradataml DataFrame.
* line - Generates line plot.
* bar - Generates bar plot.
* scatter - Generates scatter plot.
* corr - Generates correlation plot.
* wiggle - Generates a wiggle plot.
* mesh - Generates a mesh plot.
* `DataFrame.itertuples()` - iterate over teradataml DataFrame rows as namedtuples or list.
* ###### teradataml: GeoDataFrame
* New Functions
* `GeoDataFrame.plot()` - Generate the below type of plots on teradataml GeoDataFrame.
* line - Generates line plot.
* bar - Generates bar plot.
* scatter - Generates scatter plot.
* corr - Generates correlation plot.
* wiggle - Generates a wiggle plot.
* mesh - Generates a mesh plot.
* geometry - Generates plot on geospatial data.
* Plot:
* `Axis` - Genertes the axis for plot.
* `Figure` - Generates the figure for plot.
* `subplots` - Helps in generating multiple plots on a single `Figure`.
* Bring Your Own Model (BYOM) Function:
* `DataikuPredict` - Score the data in Vantage using the model trained externally in Dataiku UI and stored in Vantage.
* `async_run_status()` - Function to check the status of asynchronous run(s) using unique run id(s).
* ###### teradataml DataFrameColumn a.k.a. ColumnExpression
* _Regular Arithmetic Functions_
* `DataFrameColumn.abs()` - Computes the absolute value.
* `DataFrameColumn.ceil()` - Returns the ceiling value of the column.
* `DataFrameColumn.ceiling()` - It is an alias for `DataFrameColumn.ceil()` function.
* `DataFrameColumn.degrees()` - Converts radians value from the column to degrees.
* `DataFrameColumn.exp()` - Raises e (the base of natural logarithms) to the power of the value in the column, where e = 2.71828182845905.
* `DataFrameColumn.floor()` - Returns the largest integer equal to or less than the value in the column.
* `DataFrameColumn.ln()` - Computes the natural logarithm of values in column.
* `DataFrameColumn.log10()` - Computes the base 10 logarithm.
* `DataFrameColumn.mod()` - Returns the modulus of the column.
* `DataFrameColumn.pmod()` - It is an alias for `DataFrameColumn.mod()` function.
* `DataFrameColumn.nullifzero()` - Converts data from zero to null to avoid problems with division by zero.
* `DataFrameColumn.pow()` - Computes the power of the column raised to expression or constant.
* `DataFrameColumn.power()` - It is an alias for `DataFrameColumn.pow()` function.
* `DataFrameColumn.radians()` - Converts degree value from the column to radians.
* `DataFrameColumn.round()` - Returns the rounded off value.
* `DataFrameColumn.sign()` - Returns the sign.
* `DataFrameColumn.signum()` - It is an alias for `DataFrameColumn.sign()` function.
* `DataFrameColumn.sqrt()` - Computes the square root of values in the column.
* `DataFrameColumn.trunc()` - Provides the truncated value of columns.
* `DataFrameColumn.width_bucket()` - Returns the number of the partition to which column is assigned.
* `DataFrameColumn.zeroifnull()` - Converts data from null to zero to avoid problems with null.
* _Trigonometric Functions_
* `DataFrameColumn.acos()` - Returns the arc-cosine value.
* `DataFrameColumn.asin()` - Returns the arc-sine value.
* `DataFrameColumn.atan()` - Returns the arc-tangent value.
* `DataFrameColumn.atan2()` - Returns the arc-tangent value based on x and y coordinates.
* `DataFrameColumn.cos()` - Returns the cosine value.
* `DataFrameColumn.sin()` - Returns the sine value.
* `DataFrameColumn.tan()` - Returns the tangent value.
* _Hyperbolic Functions_
* `DataFrameColumn.acosh()` - Returns the inverse hyperbolic cosine value.
* `DataFrameColumn.asinh()` - Returns the inverse hyperbolic sine value.
* `DataFrameColumn.atanh()` - Returns the inverse hyperbolic tangent value.
* `DataFrameColumn.cosh()` - Returns the hyperbolic cosine value.
* `DataFrameColumn.sinh()` - Returns the hyperbolic sine value
* `DataFrameColumn.tanh()` - Returns the hyperbolic tangent value.
* _String Functions_
* `DataFrameColumn.ascii()` - Returns the decimal representation of the first character in column.
* `DataFrameColumn.char2hexint()` - Returns the hexadecimal representation for a character string in a column.
* `DataFrameColumn.chr()` - Returns the Latin ASCII character of a given a numeric code value in column.
* `DataFrameColumn.char()` - It is an alias for `DataFrameColumn.chr()` function.
* `DataFrameColumn.character_length()` - Returns the number of characters in the column.
* `DataFrameColumn.char_length()` - It is an alias for `DataFrameColumn.character_length()` function.
* `DataFrameColumn.edit_distance()` - Returns the minimum number of edit operations required to
transform string in a column into string specified in argument.
* `DataFrameColumn.index()` - Returns the position of a string in a column where string specified in argument starts.
* `DataFrameColumn.initcap()` - Modifies a string column and returns the string with the first character
of each word in uppercase.
* `DataFrameColumn.instr()` - Searches the string in a column for occurrences of search string passed as argument.
* `DataFrameColumn.lcase()` - Returns a character string identical to string values in column,
with all uppercase letters replaced with their lowercase equivalents.
* `DataFrameColumn.left()` - Truncates string in a column to a specified number of characters desired from
the left side of the string.
* `DataFrameColumn.length()` - It is an alias for `DataFrameColumn.character_length()` function.
* `DataFrameColumn.levenshtein()` - It is an alias for `DataFrameColumn.edit_distance()` function.
* `DataFrameColumn.locate()` - Returns the position of the first occurrence of a string in a column within
string in argument.
* `DataFrameColumn.lower()` - It is an alias for `DataFrameColumn.character_lcase()` function.
* `DataFrameColumn.lpad()` - Returns the string in a column padded to the left with the characters specified
in argument so that the resulting string has length specified in argument.
* `DataFrameColumn.ltrim()` - Returns the string in a column, with its left-most characters removed up
to the first character that is not in the string specified in argument.
* `DataFrameColumn.ngram()` - Returns the number of n-gram matches between string in a column,
and string specified in argument.
* `DataFrameColumn.nvp()` - Extracts the value of a name-value pair where the name in the pair matches
the name and the number of the occurrence specified.
* `DataFrameColumn.oreplace()` - Replaces every occurrence of search string in the column.
* `DataFrameColumn.otranslate()` - Returns string in a column with every occurrence of each character in
string in argument replaced with the corresponding character in another argument.
* `DataFrameColumn.replace()` - It is an alias for `DataFrameColumn.oreplace()` function.
* `DataFrameColumn.reverse()` - Returns the reverse of string in column.
* `DataFrameColumn.right()` - Truncates input string to a specified number of characters desired from
the right side of the string.
* `DataFrameColumn.rpad()` - Returns the string in a column padded to the right with the characters specified
in argument so the resulting string has length specified in argument.
* `DataFrameColumn.rtrim()` - Returns the string in column, with its right-most characters removed up
to the first character that is not in the string specified in argument.
* `DataFrameColumn.soundex()` - Returns a character string that represents the Soundex code for
string in a column.
* `DataFrameColumn.string_cs()` - Returns a heuristically derived integer value that can be used to determine
which KANJI1-compatible client character set was used to encode string in a column.
* `DataFrameColumn.translate()` - It is an alias for `DataFrameColumn.otranslate()` function.
* `DataFrameColumn.upper()` - Returns a character string with all lowercase letters in a column replaced
with their uppercase equivalents.
* ##### teradataml Options
* Configuration Options
* `configure.indb_install_location`
Specifies the installation location of In-DB Python package.
* ##### Updates
* Open Analytics Framework (OpenAF) APIs:
* `set_auth_token()`
* `set_auth_token()` does not accept username and password anymore. Instead, function opens up a browser session and user should authenticate in browser.
* After token expiry, teradataml will open a browser and user needs to authenticate again.
* If client machine does not have browser, then user should copy the URL posted by teradataml and authenticate themselves.
* Security fixes - `auth_token` is not set or retrieved from the `configure` option anymore.
* Manage all user environments.
* `create_env()` - supports creation of R environment.
* `remove_env()` - Supports removal of remote R environment.
* `remove_all_envs()` - Supports removal of all remote R environments.
* `remove_env()` and `remove_all_envs()` supports asynchronous call.
* UserEnv Class – Supports managing of R remote environments.
* Properties:
* `libs` - Supports listing of libraries in R remote environment.
* Methods:
* `install_lib()` - Supports installing of libraries in remote R environment.
* `uninstall_lib()` - Supports uninstalling of libraries in remote R environment.
* `update_lib()` - Supports updating of libraries in remote R environment.
* Unbounded Array Framework (UAF) Functions:
* `ArimaEstimate()`
* Added support for `CSS` algorithm via `algorithm` argument.
* ##### Bug Fixes
* Installation location of In-DB 2.0.0 package is changed. Script() will now work with both 2.0.0 and previous version.
## Release Notes:
#### teradataml 17.20.00.04
* ##### New Features/Functionality
* teradataml is now compatible with SQLAlchemy 2.0.X
* **Important notes** when user has sqlalchemy version >= 2.0:
* Users will not be able to run the `execute()` method on SQLAlchemy engine object returned by
`get_context()` and `create_context()` teradataml functions. This is due to the SQLAlchemy has
removed the support for `execute()` method on the engine object. Thus, user scripts where
`get_context().execute()` and `create_context().execute()`, is used, Teradata recommends to
replace those with either `execute_sql()` function exposed by teradataml or `exec_driver_sql()`
method on the `Connection` object returned by `get_connection()` function in teradataml.
* Now `get_connection().execute()` accepts only executable sqlalchemy object. Refer to
`sqlalchemy.engine.base.execute()` for more details.
* Teradata recommends to use either `execute_sql()` function exposed by teradataml or
`exec_driver_sql()` method on the `Connection` object returned by `get_connection()`
function in teradataml, in such cases.
* New utility function `execute_sql()` is added to execute the SQL.
* Extending compatibility for MAC with ARM processors.
* Added support for floor division (//) between two teradataml DataFrame Columns.
* Analytics Database Analytic Functions:
* `GLMPerSegment()`
* `GLMPredictPerSegment()`
* `OneClassSVM()`
* `OneClassSVMPredict()`
* `SVM()`
* `SVMPredict()`
* `TargetEncodingFit()`
* `TargetEncodingTransform()`
* `TrainTestSplit()`
* `WordEmbeddings()`
* `XGBoost()`
* `XGBoostPredict()`
* ###### teradataml Options
* Display Options
* `display.geometry_column_length`
Option to display the default length of geometry column in GeoDataFrame.
* ##### Updates
* `set_auth_token()` function can generate the client id automatically based on org_id when user do not specify it.
* Analytics Database Analytic Functions:
* `ColumnTransformer()`
* Does not allow list values for arguments - `onehotencoding_fit_data` and `ordinalencoding_fit_data`.
* `OrdidnalEncodingFit()`
* New arguments added - `category_data`, `target_column_names`, `categories_column`, `ordinal_values_column`.
* Allows the list of values for arguments - `target_column`, `start_value`, `default_value`.
* `OneHotEncodingFit()`
* New arguments added - `category_data`, `approach`, `target_columns`, `categories_column`, `category_counts`.
* Allows the list of values for arguments - `target_column`, `other_column`.
* ##### Bug Fixes
* `DataFrame.sample()` method output is now deterministic.
* `copy_to_sql()` now preserves the rows of the table even when the view content is copied to the same table name.
* `list_user_envs()` does not raise warning when no user environments found.
## Release Notes:
#### teradataml 17.20.00.03
* ##### Updates
* DataFrame.join
* New arguments `lprefix` and `rprefix` added.
* Behavior of arguments `lsuffix` and `rsuffix` will be changed in future, use new arguments instead.
* New and old affix arguments can now be used independently.
* Analytic functions can be imported regardless of context creation.
Import after create context constraint is now removed.
* `ReadNOS` and `WriteNOS` now accept dictionary value for `authorization` and `row_format` arguments.
* `WriteNOS` supports writing CSV files to external store.
* Following model cataloging APIs will be deprecated in future:
* describe_model
* delete_model
* list_models
* publish_model
* retrieve_model
* save_model
* ##### Bug Fixes
* `copy_to_sql()` bug related to NaT value has been fixed.
* Tooltip on PyCharm IDE now points to SQLE.
* `value` argument of `FillNa()`, a Vantage Analytic Library function supports special characters.
* `case` function accepts DataFrame column as value in `whens` argument.
## Release Notes:
#### teradataml 17.20.00.02
* ##### New Features/Functionality
* ###### teradataml: Open Analytics
* New Functions
* `set_auth_token()` - Sets the JWT token automatically for using Open AF API's.
* ###### teradataml Options
* Display Options
* `display.suppress_vantage_runtime_warnings`
Suppresses the VantageRuntimeWarning raised by teradataml, when set to True.
* ##### Updates
* SimpleImputeFit function arguments `stats_columns` and `stats` are made to be optional.
* New argument `table_format` is added to ReadNOS().
* Argument `full_scan` is changed to `scan_pct` in ReadNOS().
* ##### Bug Fixes
* Minor bug fix related to read_csv.
* APPLY and `DataFrame.apply()` supports hash by and local order by.
* Output column names are changed for DataFrame.dtypes and DataFrame.tdtypes.
## Release Notes:
#### teradataml 17.20.00.01
* ##### New Features/Functionality
* ###### teradataml: DataFrame
* New Functions
* `DataFrame.pivot()` - Rotate data from rows into columns to create easy-to-read DataFrames.
* `DataFrame.unpivot()` - Rotate data from columns into rows to create easy-to-read DataFrames.
* `DataFrame.drop_duplicate()` - Drop duplicate rows from teradataml DataFrame.
* New properties
* `Dataframe.is_art` - Check whether teradataml DataFrame is created on an Analytic Result Table, i.e., ART table or not.
* ###### teradataml: Unbounded Array Framework (UAF) Functions:
* New Functions
* New Functions Supported on Database Versions: 17.20.x.x
* MODEL PREPARATION AND PARAMETER ESTIMATION functions:
1. `ACF()`
2. `ArimaEstimate()`
3. `ArimaValidate()`
4. `DIFF()`
5. `LinearRegr()`
6. `MultivarRegr()`
7. `PACF()`
8. `PowerTransform()`
9. `SeasonalNormalize()`
10. `Smoothma()`
11. `UNDIFF()`
12. `Unnormalize()`
* SERIES FORECASTING functions:
1. `ArimaForecast()`
2. `DTW()`
3. `HoltWintersForecaster()`
4. `MAMean()`
5. `SimpleExp()`
* DATA PREPARATION functions:
1. `BinaryMatrixOp()`
2. `BinarySeriesOp()`
3. `GenseriesFormula()`
4. `MatrixMultiply()`
5. `Resample()`
* DIAGNOSTIC STATISTICAL TEST functions:
1. `BreuschGodfrey()`
2. `BreuschPaganGodfrey()`
3. `CumulPeriodogram()`
4. `DickeyFuller()`
5. `DurbinWatson()`
6. `FitMetrics()`
7. `GoldfeldQuandt()`
8. `Portman()`
9. `SelectionCriteria()`
10. `SignifPeriodicities()`
11. `SignifResidmean()`
12. `WhitesGeneral()`
* TEMPORAL AND SPATIAL functions:
1. `Convolve()`
2. `Convolve2()`
3. `DFFT()`
4. `DFFT2()`
5. `DFFT2Conv()`
6. `DFFTConv()`
7. `GenseriesSinusoids()`
8. `IDFFT()`
9. `IDFFT2()`
10. `LineSpec()`
11. `PowerSpec()`
* GENERAL UTILITY functions:
1. `ExtractResults()`
2. `InputValidator()`
3. `MInfo()`
4. `SInfo()`
5. `TrackingOp()`
* New Features: Inputs to Unbounded Array Framework (UAF) functions
* `TDAnalyticResult()` - Allows to prepare function output generated by UAF functions to be passed.
* `TDGenSeries()` - Allows to generate a series, that can be passed to a UAF function.
* `TDMatrix()` - Represents a Matrix in time series, that can be created from a teradataml DataFrame.
* `TDSeries()` - Represents a Series in time series, that can be created from a teradataml DataFrame.
* ##### Updates
* Native Object Store (NOS) functions support authorization by specifying authorization object.
* `display_analytic_functions()` categorizes the analytic functions based on function type.
* ColumnTransformer accepts multiple values for arguments nonlinearcombine_fit_data,
onehotencoding_fit_data, ordinalencoding_fit_data.
* ##### Bug Fixes
* Redundant warnings thrown by teradataml are suppressed.
* OpenAF supports when context is created with JWT Token.
* New argument "match_column_order" added to copy_to_sql, that allows DataFrame loading with any column order.
* `copy_to_sql` updated to map data type timezone(tzinfo) to TIMESTAMP(timezone=True), instead of VARCHAR.
* Improved performance for DataFrame.sum and DataFrameColumn.sum functions.
## Release Notes:
#### teradataml 17.20.00.00
* ##### New Features/Functionality
* ###### teradataml: Analytics Database Analytic Functions
* _New Functions_
* ###### New Functions Supported on Database Versions: 17.20.x.x
* `ANOVA()`
* `ClassificationEvaluator()`
* `ColumnTransformer()`
* `DecisionForest()`
* `GLM()`
* `GetFutileColumns()`
* `KMeans()`
* `KMeansPredict()`
* `NaiveBayesTextClassifierTrainer()`
* `NonLinearCombineFit()`
* `NonLinearCombineTransform()`
* `OrdinalEncodingFit()`
* `OrdinalEncodingTransform()`
* `RandomProjectionComponents()`
* `RandomProjectionFit()`
* `RandomProjectionTransform()`
* `RegressionEvaluator()`
* `ROC()`
* `SentimentExtractor()`
* `Silhouette()`
* `TDGLMPredict()`
* `TextParser()`
* `VectorDistance()`
* _Updates_
* `display_analytic_functions()` categorizes the analytic functions based on function type.
* Users can provide range value for columns argument.
* ###### teradataml: Open Analytics
* Manage all user environments.
* `list_base_envs()` - list the available python base versions.
* `create_env()` - create a new user environment.
* `get_env()` - get existing user environment.
* `list_user_envs()` - list the available user environments.
* `remove_env()` - delete user environment.
* `remove_all_envs()` - delete all the user environments.
* UserEnv Class – Manage individual user environment.
* Properties
* `files` - Get files in user environment.
* `libs` - Get libraries in user environment.
* Methods
* `install_file()` - Install a file in user environment.
* `remove_file()` - Remove a file in user environment.
* `install_lib()` - Install a library in user environment.
* `update_lib()` - Update a library in user environment.
* `uninstall_lib()` - Uninstall a library in user environment.
* `status()` - Check the status of
* file installation
* library installation
* library update
* library uninstallation
* `refresh()` - Refresh the environment details in local client.
* Apply Class – Execute a user script on VantageCloud Lake.
* `__init__()` - Instantiate an object of apply for script execution.
* `install_file()` - Install a file in user environment.
* `remove_file()` - Remove a file in user environment.
* `set_data()` – Reset data and related arguments.
* `execute_script()` – Executes Python script.
* ###### teradataml: DataFrame
* _New Functions_
* `DataFrame.apply()` - Execute a user defined Python function on VantageLake Cloud.
* ###### teradataml: Bring Your Own Model
* _New Functions_
* `ONNXPredict()` - Score using model trained externally on ONNX and stored in Vantage.
* ###### teradataml: Options
* _New Functions_
* set_config_params() New API to set all config params in one go.
* _New Configuration Options_
* For Open Analytics support.
* ues_url – User Environment Service URL for VantageCloud Lake.
* auth_token – Authentication token to connect to VantageCloud Lake.
* certificate_file – Path to a CA_BUNDLE file or directory with certificates of trusted CAs.
* ##### Updates
* `accumulate` argument is working for `ScaleTransform()`.
* Following functions have `accumulate` argument added on Database Versions: 17.20.x.x
* `ConvertTo()`
* `GetRowsWithoutMissingValues()`
* `GetRowsWithoutMissingValues()`
* `OutlierFilterFit()` supports multiple output.
* For `OutlierFilterFit()` function below arguments are optional in teradataml 17.20.x.x
* `lower_percentile`
* `upper_percentile`
* `outlier_method`
* `replacement_value`
* `percentile_method`
* Analytics Database analytic functions – In line help, i.e., help() for the functions
is available.
* ##### Bug Fixes
* Vantage Analytic Library FillNa() function: Now `columns` argument is required.
* `output_responses` argument in MLE function `DecisionTreePredict()`, does not allow empty string.
* teradataml closes docker sandbox environment properly.
* Users can create context using JWT token.
#### teradataml 17.10.00.02
* ##### New Features/Functionality
* ###### Database Utility
* `list_td_reserved_keywords()` - Validates if the specified string is Teradata reserved
keyword or not, else lists down all the Teradata reserved keywords.
* ##### Updates
* ###### DataFrame
* _Updates_
* Multiple columns can be selected using slice operator ([]).
* ###### Script
* _Updates_
* A warning will be raised, when Teradata reserved keyword is used in Script local mode.
* ##### Bug Fixes
* Numeric overflow issue observed for describe(), sum(), csum(), and mean() has been fixed.
* Error messages are updated for SQLE function arguments accepting multiple datatypes.
* Error messages are updated for SQLE function arguments volatile and persist arguments when
non-boolean value is provided.
* DataFrame sample() method can handle column names with special characters like space, hyphen,
period etc.
* In-DB SQLE functions can be loaded for any locale setting.
* `create_context()` - Password containing special characters requires URL encoding as per
https://docs.microfocus.com/OMi/10.62/Content/OMi/ExtGuide/ExtApps/URL_encoding.html.
teradataml has added a fix to take care of the URL encoding of the password while creating a context.
Also, a new argument is added to give a more control over the URL encoding to be done at the time of context creation.
#### teradataml 17.10.00.01
* ##### New Features/Functionality
* ###### Geospatial
The Geospatial feature in teradataml enables data manipulation, exploration and analysis on tables, views, and queries on Teradata Vantage that contains Geospatial data.
* ###### Geomtery Types
* Point
* LineString
* Polygon
* MultiPoint
* MultiLineString
* MultiPolygon
* GeometryCollection
* GeoSequence
* ###### teradataml GeoDataFrame
* Properties
* columns
* dtypes
* geometry
* iloc
* index
* loc
* shape
* size
* tdtypes
* Geospatial Specific Properties
* ###### Properties for all Types of Geometries
* boundary
* centroid
* convex_hell
* coord_dim
* dimension
* geom_type
* is_3D
* is_empty
* is_simple
* is_valid
* max_x
* max_y
* max_z
* min_x
* min_y
* min_z
* srid
* ###### Properties for Point Geometry
* x
* y
* z
* ###### Properties for LineString Geometry
* is_closed_3D
* is_closed
* is_ring
* ###### Properties for Polygon Geometry
* area
* exterior
* perimeter
* Methods
* `__getattr__()`
* `__getitem__()`
* `__init__()`
* `__repr__()`
* `assign()`
* `concat()`
* `count()`
* `drop()`
* `dropna()`
* `filter()`
* `from_query()`
* `from_table()`
* `get()`
* `get_values()`
* `groupby()`
* `head()`
* `info()`
* `join()`
* `keys()`
* `merge()`
* `sample()`
* `select()`
* `set_index()`
* `show_query()`
* `sort()`
* `sort_index()`
* `squeeze()`
* `tail()`
* `to_csv()`
* `to_pandas()`
* `to_sql()`
* Geospatial Specific Methods
* ###### Methods for All Type of Geometry
* `buffer()`
* `contains()`
* `crosses()`
* `difference()`
* `disjoint()`
* `distance()`
* `distance_3D()`
* `envelope()`
* `geom_equals()`
* `intersection()`
* `intersects()`
* `make_2D()`
* `mbb()`
* `mbr()`
* `overlaps()`
* `relates()`
* `set_exterior()`
* `set_srid()`
* `simplify()`
* `sym_difference()`
* `to_binary()`
* `to_text()`
* `touches()`
* `transform()`
* `union()`
* `within()`
* `wkb_geom_to_sql()`
* `wkt_geom_to_sql()`
* ###### Methods for Point Geometry
* `spherical_buffer()`
* `spherical_distance()`
* `spheriodal_buffer()`
* `spheriodal_distance()`
* `set_x()`
* `set_y()`
* `set_z()`
* ###### Methods for LineString Geometry
* `end_point()`
* `length()`
* `length_3D()`
* `line_interpolate_point()`
* `num_points()`
* `point()`
* `start_point()`
* ###### Methods for Polygon Geometry
* `interiors()`
* `num_interior_ring()`
* `point_on_surface()`
* ###### Methods for GeometryCollection Geometry
* `geom_component()`
* `num_geometry()`
* ###### Methods for GeoSequence Geometry
* `clip()`
* `get_final_timestamp()`
* `get_init_timestamp()`
* `get_link()`
* `get_user_field()`
* `get_user_field_count()`
* `point_heading()`
* `set_link()`
* `speed()`
* ###### Filtering Functions and Methods
* `intersects_mbb()`
* `mbb_filter()`
* `mbr_filter()`
* `within_mbb()`
* ###### teradataml GeoDataFrameColumn
* Geospatial Specific Properties
* ###### Properties for all Types of Geometries
* boundary
* centroid
* convex_hell
* coord_dim
* dimension
* geom_type
* is_3D
* is_empty
* is_simple
* is_valid
* max_x
* max_y
* max_z
* min_x
* min_y
* min_z
* srid
* ###### Properties for Point Geometry
* x
* y
* z
* ###### Properties for LineString Geometry
* is_closed_3D
* is_closed
* is_ring
* ###### Properties for Polygon Geometry
* area
* exterior
* perimeter
* Geospatial Specific Methods
* ###### Methods for All Type of Geometry
* `buffer()`
* `contains()`
* `crosses()`
* `difference()`
* `disjoint()`
* `distance()`
* `distance_3D()`
* `envelope()`
* `geom_equals()`
* `intersection()`
* `intersects()`
* `make_2D()`
* `mbb()`
* `mbr()`
* `overlaps()`
* `relates()`
* `set_exterior()`
* `set_srid()`
* `simplify()`
* `sym_difference()`
* `to_binary()`
* `to_text()`
* `touches()`
* `transform()`
* `union()`
* `within()`
* `wkb_geom_to_sql()`
* `wkt_geom_to_sql()`
* ###### Methods for Point Geometry
* `spherical_buffer()`
* `spherical_distance()`
* `spheriodal_buffer()`
* `spheriodal_distance()`
* `set_x()`
* `set_y()`
* `set_z()`
* ###### Methods for LineString Geometry
* `endpoint()`
* `length()`
* `length_3D()`
* `line_interpolate_point()`
* `num_points()`
* `point()`
* `start_point()`
* ###### Methods for Polygon Geometry
* `interiors()`
* `num_interior_ring()`
* `point_on_surface()`
* ###### Methods for GeometryCollection Geometry
* `geom_component()`
* `num_geometry()`
* ###### Methods for GeoSequence Geometry
* `clip()`
* `get_final_timestamp()`
* `get_init_timestamp()`
* `get_link()`
* `get_user_field()`
* `get_user_field_count()`
* `point_heading()`
* `set_link()`
* `speed()`
* ###### Filtering Functions and Methods
* `intersects_mbb()`
* `mbb_filter()`
* `mbr_filter()`
* `within_mbb()`
* ###### teradataml DataFrame
* _New Functions_
* `to_csv()`
* ###### teradataml: SQLE Engine Analytic Functions
* _New Functions_
* Newly added SQLE functions are accessible only after establishing the connection to Vantage.
* `display_analytic_functions()` API displays all the available SQLE Analytic functions based on database version.
* ###### Functions Supported on DatabaseVersions: 16.20.x.x, 17.10.x.x, 17.05.x.x
* `Antiselect()`
* `Attribution()`
* `DecisionForestPredict()`
* `DecisionTreePredict()`
* `GLMPredict()`
* `MovingAverage()`
* `NaiveBayesPredict()`
* `NaiveBayesTextClassifierPredict()`
* `NGramSplitter()`
* `NPath()`
* `Pack()`
* `Sessionize()`
* `StringSimilarity()`
* `SVMParsePredict()`
* `Unpack()`
* ###### Functions Supported on DatabaseVersions: 17.10.x.x
* `Antiselect()`
* `Attribution()`
* `BincoodeFit()`
* `BncodeTransform()`
* `CategoricalSummary()`
* `ChiSq()`
* `ColumnSummary()`
* `ConvertTo()`
* `DecisionForestPredict()`
* `DecisionTreePredict()`
* `GLMPredict()`
* `FillRowId()`
* `FTest()`
* `Fit()`
* `Transform()`
* `GetRowsWithMissingValues()`
* `GetRowsWithoutMissingValues()`
* `MovingAverage()`
* `Histogram()`
* `NaiveBayesPredict()`
* `NaiveBayesTextClassifierPredict()`
* `NGramSplitter()`
* `NPath()`
* `NumApply()`
* `OneHotEncodingFit()`
* `OneHotEncodingTransform()`
* `OutlierFilterFit()`
* `OutlierFilterTransform()`
* `Pack()`
* `PolynomialFeatuesFit()`
* `PolynomialFeatuesTransform()`
* `QQNorm()`
* `RoundColumns()`
* `RowNormalizeFit()`
* `RowNormalizeTransform()`
* `ScaleFit()`
* `ScaleTransform()`
* `Sessionize()`
* `SimpleImputeFit()`
* `SimpleImputeTransform()`
* `StrApply()`
* `StringSimilarity()`
* `SVMParsePredict()`
* `UniVariateStatistics()`
* `Unpack()`
* `WhichMax()`
* `WhichMin()`
* `ZTest()`
* ###### teradataml: General Functions
* _New Functions_
* Data Transfer Utility
* `read_csv()`
* ###### Operators
* _New Functions_
* Table Operators
* `read_nos()`
* `write_nos()`
* ###### teradataml: Bring Your Own Model
* _New Functions_
* Model Cataloging
* `get_license()`
* `set_byom_catalog()`
* `set_license()`
* ##### Updates
* ###### teradataml: General Functions
* Data Transfer Utility
* `copy_to_sql()` - New argument "chunksize" added to load data in chunks.
* Following Data Transfer Utility Functions updated to specify the number of Teradata sessions to open for data transfer using "open_session" argument:
* `fastexport()`
* `fastload()`
* `to_pandas()`
* ###### Operators
* Following Set Operator Functions updated to work with Geospatial data:
* `concat()`
* `td_intersect()`
* `td_expect()`
* `td_minus()`
* ###### teradataml: Bring Your Own Model
* Model cataloging APIs mentioned below are updated to use session level parameters set by `set_byom_catalog()` and `set_license()` such as table name, schema name and license details respectively.
* `delete_byom()`
* `list_byom()`
* `retrieve_byom()`
* `save_byom()`
* `view_log()` - Allows user to view BYOM logs.
* ##### Bug Fixes
* CS0733758 - `db_python_package_details()` function is fixed to support latest STO release for pip and Python aliases used.
* DataFrame `print()` issue related to `Response Row size is greater than the 1MB allowed maximum.` has been fixed to print the data with lot of columns.
* New parameter "chunksize" is added to `DataFrame.to_sql()` and `copy_to_sql()` to fix the issue where the function was failing with error - "Request requires too many SPOOL files.". Reducing the chunksize than the default one will result in successful operation.
* `remove_context()` is fixed to remove the active connection from database.
* Support added to specify the number of Teradata data transfer sessions to open for data transfer using `fastexport()` and `fastload()` functions.
* `DataFrame.to_sql()` is fixed to support temporary table when default database differs from the username.
* `DataFrame.to_pandas()` now by default support data transfer using regular method. Change is carried out for user to allow the data transfer if utility throttles are configured, i.e., TASM configuration does not support data export using FastExport.
* `save_byom()` now notifies if VARCHAR column is trimmed out if data passed to the API is greater than the length of the VARCHAR column.
* Standard error can now be captured for `DataFrame.map_row()` and `DataFrame.map_parition()` when executed in LOCAL mode.
* Vantage Analytic Library - Underlying SQL can be retrieved using newly added arguments "gen_sql"/"gen_sql_only" for the functions. Query can be viewed with the help `show_query()`.
* Documentation example has been fixed for `fastexport()` to show the correct import statement.
#### teradataml 17.00.00.05
Fixed [CS0733758] db_python_package_details() fails on recent STO release due to changes in pip and python aliases.
#### teradataml 17.00.00.04
* ##### New Features/Functionality
* ###### Analytic Functions
* Bring Your Own Analytics Functions
The BYOM feature in Vantage provides flexibility to score the data in Vantage using external models using following BYOM functions:
* `H2OPredict()` - Score using model trained externally in H2O and stored in Vantage.
* `PMMLPredict()` - Score using model trained externally in PMML and stored in Vantage.
* BYOM Model Catalog APIs
* `save_byom()` - Save externally trained models in Teradata Vantage.
* `delete_byom()` - Delete a model from the user specified table in Teradata Vantage.
* `list_byom()` - List models.
* `retrieve_byom()` - Function to retrieve a saved model.
* Vantage Analytic Library Functions
* _New Functions_
* `XmlToHtmlReport()` - Transforms XML output of VAL functions to HTML.
* ###### teradataml DataFrame
* `DataFrame.window()` - Generates Window object on a teradataml DataFrame to run window aggregate functions.
* `DataFrame.csum()` - Returns column-wise cumulative sum for rows in the partition of the dataframe.
* `DataFrame.mavg()` - Returns moving average for the current row and the preceding rows.
* `DataFrame.mdiff()` - Returns moving difference for the current row and the preceding rows.
* `DataFrame.mlinreg()` - Returns moving linear regression for the current row and the preceding rows.
* `DataFrame.msum()` - Returns moving sum for the current row and the preceding rows.
* _Regular Aggregate Functions_
* `DataFrame.corr()` - Returns the Sample Pearson product moment correlation coefficient.
* `DataFrame.covar_pop()` - Returns the population covariance.
* `DataFrame.covar_samp()` - Returns the sample covariance.
* `DataFrame.regr_avgx()` - Returns the mean of the independent variable.
* `DataFrame.regr_avgy()` - Returns the mean of the dependent variable.
* `DataFrame.regr_count()` - Returns the count of the dependent and independent variable arguments.
* `DataFrame.rege_intercept()` - Returns the intercept of the univariate linear regression line.
* `DataFrame.regr_r2()` - Returns the coefficient of determination.
* `DataFrame.regr_slope()` - Returns the slope of the univariate linear regression line through.
* `DataFrame.regr_sxx()` - Returns the sum of the squares of the independent variable expression.
* `DataFrame.regr_sxy()` - Returns the sum of the products of the independent variable and the dependent variable.
* `DataFrame.regr_syy()` - Returns the sum of the squares of the dependent variable expression.
* ###### teradataml DataFrameColumn a.k.a. ColumnExpression
* `ColumnExpression.window()` - Generates Window object on a teradataml DataFrameColumn to run window aggregate functions.
* `ColumnExpression.desc()` - Sorts ColumnExpression in descending order.
* `ColumnExpression.asc()` - Sorts ColumnExpression in ascending order.
* `ColumnExpression.distinct()` - Removes duplicate value from ColumnExpression.
* _Regular Aggregate Functions_
* `ColumnExpression.corr()` - Returns the Sample Pearson product moment correlation coefficient.
* `ColumnExpression.count()` - Returns the column-wise count.
* `ColumnExpression.covar_pop()` - Returns the population covariance.
* `ColumnExpression.covar_samp()` - Returns the sample covariance.
* `ColumnExpression.kurtosis()` - Returns kurtosis value for a column.
* `ColumnExpression.median()` - Returns column-wise median value.
* `ColumnExpression.max()` - Returns the column-wise max value.
* `ColumnExpression.mean()` - Returns the column-wise average value.
* `ColumnExpression.min()` - Returns the column-wise min value.
* `ColumnExpression.regr_avgx()` - Returns the mean of the independent variable.
* `ColumnExpression.regr_avgy()` - Returns the mean of the dependent variable.
* `ColumnExpression.regr_count()` - Returns the count of the dependent and independent variable arguments.
* `ColumnExpression.rege_intercept()` - Returns the intercept of the univariate linear regression line.
* `ColumnExpression.regr_r2()` - Returns the coefficient of determination arguments.
* `ColumnExpression.regr_slope()` - Returns the slope of the univariate linear regression line.
* `ColumnExpression.regr_sxx()` - Returns the sum of the squares of the independent variable expression.
* `ColumnExpression.regr_sxy()` - Returns the sum of the products of the independent variable and the dependent variable.
* `ColumnExpression.regr_syy()` - Returns the sum of the squares of the dependent variable expression.
* `ColumnExpression.skew()` - Returns skew value for a column.
* `ColumnExpression.std()` - Returns the column-wise population/sample standard deviation.
* `ColumnExpression.sum()` - Returns the column-wise sum.
* `ColumnExpression.var()` - Returns the column-wise population/sample variance.
* `ColumnExpression.percentile()` - Returns the column-wise percentile.
* ###### teradataml Window - Window Aggregate Functions
Following set of _Window Aggregate Functions_ return the results over a specified window which can be of any type:
* Cumulative/Expanding window
* Moving/Rolling window
* Contracting/Remaining window
* Grouping window
_Window Aggregate Functions_
* `Window.corr()` - Returns the Sample Pearson product moment correlation coefficient.
* `Window.count()` - Returns the count.
* `Window.covar_pop()` - Returns the population covariance.
* `Window.covar_samp()` - Returns the sample covariance.
* `Window.cume_dist()` - Returns the cumulative distribution of values.
* `Window.dense_Rank()` - Returns the ordered ranking of all the rows.
* `Window.first_value()` - Returns the first value of an ordered set of values.
* `Window.lag()` - Returns data from the row preceding the current row at a specified offset value.
* `Window.last_value()` - Returns the last value of an ordered set of values.
* `Window.lead()` - Returns data from the row following the current row at a specified offset value.
* `Window.max()` - Returns the column-wise max value.
* `Window.mean()` - Returns the column-wise average value.
* `Window.min()` - Returns the column-wise min value.
* `Window.percent_rank()` - Returns the relative rank of all the rows.
* `Window.rank()` - Returns the rank (1 … n) of all the rows.
* `Window.regr_avgx()` - Returns the mean of the independent variable arguments.
* `Window.regr_avgy()` - Returns the mean of the dependent variable arguments.
* `Window.regr_count()` - Returns the count of the dependent and independent variable arguments.
* `Window.rege_intercept()` - Returns the intercept of the univariate linear regression line arguments.
* `Window.regr_r2()` - Returns the coefficient of determination arguments.
* `Window.regr_slope()` - Returns the slope of the univariate linear regression line.
* `Window.regr_sxx()` - Returns the sum of the squares of the independent variable expression.
* `Window.regr_sxy()` - Returns the sum of the products of the independent variable and the dependent variable.
* `Window.regr_syy()` - Returns the sum of the squares of the dependent variable expression.
* `Window.row_number()` - Returns the sequential row number.
* `Window.std()` - Returns the column-wise population/sample standard deviation.
* `Window.sum()` - Returns the column-wise sum.
* `Window.var()` - Returns the column-wise population/sample variance.
* ###### General functions
* _New functions_
* `fastexport()` - Exports teradataml DataFrame to Pandas DataFrame using FastExport data transfer protocol.
* ###### teradataml Options
* Display Options
* `display.blob_length`
Specifies default display length of BLOB column in teradataml DataFrame.
* Configuration Options
* `configure.temp_table_database`
Specifies database name for storing the tables created internally.
* `configure.temp_view_database`
Specifies database name for storing the views created internally.
* `configure.byom_install_location`
Specifies the install location for the BYOM functions.
* `configure.val_install_location`
Specifies the install location for the Vantage Analytic Library functions.
* ##### Updates
* ###### teradataml DataFrame
* `to_pandas()` -
* Support added to transfer data to Pandas DataFrame using fastexport protocol improving the performance.
* Support added for other arguments similar to Pandas `read_sql()`:
* `coerce_float`
* `parse_dates`
* ###### Analytic functions
* Vantage Analytic Library Functions
* Support added to accept datetime.date object for literals/values in
following transformation functions:
* `FillNa()`
* `Binning()`
* `OneHotEncoder()`
* `LabelEncoder()`
* All transformation functions now supports accepting
teradatasqlalchemy datatypes as input to "datatype" argument for
casting the result.
* ##### Bug Fixes.
* CS0249633 - Support added for teradataml to work with user/database/tablename
containing period (.).
* CS0086594 - Use of dbc.tablesvx versus dbc.tablesvx in teradatasqlalchemy.
* IPython integration to print the teradataml DataFrames in pretty format.
* teradataml DataFrame APIs now support column names same as that of Teradata
reserved keywords.
* Issue has been fixed for duplicate rows being loaded via teradataml
fastload() API.
* VAL - Empty string now can be passed as input for recoding values using
LabelEncoder.
* teradataml extension with SQLAlchemy functions:
* mod() function is fixed to return correct datatype.
* sum() function is fixed to return correct datatype.
#### teradataml 17.00.00.03
- New release of SQLAlchemy1.4.x introduced backward compatibility issue. A fix has been carried out so that teradataml can support latest SQLAlchemy changes.
- Other minor bug fixes.
#### teradataml 17.00.00.02
Fixed the internal library load issue related to the GCC version discrepancies on CentOS platform.
#### teradataml 17.00.00.01
* ##### New Features/Functionality
* ###### Analytic Functions
* Vantage Analytic Library
teradataml now supports executing analytic functions offered by Vantage Analytic Library.
These functions are available via new 'valib' sub-package of teradataml.
Following functions are added as part of this:
* Association Rules:
* `Association()`
* Descriptive Statistics:
* `AdaptiveHistogram()`
* `Explore()`
* `Frequency()`
* `Histogram()`
* `Overlaps()`
* `Statistics()`
* `TextAnalyzer()`
* `Values()`
* Decision Tree:
* `DecisionTree()`
* `DecisionTreePredict()`
* `DecisionTreeEvaluator()`
* Fast K-Means Clustering:
* `KMeans()`
* `KMeansPredict()`
* Linear Regression:
* `LinReg()`
* `LinRegPredict()`
* Logistic Regression:
* `LogReg()`
* `LogRegPredict()`
* `LogRegEvaluator()`
* Factor Analysis:
* `PCA()`
* `PCAPredict()`
* `PCAEvaluator()`
* Matrix Building:
* `Matrix()`
* Statistical Tests:
* `BinomialTest()`
* `ChiSquareTest()`
* `KSTest()`
* `ParametricTest()`
* `RankTest()`
* Variable Transformation:
* `Transform()`
* Transformation Techniques supported for variable transformation:
* `Binning()` - Perform bin coding to replaces continuous numeric column with a
categorical one to produce ordinal values.
* `Derive()` - Perform free-form transformation done using arithmetic formula.
* `FillNa()` - Perform missing value/null replacement transformations.
* `LabelEncoder()` - Re-express categorical column values into a new coding scheme.
* `MinMaxScalar()` - Rescale data limiting the upper and lower boundaries.
* `OneHotEncoder()` - Re-express a categorical data element as one or more
numeric data elements, creating a binary numeric field for each
categorical data value.
* `Retain()` - Copy one or more columns into the final analytic data set.
* `Sigmoid()` - Rescale data using sigmoid or s-shaped functions.
* `ZScore()` - Rescale data using Z-Score values.
* ML Engine Functions (mle)
* Correlation2
* NaiveBayesTextClassifier2
* ###### DataFrame
* _New Functions_
* `DataFrame.map_row()` - Function to apply a user defined function to each row in the
teradataml DataFrame.
* `DataFrame.map_partition()` - Function to apply a user defined function to a group or
partition of rows in the teradataml DataFrame.
* _New Property_
* `DataFrame.tdtypes` - Get the teradataml DataFrame metadata containing column names and
corresponding teradatasqlalchemy types.
* ###### General functions
* _New functions_
* Database Utility Functions
* `db_python_package_details()` - Lists the details of Python packages installed on Vantage.
* General Utility Functions
* `print_options()`
* `view_log()`
* `setup_sandbox_env()`
* `copy_files_from_container()`
* `cleanup_sandbox_env()`
* ##### Updates
* ###### `create_context()`
* Supports all connection parameters supported by teradatasql.connect().
* ###### Script
* `test_script()` can now be executed in 'local' mode, i.e., outside of the sandbox.
* `Script.setup_sto_env()` is deprecated. Use `setup_sandbox_env()` function instead.
* Added support for using "quotechar" argument.
* ###### Analytic functions
* _Updates_
* Visit teradataml User Guide to know more about the updates done to ML Engine analytic
functions. Following type of updates are done to several functions:
* New arguments are added, which are supported only on Vantage Version 1.3.
* Default value has been updated for few function arguments.
* Few arguments were required, but now they are optional.
* ##### Minor Bug Fixes.
#### teradataml 17.00.00.00
* ##### New Features/Functionality
* ###### Model Cataloging - Functionality to catalog model metadata and related information in the Model Catalog.
* `save_model()` - Save a teradataml Analytic Function model.
* `retrieve_model()` - Retrieve a saved model.
* `list_model()` - List accessible models.
* `describe_model()` - List the details of a model.
* `delete_model()` - Remove a model from Model Catalog.
* `publish_model()` - Share a model.
* ###### Script - An interface to the SCRIPT table operator object in the Advanced SQL Engine.
Interface offers execution in two modes:
* Test/Debug - to test user scripts locally in a containerized environment.
Supporting methods:
* `setup_sto_env()` - Set up test environment.
* `test_script()` - Test user script in containerized environment.
* `set_data()` - Set test data parameters.
* In-Database Script Execution - to execute user scripts in database.
Supporting methods:
* `execute_script()` - Execute user script in Vantage.
* `install_file()` - Install or replace file in Database.
* `remove_file()` - Remove installed file from Database.
* `set_data()` - Set test data parameters.
* ###### DataFrame
* `DataFrame.show_query()` - Show underlying query for DataFrame.
* Regular Aggregates
* _New functions_
* `kurtosis()` - Calculate the kurtosis value.
* `skew()` - Calculate the skewness of the distribution.
* _Updates_\
New argument `distinct` is added to following aggregates to exclude duplicate values.
* `count()`
* `max()`
* `mean()`
* `min()`
* `sum()`
* `std()`
* New argument `population` is added to calculate the population standard deviation.
* `var()`
* New argument `population` is added to calculate the population variance.
* Time Series Aggregates
* _New functions_
* `kurtosis()` - Calculate the kurtosis value.
* `count()` - Get the total number of values.
* `max()` - Calculate the maximum value.
* `mean()` - Calculate the average value.
* `min()` - Calculate the minimum value.
* `percentile()` - Calculate the desired percentile.
* `skew()` - Calculate the skewness of the distribution.
* `sum()` - Calculate the column-wise sum value.
* `std()` - Calculate the sample and population standard deviation.
* `var()` - Calculate the sample and population standard variance.
* ###### General functions
* _New functions_
* Database Utility Functions
* `db_drop_table()`
* `db_drop_view()`
* `db_list_tables()`
* Vantage File Management Functions
* `install_file()` - Install a file in Database.
* `remove_file()` - Remove an installed file from Database.
* _Updates_
* `create_context()`
* Support added for Stored Password Protection feature.
* Kerberos authentication bug fix.
* New argument `database` added to `create_context()` API, that allows user to specify connecting database.
* ###### Analytic functions
* _New functions_
* `Betweenness`
* `Closeness`
* `FMeasure`
* `FrequentPaths`
* `IdentityMatch`
* `Interpolator`
* `ROC`
* _Updates_
* New methods are added to all analytic functions
* `show_query()`
* `get_build_time()`
* `get_prediction_type()`
* `get_target_column()`
* New properties are added to analytic function's Formula argument
* `response_column`
* `numeric_columns`
* `categorical_columns`
* `all_columns`
#### teradataml 16.20.00.06
Fixed the DataFrame data display corruption issue observed with certain analytic functions.
#### teradataml 16.20.00.05
Compatible with Vantage 1.1.1.\
The following ML Engine (`teradataml.analytics.mle`) functions have new and/or updated arguments to support the Vantage version:
* `AdaBoostPredict`
* `DecisionForestPredict`
* `DecisionTreePredict`
* `GLMPredict`
* `LDA`
* `NaiveBayesPredict`
* `NaiveBayesTextClassifierPredict`
* `SVMDensePredict`
* `SVMSparse`
* `SVMSparsePredict`
* `XGBoostPredict`
#### teradataml 16.20.00.04
* ##### Improvements
* DataFrame creation is now quicker, impacting many APIs and Analytic functions.
* Improved performance by reducing the number of intermediate queries issued to Teradata Vantage when not required.
* The number of queries reduced by combining multiple operations into a single step whenever possible and unless the user expects or demands to see the intermediate results.
* The performance improvement is almost proportional to the number of chained and unexecuted operations on a teradataml DataFrame.
* Reduced number of intermediate internal objects created on Vantage.
* ##### New Features/Functionality
* ###### General functions
* _New functions_
* `show_versions()` - to list the version of teradataml and dependencies installed.
* `fastload()` - for high performance data loading of large amounts of data into a table on Vantage. Requires `teradatasql` version `16.20.0.48` or above.
* Set operators:
* `concat`
* `td_intersect`
* `td_except`
* `td_minus`
* `case()` - to help construct SQL CASE based expressions.
* _Updates_
* `copy_to_sql`
* Added support to `copy_to_sql` to save multi-level index.
* Corrected the type mapping for index when being saved.
* `create_context()` updated to support 'JWT' logon mechanism.
* ###### Analytic functions
* _New functions_
* `NERTrainer`
* `NERExtractor`
* `NEREvaluator`
* `GLML1L2`
* `GLML1L2Predict`
* _Updates_
* Added support to categorize numeric columns as categorical while using formula - `as_categorical()` in the `teradataml.common.formula` module.
* ###### DataFrame
* Added support to create DataFrame from Volatile and Primary Time Index tables.
* `DataFrame.sample()` - to sample data.
* `DataFrame.index` - Property to access `index_label` of DataFrame.
* Functionality to process Time Series Data
* Grouping/Resampling time series data:
* `groupby_time()`
* `resample()`
* Time Series Aggregates:
* `bottom()`
* `count()`
* `describe()`
* `delta_t()`
* `mad()`
* `median()`
* `mode()`
* `first()`
* `last()`
* `top()`
* DataFrame API and method argument validation added.
* `DataFrame.info()` - Default value for `null_counts` argument updated from `None` to `False`.
* `Dataframe.merge()` updated to accept columns expressions along with column names to `on`, `left_on`, `right_on` arguments.
* ###### DataFrame Column/ColumnExpression methods
* `cast()` - to help cast the column to a specified type.
* `isin()` and `~isin()` - to check the presence of values in a column.
* ##### Removed deprecated Analytic functions
* All the deprecated Analytic functions under the `teradataml.analytics module` have been removed.
Newer versions of the functions are available under the `teradataml.analytics.mle` and the `teradataml.analytics.sqle` modules.
The modules removed are:
* `teradataml.analytics.Antiselect`
* `teradataml.analytics.Arima`
* `teradataml.analytics.ArimaPredictor`
* `teradataml.analytics.Attribution`
* `teradataml.analytics.ConfusionMatrix`
* `teradataml.analytics.CoxHazardRatio`
* `teradataml.analytics.CoxPH`
* `teradataml.analytics.CoxSurvival`
* `teradataml.analytics.DecisionForest`
* `teradataml.analytics.DecisionForestEvaluator`
* `teradataml.analytics.DecisionForestPredict`
* `teradataml.analytics.DecisionTree`
* `teradataml.analytics.DecisionTreePredict`
* `teradataml.analytics.GLM`
* `teradataml.analytics.GLMPredict`
* `teradataml.analytics.KMeans`
* `teradataml.analytics.NGrams`
* `teradataml.analytics.NPath`
* `teradataml.analytics.NaiveBayes`
* `teradataml.analytics.NaiveBayesPredict`
* `teradataml.analytics.NaiveBayesTextClassifier`
* `teradataml.analytics.NaiveBayesTextClassifierPredict`
* `teradataml.analytics.Pack`
* `teradataml.analytics.SVMSparse`
* `teradataml.analytics.SVMSparsePredict`
* `teradataml.analytics.SentenceExtractor`
* `teradataml.analytics.Sessionize`
* `teradataml.analytics.TF`
* `teradataml.analytics.TFIDF`
* `teradataml.analytics.TextTagger`
* `teradataml.analytics.TextTokenizer`
* `teradataml.analytics.Unpack`
* `teradataml.analytics.VarMax`
#### teradataml 16.20.00.03
* Fixed the garbage collection issue observed with `remove_context()` when context is created using a SQLAlchemy engine.
* Added 4 new Advanced SQL Engine (was NewSQL Engine) analytic functions supported only on Vantage 1.1:
* `Antiselect`, `Pack`, `StringSimilarity`, and `Unpack`.
* Updated the Machine Learning Engine `NGrams` function to work with Vantage 1.1.
#### teradataml 16.20.00.02
* Python version 3.4.x will no longer be supported. The Python versions supported are 3.5.x, 3.6.x, and 3.7.x.
* Major issue with the usage of formula argument in analytic functions with Python3.7 has been fixed, allowing this package to be used with Python3.7 or later.
* Configurable alias name support for analytic functions has been added.
* Support added to create_context (connect to Teradata Vantage) with different logon mechanisms.
Logon mechanisms supported are: 'TD2', 'TDNEGO', 'LDAP' & 'KRB5'.
* copy_to_sql function and DataFrame 'to_sql' methods now provide following additional functionality:
* Create Primary Time Index tables.
* Create set/multiset tables.
* New DataFrame methods are added: 'median', 'var', 'squeeze', 'sort_index', 'concat'.
* DataFrame method 'join' is now updated to make use of ColumnExpressions (df.column_name) for the 'on' clause as opposed to strings.
* Series is supported as a first class object by calling squeeze on DataFrame.
* Methods supported by teradataml Series are: 'head', 'unique', 'name', '\_\_repr__'.
* Binary operations with teradataml Series is not yet supported. Try using Columns from teradataml.DataFrames.
* Sample datasets and commands to load the same have been provided in the function examples.
* New configuration property has been added 'column_casesenitive_handler'. Useful when one needs to play with case sensitive columns.
#### teradataml 16.20.00.01
* New support has been added for Linux distributions: Red Hat 7+, Ubuntu 16.04+, CentOS 7+, SLES12+.
* 16.20.00.01 now has over 100 analytic functions. These functions have been organized into their own packages for better control over which engine to execute the analytic function on. Due to these namespace changes, the old analytic functions have been deprecated and will be removed in a future release. See the Deprecations section in the Teradata Python Package User Guide for more information.
* New DataFrame methods `shape`, `iloc`, `describe`, `get_values`, `merge`, and `tail`.
* New Series methods for NA checking (`isnull`, `notnull`) and string processing (`lower`, `strip`, `contains`).
#### teradataml 16.20.00.00
* `teradataml 16.20.00.00` is the first release version. Please refer to the _Teradata Python Package User Guide_ for a list of Limitations and Usage Considerations.
## Installation and Requirements
### Package Requirements:
* Python 3.5 or later
Note: 32-bit Python is not supported.
### Minimum System Requirements:
* Windows 7 (64Bit) or later
* macOS 10.9 (64Bit) or later
* Red Hat 7 or later versions
* Ubuntu 16.04 or later versions
* CentOS 7 or later versions
* SLES 12 or later versions
* Teradata Vantage Advanced SQL Engine:
* Advanced SQL Engine 16.20 Feature Update 1 or later
* For a Teradata Vantage system with the ML Engine:
* Teradata Machine Learning Engine 08.00.03.01 or later
### Installation
Use pip to install the Teradata Python Package for Advanced Analytics.
Platform | Command
-------------- | ---
macOS/Linux | `pip install teradataml`
Windows | `py -3 -m pip install teradataml`
When upgrading to a new version of the Teradata Python Package, you may need to use pip install's `--no-cache-dir` option to force the download of the new version.
Platform | Command
-------------- | ---
macOS/Linux | `pip install --no-cache-dir -U teradataml`
Windows | `py -3 -m pip install --no-cache-dir -U teradataml`
## Using the Teradata Python Package
Your Python script must import the `teradataml` package in order to use the Teradata Python Package:
```
>>> import teradataml as tdml
>>> from teradataml import create_context, remove_context
>>> create_context(host = 'hostname', username = 'user', password = 'password')
>>> df = tdml.DataFrame('iris')
>>> df
SepalLength SepalWidth PetalLength PetalWidth Name
0 5.1 3.8 1.5 0.3 Iris-setosa
1 6.9 3.1 5.1 2.3 Iris-virginica
2 5.1 3.5 1.4 0.3 Iris-setosa
3 5.9 3.0 4.2 1.5 Iris-versicolor
4 6.0 2.9 4.5 1.5 Iris-versicolor
5 5.0 3.5 1.3 0.3 Iris-setosa
6 5.5 2.4 3.8 1.1 Iris-versicolor
7 6.9 3.2 5.7 2.3 Iris-virginica
8 4.4 3.0 1.3 0.2 Iris-setosa
9 5.8 2.7 5.1 1.9 Iris-virginica
>>> df = df.select(['Name', 'SepalLength', 'PetalLength'])
>>> df
Name SepalLength PetalLength
0 Iris-versicolor 6.0 4.5
1 Iris-versicolor 5.5 3.8
2 Iris-virginica 6.9 5.7
3 Iris-setosa 5.1 1.4
4 Iris-setosa 5.1 1.5
5 Iris-virginica 5.8 5.1
6 Iris-virginica 6.9 5.1
7 Iris-setosa 5.1 1.4
8 Iris-virginica 7.7 6.7
9 Iris-setosa 5.0 1.3
>>> df = df[(df.Name == 'Iris-setosa') & (df.PetalLength > 1.5)]
>>> df
Name SepalLength PetalLength
0 Iris-setosa 4.8 1.9
1 Iris-setosa 5.4 1.7
2 Iris-setosa 5.7 1.7
3 Iris-setosa 5.0 1.6
4 Iris-setosa 5.1 1.9
5 Iris-setosa 4.8 1.6
6 Iris-setosa 4.7 1.6
7 Iris-setosa 5.1 1.6
8 Iris-setosa 5.1 1.7
9 Iris-setosa 4.8 1.6
```
## Documentation
General product information, including installation instructions, is available in the [Teradata Documentation website](https://docs.teradata.com/search/documents?query=package+python+-lake&filters=category~%2522Programming+Reference%2522_%2522User+Guide%2522*prodname~%2522Teradata+Package+for+Python%2522_%2522Teradata+Python+Package%2522&sort=last_update&virtual-field=title_only&content-lang=)
## License
Use of the Teradata Python Package is governed by the *License Agreement for the Teradata Python Package for Advanced Analytics*.
After installation, the `LICENSE` and `LICENSE-3RD-PARTY` files are located in the `teradataml` directory of the Python installation directory.
Raw data
{
"_id": null,
"home_page": "http://www.teradata.com/",
"name": "teradataml",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "Teradata",
"author": "Teradata Corporation",
"author_email": null,
"download_url": null,
"platform": "MacOS X, Windows, Linux",
"description": "## Teradata Python package for Advanced Analytics.\n\nteradataml makes available to Python users a collection of analytic functions that reside on Teradata Vantage. This allows users to perform analytics on Teradata Vantage with no SQL coding. In addition, the teradataml library provides functions for scaling data manipulation and transformation, data filtering and sub-setting, and can be used in conjunction with other open-source python libraries.\n\nFor community support, please visit the [Teradata Community](https://support.teradata.com/community?id=community_forum&sys_id=14fe131e1bf7f304682ca8233a4bcb1d).\n\nFor Teradata customer support, please visit [Teradata Support](https://support.teradata.com/csm).\n\nCopyright 2024, Teradata. All Rights Reserved.\n\n### Table of Contents\n* [Release Notes](#release-notes)\n* [Installation and Requirements](#installation-and-requirements)\n* [Using the Teradata Python Package](#using-the-teradata-python-package)\n* [Documentation](#documentation)\n* [License](#license)\n\n## Release Notes:\n\n#### teradataml 20.00.00.03\n\n* teradataml no longer supports setting the `auth_token` using `set_config_params()`. Users should use `set_auth_token()` to set the token. \n\n* ##### New Features/Functionality\n * ###### teradataml: DataFrame\n * New Function\n * `alias()` - Creates a DataFrame with alias name.\n * New Properties\n * `db_object_name` - Get the underlying database object name, on which DataFrame is created.\n\n * ###### teradataml: GeoDataFrame\n * New Function\n * `alias()` - Creates a GeoDataFrame with alias name.\n\n * ###### teradataml: DataFrameColumn a.k.a. ColumnExpression\n * _Arithmetic Functions_\n * `DataFrameColumn.isnan()` - Function evaluates expression to determine if the floating-point\n argument is a NaN (Not-a-Number) value.\n * `DataFrameColumn.isinf()` - Function evaluates expression to determine if the floating-point\n argument is an infinite number.\n * `DataFrameColumn.isfinite()` - Function evaluates expression to determine if it is a finite\n floating value.\n\n * ###### FeatureStore - handles feature management within the Vantage environment\n * FeatureStore Components\n * Feature - Represents a feature which is used in ML Modeling. \n * Entity - Represents the columns which serves as uniqueness for the data used in ML Modeling. \n * DataSource - Represents the source of Data.\n * FeatureGroup - Collection of Feature, Entity and DataSource.\n * Methods\n * `apply()` - Adds Feature, Entity, DataSource to a FeatureGroup.\n * `from_DataFrame()` - Creates a FeatureGroup from teradataml DataFrame.\n * `from_query()` - Creates a FeatureGroup using a SQL query.\n * `remove()` - Removes Feature, Entity, or DataSource from a FeatureGroup.\n * `reset_labels()` - Removes the labels assigned to the FeatureGroup, that are set using `set_labels()`.\n * `set_labels()` - Sets the Features as labels for a FeatureGroup.\n * Properties\n * `features` - Get the features of a FeatureGroup.\n * `labels` - Get the labels of FeatureGroup.\n * FeatureStore \n * Methods\n * `apply()` - Adds Feature, Entity, DataSource, FeatureGroup to FeatureStore.\n * `archive_data_source()` - Archives a specified DataSource from a FeatureStore.\n * `archive_entity()` - Archives a specified Entity from a FeatureStore.\n * `archive_feature()` - Archives a specified Feature from a FeatureStore.\n * `archive_feature_group()` - Archives a specified FeatureGroup from a FeatureStore. Method archives underlying Feature, Entity, DataSource also.\n * `delete_data_source()` - Deletes an archived DataSource.\n * `delete_entity()` - Deletes an archived Entity.\n * `delete_feature()` - Deletes an archived Feature.\n * `delete_feature_group()` - Deletes an archived FeatureGroup. \n * `get_data_source()` - Get the DataSources associated with FeatureStore.\n * `get_dataset()` - Get the teradataml DataFrame based on Features, Entities and DataSource from FeatureGroup.\n * `get_entity()` - Get the Entity associated with FeatureStore.\n * `get_feature()` - Get the Feature associated with FeatureStore.\n * `get_feature_group()` - Get the FeatureGroup associated with FeatureStore.\n * `list_data_sources()` - List DataSources.\n * `list_entities()` - List Entities.\n * `list_feature_groups()` - List FeatureGroups.\n * `list_features()` - List Features.\n * `list_repos()` - List available repos which are configured for FeatureStore. \n * `repair()` - Repairs the underlying FeatureStore schema on database. \n * `set_features_active()` - Marks the Features as active.\n * `set_features_inactive()` - Marks the Features as inactive.\n * `setup()` - Setup the FeatureStore for a repo.\n * Property\n * `repo` - Property for FeatureStore repo.\n * `grant` - Property to Grant access on FeatureStore to user.\n * `revoke` - Property to Revoke access on FeatureStore from user.\n\n * ###### teradataml: Table Operator Functions\n * `Image2Matrix()` - Converts an image into a matrix.\n\n * ###### teradataml: SQLE Engine Analytic Functions\n * New Analytics Database Analytic Functions:\n * `CFilter()`\n * `NaiveBayes()`\n * `TDNaiveBayesPredict()`\n * `Shap()`\n * `SMOTE()`\n\n * ###### teradataml: Unbounded Array Framework (UAF) Functions\n * New Unbounded Array Framework(UAF) Functions:\n * `CopyArt()`\n\n * ###### General functions\n * Vantage File Management Functions\n * `list_files()` - List the installed files in Database.\n\n * ###### OpensourceML: LightGBM\n * teradataml adds support for lightGBM package through `OpensourceML` (`OpenML`) feature.\n The following functionality is added in the current release:\n * `td_lightgbm` - Interface object to run lightgbm functions and classes through Teradata Vantage.\n Example usage below:\n ```\n from teradataml import td_lightgbm, DataFrame\n\n df_train = DataFrame(\"multi_model_classification\")\n\n feature_columns = [\"col1\", \"col2\", \"col3\", \"col4\"]\n label_columns = [\"label\"]\n part_columns = [\"partition_column_1\", \"partition_column_2\"]\n\n df_x = df_train.select(feature_columns)\n df_y = df_train.select(label_columns)\n\n # Dataset creation.\n # Single model case.\n obj_s = td_lightgbm.Dataset(df_x, df_y, silent=True, free_raw_data=False)\n\n # Multi model case.\n obj_m = td_lightgbm.Dataset(df_x, df_y, free_raw_data=False, partition_columns=part_columns)\n obj_m_v = td_lightgbm.Dataset(df_x, df_y, free_raw_data=False, partition_columns=part_columns)\n\n ## Model training.\n # Single model case.\n opt = td_lightgbm.train(params={}, train_set = obj_s, num_boost_round=30)\n\n opt.predict(data=df_x, num_iteration=20, pred_contrib=True)\n\n # Multi model case.\n opt = td_lightgbm.train(params={}, train_set = obj_m, num_boost_round=30,\n callbacks=[td_lightgbm.record_evaluation(rec)],\n valid_sets=[obj_m_v, obj_m_v])\n\n # Passing `label` argument to get it returned in output DataFrame.\n opt.predict(data=df_x, label=df_y, num_iteration=20)\n\n ```\n * Added support for accessing scikit-learn APIs using exposed inteface object `td_lightgbm`.\n\n Refer Teradata Python Package User Guide for more details of this feature, arguments, usage, examples and supportability in Vantage.\n\n * ###### teradataml: Functions\n * `register()` - Registers a user defined function (UDF).\n * `call_udf()` - Calls a registered user defined function (UDF) and returns ColumnExpression.\n * `list_udfs()` - List all the UDFs registered using 'register()' function.\n * `deregister()` - Deregisters a user defined function (UDF).\n\n * ###### teradataml: Options\n * Configuration Options\n * `table_operator` - Specifies the name of table operator.\n\n* ##### Updates\n * ###### General functions\n * `set_auth_token()` - Added `base_url` parameter which accepts the CCP url. \n 'ues_url' will be deprecated in future and users\n will need to specify 'base_url' instead.\n\n * ###### teradataml: DataFrame function\n * `join()`\n * Now supports compound ColumExpression having more than one binary operator in `on` argument.\n * Now supports ColumExpression containing FunctionExpression(s) in `on` argument.\n * self-join now expects aliased DataFrame in `other` argument.\n\n * ###### teradataml: GeoDataFrame function\n * `join()`\n * Now supports compound ColumExpression having more than one binary operator in `on` argument.\n * Now supports ColumExpression containing FunctionExpression(s) in `on` argument.\n * self-join now expects aliased DataFrame in `other` argument.\n\n * ###### teradataml: Unbounded Array Framework (UAF) Functions\n * `SAX()` - Default value added for `window_size` and `output_frequency`.\n * `DickeyFuller()`\n * Supports TDAnalyticResult as input.\n * Default value added for `max_lags`.\n * Removed parameter `drift_trend_formula`.\n * Updated permitted values for `algorithm`.\n\n * ##### teradataml: AutoML\n * `AutoML`, `AutoRegressor` and `AutoClassifier`\n * Now supports DECIMAL datatype as input.\n\n * ##### teradataml: SQLE Engine Analytic Functions\n * `TextParser()`\n * Argument name `covert_to_lowercase` changed to `convert_to_lowercase`.\n\n* ##### Bug Fixes\n * `db_list_tables()` now returns correct results when '%' is used.\n\n#### teradataml 20.00.00.02\n\n* teradataml will no longer be supported with SQLAlchemy < 2.0.\n* teradataml no longer shows the warnings from Vantage by default. \n * Users should set `display.suppress_vantage_runtime_warnings` to `False` to display warnings.\n\n* ##### New Features/Functionality\n * ##### teradataml: SQLE Engine Analytic Functions\n * New Analytics Database Analytic Functions:\n * `TFIDF()`\n * `Pivoting()`\n * `UnPivoting()`\n * New Unbounded Array Framework(UAF) Functions:\n * `AutoArima()`\n * `DWT()`\n * `DWT2D()`\n * `FilterFactory1d()` \n * `IDWT()`\n * `IDWT2D()`\n * `IQR()`\n * `Matrix2Image()`\n * `SAX()`\n * `WindowDFFT()`\n * ###### teradataml: Functions\n * `udf()` - Creates a user defined function (UDF) and returns ColumnExpression.\n * `set_session_param()` is added to set the database session parameters. \n * `unset_session_param()` is added to unset database session parameters.\n\n * ###### teradataml: DataFrame\n * `materialize()` - Persists DataFrame into database for current session.\n * `create_temp_view()` - Creates a temporary view for session on the DataFrame.\n\n * ###### teradataml DataFrameColumn a.k.a. ColumnExpression\n * _Date Time Functions_\n * `DataFrameColumn.to_timestamp()` - Converts string or integer value to a TIMESTAMP data type or TIMESTAMP WITH TIME ZONE data type.\n * `DataFrameColumn.extract()` - Extracts date component to a numeric value.\n * `DataFrameColumn.to_interval()` - Converts a numeric value or string value into an INTERVAL_DAY_TO_SECOND or INTERVAL_YEAR_TO_MONTH value.\n * _String Functions_\n * `DataFrameColumn.parse_url()` - Extracts a part from a URL.\n * _Arithmetic Functions_\n * `DataFrameColumn.log` - Returns the logarithm value of the column with respect to 'base'.\n\n * ##### teradataml: AutoML\n * New methods added for `AutoML()`, `AutoRegressor()` and `AutoClassifier()`:\n * `evaluate()` - Performs evaluation on the data using the best model or the model of users choice\n from the leaderboard.\n * `load()`: Loads the saved model from database.\n * `deploy()`: Saves the trained model inside database.\n * `remove_saved_model()`: Removes the saved model in database.\n * `model_hyperparameters()`: Returns the hyperparameter of fitted or loaded models.\n\n* ##### Updates\n * ##### teradataml: AutoML\n * `AutoML()`, `AutoRegressor()`\n * New performance metrics added for task type regression i.e., \"MAPE\", \"MPE\", \"ME\", \"EV\", \"MPD\" and \"MGD\".\n * `AutoML()`, `AutoRegressor()` and `AutoClassifier`\n * New arguments added: `volatile`, `persist`.\n * `predict()` - Data input is now mandatory for generating predictions. Default model \n evaluation is now removed.\n * `DataFrameColumn.cast()`: Accepts 2 new arguments `format` and `timezone`.\n * `DataFrame.assign()`: Accepts ColumnExpressions returned by `udf()`.\n\n * ##### teradataml: Options\n * `set_config_params()`\n * Following arguments will be deprecated in the future:\n * `ues_url`\n * `auth_token`\n\n * #### teradata DataFrame\n * `to_pandas()` - Function returns the pandas dataframe with Decimal columns types as float instead of object.\n If user want datatype to be object, set argument `coerce_float` to False.\n\n * ###### Database Utility\n * `list_td_reserved_keywords()` - Accepts a list of strings as argument.\n\n * ##### Updates to existing UAF Functions:\n * `ACF()` - `round_results` parameter removed as it was used for internal testing.\n * `BreuschGodfrey()` - Added default_value 0.05 for parameter `significance_level`.\n * `GoldfeldQuandt()` - \n * Removed parameters `weights` and `formula`.\n Replaced parameter `orig_regr_paramcnt` with `const_term`.\n Changed description for parameter `algorithm`. Please refer document for more details.\n * Note: This will break backward compatibility.\n * `HoltWintersForecaster()` - Default value of parameter `seasonal_periods` removed.\n * `IDFFT2()` - Removed parameter `output_fmt_row_major` as it is used for internal testing.\n * `Resample()` - Added parameter `output_fmt_index_style`.\n\n* ##### Bug Fixes\n * KNN `predict()` function can now predict on test data which does not contain target column.\n * Metrics functions are supported on the Lake system.\n * The following OpensourceML functions from different sklearn modules in single model case are fixed.\n * `sklearn.ensemble`:\n * ExtraTreesClassifier - `apply()`\n * ExtraTreesRegressor - `apply()`\n * RandomForestClassifier - `apply()`\n * RandomForestRegressor - `apply()`\n * `sklearn.impute`:\n * SimpleImputer - `transform()`, `fit_transform()`, `inverse_transform()`\n * MissingIndicator - `transform()`, `fit_transform()`\n * `sklearn.kernel_approximations`:\n * Nystroem - `transform()`, `fit_transform()`\n * PolynomialCountSketch - `transform()`, `fit_transform()`\n * RBFSampler - `transform()`, `fit_transform()`\n * `sklearn.neighbors`:\n * KNeighborsTransformer - `transform()`, `fit_transform()`\n * RadiusNeighborsTransformer - `transform()`, `fit_transform()`\n * `sklearn.preprocessing`:\n * KernelCenterer - `transform()`\n * OneHotEncoder - `transform()`, `inverse_transform()`\n * The following OpensourceML functions from different sklearn modules in multi model case are fixed.\n * `sklearn.feature_selection`:\n * SelectFpr - `transform()`, `fit_transform()`, `inverse_transform()`\n * SelectFdr - `transform()`, `fit_transform()`, `inverse_transform()`\n * SelectFromModel - `transform()`, `fit_transform()`, `inverse_transform()`\n * SelectFwe - `transform()`, `fit_transform()`, `inverse_transform()`\n * RFECV - `transform()`, `fit_transform()`, `inverse_transform()`\n * `sklearn.clustering`:\n * Birch - `transform()`, `fit_transform()`\n * OpensourceML returns teradataml objects for model attributes and functions instead of sklearn\n objects so that the user can perform further operations like `score()`, `predict()` etc on top\n of the returned objects.\n * AutoML `predict()` function now generates correct ROC-AUC value for positive class.\n * `deploy()` method of `Script` and `Apply` classes retries model deployment if there is any \n intermittent network issues.\n\n#### teradataml 20.00.00.01\n* teradataml no longer supports Python versions less than 3.8.\n\n* ##### New Features/Functionality\n * ##### Personal Access Token (PAT) support in teradataml\n * `set_auth_token()` - teradataml now supports authentication via PAT in addition to \n OAuth 2.0 Device Authorization Grant (formerly known as the Device Flow).\n * It accepts UES URL, Personal AccessToken (PAT) and Private Key file generated from VantageCloud Lake Console \n and optional argument `username` and `expiration_time` in seconds. \n\n* ##### Updates\n * ##### teradataml: SQLE Engine Analytic Functions\n * `ANOVA()`\n * New arguments added: `group_name_column`, `group_value_name`, `group_names`, `num_groups` for data containing group values and group names. \n * `FTest()`\n * New arguments added: `sample_name_column`, `sample_name_value`, `first_sample_name`, `second_sample_name`.\n * `GLM()`\n * Supports stepwise regression and accept new arguments `stepwise_direction`, `max_steps_num` and `initial_stepwise_columns`.\n * New arguments added: `attribute_data`, `parameter_data`, `iteration_mode` and `partition_column`.\n * `GetFutileColumns()`\n * Arguments `category_summary_column` and `threshold_value` are now optional.\n * `KMeans()`\n * New argument added: `initialcentroids_method`.\n * `NonLinearCombineFit()`\n * Argument `result_column` is now optional.\n * `ROC()`\n * Argument `positive_class` is now optional. \n * `SVMPredict()`\n * New argument added: `model_type`. \n * `ScaleFit()`\n * New arguments added: `ignoreinvalid_locationscale`, `unused_attributes`, `attribute_name_column`, `attribute_value_column`.\n * Arguments `attribute_name_column`, `attribute_value_column` and `target_attributes` are supported for sparse input.\n * Arguments `attribute_data`, `parameter_data` and `partition_column` are supported for partitioning.\n * `ScaleTransform()`\n * New arguments added: `attribute_name_column` and `attribute_value_column` support for sparse input.\n * `TDGLMPredict()`\n * New arguments added: `family` and `partition_column`.\n * `XGBoost()`\n * New argument `base_score` is added for initial prediction value for all data points.\n * `XGBoostPredict()`\n * New argument `detailed` is added for detailed information of each prediction.\n * `ZTest()`\n * New arguments added: `sample_name_column`, `sample_value_column`, `first_sample_name` and `second_sample_name`.\n * ##### teradataml: AutoML\n * `AutoML()`, `AutoRegressor()` and `AutoClassifier()`\n * New argument `max_models` is added as an early stopping criterion to limit the maximum number of models to be trained.\n * ##### teradataml: DataFrame functions\n * `DataFrame.agg()` \n * Accepts ColumnExpressions and list of ColumnExpressions as arguments.\n * ##### teradataml: General Functions\n * Data Transfer Utility\n * `fastload()` - Improved error and warning table handling with below-mentioned new arguments.\n * `err_staging_db`\n * `err_tbl_name`\n * `warn_tbl_name`\n * `err_tbl_1_suffix`\n * `err_tbl_2_suffix`\n * `fastload()` - Change in behaviour of `save_errors` argument.\n When `save_errors` is set to `True`, error information will be available in two persistent tables `ERR_1` and `ERR_2`.\n When `save_errors` is set to `False`, error information will be available in single pandas dataframe.\n * Garbage collector location is now configurable. \n User can set configure.local_storage to a desired location.\n\n* ##### Bug Fixes\n * UAF functions now work if the database name has special characters.\n * OpensourceML can now read and process NULL/nan values.\n * Boolean values output will now be returned as VARBYTE column with 0 or 1 values in OpensourceML.\n * Fixed bug for `Apply`'s `deploy()`.\n * Issue with volatile table creation is fixed where it is created in the right database, i.e., user's spool space, regardless of the temp database specified.\n * `ColumnTransformer` function now processes its arguments in the order they are passed.\n\n#### teradataml 20.00.00.00\n* ##### New Features/Functionality\n * ###### teradataml OpenML: Run Opensource packages through Teradata Vantage\n `OpenML` dynamically exposes opensource packages through Teradata Vantage. `OpenML` provides an\n interface object through which exposed classes and functions of opensource packages can be accessed\n with the same syntax and arguments. \n The following functionality is added in the current release:\n * `td_sklearn` - Interface object to run scikit-learn functions and classes through Teradata Vantage.\n Example usage below:\n ```\n from teradataml import td_sklearn, DataFrame\n\n df_train = DataFrame(\"multi_model_classification\")\n\n feature_columns = [\"col1\", \"col2\", \"col3\", \"col4\"]\n label_columns = [\"label\"]\n part_columns = [\"partition_column_1\", \"partition_column_2\"]\n\n linear_svc = td_sklearn.LinearSVC()\n ```\n * `OpenML` is supported in both Teradata Vantage Enterprise and Teradata Vantage Lake.\n * Argument Support:\n * `Use of X and y arguments` - Scikit-learn users are familiar with using `X` and `y` as argument names\n which take data as pandas DataFrames, numpy arrays or lists etc. However, in OpenML, we pass \n teradataml DataFrames for arguments `X` and `y`.\n ```\n df_x = df_train.select(feature_columns)\n df_y = df_train.select(label_columns)\n\n linear_svc = linear_svc.fit(X=df_x, y=df_y)\n ```\n * `Additional support for data, feature_columns, label_columns and group_columns arguments` -\n Apart from traditional arguments, OpenML supports additional arguments - `data`,\n `feature_columns`, `label_columns` and `group_columns`. These are used as alternatives to `X`, `y`\n and `groups`.\n ```\n linear_svc = linear_svc.fit(data=df_train, feature_columns=feature_columns, label_colums=label_columns)\n ```\n * `Support for classification and regression metrics` - Metrics functions for classification and\n regression in `sklearn.metrics` module are supported. Other metrics functions' support will be added\n in future releases.\n * `Distributed Modeling and partition_columns argument support` - Existing scikit-learn supports \n only single model generation. However, OpenML supports both single model use case and distributed\n (multi) model use case. For this, user has to additionally pass `partition_columns` argument to \n existing `fit()`, `predict()` or any other function to be run. This will generate multiple models\n for multiple partitions, using the data in corresponding partition.\n ```\n df_x_1 = df_train.select(feature_columns + part_columns)\n linear_svc = linear_svc.fit(X=df_x_1, y=df_y, partition_columns=part_columns) \n ```\n * `Support for load and deploy models` - OpenML provides additional support for saving (deploying) the\n trained models. These models can be loaded later to perform operations like prediction, score etc. The\n following functions are provided by OpenML:\n * `<obj>.deploy()` - Used to deploy/save the model created and/or trained by OpenML.\n * `td_sklearn.deploy()` - Used to deploy/save the model created and/or trained outside teradataml.\n * `td_sklearn.load()` - Used to load the saved models.\n\n <br>Refer Teradata Python Package User Guide for more details of this feature, arguments, usage, examples and supportability in both VantageCloud Enterprise and VantageCloud Lake.\n\n * ###### teradataml: AutoML - Automated end to end Machine Learning flow.\n AutoML is an approach to automate the process of building, training, and validating machine learning models. \n It involves automation of various aspects of the machine learning workflow, such as feature exploration, \n feature engineering, data preparation, model training and evaluation for given dataset.\n teradataml AutoML feature offers best model identification, model leaderboard generation, parallel execution, \n early stopping feature, model evaluation, model prediction, live logging, customization on default process.\n * `AutoML`\n AutoML is a generic algorithm that supports all three tasks, i.e. 'Regression',\n 'Binary Classification' and 'Multiclass Classification'. \n * Methods of AutoML\n * `__init__()` - Instantiate an object of AutoML with given parameters.\n * `fit()` - Perform fit on specified data and target column.\n * `leaderboard()` - Get the leaderboard for the AutoML. Presents diverse models, feature \n selection method, and performance metrics.\n * `leader()` - Show best performing model and its details such as feature \n selection method, and performance metrics.\n * `predict()` - Perform prediction on the data using the best model or the model of users \n choice from the leaderboard.\n * `generate_custom_config()` - Generate custom config JSON file required for customized \n run of AutoML.\n * `AutoRegressor`\n AutoRegressor is a special purpose AutoML feature to run regression specific tasks. \n * Methods of AutoRegressor\n * `__init__()` - Instantiate an object of AutoRegressor with given parameters.\n * `fit()` - Perform fit on specified data and target column.\n * `leaderboard()` - Get the leaderboard for the AutoRegressor. Presents diverse models, feature \n selection method, and performance metrics.\n * `leader()` - Show best performing model and its details such as feature \n selection method, and performance metrics.\n * `predict()` - Perform prediction on the data using the best model or the model of users \n choice from the leaderboard.\n * `generate_custom_config()` - Generate custom config JSON file required for customized \n run of AutoRegressor.\n * `AutoClassifier`\n AutoClassifier is a special purpose AutoML feature to run classification specific tasks.\n * Methods of AutoClassifier\n * `__init__()` - Instantiate an object of AutoClassifier with given parameters.\n * `fit()` - Perform fit on specified data and target column.\n * `leaderboard()` - Get the leaderboard for the AutoClassifier. Presents diverse models, feature \n selection method, and performance metrics.\n * `leader()` - Show best performing model and its details such as feature \n selection method, and performance metrics.\n * `predict()` - Perform prediction on the data using the best model or the model of users \n choice from the leaderboard.\n * `generate_custom_config()` - Generate custom config JSON file required for customized \n run of AutoClassifier.\n\n * ###### teradataml: DataFrame\n * `fillna` - Replace the null values in a column with the value specified. \n * Data Manipulation\n * `cube()`- Analyzes data by grouping it into multiple dimensions.\n * `rollup()` - Analyzes a set of data across a single dimension with more than one level of detail.\n * `replace()` - Replaces the values for columns.\n\n * ###### teradataml: Script and Apply\n * `deploy()` - Function deploys the model, generated after `execute_script()`, in database or user\n environment in lake. The function is available in both Script and Apply.\n\n * ###### teradataml: DataFrameColumn\n * `fillna` - Replaces every occurrence of null value in column with the value specified.\n\n* ###### teradataml DataFrameColumn a.k.a. ColumnExpression\n * _Date Time Functions_\n * `DataFrameColumn.week_start()` - Returns the first date or timestamp of the week that begins immediately before the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.week_begin()` - It is an alias for `DataFrameColumn.week_start()` function.\n * `DataFrameColumn.week_end()` - Returns the last date or timestamp of the week that ends immediately after the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.month_start()` - Returns the first date or timestamp of the month that begins immediately before the specified date or timestamp value in a column or as a literal.\n * `DataFrameColumn.month_begin()` - It is an alias for `DataFrameColumn.month_start()` function.\n * `DataFrameColumn.month_end()` - Returns the last date or timestamp of the month that ends immediately after the specified date or timestamp value in a column or as a literal.\n * `DataFrameColumn.year_start()` - Returns the first date or timestamp of the year that begins immediately before the specified date or timestamp value in a column or as a literal.\n * `DataFrameColumn.year_begin()` - It is an alias for `DataFrameColumn.year_start()` function.\n * `DataFrameColumn.year_end()` - Returns the last date or timestamp of the year that ends immediately after the specified date or timestamp value in a column or as a literal.\n * `DataFrameColumn.quarter_start()` - Returns the first date or timestamp of the quarter that begins immediately before the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.quarter_begin()` - It is an alias for `DataFrameColumn.quarter_start()` function.\n * `DataFrameColumn.quarter_end()` - Returns the last date or timestamp of the quarter that ends immediately after the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.last_sunday()` - Returns the date or timestamp of Sunday that falls immediately before the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.last_monday()` - Returns the date or timestamp of Monday that falls immediately before the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.last_tuesday()` - Returns the date or timestamp of Tuesday that falls immediately before the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.last_wednesday()` - Returns the date or timestamp of Wednesday that falls immediately before specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.last_thursday()`- Returns the date or timestamp of Thursday that falls immediately before specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.last_friday()` - Returns the date or timestamp of Friday that falls immediately before specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.last_saturday()` - Returns the date or timestamp of Saturday that falls immediately before specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.day_of_week()` - Returns the number of days from the beginning of the week to the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.day_of_month()` - Returns the number of days from the beginning of the month to the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.day_of_year()` - Returns the number of days from the beginning of the year to the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.day_of_calendar()` - Returns the number of days from the beginning of the business calendar to the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.week_of_month()` - Returns the number of weeks from the beginning of the month to the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.week_of_quarter()` - Returns the number of weeks from the beginning of the quarter to the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.week_of_year()` - Returns the number of weeks from the beginning of the year to the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.week_of_calendar()` - Returns the number of weeks from the beginning of the calendar to the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.month_of_year()` - Returns the number of months from the beginning of the year to the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.month_of_calendar()` - Returns the number of months from the beginning of the calendar to the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.month_of_quarter()` - Returns the number of months from the beginning of the quarter to the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.quarter_of_year()` - Returns the number of quarters from the beginning of the year to the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.quarter_of_calendar()` - Returns the number of quarters from the beginning of the calendar to the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.year_of_calendar()` - Returns the year of the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.day_occurrence_of_month()` - Returns the nth occurrence of the weekday in the month for the date to the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.year()` - Returns the integer value for year in the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.month()` - Returns the integer value for month in the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.hour()` - Returns the integer value for hour in the specified timestamp value in a column as a literal.\n * `DataFrameColumn.minute()` - Returns the integer value for minute in the specified timestamp value in a column as a literal.\n * `DataFrameColumn.second()` - Returns the integer value for seconds in the specified timestamp value in a column as a literal.\n * `DataFrameColumn.week()` - Returns the number of weeks from the beginning of the year to the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.next_day()` - Returns the date of the first weekday specified as 'day_value' that is later than the specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.months_between()` - Returns the number of months between value in specified date or timestamp value in a column as a literal and date or timestamp value in argument.\n * `DataFrameColumn.add_months()` - Adds an integer number of months to specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.oadd_months()` - Adds an integer number of months, date or timestamp value in specified date or timestamp value in a column as a literal.\n * `DataFrameColumn.to_date()` - Function converts a string-like representation of a DATE or PERIOD type to Date type.\n * _String Functions_\n * `DataFrameColumn.concat()` - Function to concatenate the columns with a separator.\n * `DataFrameColumn.like()` - Function to match the string pattern. String match is case sensitive. \n * `DataFrameColumn.ilike()` - Function to match the string pattern. String match is not case sensitive.\n * `DataFrameColumn.substr()` - Returns the substring from a string column. \n * `DataFrameColumn.startswith()` - Function to check if the column value starts with the specified value or not. \n * `DataFrameColumn.endswith()` - Function to check if the column value ends with the specified value or not.\n * `DataFrameColumn.format()` - Function to format the values in column based on formatter.\n * `DataFrameColumn.to_char()` - Function converts numeric type or datetype to character type.\n * `DataFrameColumn.trim()` - Function trims the string values in the column.\n * _Regular Arithmetic Functions_\n * `DataFrameColumn.cbrt()` - Computes the cube root of values in the column.\n * `DataFrameColumn.hex()` - Computes the Hexadecimal from decimal for the values in the column.\n * `DataframeColumn.hypot()` - Computes the decimal from Hexadecimal for the values in the column.\n * `DataFrameColumn.unhex()` - computes the hypotenuse for the values between two columns.\n * _Bit Byte Manipulation Functions_\n * `DataFrameColumn.from_byte()` - Encodes a sequence of bits into a sequence of characters.\n * _Comparison Functions_\n * `DataFrameColumn.greatest()` - Returns the greatest values from columns.\n * `DataFrameColumn.least()` - Returns the least values from columns.\n * Behaviour of `DataFrameColumn.replace()` is changed.\n * Behaviour of `DataFrameColumn.to_byte()` is changed. It now decodes a sequence of characters in a given encoding into a sequence of bits.\n * Behaviour of `DataFrameColumn.trunc()` is changed. It now accepts Date type columns. \n\n* ##### Bug Fixes\n * Argument `url_encode` is no longer used in `create_context()` and is deprecated. \n * **Important notes**\n * Users do not need to encode password even if password contain special characters.\n * Pass the password to the `create_context()` function argument `password` as it is without changing special characters.\n * `fillna()` in VAL transformation allows to replace NULL values with empty string.\n\n* ##### Updates\n * Support for following deprecated functionality is removed:\n * ML Engine functions\n * STO and APPLY sandbox feature support for testing the script. \n * sandbox_container_utils is removed. Following methods can no longer be used:\n * `setup_sandbox_env()`\n * `copy_files_from_container()`\n * `cleanup_sandbox_env()`\n * Model Cataloging APIs can no longer be used: \n * `describe_model()`\n * `delete_model()`\n * `list_models()`\n * `publish_model()`\n * `retrieve_model()`\n * `save_model()`\n * `DataFrame.join()`\n * Arguments `lsuffix` and `rsuffix` now add suffixes to new column names for join operation.\n * `DataFrame.describe()`\n * New argument `columns` is added to generate statistics on only those columns instead of all applicable columns.\n * `DataFrame.groupby()`\n * Supports `CUBE` and `ROLLUP` with additional optional argument `option`.\n * `DataFrame.column.window()` \n * Supports ColumnExpressions for `partition_columns` and `order_columns` arguments.\n * `DataFrame.column.contains()` allows ColumnExpressions for `pattern` argument.\n * `DataFrame.window()` \n * Supports ColumnExpressions for `partition_columns` and `order_columns` arguments.\n\n#### teradataml 17.20.00.07\n* ##### New Features/Functionality\n * ###### Open Analytics Framework (OpenAF) APIs:\n * Manage all user environments.\n * `create_env()`:\n * new argument `conda_env` is added to create a conda environment.\n * `list_user_envs()`:\n * User can list conda environment(s) by using filter with new argument `conda_env`.\n * Conda environment(s) can be managed using APIs for installing , updating, removing files/libraries. \n* ##### Bug Fixes\n * `columns` argument for `FillNa` function is made optional.\n\n#### teradataml 17.20.00.06\n* ##### New Features/Functionality\n* ###### teradataml DataFrameColumn a.k.a. ColumnExpression\n * `ColumnExpression.nulls_first()` - Displays NULL values at first.\n * `ColumnExpression.nulls_last()` - Displays NULL values at last.\n * _Bit Byte Manipulation Functions_\n * `DataFrameColumn.bit_and()` - Returns the logical AND operation on the bits from\n the column and corresponding bits from the argument.\n * `DataFrameColumn.bit_get()` - Returns the bit specified by input argument from the column and \n returns either 0 or 1 to indicate the value of that bit.\n * `DataFrameColumn.bit_or()` - Returns the logical OR operation on the bits from the column and \n corresponding bits from the argument.\n * `DataFrameColumn.bit_xor()` - Returns the bitwise XOR operation on the binary representation of the\n column and corresponding bits from the argument.\n * `DataFrameColumn.bitand()` - It is an alias for `DataFrameColumn.bit_and()` function.\n * `DataFrameColumn.bitnot()` - Returns a bitwise complement on the binary representation of the column.\n * `DataFrameColumn.bitor()` - It is an alias for `DataFrameColumn.bit_or()` function.\n * `DataFrameColumn.bitwise_not()` - It is an alias for `DataFrameColumn.bitnot()` function.\n * `DataFrameColumn.bitwiseNOT()` - It is an alias for `DataFrameColumn.bitnot()` function.\n * `DataFrameColumn.bitxor()` - It is an alias for `DataFrameColumn.bit_xor()` function.\n * `DataFrameColumn.countset()` - Returns the count of the binary bits within the column that are either set to 1 \n or set to 0, depending on the input argument value.\n * `DataFrameColumn.getbit()` - It is an alias for `DataFrameColumn.bit_get()` function.\n * `DataFrameColumn.rotateleft()` - Returns an expression rotated to the left by the specified number of bits,\n with the most significant bits wrapping around to the right.\n * `DataFrameColumn.rotateright()` - Returns an expression rotated to the right by the specified number of bits,\n with the least significant bits wrapping around to the left.\n * `DataFrameColumn.setbit()` - Sets the value of the bit specified by input argument to the value\n of column.\n * `DataFrameColumn.shiftleft()` - Returns the expression when value in column is shifted by the specified\n number of bits to the left.\n * `DataFrameColumn.shiftright()` - Returns the expression when column expression is shifted by the specified\n number of bits to the right.\n * `DataFrameColumn.subbitstr()` - Extracts a bit substring from the column expression based on the specified \n bit position.\n * `DataFrameColumn.to_byte()` - Converts a numeric data type to the Vantage byte representation\n (byte value) of the column expression value.\n\n * _Regular Expression Functions_\n * `DataFrameColumn.regexp_instr()` - Searches string value in column for a match to value specified in argument.\n * `DataFrameColumn.regexp_replace()` - Replaces the portions of string value in a column that matches the value \n specified regex string and replaces with the replace string.\n * `DataFrameColumn.regexp_similar()` - Compares value in column to value in argument and returns integer value.\n * `DataFrameColumn.regexp_substr()` - Extracts a substring from column that matches a regular expression \n specified in the input argument.\n\n* ###### Open Analytics Framework (OpenAF) APIs:\n * Manage all user environments.\n * `create_env()`:\n * User can create one or more user environments using newly added argument `template` by providing specifications in template json file. New feature allows user to create complete user environment, including file and library installation, in just single function call.\n * UserEnv Class \u2013 Manage individual user environment.\n * Properties:\n * `models` - Supports listing of models in user environment.\n * Methods:\n * `install_model()` - Install a model in user environment.\n * `uninstall_model()` - Uninstall a model from user environment.\n * `snapshot()`- Take the snapshot of the user environment.\n\n* ###### teradataml: Bring Your Own Model\n * _New Functions_\n * `DataRobotPredict()` - Score the data in Vantage using the model trained externally in datarobot and stored \n in Vantage.\n\n* ##### Updates\n * `DataFrame.describe()`\n * Method now accepts an argument `statistics`, which specifies the aggregate operation to be performed. \n * `DataFrame.sort()` \n * Method now accepts ColumnExpressions as well.\n * Enables sorting using NULLS FIRST and NULLS LAST.\n * `view_log()` downloads the Apply query logs based on query id.\n * Arguments which accepts floating numbers will accept integers also for `Analytics Database Analytic Functions`.\n * Argument `ignore_nulls` added to `DataFrame.plot()` to ignore the null values while plotting the data.\n * `Dataframe.sample()` \n * Method supports column stratification. \n\n* ##### Bug Fixes\n * `DataFrameColumn.cast()` accepts all teradatasqlalchemy types.\n * Minor bug fix related to `DataFrame.merge()`.\n\n#### teradataml 17.20.00.05\n* ##### New Features/Functionality\n * ###### teradataml: Hyperparameter-Tuning - Technique to identify best model parameters.\n Hyperparameter tuning is an\u00a0optimization method to determine the optimal\u00a0set of \n hyperparameters\u00a0for the given\u00a0dataset and learning model. teradataml hyperparameter tuning feature\n offers best model identification, parallel execution, early stopping feature, best data identification, \n model evaluation, model prediction, live logging, input data hyper-parameterization, input data sampling, \n numerous scoring functions, hyper-parameterization for non-model trainer functions. \n * `GridSearch`\n GridSearch is an exhaustive search algorithm that covers all possible\n parameter values to identify optimal hyperparameters.\n * Methods of GridSearch\n * `__init__()` - Instantiate an object of GridSearch for given model function and parameters.\n * `evaluate()` - Function to perform evaluation on the given teradataml DataFrame using default model.\n * `fit()` - Function to perform hyperparameter-tuning for given hyperparameters and model on teradataml DataFrame.\n * `get_error_log()` - Useful to get the error log if model execution failed, using the model identifier.\n * `get_input_data()` - Useful to get the input data using the data identifier, when input data is also parameterized.\n * `get_model()` - Returns the trained model for the given model identifier.\n * `get_parameter_grid()` - Returns the hyperparameter space used for hyperparameter optimization.\n * `is_running()` - Returns the execution status of hyperaparameter tuning.\n * `predict()` - Function to perform prediction on the given teradataml DataFrame using default model.\n * `set_model()` - Function to update the default model.\n * Properties of GridSearch\n * `best_data_id` - Returns the best data identifier used for model training.\n * `best_model` - Returns the best trained model.\n * `best_model_id` - Returns the identifier for best model.\n * `best_params_` - Returns the best set of hyperparameter.\n * `best_sampled_data_` - Returns the best sampled data used to train the best model.\n * `best_score_` - Returns the best trained model score.\n * `model_stats` - Returns the model evaluation reports.\n * `models` - Returns the metadata of all the models.\n * `RandomSearch`\n RandomSearch algorithm performs random sampling on hyperparameter \n space to identify optimal hyperparameters.\n * Methods of RandomSearch\n * `__init__()` - Instantiate an object of RandomSearch for given model function and parameters.\n * `evaluate()` - Function to perform evaluation on the given teradataml DataFrame using default model.\n * `fit()` - Function to perform hyperparameter-tuning for given hyperparameters and model on teradataml DataFrame.\n * `get_error_log()` - Useful to get the error log if model execution failed, using the model identifier.\n * `get_input_data()` - Useful to get the input data using the data identifier, when input data is also parameterized.\n * `get_model()` - Returns the trained model for the given model identifier.\n * `get_parameter_grid()` - Returns the hyperparameter space used for hyperparameter optimization.\n * `is_running()` - Returns the execution status of hyperaparameter tuning.\n * `predict()` - Function to perform prediction on the given teradataml DataFrame using default model.\n * `set_model()` - Function to update the default model.\n * Properties of GridSearch \n * `best_data_id` - Returns the best data identifier used for model training.\n * `best_model` - Returns the best trained model.\n * `best_model_id` - Returns the identifier for best model.\n * `best_params_` - Returns the best set of hyperparameter.\n * `best_sampled_data_` - Returns the best sampled data used to train the best model.\n * `best_score_` - Returns the best trained model score.\n * `model_stats` - Returns the model evaluation reports.\n * `models` - Returns the metadata of all the models.\n\n * ###### teradataml: Analytic Functions\n teradataml currently has different functions to generate a model, predict, transform and evaluate. All these functions are needed to be invoked individually, i.e., predict(), evaluate(), transform() cannot be invoked using the model trainer function output. Enhancement done to this feature now enables user to invoke these functions as methods of the model trainer function. Below is the list of functions, updated with this enhancement:\n * Analytics Database Analytic Functions\n * `BincodeFit()` - Supports `transform()` method.\n * `DecisionForest()` - Supports `predict()`, `evaluate()` methods.\n * `Fit()` - Supports `transform()` method. \n * `GLM()` - Supports `predict()`, `evaluate()` methods. \n * `GLMPerSegment()` - Supports `predict()`, `evaluate()` methods. \n * `KMeans()` - Supports `predict()` method.\n * `KNN()` - Supports `predict()`, `evaluate()` methods. \n * `NaiveBayesTextClassifierTrainer()` - Supports `predict()`, `evaluate()` methods. \n * `NonLinearCombineFit()` - Supports `transform()` method. \n * `OneClassSVM()` - Supports `predict()` method.\n * `OneHotEncodingFit()` - Supports `transform()` method. \n * `OrdinalEncodingFit()` - Supports `transform()` method. \n * `OutlierFilterFit()` - Supports `transform()` method. \n * `PolynomialFeaturesFit()` - Supports `transform()` method. \n * `RandomProjectionFit()` - Supports `transform()` method. \n * `RowNormalizeFit()` - Supports `transform()` method. \n * `ScaleFit()` - Supports `transform()` method. \n * `SimpleImputeFit()` - Supports `transform()` method. \n * `SVM()` - Supports `predict()`, `evaluate()` methods. \n * `TargetEncodingFit()` - Supports `transform()` method. \n * `XGBoost()` - Supports `predict()`, `evaluate()` methods. \n * Time Series Analytic (UAF) Functions\n * `ArimaEstimate()` - Supports `forecast()`, `validate()` methods.\n * `DFFT()` - Supports `convolve()`, `inverse()` methods. \n * `IDFFT()` - Supports `inverse()` method. \n * `DFFT2()` - Supports `convolve()`, `inverse()` methods. \n * `IDFFT2()` - Supports `inverse()` method. \n * `DIFF()` - Supports `inverse()` method. \n * `UNDIFF()` - Supports `inverse()` method. \n * `SeasonalNormalize()` - Supports `inverse()` method.\n\n * ###### teradataml: DataFrame\n * New Functions\n * `DataFrame.plot()` - Generates the below type of plots on teradataml DataFrame.\n * line - Generates line plot.\n * bar - Generates bar plot.\n * scatter - Generates scatter plot.\n * corr - Generates correlation plot.\n * wiggle - Generates a wiggle plot.\n * mesh - Generates a mesh plot.\n * `DataFrame.itertuples()` - iterate over teradataml DataFrame rows as namedtuples or list.\n * ###### teradataml: GeoDataFrame\n * New Functions\n * `GeoDataFrame.plot()` - Generate the below type of plots on teradataml GeoDataFrame.\n * line - Generates line plot.\n * bar - Generates bar plot.\n * scatter - Generates scatter plot.\n * corr - Generates correlation plot.\n * wiggle - Generates a wiggle plot.\n * mesh - Generates a mesh plot.\n * geometry - Generates plot on geospatial data. \n * Plot:\n * `Axis` - Genertes the axis for plot.\n * `Figure` - Generates the figure for plot.\n * `subplots` - Helps in generating multiple plots on a single `Figure`. \n * Bring Your Own Model (BYOM) Function:\n * `DataikuPredict` - Score the data in Vantage using the model trained externally in Dataiku UI and stored in Vantage.\n * `async_run_status()` - Function to check the status of asynchronous run(s) using unique run id(s).\n\n * ###### teradataml DataFrameColumn a.k.a. ColumnExpression\n * _Regular Arithmetic Functions_\n * `DataFrameColumn.abs()` - Computes the absolute value.\n * `DataFrameColumn.ceil()` - Returns the ceiling value of the column.\n * `DataFrameColumn.ceiling()` - It is an alias for `DataFrameColumn.ceil()` function.\n * `DataFrameColumn.degrees()` - Converts radians value from the column to degrees.\n * `DataFrameColumn.exp()` - Raises e (the base of natural logarithms) to the power of the value in the column, where e = 2.71828182845905.\n * `DataFrameColumn.floor()` - Returns the largest integer equal to or less than the value in the column.\n * `DataFrameColumn.ln()` - Computes the natural logarithm of values in column.\n * `DataFrameColumn.log10()` - Computes the base 10 logarithm.\n * `DataFrameColumn.mod()` - Returns the modulus of the column.\n * `DataFrameColumn.pmod()` - It is an alias for `DataFrameColumn.mod()` function.\n * `DataFrameColumn.nullifzero()` - Converts data from zero to null to avoid problems with division by zero.\n * `DataFrameColumn.pow()` - Computes the power of the column raised to expression or constant.\n * `DataFrameColumn.power()` - It is an alias for `DataFrameColumn.pow()` function.\n * `DataFrameColumn.radians()` - Converts degree value from the column to radians.\n * `DataFrameColumn.round()` - Returns the rounded off value.\n * `DataFrameColumn.sign()` - Returns the sign.\n * `DataFrameColumn.signum()` - It is an alias for `DataFrameColumn.sign()` function.\n * `DataFrameColumn.sqrt()` - Computes the square root of values in the column.\n * `DataFrameColumn.trunc()` - Provides the truncated value of columns.\n * `DataFrameColumn.width_bucket()` - Returns the number of the partition to which column is assigned.\n * `DataFrameColumn.zeroifnull()` - Converts data from null to zero to avoid problems with null.\n * _Trigonometric Functions_\n * `DataFrameColumn.acos()` - Returns the arc-cosine value.\n * `DataFrameColumn.asin()` - Returns the arc-sine value.\n * `DataFrameColumn.atan()` - Returns the arc-tangent value.\n * `DataFrameColumn.atan2()` - Returns the arc-tangent value based on x and y coordinates.\n * `DataFrameColumn.cos()` - Returns the cosine value.\n * `DataFrameColumn.sin()` - Returns the sine value.\n * `DataFrameColumn.tan()` - Returns the tangent value.\n * _Hyperbolic Functions_\n * `DataFrameColumn.acosh()` - Returns the inverse hyperbolic cosine value. \n * `DataFrameColumn.asinh()` - Returns the inverse hyperbolic sine value.\n * `DataFrameColumn.atanh()` - Returns the inverse hyperbolic tangent value.\n * `DataFrameColumn.cosh()` - Returns the hyperbolic cosine value.\n * `DataFrameColumn.sinh()` - Returns the hyperbolic sine value\n * `DataFrameColumn.tanh()` - Returns the hyperbolic tangent value.\n * _String Functions_\n * `DataFrameColumn.ascii()` - Returns the decimal representation of the first character in column.\n * `DataFrameColumn.char2hexint()` - Returns the hexadecimal representation for a character string in a column.\n * `DataFrameColumn.chr()` - Returns the Latin ASCII character of a given a numeric code value in column.\n * `DataFrameColumn.char()` - It is an alias for `DataFrameColumn.chr()` function.\n * `DataFrameColumn.character_length()` - Returns the number of characters in the column.\n * `DataFrameColumn.char_length()` - It is an alias for `DataFrameColumn.character_length()` function.\n * `DataFrameColumn.edit_distance()` - Returns the minimum number of edit operations required to \n transform string in a column into string specified in argument.\n * `DataFrameColumn.index()` - Returns the position of a string in a column where string specified in argument starts.\n * `DataFrameColumn.initcap()` - Modifies a string column and returns the string with the first character\n of each word in uppercase.\n * `DataFrameColumn.instr()` - Searches the string in a column for occurrences of search string passed as argument.\n * `DataFrameColumn.lcase()` - Returns a character string identical to string values in column,\n with all uppercase letters replaced with their lowercase equivalents.\n * `DataFrameColumn.left()` - Truncates string in a column to a specified number of characters desired from\n the left side of the string.\n * `DataFrameColumn.length()` - It is an alias for `DataFrameColumn.character_length()` function.\n * `DataFrameColumn.levenshtein()` - It is an alias for `DataFrameColumn.edit_distance()` function.\n * `DataFrameColumn.locate()` - Returns the position of the first occurrence of a string in a column within \n string in argument. \n * `DataFrameColumn.lower()` - It is an alias for `DataFrameColumn.character_lcase()` function.\n * `DataFrameColumn.lpad()` - Returns the string in a column padded to the left with the characters specified \n in argument so that the resulting string has length specified in argument.\n * `DataFrameColumn.ltrim()` - Returns the string in a column, with its left-most characters removed up\n to the first character that is not in the string specified in argument.\n * `DataFrameColumn.ngram()` - Returns the number of n-gram matches between string in a column,\n and string specified in argument.\n * `DataFrameColumn.nvp()` - Extracts the value of a name-value pair where the name in the pair matches\n the name and the number of the occurrence specified.\n * `DataFrameColumn.oreplace()` - Replaces every occurrence of search string in the column.\n * `DataFrameColumn.otranslate()` - Returns string in a column with every occurrence of each character in\n string in argument replaced with the corresponding character in another argument.\n * `DataFrameColumn.replace()` - It is an alias for `DataFrameColumn.oreplace()` function.\n * `DataFrameColumn.reverse()` - Returns the reverse of string in column.\n * `DataFrameColumn.right()` - Truncates input string to a specified number of characters desired from\n the right side of the string.\n * `DataFrameColumn.rpad()` - Returns the string in a column padded to the right with the characters specified \n in argument so the resulting string has length specified in argument.\n * `DataFrameColumn.rtrim()` - Returns the string in column, with its right-most characters removed up\n to the first character that is not in the string specified in argument.\n * `DataFrameColumn.soundex()` - Returns a character string that represents the Soundex code for\n string in a column.\n * `DataFrameColumn.string_cs()` - Returns a heuristically derived integer value that can be used to determine\n which KANJI1-compatible client character set was used to encode string in a column.\n * `DataFrameColumn.translate()` - It is an alias for `DataFrameColumn.otranslate()` function.\n * `DataFrameColumn.upper()` - Returns a character string with all lowercase letters in a column replaced \n with their uppercase equivalents.\n\n * ##### teradataml Options\n * Configuration Options\n * `configure.indb_install_location`\n Specifies the installation location of In-DB Python package.\n\n* ##### Updates\n * Open Analytics Framework (OpenAF) APIs:\n * `set_auth_token()`\n * `set_auth_token()` does not accept username and password anymore. Instead, function opens up a browser session and user should authenticate in browser.\n * After token expiry, teradataml will open a browser and user needs to authenticate again.\n * If client machine does not have browser, then user should copy the URL posted by teradataml and authenticate themselves.\n * Security fixes - `auth_token` is not set or retrieved from the `configure` option anymore.\n * Manage all user environments.\n * `create_env()` - supports creation of R environment.\n * `remove_env()` - Supports removal of remote R environment.\n * `remove_all_envs()` - Supports removal of all remote R environments.\n * `remove_env()` and `remove_all_envs()` supports asynchronous call.\n * UserEnv Class \u2013 Supports managing of R remote environments.\n * Properties:\n * `libs` - Supports listing of libraries in R remote environment.\n * Methods:\n * `install_lib()` - Supports installing of libraries in remote R environment.\n * `uninstall_lib()` - Supports uninstalling of libraries in remote R environment.\n * `update_lib()` - Supports updating of libraries in remote R environment.\n * Unbounded Array Framework (UAF) Functions:\n * `ArimaEstimate()`\n * Added support for `CSS` algorithm via `algorithm` argument.\n\n* ##### Bug Fixes\n * Installation location of In-DB 2.0.0 package is changed. Script() will now work with both 2.0.0 and previous version. \n\n## Release Notes:\n#### teradataml 17.20.00.04\n* ##### New Features/Functionality\n * teradataml is now compatible with SQLAlchemy 2.0.X\n * **Important notes** when user has sqlalchemy version >= 2.0: \n * Users will not be able to run the `execute()` method on SQLAlchemy engine object returned by \n `get_context()` and `create_context()` teradataml functions. This is due to the SQLAlchemy has\n removed the support for `execute()` method on the engine object. Thus, user scripts where \n `get_context().execute()` and `create_context().execute()`, is used, Teradata recommends to\n replace those with either `execute_sql()` function exposed by teradataml or `exec_driver_sql()` \n method on the `Connection` object returned by `get_connection()` function in teradataml.\n * Now `get_connection().execute()` accepts only executable sqlalchemy object. Refer to \n `sqlalchemy.engine.base.execute()` for more details.\n * Teradata recommends to use either `execute_sql()` function exposed by teradataml or \n `exec_driver_sql()` method on the `Connection` object returned by `get_connection()` \n function in teradataml, in such cases.\n * New utility function `execute_sql()` is added to execute the SQL. \n * Extending compatibility for MAC with ARM processors.\n * Added support for floor division (//) between two teradataml DataFrame Columns.\n * Analytics Database Analytic Functions:\n * `GLMPerSegment()`\n * `GLMPredictPerSegment()`\n * `OneClassSVM()`\n * `OneClassSVMPredict()`\n * `SVM()`\n * `SVMPredict()`\n * `TargetEncodingFit()`\n * `TargetEncodingTransform()`\n * `TrainTestSplit()`\n * `WordEmbeddings()`\n * `XGBoost()`\n * `XGBoostPredict()`\n\n * ###### teradataml Options\n * Display Options\n * `display.geometry_column_length`\n Option to display the default length of geometry column in GeoDataFrame.\n\n * ##### Updates\n * `set_auth_token()` function can generate the client id automatically based on org_id when user do not specify it.\n * Analytics Database Analytic Functions:\n * `ColumnTransformer()` \n * Does not allow list values for arguments - `onehotencoding_fit_data` and `ordinalencoding_fit_data`.\n * `OrdidnalEncodingFit()`\n * New arguments added - `category_data`, `target_column_names`, `categories_column`, `ordinal_values_column`.\n * Allows the list of values for arguments - `target_column`, `start_value`, `default_value`.\n * `OneHotEncodingFit()`\n * New arguments added - `category_data`, `approach`, `target_columns`, `categories_column`, `category_counts`.\n * Allows the list of values for arguments - `target_column`, `other_column`.\n\n * ##### Bug Fixes\n * `DataFrame.sample()` method output is now deterministic.\n * `copy_to_sql()` now preserves the rows of the table even when the view content is copied to the same table name.\n * `list_user_envs()` does not raise warning when no user environments found.\n\n## Release Notes:\n#### teradataml 17.20.00.03\n\n * ##### Updates\n * DataFrame.join \n * New arguments `lprefix` and `rprefix` added.\n * Behavior of arguments `lsuffix` and `rsuffix` will be changed in future, use new arguments instead.\n * New and old affix arguments can now be used independently.\n * Analytic functions can be imported regardless of context creation. \n Import after create context constraint is now removed.\n * `ReadNOS` and `WriteNOS` now accept dictionary value for `authorization` and `row_format` arguments.\n * `WriteNOS` supports writing CSV files to external store.\n * Following model cataloging APIs will be deprecated in future:\n * describe_model\n * delete_model\n * list_models\n * publish_model\n * retrieve_model\n * save_model\n\n * ##### Bug Fixes\n * `copy_to_sql()` bug related to NaT value has been fixed.\n * Tooltip on PyCharm IDE now points to SQLE.\n * `value` argument of `FillNa()`, a Vantage Analytic Library function supports special characters.\n * `case` function accepts DataFrame column as value in `whens` argument.\n\n## Release Notes:\n#### teradataml 17.20.00.02\n* ##### New Features/Functionality\n * ###### teradataml: Open Analytics\n * New Functions\n * `set_auth_token()` - Sets the JWT token automatically for using Open AF API's.\n\n * ###### teradataml Options\n * Display Options\n * `display.suppress_vantage_runtime_warnings`\n Suppresses the VantageRuntimeWarning raised by teradataml, when set to True.\n\n * ##### Updates\n * SimpleImputeFit function arguments `stats_columns` and `stats` are made to be optional.\n * New argument `table_format` is added to ReadNOS().\n * Argument `full_scan` is changed to `scan_pct` in ReadNOS(). \n\n * ##### Bug Fixes\n * Minor bug fix related to read_csv.\n * APPLY and `DataFrame.apply()` supports hash by and local order by.\n * Output column names are changed for DataFrame.dtypes and DataFrame.tdtypes.\n\n## Release Notes:\n#### teradataml 17.20.00.01\n* ##### New Features/Functionality\n * ###### teradataml: DataFrame\n * New Functions\n * `DataFrame.pivot()` - Rotate data from rows into columns to create easy-to-read DataFrames.\n * `DataFrame.unpivot()` - Rotate data from columns into rows to create easy-to-read DataFrames.\n * `DataFrame.drop_duplicate()` - Drop duplicate rows from teradataml DataFrame.\n * New properties \n * `Dataframe.is_art` - Check whether teradataml DataFrame is created on an Analytic Result Table, i.e., ART table or not.\n\n * ###### teradataml: Unbounded Array Framework (UAF) Functions:\n * New Functions\n * New Functions Supported on Database Versions: 17.20.x.x\n * MODEL PREPARATION AND PARAMETER ESTIMATION functions:\n\t\t\t 1. `ACF()`\n\t\t\t 2. `ArimaEstimate()`\n\t\t\t 3. `ArimaValidate()`\n\t\t\t 4. `DIFF()`\n\t\t\t 5. `LinearRegr()`\n\t\t\t 6. `MultivarRegr()`\n\t\t\t 7. `PACF()`\n\t\t\t 8. `PowerTransform()`\n\t\t\t 9. `SeasonalNormalize()`\n\t\t\t 10. `Smoothma()`\n\t\t\t 11. `UNDIFF()`\n\t\t\t 12. `Unnormalize()`\n\t\t* SERIES FORECASTING functions:\n\t\t\t 1. `ArimaForecast()`\n\t\t\t 2. `DTW()`\n\t\t\t 3. `HoltWintersForecaster()`\n\t\t\t 4. `MAMean()`\n\t\t\t 5. `SimpleExp()`\n\t\t* DATA PREPARATION functions:\n\t\t\t 1. `BinaryMatrixOp()`\n\t\t\t 2. `BinarySeriesOp()`\n\t\t\t 3. `GenseriesFormula()`\n\t\t\t 4. `MatrixMultiply()`\n\t\t\t 5. `Resample()`\n\t\t* DIAGNOSTIC STATISTICAL TEST functions:\n\t\t\t 1. `BreuschGodfrey()`\n\t\t\t 2. `BreuschPaganGodfrey()`\n\t\t\t 3. `CumulPeriodogram()`\n\t\t\t 4. `DickeyFuller()`\n\t\t\t 5. `DurbinWatson()`\n\t\t\t 6. `FitMetrics()`\n\t\t\t 7. `GoldfeldQuandt()`\n\t\t\t 8. `Portman()`\n\t\t\t 9. `SelectionCriteria()`\n\t\t\t 10. `SignifPeriodicities()`\n\t\t\t 11. `SignifResidmean()`\n\t\t\t 12. `WhitesGeneral()`\n\t\t* TEMPORAL AND SPATIAL functions:\n\t\t\t 1. `Convolve()`\n\t\t\t 2. `Convolve2()`\n\t\t\t 3. `DFFT()`\n\t\t\t 4. `DFFT2()`\n\t\t\t 5. `DFFT2Conv()`\n\t\t\t 6. `DFFTConv()`\n\t\t\t 7. `GenseriesSinusoids()`\n\t\t\t 8. `IDFFT()`\n\t\t\t 9. `IDFFT2()`\n\t\t\t 10. `LineSpec()`\n\t\t\t 11. `PowerSpec()`\n\t\t* GENERAL UTILITY functions:\n\t\t\t 1. `ExtractResults()`\n\t\t\t 2. `InputValidator()`\n\t\t\t 3. `MInfo()`\n\t\t\t 4. `SInfo()`\n\t\t\t 5. `TrackingOp()`\n\n * New Features: Inputs to Unbounded Array Framework (UAF) functions\n * `TDAnalyticResult()` - Allows to prepare function output generated by UAF functions to be passed.\n * `TDGenSeries()` - Allows to generate a series, that can be passed to a UAF function.\n * `TDMatrix()` - Represents a Matrix in time series, that can be created from a teradataml DataFrame.\n * `TDSeries()` - Represents a Series in time series, that can be created from a teradataml DataFrame.\n\n * ##### Updates \n * Native Object Store (NOS) functions support authorization by specifying authorization object.\n * `display_analytic_functions()` categorizes the analytic functions based on function type.\n * ColumnTransformer accepts multiple values for arguments nonlinearcombine_fit_data, \n onehotencoding_fit_data, ordinalencoding_fit_data.\n\n * ##### Bug Fixes\n * Redundant warnings thrown by teradataml are suppressed.\n * OpenAF supports when context is created with JWT Token.\n * New argument \"match_column_order\" added to copy_to_sql, that allows DataFrame loading with any column order.\n * `copy_to_sql` updated to map data type timezone(tzinfo) to TIMESTAMP(timezone=True), instead of VARCHAR.\n * Improved performance for DataFrame.sum and DataFrameColumn.sum functions.\n\n## Release Notes:\n#### teradataml 17.20.00.00\n* ##### New Features/Functionality\n * ###### teradataml: Analytics Database Analytic Functions\n * _New Functions_ \n * ###### New Functions Supported on Database Versions: 17.20.x.x\n * `ANOVA()`\u200b\n * `ClassificationEvaluator()`\u200b\n * `ColumnTransformer()`\u200b\n * `DecisionForest()`\n * `GLM\u200b()`\n * `GetFutileColumns()`\n * `KMeans()`\u200b\n * `KMeansPredict()`\u200b\u200b\n * `NaiveBayesTextClassifierTrainer()`\u200b\n * `NonLinearCombineFit()`\u200b\n * `NonLinearCombineTransform()`\u200b\n * `OrdinalEncodingFit\u200b()`\n * `OrdinalEncodingTransform()`\u200b\n * `RandomProjectionComponents\u200b()`\n * `RandomProjectionFit\u200b()`\n * `RandomProjectionTransform()`\u200b\n * `RegressionEvaluator\u200b()`\n * `ROC\u200b()`\n * `SentimentExtractor()`\u200b\n * `Silhouette\u200b()`\n * `TDGLMPredict\u200b()`\n * `TextParser\u200b()`\n * `VectorDistance()`\n * _Updates_\n * `display_analytic_functions()` categorizes the analytic functions based on function type.\n * Users can provide range value for columns argument.\n\n * ###### teradataml: Open Analytics\n * Manage all user environments.\n * `list_base_envs()` - list the available python base versions.\u200b\n * `create_env()` - create a new user environment. \u200b\n * `get_env()` - get existing user environment.\n * `list_user_envs()` - list the available user environments.\u200b\n * `remove_env()` - delete user environment.\u200b\n * `remove_all_envs()` - delete all the user environments.\n * UserEnv Class \u2013 Manage individual user environment. \n * Properties\n * `files` - Get files in user environment. \n * `libs` - Get libraries in user environment.\n * Methods\n * `install_file()` - Install a file in user environment.\u200b\n * `remove_file()` - Remove a file in user environment.\u200b\n * `install_lib()` - Install a library in user environment.\u200b\n * `update_lib()` - Update a library in user environment.\u200b\n * `uninstall_lib()` - Uninstall a library in user environment.\u200b\n * `status()` - Check the status of\u200b\n * file installation\u200b\n * library installation\u200b\n * library update\u200b\n * library uninstallation\u200b\n * `refresh()` - Refresh the environment details in local client.\n * Apply Class \u2013 Execute a user script on VantageCloud Lake.\u200b\n * `__init__()` - Instantiate an object of apply for script execution.\u200b\n * `install_file()` - Install a file in user environment.\u200b\n * `remove_file()` - Remove a file in user environment.\u200b\n * `set_data()` \u2013 Reset data and related arguments.\u200b\n * `execute_script()` \u2013 Executes Python script.\n\n * ###### teradataml: DataFrame\n * _New Functions_\n * `DataFrame.apply()` - Execute a user defined Python function on VantageLake Cloud.\n\n * ###### teradataml: Bring Your Own Model\n * _New Functions_\n * `ONNXPredict()` - Score using model trained externally on ONNX and stored in Vantage.\n\n * ###### teradataml: Options\n * _New Functions_\n * set_config_params() New API to set all config params in one go.\n * _New Configuration Options_\n * For Open Analytics support.\u200b\n * ues_url \u2013 User Environment Service URL for VantageCloud Lake.\u200b\n * auth_token \u2013 Authentication token to connect to VantageCloud Lake.\n * certificate_file \u2013 Path to a CA_BUNDLE file or directory with certificates of trusted CAs.\n\n * ##### Updates\n * `accumulate` argument is working for `ScaleTransform()`.\n * Following functions have `accumulate` argument added on Database Versions: 17.20.x.x\n * `ConvertTo()`\n * `GetRowsWithoutMissingValues()`\n * `GetRowsWithoutMissingValues()`\n * `OutlierFilterFit()` supports multiple output.\n * For `OutlierFilterFit()` function below arguments are optional in teradataml 17.20.x.x\n * `lower_percentile`\n * `upper_percentile`\n * `outlier_method`\n * `replacement_value`\n * `percentile_method`\n * Analytics Database analytic functions \u2013 In line help, i.e., help() for the functions\n is available.\u200b\n\n * ##### Bug Fixes\n * Vantage Analytic Library FillNa() function: Now `columns` argument is required.\n * `output_responses` argument in MLE function `DecisionTreePredict()`, does not allow empty string.\n * teradataml closes docker sandbox environment properly.\n * Users can create context using JWT token.\n\n#### teradataml 17.10.00.02\n* ##### New Features/Functionality\n * ###### Database Utility\n * `list_td_reserved_keywords()` - Validates if the specified string is Teradata reserved\n keyword or not, else lists down all the Teradata reserved keywords.\n\n* ##### Updates\n * ###### DataFrame\n * _Updates_\n * Multiple columns can be selected using slice operator ([]).\n\n * ###### Script\n * _Updates_\n * A warning will be raised, when Teradata reserved keyword is used in Script local mode.\n\n* ##### Bug Fixes\n * Numeric overflow issue observed for describe(), sum(), csum(), and mean() has been fixed.\n * Error messages are updated for SQLE function arguments accepting multiple datatypes.\n * Error messages are updated for SQLE function arguments volatile and persist arguments when \n non-boolean value is provided.\n * DataFrame sample() method can handle column names with special characters like space, hyphen, \n period etc.\n * In-DB SQLE functions can be loaded for any locale setting.\n * `create_context()` - Password containing special characters requires URL encoding as per\n https://docs.microfocus.com/OMi/10.62/Content/OMi/ExtGuide/ExtApps/URL_encoding.html. \n teradataml has added a fix to take care of the URL encoding of the password while creating a context. \n Also, a new argument is added to give a more control over the URL encoding to be done at the time of context creation.\n\n#### teradataml 17.10.00.01\n* ##### New Features/Functionality\n * ###### Geospatial\n The Geospatial feature in teradataml enables data manipulation, exploration and analysis on tables, views, and queries on Teradata Vantage that contains Geospatial data.\n * ###### Geomtery Types\n * Point\n * LineString\n * Polygon\n * MultiPoint\n * MultiLineString\n * MultiPolygon\n * GeometryCollection\n * GeoSequence\n * ###### teradataml GeoDataFrame\n * Properties\n * columns\n * dtypes\n * geometry\n * iloc\n * index\n * loc\n * shape\n * size\n * tdtypes\n * Geospatial Specific Properties\n * ###### Properties for all Types of Geometries\n * boundary\n * centroid\n * convex_hell\n * coord_dim\n * dimension\n * geom_type\n * is_3D\n * is_empty\n * is_simple\n * is_valid\n * max_x\n * max_y\n * max_z\n * min_x\n * min_y\n * min_z\n * srid\n * ###### Properties for Point Geometry\n * x\n * y\n * z\n * ###### Properties for LineString Geometry\n * is_closed_3D\n * is_closed\n * is_ring\n * ###### Properties for Polygon Geometry\n * area\n * exterior\n * perimeter\n * Methods\n * `__getattr__()`\n * `__getitem__()`\n * `__init__()`\n * `__repr__()`\n * `assign()`\n * `concat()`\n * `count()`\n * `drop()`\n * `dropna()`\n * `filter()`\n * `from_query()`\n * `from_table()`\n * `get()`\n * `get_values()`\n * `groupby()`\n * `head()`\n * `info()`\n * `join()`\n * `keys()`\n * `merge()`\n * `sample()`\n * `select()`\n * `set_index()`\n * `show_query()`\n * `sort()`\n * `sort_index()`\n * `squeeze()`\n * `tail()`\n * `to_csv()`\n * `to_pandas()`\n * `to_sql()` \n * Geospatial Specific Methods\n * ###### Methods for All Type of Geometry\n * `buffer()`\n * `contains()`\n * `crosses()`\n * `difference()`\n * `disjoint()`\n * `distance()`\n * `distance_3D()`\n * `envelope()`\n * `geom_equals()`\n * `intersection()`\n * `intersects()`\n * `make_2D()`\n * `mbb()`\n * `mbr()`\n * `overlaps()`\n * `relates()`\n * `set_exterior()`\n * `set_srid()`\n * `simplify()`\n * `sym_difference()`\n * `to_binary()`\n * `to_text()`\n * `touches()`\n * `transform()`\n * `union()`\n * `within()`\n * `wkb_geom_to_sql()`\n * `wkt_geom_to_sql()`\n * ###### Methods for Point Geometry\n * `spherical_buffer()`\n * `spherical_distance()`\n * `spheriodal_buffer()`\n * `spheriodal_distance()`\n * `set_x()`\n * `set_y()`\n * `set_z()`\n * ###### Methods for LineString Geometry\n * `end_point()`\n * `length()`\n * `length_3D()`\n * `line_interpolate_point()`\n * `num_points()`\n * `point()`\n * `start_point()`\n * ###### Methods for Polygon Geometry\n * `interiors()`\n * `num_interior_ring()`\n * `point_on_surface()`\n * ###### Methods for GeometryCollection Geometry\n * `geom_component()`\n * `num_geometry()`\n * ###### Methods for GeoSequence Geometry\n * `clip()`\n * `get_final_timestamp()`\n * `get_init_timestamp()`\n * `get_link()`\n * `get_user_field()`\n * `get_user_field_count()`\n * `point_heading()`\n * `set_link()`\n * `speed()`\n * ###### Filtering Functions and Methods\n * `intersects_mbb()`\n * `mbb_filter()`\n * `mbr_filter()`\n * `within_mbb()`\n * ###### teradataml GeoDataFrameColumn\n * Geospatial Specific Properties\n * ###### Properties for all Types of Geometries\n * boundary\n * centroid\n * convex_hell\n * coord_dim\n * dimension\n * geom_type\n * is_3D\n * is_empty\n * is_simple\n * is_valid\n * max_x\n * max_y\n * max_z\n * min_x\n * min_y\n * min_z\n * srid\n * ###### Properties for Point Geometry\n * x\n * y\n * z\n * ###### Properties for LineString Geometry\n * is_closed_3D\n * is_closed\n * is_ring\n * ###### Properties for Polygon Geometry\n * area\n * exterior\n * perimeter\n * Geospatial Specific Methods\n * ###### Methods for All Type of Geometry\n * `buffer()`\n * `contains()`\n * `crosses()`\n * `difference()`\n * `disjoint()`\n * `distance()`\n * `distance_3D()`\n * `envelope()`\n * `geom_equals()`\n * `intersection()`\n * `intersects()`\n * `make_2D()`\n * `mbb()`\n * `mbr()`\n * `overlaps()`\n * `relates()`\n * `set_exterior()`\n * `set_srid()`\n * `simplify()`\n * `sym_difference()`\n * `to_binary()`\n * `to_text()`\n * `touches()`\n * `transform()`\n * `union()`\n * `within()`\n * `wkb_geom_to_sql()`\n * `wkt_geom_to_sql()`\n * ###### Methods for Point Geometry\n * `spherical_buffer()`\n * `spherical_distance()`\n * `spheriodal_buffer()`\n * `spheriodal_distance()`\n * `set_x()`\n * `set_y()`\n * `set_z()`\n * ###### Methods for LineString Geometry\n * `endpoint()`\n * `length()`\n * `length_3D()`\n * `line_interpolate_point()`\n * `num_points()`\n * `point()`\n * `start_point()`\n * ###### Methods for Polygon Geometry\n * `interiors()`\n * `num_interior_ring()`\n * `point_on_surface()`\n * ###### Methods for GeometryCollection Geometry\n * `geom_component()`\n * `num_geometry()`\n * ###### Methods for GeoSequence Geometry\n * `clip()`\n * `get_final_timestamp()`\n * `get_init_timestamp()`\n * `get_link()`\n * `get_user_field()`\n * `get_user_field_count()`\n * `point_heading()`\n * `set_link()`\n * `speed()`\n * ###### Filtering Functions and Methods\n * `intersects_mbb()`\n * `mbb_filter()`\n * `mbr_filter()`\n * `within_mbb()`\n\n * ###### teradataml DataFrame\n * _New Functions_\n * `to_csv()`\n\n * ###### teradataml: SQLE Engine Analytic Functions\n * _New Functions_\n * Newly added SQLE functions are accessible only after establishing the connection to Vantage.\n * `display_analytic_functions()` API displays all the available SQLE Analytic functions based on database version. \n * ###### Functions Supported on DatabaseVersions: 16.20.x.x, 17.10.x.x, 17.05.x.x\n * `Antiselect()`\n * `Attribution()`\n * `DecisionForestPredict()`\n * `DecisionTreePredict()`\n * `GLMPredict()`\n * `MovingAverage()`\n * `NaiveBayesPredict()`\n * `NaiveBayesTextClassifierPredict()`\n * `NGramSplitter()`\n * `NPath()`\n * `Pack()`\n * `Sessionize()`\n * `StringSimilarity()`\n * `SVMParsePredict()`\n * `Unpack()`\n * ###### Functions Supported on DatabaseVersions: 17.10.x.x\n * `Antiselect()`\n * `Attribution()`\n * `BincoodeFit()`\n * `BncodeTransform()`\n * `CategoricalSummary()`\n * `ChiSq()`\n * `ColumnSummary()`\n * `ConvertTo()`\n * `DecisionForestPredict()`\n * `DecisionTreePredict()`\n * `GLMPredict()`\n * `FillRowId()`\n * `FTest()`\n * `Fit()`\n * `Transform()`\n * `GetRowsWithMissingValues()`\n * `GetRowsWithoutMissingValues()`\n * `MovingAverage()`\n * `Histogram()`\n * `NaiveBayesPredict()` \n * `NaiveBayesTextClassifierPredict()`\n * `NGramSplitter()`\n * `NPath()`\n * `NumApply()`\n * `OneHotEncodingFit()`\n * `OneHotEncodingTransform()`\n * `OutlierFilterFit()`\n * `OutlierFilterTransform()`\n * `Pack()`\n * `PolynomialFeatuesFit()`\n * `PolynomialFeatuesTransform()`\n * `QQNorm()`\n * `RoundColumns()`\n * `RowNormalizeFit()`\n * `RowNormalizeTransform()`\n * `ScaleFit()`\n * `ScaleTransform()` \n * `Sessionize()`\n * `SimpleImputeFit()`\n * `SimpleImputeTransform()`\n * `StrApply()`\n * `StringSimilarity()`\n * `SVMParsePredict()`\n * `UniVariateStatistics()`\n * `Unpack()`\n * `WhichMax()`\n * `WhichMin()`\n * `ZTest()`\n\n * ###### teradataml: General Functions\n * _New Functions_\n * Data Transfer Utility\n * `read_csv()`\n\n * ###### Operators\n * _New Functions_\n * Table Operators\n * `read_nos()`\n * `write_nos()`\n\n * ###### teradataml: Bring Your Own Model\n * _New Functions_\n * Model Cataloging\n * `get_license()`\n * `set_byom_catalog()`\n * `set_license()`\n\n* ##### Updates\n * ###### teradataml: General Functions\n * Data Transfer Utility\n * `copy_to_sql()` - New argument \"chunksize\" added to load data in chunks.\n * Following Data Transfer Utility Functions updated to specify the number of Teradata sessions to open for data transfer using \"open_session\" argument:\n * `fastexport()`\n * `fastload()`\n * `to_pandas()`\n\n * ###### Operators\n * Following Set Operator Functions updated to work with Geospatial data:\n * `concat()`\n * `td_intersect()`\n * `td_expect()`\n * `td_minus()`\n\n * ###### teradataml: Bring Your Own Model\n * Model cataloging APIs mentioned below are updated to use session level parameters set by `set_byom_catalog()` and `set_license()` such as table name, schema name and license details respectively.\n * `delete_byom()`\n * `list_byom()`\n * `retrieve_byom()`\n * `save_byom()`\n * `view_log()` - Allows user to view BYOM logs.\n\n* ##### Bug Fixes\n * CS0733758 - `db_python_package_details()` function is fixed to support latest STO release for pip and Python aliases used.\n * DataFrame `print()` issue related to `Response Row size is greater than the 1MB allowed maximum.` has been fixed to print the data with lot of columns.\n * New parameter \"chunksize\" is added to `DataFrame.to_sql()` and `copy_to_sql()` to fix the issue where the function was failing with error - \"Request requires too many SPOOL files.\". Reducing the chunksize than the default one will result in successful operation.\n * `remove_context()` is fixed to remove the active connection from database.\n * Support added to specify the number of Teradata data transfer sessions to open for data transfer using `fastexport()` and `fastload()` functions.\n * `DataFrame.to_sql()` is fixed to support temporary table when default database differs from the username. \n * `DataFrame.to_pandas()` now by default support data transfer using regular method. Change is carried out for user to allow the data transfer if utility throttles are configured, i.e., TASM configuration does not support data export using FastExport.\n * `save_byom()` now notifies if VARCHAR column is trimmed out if data passed to the API is greater than the length of the VARCHAR column.\n * Standard error can now be captured for `DataFrame.map_row()` and `DataFrame.map_parition()` when executed in LOCAL mode.\n * Vantage Analytic Library - Underlying SQL can be retrieved using newly added arguments \"gen_sql\"/\"gen_sql_only\" for the functions. Query can be viewed with the help `show_query()`.\n * Documentation example has been fixed for `fastexport()` to show the correct import statement.\n\n\n#### teradataml 17.00.00.05\nFixed [CS0733758] db_python_package_details() fails on recent STO release due to changes in pip and python aliases.\n\n\n#### teradataml 17.00.00.04\n* ##### New Features/Functionality\n * ###### Analytic Functions\n * Bring Your Own Analytics Functions\n The BYOM feature in Vantage provides flexibility to score the data in Vantage using external models using following BYOM functions:\n * `H2OPredict()` - Score using model trained externally in H2O and stored in Vantage.\n * `PMMLPredict()` - Score using model trained externally in PMML and stored in Vantage.\n * BYOM Model Catalog APIs\n * `save_byom()` - Save externally trained models in Teradata Vantage.\n * `delete_byom()` - Delete a model from the user specified table in Teradata Vantage.\n * `list_byom()` - List models.\n * `retrieve_byom()` - Function to retrieve a saved model.\n * Vantage Analytic Library Functions\n * _New Functions_\n * `XmlToHtmlReport()` - Transforms XML output of VAL functions to HTML.\n * ###### teradataml DataFrame\n * `DataFrame.window()` - Generates Window object on a teradataml DataFrame to run window aggregate functions.\n * `DataFrame.csum()` - Returns column-wise cumulative sum for rows in the partition of the dataframe.\n * `DataFrame.mavg()` - Returns moving average for the current row and the preceding rows.\n * `DataFrame.mdiff()` - Returns moving difference for the current row and the preceding rows.\n * `DataFrame.mlinreg()` - Returns moving linear regression for the current row and the preceding rows.\n * `DataFrame.msum()` - Returns moving sum for the current row and the preceding rows.\n * _Regular Aggregate Functions_\n * `DataFrame.corr()` - Returns the Sample Pearson product moment correlation coefficient.\n * `DataFrame.covar_pop()` - Returns the population covariance.\n * `DataFrame.covar_samp()` - Returns the sample covariance.\n * `DataFrame.regr_avgx()` - Returns the mean of the independent variable.\n * `DataFrame.regr_avgy()` - Returns the mean of the dependent variable.\n * `DataFrame.regr_count()` - Returns the count of the dependent and independent variable arguments.\n * `DataFrame.rege_intercept()` - Returns the intercept of the univariate linear regression line.\n * `DataFrame.regr_r2()` - Returns the coefficient of determination.\n * `DataFrame.regr_slope()` - Returns the slope of the univariate linear regression line through.\n * `DataFrame.regr_sxx()` - Returns the sum of the squares of the independent variable expression.\n * `DataFrame.regr_sxy()` - Returns the sum of the products of the independent variable and the dependent variable.\n * `DataFrame.regr_syy()` - Returns the sum of the squares of the dependent variable expression.\n * ###### teradataml DataFrameColumn a.k.a. ColumnExpression\n * `ColumnExpression.window()` - Generates Window object on a teradataml DataFrameColumn to run window aggregate functions.\n * `ColumnExpression.desc()` - Sorts ColumnExpression in descending order.\n * `ColumnExpression.asc()` - Sorts ColumnExpression in ascending order.\n * `ColumnExpression.distinct()` - Removes duplicate value from ColumnExpression.\n * _Regular Aggregate Functions_\n * `ColumnExpression.corr()` - Returns the Sample Pearson product moment correlation coefficient.\n * `ColumnExpression.count()` - Returns the column-wise count.\n * `ColumnExpression.covar_pop()` - Returns the population covariance.\n * `ColumnExpression.covar_samp()` - Returns the sample covariance.\n * `ColumnExpression.kurtosis()` - Returns kurtosis value for a column.\n * `ColumnExpression.median()` - Returns column-wise median value.\n * `ColumnExpression.max()` - Returns the column-wise max value.\n * `ColumnExpression.mean()` - Returns the column-wise average value.\n * `ColumnExpression.min()` - Returns the column-wise min value.\n * `ColumnExpression.regr_avgx()` - Returns the mean of the independent variable.\n * `ColumnExpression.regr_avgy()` - Returns the mean of the dependent variable.\n * `ColumnExpression.regr_count()` - Returns the count of the dependent and independent variable arguments.\n * `ColumnExpression.rege_intercept()` - Returns the intercept of the univariate linear regression line.\n * `ColumnExpression.regr_r2()` - Returns the coefficient of determination arguments.\n * `ColumnExpression.regr_slope()` - Returns the slope of the univariate linear regression line.\n * `ColumnExpression.regr_sxx()` - Returns the sum of the squares of the independent variable expression.\n * `ColumnExpression.regr_sxy()` - Returns the sum of the products of the independent variable and the dependent variable.\n * `ColumnExpression.regr_syy()` - Returns the sum of the squares of the dependent variable expression.\n * `ColumnExpression.skew()` - Returns skew value for a column.\n * `ColumnExpression.std()` - Returns the column-wise population/sample standard deviation.\n * `ColumnExpression.sum()` - Returns the column-wise sum.\n * `ColumnExpression.var()` - Returns the column-wise population/sample variance.\n * `ColumnExpression.percentile()` - Returns the column-wise percentile.\n * ###### teradataml Window - Window Aggregate Functions\n Following set of _Window Aggregate Functions_ return the results over a specified window which can be of any type:\n * Cumulative/Expanding window\n * Moving/Rolling window\n * Contracting/Remaining window\n * Grouping window\n _Window Aggregate Functions_\n * `Window.corr()` - Returns the Sample Pearson product moment correlation coefficient.\n * `Window.count()` - Returns the count.\n * `Window.covar_pop()` - Returns the population covariance.\n * `Window.covar_samp()` - Returns the sample covariance.\n * `Window.cume_dist()` - Returns the cumulative distribution of values.\n * `Window.dense_Rank()` - Returns the ordered ranking of all the rows.\n * `Window.first_value()` - Returns the first value of an ordered set of values.\n * `Window.lag()` - Returns data from the row preceding the current row at a specified offset value.\n * `Window.last_value()` - Returns the last value of an ordered set of values.\n * `Window.lead()` - Returns data from the row following the current row at a specified offset value.\n * `Window.max()` - Returns the column-wise max value.\n * `Window.mean()` - Returns the column-wise average value.\n * `Window.min()` - Returns the column-wise min value.\n * `Window.percent_rank()` - Returns the relative rank of all the rows.\n * `Window.rank()` - Returns the rank (1 \u2026 n) of all the rows.\n * `Window.regr_avgx()` - Returns the mean of the independent variable arguments.\n * `Window.regr_avgy()` - Returns the mean of the dependent variable arguments.\n * `Window.regr_count()` - Returns the count of the dependent and independent variable arguments.\n * `Window.rege_intercept()` - Returns the intercept of the univariate linear regression line arguments.\n * `Window.regr_r2()` - Returns the coefficient of determination arguments.\n * `Window.regr_slope()` - Returns the slope of the univariate linear regression line.\n * `Window.regr_sxx()` - Returns the sum of the squares of the independent variable expression.\n * `Window.regr_sxy()` - Returns the sum of the products of the independent variable and the dependent variable.\n * `Window.regr_syy()` - Returns the sum of the squares of the dependent variable expression.\n * `Window.row_number()` - Returns the sequential row number.\n * `Window.std()` - Returns the column-wise population/sample standard deviation.\n * `Window.sum()` - Returns the column-wise sum.\n * `Window.var()` - Returns the column-wise population/sample variance.\n * ###### General functions\n * _New functions_\n * `fastexport()` - Exports teradataml DataFrame to Pandas DataFrame using FastExport data transfer protocol.\n * ###### teradataml Options\n * Display Options\n * `display.blob_length`\n Specifies default display length of BLOB column in teradataml DataFrame.\n * Configuration Options\n * `configure.temp_table_database`\n Specifies database name for storing the tables created internally.\n * `configure.temp_view_database`\n Specifies database name for storing the views created internally.\n * `configure.byom_install_location`\n Specifies the install location for the BYOM functions.\n * `configure.val_install_location`\n Specifies the install location for the Vantage Analytic Library functions.\n* ##### Updates\n * ###### teradataml DataFrame\n * `to_pandas()` - \n * Support added to transfer data to Pandas DataFrame using fastexport protocol improving the performance.\n * Support added for other arguments similar to Pandas `read_sql()`:\n * `coerce_float`\n * `parse_dates`\n * ###### Analytic functions\n * Vantage Analytic Library Functions\n * Support added to accept datetime.date object for literals/values in \n following transformation functions:\n * `FillNa()`\n * `Binning()`\n * `OneHotEncoder()`\n * `LabelEncoder()`\n * All transformation functions now supports accepting \n teradatasqlalchemy datatypes as input to \"datatype\" argument for \n casting the result.\n* ##### Bug Fixes.\n * CS0249633 - Support added for teradataml to work with user/database/tablename\n containing period (.).\n * CS0086594 - Use of dbc.tablesvx versus dbc.tablesvx in teradatasqlalchemy.\n * IPython integration to print the teradataml DataFrames in pretty format.\n * teradataml DataFrame APIs now support column names same as that of Teradata \n reserved keywords.\n * Issue has been fixed for duplicate rows being loaded via teradataml \n fastload() API.\n * VAL - Empty string now can be passed as input for recoding values using \n LabelEncoder.\n * teradataml extension with SQLAlchemy functions:\n * mod() function is fixed to return correct datatype.\n * sum() function is fixed to return correct datatype.\n\n\n#### teradataml 17.00.00.03\n- New release of SQLAlchemy1.4.x introduced backward compatibility issue. A fix has been carried out so that teradataml can support latest SQLAlchemy changes.\n- Other minor bug fixes.\n\n#### teradataml 17.00.00.02\nFixed the internal library load issue related to the GCC version discrepancies on CentOS platform.\n\n#### teradataml 17.00.00.01\n* ##### New Features/Functionality\n * ###### Analytic Functions\n * Vantage Analytic Library\n teradataml now supports executing analytic functions offered by Vantage Analytic Library.\n These functions are available via new 'valib' sub-package of teradataml.\n Following functions are added as part of this:\n * Association Rules:\n * `Association()`\n * Descriptive Statistics:\n * `AdaptiveHistogram()`\n * `Explore()`\n * `Frequency()`\n * `Histogram()`\n * `Overlaps()`\n * `Statistics()`\n * `TextAnalyzer()`\n * `Values()`\n * Decision Tree:\n * `DecisionTree()`\n * `DecisionTreePredict()`\n * `DecisionTreeEvaluator()`\n * Fast K-Means Clustering:\n * `KMeans()`\n * `KMeansPredict()`\n * Linear Regression:\n * `LinReg()`\n * `LinRegPredict()`\n * Logistic Regression:\n * `LogReg()`\n * `LogRegPredict()`\n * `LogRegEvaluator()`\n * Factor Analysis:\n * `PCA()`\n * `PCAPredict()`\n * `PCAEvaluator()`\n * Matrix Building:\n * `Matrix()`\n * Statistical Tests:\n * `BinomialTest()`\n * `ChiSquareTest()`\n * `KSTest()`\n * `ParametricTest()`\n * `RankTest()`\n * Variable Transformation:\n * `Transform()`\n * Transformation Techniques supported for variable transformation:\n * `Binning()` - Perform bin coding to replaces continuous numeric column with a\n categorical one to produce ordinal values.\n * `Derive()` - Perform free-form transformation done using arithmetic formula.\n * `FillNa()` - Perform missing value/null replacement transformations.\n * `LabelEncoder()` - Re-express categorical column values into a new coding scheme.\n * `MinMaxScalar()` - Rescale data limiting the upper and lower boundaries.\n * `OneHotEncoder()` - Re-express a categorical data element as one or more\n numeric data elements, creating a binary numeric field for each\n categorical data value.\n * `Retain()` - Copy one or more columns into the final analytic data set.\n * `Sigmoid()` - Rescale data using sigmoid or s-shaped functions.\n * `ZScore()` - Rescale data using Z-Score values.\n * ML Engine Functions (mle)\n * Correlation2\n * NaiveBayesTextClassifier2\n * ###### DataFrame\n * _New Functions_\n * `DataFrame.map_row()` - Function to apply a user defined function to each row in the\n teradataml DataFrame.\n * `DataFrame.map_partition()` - Function to apply a user defined function to a group or\n partition of rows in the teradataml DataFrame.\n * _New Property_\n * `DataFrame.tdtypes` - Get the teradataml DataFrame metadata containing column names and\n corresponding teradatasqlalchemy types.\n * ###### General functions\n * _New functions_\n * Database Utility Functions\n * `db_python_package_details()` - Lists the details of Python packages installed on Vantage.\n * General Utility Functions\n * `print_options()`\n * `view_log()`\n * `setup_sandbox_env()`\n * `copy_files_from_container()`\n * `cleanup_sandbox_env()`\n* ##### Updates\n * ###### `create_context()`\n * Supports all connection parameters supported by teradatasql.connect().\n * ###### Script\n * `test_script()` can now be executed in 'local' mode, i.e., outside of the sandbox.\n * `Script.setup_sto_env()` is deprecated. Use `setup_sandbox_env()` function instead.\n * Added support for using \"quotechar\" argument.\n * ###### Analytic functions\n * _Updates_\n * Visit teradataml User Guide to know more about the updates done to ML Engine analytic\n functions. Following type of updates are done to several functions:\n * New arguments are added, which are supported only on Vantage Version 1.3.\n * Default value has been updated for few function arguments.\n * Few arguments were required, but now they are optional.\n* ##### Minor Bug Fixes.\n\n#### teradataml 17.00.00.00\n* ##### New Features/Functionality\n * ###### Model Cataloging - Functionality to catalog model metadata and related information in the Model Catalog.\n * `save_model()` - Save a teradataml Analytic Function model.\n * `retrieve_model()` - Retrieve a saved model.\n * `list_model()` - List accessible models.\n * `describe_model()` - List the details of a model.\n * `delete_model()` - Remove a model from Model Catalog.\n * `publish_model()` - Share a model.\n * ###### Script - An interface to the SCRIPT table operator object in the Advanced SQL Engine.\n Interface offers execution in two modes:\n * Test/Debug - to test user scripts locally in a containerized environment.\n Supporting methods:\n * `setup_sto_env()` - Set up test environment.\n * `test_script()` - Test user script in containerized environment.\n * `set_data()` - Set test data parameters.\n * In-Database Script Execution - to execute user scripts in database.\n Supporting methods:\n * `execute_script()` - Execute user script in Vantage.\n * `install_file()` - Install or replace file in Database.\n * `remove_file()` - Remove installed file from Database.\n * `set_data()` - Set test data parameters.\n * ###### DataFrame\n * `DataFrame.show_query()` - Show underlying query for DataFrame.\n * Regular Aggregates\n * _New functions_\n * `kurtosis()` - Calculate the kurtosis value.\n * `skew()` - Calculate the skewness of the distribution.\n * _Updates_\\\n New argument `distinct` is added to following aggregates to exclude duplicate values.\n * `count()`\n * `max()`\n * `mean()`\n * `min()`\n * `sum()`\n * `std()`\n * New argument `population` is added to calculate the population standard deviation.\n * `var()`\n * New argument `population` is added to calculate the population variance.\n * Time Series Aggregates\n * _New functions_\n * `kurtosis()` - Calculate the kurtosis value.\n * `count()` - Get the total number of values.\n * `max()` - Calculate the maximum value.\n * `mean()` - Calculate the average value.\n * `min()` - Calculate the minimum value.\n * `percentile()` - Calculate the desired percentile.\n * `skew()` - Calculate the skewness of the distribution.\n * `sum()` - Calculate the column-wise sum value.\n * `std()` - Calculate the sample and population standard deviation.\n * `var()` - Calculate the sample and population standard variance.\n * ###### General functions\n * _New functions_\n * Database Utility Functions\n * `db_drop_table()`\n * `db_drop_view()`\n * `db_list_tables()`\n * Vantage File Management Functions\n * `install_file()` - Install a file in Database.\n * `remove_file()` - Remove an installed file from Database.\n * _Updates_\n * `create_context()`\n * Support added for Stored Password Protection feature.\n * Kerberos authentication bug fix.\n * New argument `database` added to `create_context()` API, that allows user to specify connecting database.\n * ###### Analytic functions\n * _New functions_\n * `Betweenness`\n * `Closeness`\n * `FMeasure`\n * `FrequentPaths`\n * `IdentityMatch`\n * `Interpolator`\n * `ROC`\n * _Updates_\n * New methods are added to all analytic functions\n * `show_query()`\n * `get_build_time()`\n * `get_prediction_type()`\n * `get_target_column()`\n * New properties are added to analytic function's Formula argument\n * `response_column`\n * `numeric_columns`\n * `categorical_columns`\n * `all_columns`\n\n#### teradataml 16.20.00.06\nFixed the DataFrame data display corruption issue observed with certain analytic functions.\n\n#### teradataml 16.20.00.05\nCompatible with Vantage 1.1.1.\\\nThe following ML Engine (`teradataml.analytics.mle`) functions have new and/or updated arguments to support the Vantage version:\n* `AdaBoostPredict`\n* `DecisionForestPredict`\n* `DecisionTreePredict`\n* `GLMPredict`\n* `LDA`\n* `NaiveBayesPredict`\n* `NaiveBayesTextClassifierPredict`\n* `SVMDensePredict`\n* `SVMSparse`\n* `SVMSparsePredict`\n* `XGBoostPredict`\n\n#### teradataml 16.20.00.04\n* ##### Improvements\n * DataFrame creation is now quicker, impacting many APIs and Analytic functions.\n * Improved performance by reducing the number of intermediate queries issued to Teradata Vantage when not required.\n * The number of queries reduced by combining multiple operations into a single step whenever possible and unless the user expects or demands to see the intermediate results.\n * The performance improvement is almost proportional to the number of chained and unexecuted operations on a teradataml DataFrame.\n * Reduced number of intermediate internal objects created on Vantage.\n* ##### New Features/Functionality\n * ###### General functions\n * _New functions_\n * `show_versions()` - to list the version of teradataml and dependencies installed.\n * `fastload()` - for high performance data loading of large amounts of data into a table on Vantage. Requires `teradatasql` version `16.20.0.48` or above.\n * Set operators:\n * `concat`\n * `td_intersect`\n * `td_except`\n * `td_minus`\n * `case()` - to help construct SQL CASE based expressions.\n * _Updates_\n * `copy_to_sql`\n * Added support to `copy_to_sql` to save multi-level index.\n * Corrected the type mapping for index when being saved.\n * `create_context()` updated to support 'JWT' logon mechanism.\n * ###### Analytic functions\n * _New functions_\n * `NERTrainer`\n * `NERExtractor`\n * `NEREvaluator`\n * `GLML1L2`\n * `GLML1L2Predict`\n * _Updates_\n * Added support to categorize numeric columns as categorical while using formula - `as_categorical()` in the `teradataml.common.formula` module.\n * ###### DataFrame\n * Added support to create DataFrame from Volatile and Primary Time Index tables.\n * `DataFrame.sample()` - to sample data.\n * `DataFrame.index` - Property to access `index_label` of DataFrame.\n * Functionality to process Time Series Data\n * Grouping/Resampling time series data:\n * `groupby_time()`\n * `resample()`\n * Time Series Aggregates:\n * `bottom()`\n * `count()`\n * `describe()`\n * `delta_t()`\n * `mad()`\n * `median()`\n * `mode()`\n * `first()`\n * `last()`\n * `top()`\n * DataFrame API and method argument validation added.\n * `DataFrame.info()` - Default value for `null_counts` argument updated from `None` to `False`.\n * `Dataframe.merge()` updated to accept columns expressions along with column names to `on`, `left_on`, `right_on` arguments.\n * ###### DataFrame Column/ColumnExpression methods\n * `cast()` - to help cast the column to a specified type.\n * `isin()` and `~isin()` - to check the presence of values in a column.\n* ##### Removed deprecated Analytic functions\n * All the deprecated Analytic functions under the `teradataml.analytics module` have been removed.\n Newer versions of the functions are available under the `teradataml.analytics.mle` and the `teradataml.analytics.sqle` modules.\n The modules removed are:\n * `teradataml.analytics.Antiselect`\n * `teradataml.analytics.Arima`\n * `teradataml.analytics.ArimaPredictor`\n * `teradataml.analytics.Attribution`\n * `teradataml.analytics.ConfusionMatrix`\n * `teradataml.analytics.CoxHazardRatio`\n * `teradataml.analytics.CoxPH`\n * `teradataml.analytics.CoxSurvival`\n * `teradataml.analytics.DecisionForest`\n * `teradataml.analytics.DecisionForestEvaluator`\n * `teradataml.analytics.DecisionForestPredict`\n * `teradataml.analytics.DecisionTree`\n * `teradataml.analytics.DecisionTreePredict`\n * `teradataml.analytics.GLM`\n * `teradataml.analytics.GLMPredict`\n * `teradataml.analytics.KMeans`\n * `teradataml.analytics.NGrams`\n * `teradataml.analytics.NPath`\n * `teradataml.analytics.NaiveBayes`\n * `teradataml.analytics.NaiveBayesPredict`\n * `teradataml.analytics.NaiveBayesTextClassifier`\n * `teradataml.analytics.NaiveBayesTextClassifierPredict`\n * `teradataml.analytics.Pack`\n * `teradataml.analytics.SVMSparse`\n * `teradataml.analytics.SVMSparsePredict`\n * `teradataml.analytics.SentenceExtractor`\n * `teradataml.analytics.Sessionize`\n * `teradataml.analytics.TF`\n * `teradataml.analytics.TFIDF`\n * `teradataml.analytics.TextTagger`\n * `teradataml.analytics.TextTokenizer`\n * `teradataml.analytics.Unpack`\n * `teradataml.analytics.VarMax`\n\n#### teradataml 16.20.00.03\n* Fixed the garbage collection issue observed with `remove_context()` when context is created using a SQLAlchemy engine.\n* Added 4 new Advanced SQL Engine (was NewSQL Engine) analytic functions supported only on Vantage 1.1:\n * `Antiselect`, `Pack`, `StringSimilarity`, and `Unpack`.\n* Updated the Machine Learning Engine `NGrams` function to work with Vantage 1.1.\n\n#### teradataml 16.20.00.02\n* Python version 3.4.x will no longer be supported. The Python versions supported are 3.5.x, 3.6.x, and 3.7.x.\n* Major issue with the usage of formula argument in analytic functions with Python3.7 has been fixed, allowing this package to be used with Python3.7 or later.\n* Configurable alias name support for analytic functions has been added.\n* Support added to create_context (connect to Teradata Vantage) with different logon mechanisms.\n Logon mechanisms supported are: 'TD2', 'TDNEGO', 'LDAP' & 'KRB5'.\n* copy_to_sql function and DataFrame 'to_sql' methods now provide following additional functionality:\n * Create Primary Time Index tables.\n * Create set/multiset tables.\n* New DataFrame methods are added: 'median', 'var', 'squeeze', 'sort_index', 'concat'.\n* DataFrame method 'join' is now updated to make use of ColumnExpressions (df.column_name) for the 'on' clause as opposed to strings.\n* Series is supported as a first class object by calling squeeze on DataFrame.\n * Methods supported by teradataml Series are: 'head', 'unique', 'name', '\\_\\_repr__'.\n * Binary operations with teradataml Series is not yet supported. Try using Columns from teradataml.DataFrames.\n* Sample datasets and commands to load the same have been provided in the function examples.\n* New configuration property has been added 'column_casesenitive_handler'. Useful when one needs to play with case sensitive columns.\n\n#### teradataml 16.20.00.01\n* New support has been added for Linux distributions: Red Hat 7+, Ubuntu 16.04+, CentOS 7+, SLES12+.\n* 16.20.00.01 now has over 100 analytic functions. These functions have been organized into their own packages for better control over which engine to execute the analytic function on. Due to these namespace changes, the old analytic functions have been deprecated and will be removed in a future release. See the Deprecations section in the Teradata Python Package User Guide for more information.\n* New DataFrame methods `shape`, `iloc`, `describe`, `get_values`, `merge`, and `tail`.\n* New Series methods for NA checking (`isnull`, `notnull`) and string processing (`lower`, `strip`, `contains`).\n\n#### teradataml 16.20.00.00\n* `teradataml 16.20.00.00` is the first release version. Please refer to the _Teradata Python Package User Guide_ for a list of Limitations and Usage Considerations.\n\n## Installation and Requirements\n\n### Package Requirements:\n* Python 3.5 or later\n\nNote: 32-bit Python is not supported.\n\n### Minimum System Requirements:\n* Windows 7 (64Bit) or later\n* macOS 10.9 (64Bit) or later\n* Red Hat 7 or later versions\n* Ubuntu 16.04 or later versions\n* CentOS 7 or later versions\n* SLES 12 or later versions\n* Teradata Vantage Advanced SQL Engine:\n * Advanced SQL Engine 16.20 Feature Update 1 or later\n* For a Teradata Vantage system with the ML Engine:\n * Teradata Machine Learning Engine 08.00.03.01 or later\n\n### Installation\n\nUse pip to install the Teradata Python Package for Advanced Analytics.\n\nPlatform | Command\n-------------- | ---\nmacOS/Linux | `pip install teradataml`\nWindows | `py -3 -m pip install teradataml`\n\nWhen upgrading to a new version of the Teradata Python Package, you may need to use pip install's `--no-cache-dir` option to force the download of the new version.\n\nPlatform | Command\n-------------- | ---\nmacOS/Linux | `pip install --no-cache-dir -U teradataml`\nWindows | `py -3 -m pip install --no-cache-dir -U teradataml`\n\n## Using the Teradata Python Package\n\nYour Python script must import the `teradataml` package in order to use the Teradata Python Package:\n\n```\n>>> import teradataml as tdml\n>>> from teradataml import create_context, remove_context\n>>> create_context(host = 'hostname', username = 'user', password = 'password')\n>>> df = tdml.DataFrame('iris')\n>>> df\n\n SepalLength SepalWidth PetalLength PetalWidth Name\n0 5.1 3.8 1.5 0.3 Iris-setosa\n1 6.9 3.1 5.1 2.3 Iris-virginica\n2 5.1 3.5 1.4 0.3 Iris-setosa\n3 5.9 3.0 4.2 1.5 Iris-versicolor\n4 6.0 2.9 4.5 1.5 Iris-versicolor\n5 5.0 3.5 1.3 0.3 Iris-setosa\n6 5.5 2.4 3.8 1.1 Iris-versicolor\n7 6.9 3.2 5.7 2.3 Iris-virginica\n8 4.4 3.0 1.3 0.2 Iris-setosa\n9 5.8 2.7 5.1 1.9 Iris-virginica\n\n>>> df = df.select(['Name', 'SepalLength', 'PetalLength'])\n>>> df\n\n Name SepalLength PetalLength\n0 Iris-versicolor 6.0 4.5\n1 Iris-versicolor 5.5 3.8\n2 Iris-virginica 6.9 5.7\n3 Iris-setosa 5.1 1.4\n4 Iris-setosa 5.1 1.5\n5 Iris-virginica 5.8 5.1\n6 Iris-virginica 6.9 5.1\n7 Iris-setosa 5.1 1.4\n8 Iris-virginica 7.7 6.7\n9 Iris-setosa 5.0 1.3\n\n>>> df = df[(df.Name == 'Iris-setosa') & (df.PetalLength > 1.5)]\n>>> df\n\n Name SepalLength PetalLength\n0 Iris-setosa 4.8 1.9\n1 Iris-setosa 5.4 1.7\n2 Iris-setosa 5.7 1.7\n3 Iris-setosa 5.0 1.6\n4 Iris-setosa 5.1 1.9\n5 Iris-setosa 4.8 1.6\n6 Iris-setosa 4.7 1.6\n7 Iris-setosa 5.1 1.6\n8 Iris-setosa 5.1 1.7\n9 Iris-setosa 4.8 1.6\n```\n\n## Documentation\n\nGeneral product information, including installation instructions, is available in the [Teradata Documentation website](https://docs.teradata.com/search/documents?query=package+python+-lake&filters=category~%2522Programming+Reference%2522_%2522User+Guide%2522*prodname~%2522Teradata+Package+for+Python%2522_%2522Teradata+Python+Package%2522&sort=last_update&virtual-field=title_only&content-lang=)\n\n## License\n\nUse of the Teradata Python Package is governed by the *License Agreement for the Teradata Python Package for Advanced Analytics*. \nAfter installation, the `LICENSE` and `LICENSE-3RD-PARTY` files are located in the `teradataml` directory of the Python installation directory.\n\n\n",
"bugtrack_url": null,
"license": "Teradata License Agreement",
"summary": "Teradata Vantage Python package for Advanced Analytics",
"version": "20.0.0.3",
"project_urls": {
"Homepage": "http://www.teradata.com/"
},
"split_keywords": [
"teradata"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8d5f3074830669b73d8b085566ddd4ea296ee2194e3adaee41eca5ea7527e5ee",
"md5": "fa8a72a159e12a4e45a7a3e752c72b11",
"sha256": "db9526f37df6fca1437194b41007df18282c56137f872562fb7d48a9ae4410c0"
},
"downloads": -1,
"filename": "teradataml-20.0.0.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "fa8a72a159e12a4e45a7a3e752c72b11",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 8179211,
"upload_time": "2024-10-30T08:16:07",
"upload_time_iso_8601": "2024-10-30T08:16:07.683855Z",
"url": "https://files.pythonhosted.org/packages/8d/5f/3074830669b73d8b085566ddd4ea296ee2194e3adaee41eca5ea7527e5ee/teradataml-20.0.0.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-30 08:16:07",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "teradataml"
}
JSON