      
# Zoish
Zoish is a Python package that simplifies the machine learning process by using SHAP values for feature importance. It integrates with a range of machine learning models, provides feature selection to enhance performance, and improves model interpretability. With Zoish, users can also visualize feature importance through SHAP summary and bar plots, creating an efficient and user-friendly environment for machine learning development.
# Introduction
Zoish is a powerful tool for streamlining your machine learning pipeline by leveraging SHAP (SHapley Additive exPlanations) values for feature selection. Designed to work seamlessly with binary and multi-class classification models as well as regression models from sklearn, Zoish is also compatible with gradient boosting frameworks such as CatBoost, LightGBM and GPBoost.
## Features
- **Model Flexibility:** Zoish exhibits outstanding flexibility as it can work with most of the estimators and others supported by and even GPBoost or a superior estimator emerging from a tree-based optimization process. This enables it to integrate seamlessly into binary or multi-class Sklearn classification models, all Sklearn regression models, as well as with advanced gradient boosting frameworks such as CatBoost, LightGBM and GPBoost.
- **Feature Selection:** By utilizing SHAP values, Zoish efficiently determines the most influential features for your predictive models. This improves the interpretability of your model and can potentially enhance model performance by reducing overfitting.
- **Visualization:** Zoish includes capabilities for plotting important features using SHAP summary plots and SHAP bar plots, providing a clear and visual representation of feature importance.
## Dependencies
The core dependency of Zoish is the `shap` package, which is used to compute the SHAP values for tree based machine learning model and others too. SHAP values are a unified measure of feature importance and they offer an improved interpretation of machine learning models. They are based on the concept of cooperative game theory and provide a fair allocation of the contribution of each feature to the prediction of each instance.
## Installation
To install Zoish, use pip:
## Installation
Zoish package is available on PyPI and can be installed with pip:
```sh
pip install zoish
```
For log configuration in development environment use
```sh
export env=dev
```
For log configuration in production environment use
```sh
export env=prod
```
# Examples
```
# Built-in libraries
import pandas as pd
# Scikit-learn libraries for model selection, metrics, pipeline, impute, preprocessing, compose, and ensemble
from sklearn.compose import ColumnTransformer
from sklearn.ensemble import RandomForestClassifier
from sklearn.feature_selection import SelectFromModel
from sklearn.impute import SimpleImputer
from sklearn.metrics import classification_report, confusion_matrix, f1_score, make_scorer
from sklearn.model_selection import GridSearchCV, KFold, train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
# Other libraries
from category_encoders import TargetEncoder
from xgboost import XGBClassifier
from zoish.feature_selectors.shap_selectors import ShapFeatureSelector, ShapPlotFeatures
import logging
from zoish import logger
logger.setLevel(logging.ERROR)
from feature_engine.imputation import (
CategoricalImputer,
MeanMedianImputer
)
# Set logging level
logger.setLevel(logging.ERROR)
```
#### Example: Audiology (Standardized) Data Set
###### https://archive.ics.uci.edu/ml/datasets/Audiology+%28Standardized%29
#### Read data
```
urldata = "https://archive.ics.uci.edu/ml/machine-learning-databases/lymphography/lymphography.data"
urlname = "https://archive.ics.uci.edu/ml/machine-learning-databases/lung-cancer/lung-cancer.names"
# column names
col_names = [
"class",
"lymphatics",
"block of affere",
"bl. of lymph. c",
"bl. of lymph. s",
"by pass",
"extravasates",
"regeneration of",
"early uptake in",
"lym.nodes dimin",
"lym.nodes enlar",
"changes in lym.",
"defect in node",
"changes in node",
"special forms",
"dislocation of",
"exclusion of no",
"no. of nodes in",
]
data = pd.read_csv(urldata,names=col_names)
data.head()
```
#### Define labels and train-test split
```
data.loc[(data["class"] == 1) | (data["class"] == 2), "class"] = 0
data.loc[data["class"] == 3, "class"] = 1
data.loc[data["class"] == 4, "class"] = 2
data["class"] = data["class"].astype(int)
```
#### Train test split
```
X = data.loc[:, data.columns != "class"]
y = data.loc[:, data.columns == "class"]
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.33, random_state=42
)
```
#### Defining the feature pipeline steps:
Here, we use an untuned XGBClassifier model with the ShapFeatureSelector.In the next section, we will repeat the same process but with a tuned XGBClassifier. The aim is to demonstrate that a better estimator can yield improved results when used with the ShapFeatureSelector.
```
estimator_for_feature_selector= XGBClassifier()
estimator_for_feature_selector.fit(X_train, y_train)
shap_feature_selector = ShapFeatureSelector(model=estimator_for_feature_selector, num_features=5, cv = 5, scoring='accuracy', direction='maximum', n_iter=10, algorithm='auto')
# Define pre-processing for numeric columns (float and integer types)
numeric_features = X_train.select_dtypes(include=['int64', 'float64']).columns
numeric_transformer = Pipeline(steps=[
('imputer', SimpleImputer(strategy='mean')),
('scaler', StandardScaler())])
# Define pre-processing for categorical features
categorical_features = X_train.select_dtypes(include=['object']).columns
categorical_transformer = Pipeline(steps=[
('imputer', SimpleImputer(strategy='constant', fill_value='missing')),
('encoder', TargetEncoder(handle_missing='return_nan'))])
# Combine preprocessing into one column transformer
preprocessor = ColumnTransformer(
transformers=[
('num', numeric_transformer, numeric_features),
('cat', categorical_transformer, categorical_features)])
# Feature Selection using ShapSelector
feature_selection = shap_feature_selector
# Classifier model
classifier = RandomForestClassifier(n_estimators=100)
# Create a pipeline that combines the preprocessor with a feature selection and a classifier
pipeline = Pipeline(steps=[('preprocessor', preprocessor),
('feature_selection', feature_selection),
('classifier', classifier)])
# Fit the model
pipeline.fit(X_train, y_train)
# Predict on test data
y_test_pred = pipeline.predict(X_test)
# Output first 10 predictions
print(y_test_pred[:10])
```
#### Check performance of the Pipeline
```
print("F1 score : ")
print(f1_score(y_test, y_test_pred,average='micro'))
print("Classification report : ")
print(classification_report(y_test, y_test_pred))
print("Confusion matrix : ")
print(confusion_matrix(y_test, y_test_pred))
```
#### Use better estimator:
In this iteration, we will utilize the optimally tuned estimator with the ShapFeatureSelector, which is expected to yield improved results."
```
int_cols = X_train.select_dtypes(include=['int']).columns.tolist()
# Define the XGBClassifier
xgb_clf = XGBClassifier()
# Define the parameter grid for XGBClassifier
param_grid = {
'learning_rate': [0.01, 0.1],
'max_depth': [ 4, 5],
'min_child_weight': [1, 2, 3],
'gamma': [0, 0.1, 0.2],
}
# Define the scoring function
scoring = make_scorer(f1_score, average='micro') # Use 'micro' average in case of multiclass target
# Set up GridSearchCV
grid_search = GridSearchCV(xgb_clf, param_grid, cv=5, scoring=scoring, verbose=1)
grid_search.fit(X_train, y_train)
# Fit the GridSearchCV object
estimator_for_feature_selector= grid_search.best_estimator_
shap_feature_selector = ShapFeatureSelector(model=estimator_for_feature_selector, num_features=5, scoring='accuracy', algorithm='auto',cv = 5, n_iter=10, direction='maximum')
pipeline =Pipeline([
# int missing values imputers
('floatimputer', MeanMedianImputer(
imputation_method='mean', variables=int_cols)),
('shap_feature_selector', shap_feature_selector),
('classfier', RandomForestClassifier(n_estimators=100))
])
# Fit the model
pipeline.fit(X_train, y_train)
# Predict on test data
y_test_pred = pipeline.predict(X_test)
# Output first 10 predictions
print(y_test_pred[:10])
```
#### Performance has improved
```
print("F1 score : ")
print(f1_score(y_test, y_test_pred,average='micro'))
print("Classification report : ")
print(classification_report(y_test, y_test_pred))
print("Confusion matrix : ")
print(confusion_matrix(y_test, y_test_pred))
#### Shap related plots
```
#### Plot the features importance
```
plot_factory = ShapPlotFeatures(shap_feature_selector)
```
#### Summary Plot of the selected features
```
plot_factory.summary_plot()
```
#### Summary Plot of the all features
```
plot_factory.summary_plot_full()
```
#### Bar Plot of the selected features
```
plot_factory.bar_plot()
```
#### Bar Plot of the all features
```
plot_factory.bar_plot_full()
```
More examples are available in the [examples](https://github.com/drhosseinjavedani/zoish/tree/main/examples).
## License
Licensed under the [BSD 2-Clause](https://opensource.org/licenses/BSD-2-Clause) License.
Raw data
{
"_id": null,
"home_page": "https://github.com/TorkamaniLab/zoish",
"name": "zoish",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "Auto ML,Feature Selection,Pipeline,Machine learning,shap",
"author": "drhosseinjavedani",
"author_email": "h.javedani@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/ff/e7/ac29f5bca9794dd5ebbf1dd471b838003bada6660e775eed81f88012c618/zoish-5.0.4.tar.gz",
"platform": null,
"description": "\n       \n\n# Zoish\n\nZoish is a Python package that simplifies the machine learning process by using SHAP values for feature importance. It integrates with a range of machine learning models, provides feature selection to enhance performance, and improves model interpretability. With Zoish, users can also visualize feature importance through SHAP summary and bar plots, creating an efficient and user-friendly environment for machine learning development.\n\n# Introduction\n\nZoish is a powerful tool for streamlining your machine learning pipeline by leveraging SHAP (SHapley Additive exPlanations) values for feature selection. Designed to work seamlessly with binary and multi-class classification models as well as regression models from sklearn, Zoish is also compatible with gradient boosting frameworks such as CatBoost, LightGBM and GPBoost.\n\n## Features\n\n- **Model Flexibility:** Zoish exhibits outstanding flexibility as it can work with most of the estimators and others supported by and even GPBoost or a superior estimator emerging from a tree-based optimization process. This enables it to integrate seamlessly into binary or multi-class Sklearn classification models, all Sklearn regression models, as well as with advanced gradient boosting frameworks such as CatBoost, LightGBM and GPBoost.\n \n- **Feature Selection:** By utilizing SHAP values, Zoish efficiently determines the most influential features for your predictive models. This improves the interpretability of your model and can potentially enhance model performance by reducing overfitting.\n\n- **Visualization:** Zoish includes capabilities for plotting important features using SHAP summary plots and SHAP bar plots, providing a clear and visual representation of feature importance.\n\n## Dependencies\n\nThe core dependency of Zoish is the `shap` package, which is used to compute the SHAP values for tree based machine learning model and others too. SHAP values are a unified measure of feature importance and they offer an improved interpretation of machine learning models. They are based on the concept of cooperative game theory and provide a fair allocation of the contribution of each feature to the prediction of each instance.\n\n## Installation\n\nTo install Zoish, use pip:\n\n## Installation\n\nZoish package is available on PyPI and can be installed with pip:\n\n```sh\npip install zoish\n```\n\nFor log configuration in development environment use \n\n```sh\nexport env=dev\n\n```\n\nFor log configuration in production environment use \n\n```sh\nexport env=prod\n```\n\n# Examples \n\n```\n\n# Built-in libraries\nimport pandas as pd\n\n# Scikit-learn libraries for model selection, metrics, pipeline, impute, preprocessing, compose, and ensemble\nfrom sklearn.compose import ColumnTransformer\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.feature_selection import SelectFromModel\nfrom sklearn.impute import SimpleImputer\nfrom sklearn.metrics import classification_report, confusion_matrix, f1_score, make_scorer\nfrom sklearn.model_selection import GridSearchCV, KFold, train_test_split\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.preprocessing import StandardScaler\n\n# Other libraries\nfrom category_encoders import TargetEncoder\nfrom xgboost import XGBClassifier\nfrom zoish.feature_selectors.shap_selectors import ShapFeatureSelector, ShapPlotFeatures\nimport logging\nfrom zoish import logger\nlogger.setLevel(logging.ERROR)\nfrom feature_engine.imputation import (\n CategoricalImputer,\n MeanMedianImputer\n )\n\n# Set logging level\nlogger.setLevel(logging.ERROR)\n\n```\n\n#### Example: Audiology (Standardized) Data Set\n###### https://archive.ics.uci.edu/ml/datasets/Audiology+%28Standardized%29\n\n\n#### Read data\n\n```\nurldata = \"https://archive.ics.uci.edu/ml/machine-learning-databases/lymphography/lymphography.data\"\nurlname = \"https://archive.ics.uci.edu/ml/machine-learning-databases/lung-cancer/lung-cancer.names\"\n# column names\ncol_names = [\n \"class\",\n \"lymphatics\",\n \"block of affere\",\n \"bl. of lymph. c\",\n \"bl. of lymph. s\",\n \"by pass\",\n \"extravasates\",\n \"regeneration of\",\n \"early uptake in\",\n \"lym.nodes dimin\",\n \"lym.nodes enlar\",\n \"changes in lym.\",\n \"defect in node\",\n \"changes in node\",\n \"special forms\",\n \"dislocation of\",\n \"exclusion of no\",\n \"no. of nodes in\",\n\n]\n\ndata = pd.read_csv(urldata,names=col_names)\ndata.head()\n\n```\n\n#### Define labels and train-test split\n\n\n```\n\n\ndata.loc[(data[\"class\"] == 1) | (data[\"class\"] == 2), \"class\"] = 0\ndata.loc[data[\"class\"] == 3, \"class\"] = 1\ndata.loc[data[\"class\"] == 4, \"class\"] = 2\ndata[\"class\"] = data[\"class\"].astype(int)\n```\n\n#### Train test split\n\n```\nX = data.loc[:, data.columns != \"class\"]\ny = data.loc[:, data.columns == \"class\"]\n\nX_train, X_test, y_train, y_test = train_test_split(\n X, y, test_size=0.33, random_state=42\n)\n```\n\n#### Defining the feature pipeline steps:\nHere, we use an untuned XGBClassifier model with the ShapFeatureSelector.In the next section, we will repeat the same process but with a tuned XGBClassifier. The aim is to demonstrate that a better estimator can yield improved results when used with the ShapFeatureSelector.\n\n```\nestimator_for_feature_selector= XGBClassifier() \nestimator_for_feature_selector.fit(X_train, y_train)\nshap_feature_selector = ShapFeatureSelector(model=estimator_for_feature_selector, num_features=5, cv = 5, scoring='accuracy', direction='maximum', n_iter=10, algorithm='auto')\n \n# Define pre-processing for numeric columns (float and integer types)\nnumeric_features = X_train.select_dtypes(include=['int64', 'float64']).columns\nnumeric_transformer = Pipeline(steps=[\n ('imputer', SimpleImputer(strategy='mean')),\n ('scaler', StandardScaler())])\n\n# Define pre-processing for categorical features\ncategorical_features = X_train.select_dtypes(include=['object']).columns\ncategorical_transformer = Pipeline(steps=[\n ('imputer', SimpleImputer(strategy='constant', fill_value='missing')),\n ('encoder', TargetEncoder(handle_missing='return_nan'))])\n\n# Combine preprocessing into one column transformer\npreprocessor = ColumnTransformer(\n transformers=[\n ('num', numeric_transformer, numeric_features),\n ('cat', categorical_transformer, categorical_features)])\n\n# Feature Selection using ShapSelector \nfeature_selection = shap_feature_selector \n\n# Classifier model\nclassifier = RandomForestClassifier(n_estimators=100)\n\n# Create a pipeline that combines the preprocessor with a feature selection and a classifier\npipeline = Pipeline(steps=[('preprocessor', preprocessor),\n ('feature_selection', feature_selection),\n ('classifier', classifier)])\n\n# Fit the model\npipeline.fit(X_train, y_train)\n\n# Predict on test data\ny_test_pred = pipeline.predict(X_test)\n\n# Output first 10 predictions\nprint(y_test_pred[:10])\n```\n\n#### Check performance of the Pipeline\n\n```\n\nprint(\"F1 score : \")\nprint(f1_score(y_test, y_test_pred,average='micro'))\nprint(\"Classification report : \")\nprint(classification_report(y_test, y_test_pred))\nprint(\"Confusion matrix : \")\nprint(confusion_matrix(y_test, y_test_pred))\n\n\n```\n\n\n#### Use better estimator:\nIn this iteration, we will utilize the optimally tuned estimator with the ShapFeatureSelector, which is expected to yield improved results.\"\n\n```\nint_cols = X_train.select_dtypes(include=['int']).columns.tolist()\n\n\n# Define the XGBClassifier\nxgb_clf = XGBClassifier()\n\n# Define the parameter grid for XGBClassifier\nparam_grid = {\n 'learning_rate': [0.01, 0.1],\n 'max_depth': [ 4, 5],\n 'min_child_weight': [1, 2, 3],\n 'gamma': [0, 0.1, 0.2],\n}\n\n# Define the scoring function\nscoring = make_scorer(f1_score, average='micro') # Use 'micro' average in case of multiclass target\n\n# Set up GridSearchCV\ngrid_search = GridSearchCV(xgb_clf, param_grid, cv=5, scoring=scoring, verbose=1)\ngrid_search.fit(X_train, y_train)\n# Fit the GridSearchCV object\nestimator_for_feature_selector= grid_search.best_estimator_ \nshap_feature_selector = ShapFeatureSelector(model=estimator_for_feature_selector, num_features=5, scoring='accuracy', algorithm='auto',cv = 5, n_iter=10, direction='maximum')\n \n\npipeline =Pipeline([\n # int missing values imputers\n ('floatimputer', MeanMedianImputer(\n imputation_method='mean', variables=int_cols)),\n \n ('shap_feature_selector', shap_feature_selector),\n ('classfier', RandomForestClassifier(n_estimators=100))\n\n\n ])\n\n\n# Fit the model\npipeline.fit(X_train, y_train)\n\n# Predict on test data\ny_test_pred = pipeline.predict(X_test)\n\n# Output first 10 predictions\nprint(y_test_pred[:10])\n \n```\n\n#### Performance has improved\n\n```\n\nprint(\"F1 score : \")\nprint(f1_score(y_test, y_test_pred,average='micro'))\nprint(\"Classification report : \")\nprint(classification_report(y_test, y_test_pred))\nprint(\"Confusion matrix : \")\nprint(confusion_matrix(y_test, y_test_pred))\n\n#### Shap related plots\n\n```\n\n#### Plot the features importance\n```\nplot_factory = ShapPlotFeatures(shap_feature_selector) \n\n```\n\n#### Summary Plot of the selected features\n```\nplot_factory.summary_plot()\n\n```\n\n\n#### Summary Plot of the all features\n```\nplot_factory.summary_plot_full()\n\n```\n\n\n#### Bar Plot of the selected features\n\n```\nplot_factory.bar_plot()\n```\n\n\n#### Bar Plot of the all features\n\n```\nplot_factory.bar_plot_full()\n```\n\n\nMore examples are available in the [examples](https://github.com/drhosseinjavedani/zoish/tree/main/examples). \n\n## License\nLicensed under the [BSD 2-Clause](https://opensource.org/licenses/BSD-2-Clause) License.\n\n",
"bugtrack_url": null,
"license": "BSD-3-Clause license",
"summary": "Zoish is a Python package that streamlines machine learning by leveraging SHAP values for feature selection and interpretability, making model development more efficient and user-friendly.",
"version": "5.0.4",
"project_urls": {
"Homepage": "https://github.com/TorkamaniLab/zoish"
},
"split_keywords": [
"auto ml",
"feature selection",
"pipeline",
"machine learning",
"shap"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "bc878cbc2370a59e3f89cbe925730c3c479c06b4d16f372c86c2579fe1ca977b",
"md5": "7c28e6238715cfdade540a7d913372c5",
"sha256": "7712d3efc2861712201493754f095d56dedaec1a644061bc0f2158a1603721a5"
},
"downloads": -1,
"filename": "zoish-5.0.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7c28e6238715cfdade540a7d913372c5",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 36387,
"upload_time": "2024-02-07T07:24:25",
"upload_time_iso_8601": "2024-02-07T07:24:25.188604Z",
"url": "https://files.pythonhosted.org/packages/bc/87/8cbc2370a59e3f89cbe925730c3c479c06b4d16f372c86c2579fe1ca977b/zoish-5.0.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ffe7ac29f5bca9794dd5ebbf1dd471b838003bada6660e775eed81f88012c618",
"md5": "c0e19e8f8ee95ddb245edf067e908f0b",
"sha256": "4f93f67fda9b6ce6058b1ccaee69e555791499cf12c6f479ac5b284b46e5152f"
},
"downloads": -1,
"filename": "zoish-5.0.4.tar.gz",
"has_sig": false,
"md5_digest": "c0e19e8f8ee95ddb245edf067e908f0b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 17715,
"upload_time": "2024-02-07T07:24:26",
"upload_time_iso_8601": "2024-02-07T07:24:26.922787Z",
"url": "https://files.pythonhosted.org/packages/ff/e7/ac29f5bca9794dd5ebbf1dd471b838003bada6660e775eed81f88012c618/zoish-5.0.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-07 07:24:26",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "TorkamaniLab",
"github_project": "zoish",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "zoish"
}
JSON
