imbalanced-metrics


Nameimbalanced-metrics JSON
Version 0.0.3 PyPI version JSON
download
home_pagehttps://github.com/paobranco/ImbalanceMetrics
SummaryPerfromance metrics for imbalanced classification and imbalanced regression tasks
upload_time2023-04-14 03:05:07
maintainer
docs_urlNone
authorSadid Rafsun Tulon, Jean-Gabriel Gaudreault, Paula Branco
requires_python
license
keywords imbalanced learning classification regression metrics
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls. 
            ## ImbalanceMetrics

[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)

## Description
The "ImbalanceMetrics" Python package provides an extensive set of evaluation metrics designed for assessing the performance of machine learning models on imbalanced datasets, where traditional accuracy and error rate measurements may be inadequate. The package incorporates several evaluation metrics that tackle the challenges specific to imbalanced domains, offering a more accurate evaluation of model performance.
<br>

## Requirements
1. Python 3
2. scikit-learn
3. NumPy
4. Pandas
5. Smogn

## Installation
```python
## install pypi release
pip install imbalanced-metrics

## install developer version
pip install git+https://github.com/paobranco/ImbalanceMetrics.git
```

## Usage (Classification)
```python
import pandas as pd
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split
from imbalanced_metrics import classification_metrics as cm
df = pd.read_csv('poker-9_vs_7(processed).csv', header=None)
X,y=df.drop(columns=[10]),df[10]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33)
clf = DecisionTreeClassifier(max_depth=100)
clf.fit(X_train,y_train)
y_pred=clf.predict(X_test)
y_proba=clf.predict_proba(X_test)
gmean = cm.gmean_score(y_test, y_pred)
p_dav,r_dav,pra_dav=cm.pr_davis(y_test,y_proba,True) # By default 1 as positive
p_dav,r_dav,pra_dav=cm.pr_davis(y_test,y_proba,True,pos_label=0) # 0 as positive
p_man,r_man,pra_man=cm.pr_manning(y_test,y_proba,True)
cv_davis=cm.cross_validate_auc(clf,X,y,cm.pr_davis,6)
```

## Usage (Regression)
```python
import pandas as pd
from sklearn.svm import SVR
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from imbalanced_metrics import regression_metrics as rm
df = pd.read_csv('housing(processed).csv')
X,y=df.drop(columns="SalePrice"),df["SalePrice"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33)
sc = StandardScaler()
y_train = sc.fit_transform(y_train.values.reshape(-1, 1))
y_test = sc.transform (y_test.values.reshape(-1, 1))
reg = SVR().fit(X_train, y_train)
y_pred = reg.predict(X_test)
y_test=y_test.reshape(-1)
y_pred=y_pred.reshape(-1)
wmse = rm.phi_weighted_mse (y_test , y_pred)
wmae = rm.phi_weighted_mae (y_test , y_pred)
wr2 = rm.phi_weighted_r2 (y_test , y_pred)
wrmse = rm.phi_weighted_root_mse (y_test , y_pred) 
ser_t = rm.ser_t(y_test,y_pred,t=.7)
sera= rm.sera(y_test,y_pred,return_err = True)
```

## Contributions

ImablanceMetrics is open for improvements and maintenance. Your help is valued to make the package better for everyone.

## License

Licensed under the General Public License v3.0 (GPLv3).

## Reference

- Ribeiro, R.P.: Utility-based regression. Ph.D. thesis, Dep. Computer Science, Faculty of Sciences - University of Porto (2011)
- Branco, P., Ribeiro, R.P., Torgo, L.: UBL: an R package for utility-based learning (2016), https://arxiv.org/abs/1604.08079
- Branco, P., Torgo, L., Ribeiro, R.: A survey of predictive modelling under imbalanced distributions (2015)
- Branco, P., Torgo, L., Ribeiro, R.P.: A survey of predictive modeling on imbalanced domains. ACM Computing Surveys (CSUR) 49(2), 1–50 (2016)
- Branco, P., Torgo, L., Ribeiro, R.P.: SMOGN: a pre-processing approach for imbalanced regression. In: First international workshop on learning with imbalanced domains: Theory and applications. pp. 36–50. PMLR (2017)
- Cordón, I., García, S., Fernández, A., Herrera, F.: Imbalance: Oversampling algorithms for imbalanced classification in r. Knowledge-Based Systems 161, 329–341 (2018), https://doi.org/10.1016/j.knosys.2018.07.035
- Davis, J., Goadrich, M.: The relationship between precision-recall and roc curves. vol. 06 (06 2006). https://doi.org/10.1145/1143844.1143874
- Derrac, J., Garcia, S., Sanchez, L., Herrera, F.: Keel data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework. J. Mult. Valued Logic Soft Comput 17 (2015)
- Gaudreault, J.G., Branco, P., Gama, J.: An analysis of performance metrics for imbalanced classification. In: Discovery Science: 24th International Conference, DS 2021, Halifax, NS, Canada, October 11–13, 2021, Proceedings. pp. 67–77 (2021)
- Kubat, M., Matwin, S., et al.: Addressing the curse of imbalanced training sets: one-sided selection. In: Icml. vol. 97, p. 179. Citeseer (1997)
- Kunz, N.: SMOGN: Synthetic minority over-sampling technique for regression with gaussian noise (2020), https://pypi.org/project/smogn
- Lemaître, G., Nogueira, F., Aridas, C.K.: Imbalanced-learn: A python toolbox to tackle the curse of imbalanced datasets in machine learning. JMLR 18(17), 1–5 (2017),  http://jmlr.org/papers/v18/16-365.html
- Ribeiro, R.: Utility-based Regression. Ph.D. thesis, Dep. Computer Science, Faculty of Sciences - University of Porto (2011)
- Ribeiro, R., Moniz, N.: Imbalanced regression and extreme value prediction. Machine Learning 109,1–33 (09 2020). https://doi.org/10.1007/s10994-020-05900-9
- berreergun: Ironpy. https://github.com/berreergun/IRonPy (2021)

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/paobranco/ImbalanceMetrics",
    "name": "imbalanced-metrics",
    "maintainer": "",
    "docs_url": null,
    "requires_python": "",
    "maintainer_email": "",
    "keywords": "imbalanced learning,classification,regression,metrics",
    "author": "Sadid Rafsun Tulon, Jean-Gabriel Gaudreault, Paula Branco",
    "author_email": "stulo080@uottawa.ca, j.gaudreault@uottawa.ca, pbranco@uottawa.ca",
    "download_url": "https://files.pythonhosted.org/packages/4b/cb/52d85e50ec064a305797955a2a02cc30549d44db5f843d4dc83877f91a4c/imbalanced_metrics-0.0.3.tar.gz",
    "platform": null,
    "description": "## ImbalanceMetrics\r\n\r\n[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)\r\n\r\n## Description\r\nThe \"ImbalanceMetrics\" Python package provides an extensive set of evaluation metrics designed for assessing the performance of machine learning models on imbalanced datasets, where traditional accuracy and error rate measurements may be inadequate. The package incorporates several evaluation metrics that tackle the challenges specific to imbalanced domains, offering a more accurate evaluation of model performance.\r\n<br>\r\n\r\n## Requirements\r\n1. Python 3\r\n2. scikit-learn\r\n3. NumPy\r\n4. Pandas\r\n5. Smogn\r\n\r\n## Installation\r\n```python\r\n## install pypi release\r\npip install imbalanced-metrics\r\n\r\n## install developer version\r\npip install git+https://github.com/paobranco/ImbalanceMetrics.git\r\n```\r\n\r\n## Usage (Classification)\r\n```python\r\nimport pandas as pd\r\nfrom sklearn.tree import DecisionTreeClassifier\r\nfrom sklearn.model_selection import train_test_split\r\nfrom imbalanced_metrics import classification_metrics as cm\r\ndf = pd.read_csv('poker-9_vs_7(processed).csv', header=None)\r\nX,y=df.drop(columns=[10]),df[10]\r\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33)\r\nclf = DecisionTreeClassifier(max_depth=100)\r\nclf.fit(X_train,y_train)\r\ny_pred=clf.predict(X_test)\r\ny_proba=clf.predict_proba(X_test)\r\ngmean = cm.gmean_score(y_test, y_pred)\r\np_dav,r_dav,pra_dav=cm.pr_davis(y_test,y_proba,True) # By default 1 as positive\r\np_dav,r_dav,pra_dav=cm.pr_davis(y_test,y_proba,True,pos_label=0) # 0 as positive\r\np_man,r_man,pra_man=cm.pr_manning(y_test,y_proba,True)\r\ncv_davis=cm.cross_validate_auc(clf,X,y,cm.pr_davis,6)\r\n```\r\n\r\n## Usage (Regression)\r\n```python\r\nimport pandas as pd\r\nfrom sklearn.svm import SVR\r\nfrom sklearn.model_selection import train_test_split\r\nfrom sklearn.preprocessing import StandardScaler\r\nfrom imbalanced_metrics import regression_metrics as rm\r\ndf = pd.read_csv('housing(processed).csv')\r\nX,y=df.drop(columns=\"SalePrice\"),df[\"SalePrice\"]\r\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33)\r\nsc = StandardScaler()\r\ny_train = sc.fit_transform(y_train.values.reshape(-1, 1))\r\ny_test = sc.transform (y_test.values.reshape(-1, 1))\r\nreg = SVR().fit(X_train, y_train)\r\ny_pred = reg.predict(X_test)\r\ny_test=y_test.reshape(-1)\r\ny_pred=y_pred.reshape(-1)\r\nwmse = rm.phi_weighted_mse (y_test , y_pred)\r\nwmae = rm.phi_weighted_mae (y_test , y_pred)\r\nwr2 = rm.phi_weighted_r2 (y_test , y_pred)\r\nwrmse = rm.phi_weighted_root_mse (y_test , y_pred) \r\nser_t = rm.ser_t(y_test,y_pred,t=.7)\r\nsera= rm.sera(y_test,y_pred,return_err = True)\r\n```\r\n\r\n## Contributions\r\n\r\nImablanceMetrics is open for improvements and maintenance. Your help is valued to make the package better for everyone.\r\n\r\n## License\r\n\r\nLicensed under the General Public License v3.0 (GPLv3).\r\n\r\n## Reference\r\n\r\n- Ribeiro, R.P.: Utility-based regression. Ph.D. thesis, Dep. Computer Science, Faculty of Sciences - University of Porto (2011)\r\n- Branco, P., Ribeiro, R.P., Torgo, L.: UBL: an R package for utility-based learning (2016), https://arxiv.org/abs/1604.08079\r\n- Branco, P., Torgo, L., Ribeiro, R.: A survey of predictive modelling under imbalanced distributions (2015)\r\n- Branco, P., Torgo, L., Ribeiro, R.P.: A survey of predictive modeling on imbalanced domains. ACM Computing Surveys (CSUR) 49(2), 1\u00e2\u20ac\u201c50 (2016)\r\n- Branco, P., Torgo, L., Ribeiro, R.P.: SMOGN: a pre-processing approach for imbalanced regression. In: First international workshop on learning with imbalanced domains: Theory and applications. pp. 36\u00e2\u20ac\u201c50. PMLR (2017)\r\n- Cord\u00c3\u00b3n, I., Garc\u00c3\u00ada, S., Fern\u00c3\u00a1ndez, A., Herrera, F.: Imbalance: Oversampling algorithms for imbalanced classification in r. Knowledge-Based Systems 161, 329\u00e2\u20ac\u201c341 (2018), https://doi.org/10.1016/j.knosys.2018.07.035\r\n- Davis, J., Goadrich, M.: The relationship between precision-recall and roc curves. vol. 06 (06 2006). https://doi.org/10.1145/1143844.1143874\r\n- Derrac, J., Garcia, S., Sanchez, L., Herrera, F.: Keel data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework. J. Mult. Valued Logic Soft Comput 17 (2015)\r\n- Gaudreault, J.G., Branco, P., Gama, J.: An analysis of performance metrics for imbalanced classification. In: Discovery Science: 24th International Conference, DS 2021, Halifax, NS, Canada, October 11\u00e2\u20ac\u201c13, 2021, Proceedings. pp. 67\u00e2\u20ac\u201c77 (2021)\r\n- Kubat, M., Matwin, S., et al.: Addressing the curse of imbalanced training sets: one-sided selection. In: Icml. vol. 97, p. 179. Citeseer (1997)\r\n- Kunz, N.: SMOGN: Synthetic minority over-sampling technique for regression with gaussian noise (2020), https://pypi.org/project/smogn\r\n- Lema\u00c3\u00aetre, G., Nogueira, F., Aridas, C.K.: Imbalanced-learn: A python toolbox to tackle the curse of imbalanced datasets in machine learning. JMLR 18(17), 1\u00e2\u20ac\u201c5 (2017),  http://jmlr.org/papers/v18/16-365.html\r\n- Ribeiro, R.: Utility-based Regression. Ph.D. thesis, Dep. Computer Science, Faculty of Sciences - University of Porto (2011)\r\n- Ribeiro, R., Moniz, N.: Imbalanced regression and extreme value prediction. Machine Learning 109,1\u00e2\u20ac\u201c33 (09 2020). https://doi.org/10.1007/s10994-020-05900-9\r\n- berreergun: Ironpy. https://github.com/berreergun/IRonPy (2021)\r\n",
    "bugtrack_url": null,
    "license": "",
    "summary": "Perfromance metrics for imbalanced classification and imbalanced regression tasks",
    "version": "0.0.3",
    "split_keywords": [
        "imbalanced learning",
        "classification",
        "regression",
        "metrics"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "53ed7951b484f03f47fa55d10e2f61140279f0065521b7d1504eedfc408d1d00",
                "md5": "b3ed35edc25e7021b13c9d0a37c126df",
                "sha256": "718796d12f09e92c18167782bdf793448eb5e7c62dec08a9df70febe6e84506e"
            },
            "downloads": -1,
            "filename": "imbalanced_metrics-0.0.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "b3ed35edc25e7021b13c9d0a37c126df",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 20255,
            "upload_time": "2023-04-14T03:05:05",
            "upload_time_iso_8601": "2023-04-14T03:05:05.429623Z",
            "url": "https://files.pythonhosted.org/packages/53/ed/7951b484f03f47fa55d10e2f61140279f0065521b7d1504eedfc408d1d00/imbalanced_metrics-0.0.3-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "4bcb52d85e50ec064a305797955a2a02cc30549d44db5f843d4dc83877f91a4c",
                "md5": "b4a0c73127318c48125ca5ff3612e496",
                "sha256": "e897fb1583f47bd0aa4a09671c409ff63962ed127457fcb49ee9ada561041142"
            },
            "downloads": -1,
            "filename": "imbalanced_metrics-0.0.3.tar.gz",
            "has_sig": false,
            "md5_digest": "b4a0c73127318c48125ca5ff3612e496",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 22006,
            "upload_time": "2023-04-14T03:05:07",
            "upload_time_iso_8601": "2023-04-14T03:05:07.520432Z",
            "url": "https://files.pythonhosted.org/packages/4b/cb/52d85e50ec064a305797955a2a02cc30549d44db5f843d4dc83877f91a4c/imbalanced_metrics-0.0.3.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-04-14 03:05:07",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "github_user": "paobranco",
    "github_project": "ImbalanceMetrics",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "lcname": "imbalanced-metrics"
}
Sadid Rafsun Tulon, Jean-Gabriel Gaudreault, Paula Branco
Elapsed time: 0.05530s