## deforce: Derivative-Free Algorithms for Optimizing Cascade Forward Neural Networks
---
[](https://github.com/thieu1995/deforce/releases)
[](https://pypi.python.org/pypi/deforce)
[](https://badge.fury.io/py/deforce)
[](https://pepy.tech/project/deforce)
[](https://github.com/thieu1995/deforce/actions/workflows/publish-package.yaml)
[](https://deforce.readthedocs.io/en/latest/?badge=latest)
[](https://t.me/+fRVCJGuGJg1mNDg1)
[](https://git-scm.com/book/en/v2/GitHub-Contributing-to-a-Project)
[](https://doi.org/10.5281/zenodo.10935437)
[](https://www.gnu.org/licenses/gpl-3.0)
`deforce` (DErivative Free Optimization foR Cascade forward nEural networks) is a Python library that implements
variants and the traditional version of Cascade Forward Neural Networks. These include Derivative Free-optimized CFN
models (such as genetic algorithm, particle swarm optimization, whale optimization algorithm, teaching learning
optimization, differential evolution, ...) and Gradient Descent-optimized CFN models (such as stochastic gradient
descent, Adam optimizer, Adelta optimizer, ...). It provides a comprehensive list of optimizers for training CFN
models and is also compatible with the Scikit-Learn library. With deforce, you can perform searches and
hyperparameter tuning for traditional CFN networks using the features provided by the Scikit-Learn library.
* **Free software:** GNU General Public License (GPL) V3 license
* **Provided Estimator**: `CfnRegressor`, `CfnClassifier`, `DfoCfnRegressor`, `DfoCfnClassifier`, `DfoTuneCfn`
* **Total DFO-based CFN Regressor**: > 200 Models
* **Total DFO-based CFN Classifier**: > 200 Models
* **Total GD-based CFN Regressor**: 12 Models
* **Total GD-based CFN Classifier**: 12 Models
* **Supported performance metrics**: >= 67 (47 regressions and 20 classifications)
* **Supported objective functions**: >= 67 (47 regressions and 20 classifications)
* **Documentation:** https://deforce.readthedocs.io
* **Python versions:** >= 3.8.x
* **Dependencies:** numpy, scipy, scikit-learn, pandas, mealpy, permetrics, torch, skorch
# Citation Request
If you want to understand how to use Derivative Free-optimized Cascade Forward Neural Network, you
need to read the paper titled **"Optimization of neural-network model using a meta-heuristic algorithm for the estimation of dynamic Poisson’s ratio of selected rock types"**.
The paper can be accessed at the following [link](https://doi.org/10.1038%2Fs41598-023-38163-0)
Please include these citations if you plan to use this library:
```bibtex
@software{thieu_deforce_2024,
author = {Van Thieu, Nguyen},
title = {{deforce: Derivative-Free Algorithms for Optimizing Cascade Forward Neural Networks}},
url = {https://github.com/thieu1995/deforce},
doi = {10.5281/zenodo.10935437},
year = {2024}
}
@article{van2023mealpy,
title={MEALPY: An open-source library for latest meta-heuristic algorithms in Python},
author={Van Thieu, Nguyen and Mirjalili, Seyedali},
journal={Journal of Systems Architecture},
year={2023},
publisher={Elsevier},
doi={10.1016/j.sysarc.2023.102871}
}
@article{van2023groundwater,
title={Groundwater level modeling using Augmented Artificial Ecosystem Optimization},
author={Van Thieu, Nguyen and Barma, Surajit Deb and Van Lam, To and Kisi, Ozgur and Mahesha, Amai},
journal={Journal of Hydrology},
volume={617},
pages={129034},
year={2023},
publisher={Elsevier}
}
```
# Installation
* Install the [current PyPI release](https://pypi.python.org/pypi/deforce):
```sh
$ pip install deforce
```
After installation, check the installed version by:
```sh
$ python
>>> import deforce
>>> deforce.__version__
```
### Examples
Please check [documentation website](https://deforce.readthedocs.io/) and [examples folder](examples).
1) `deforce` provides this useful classes
```python
from deforce import DataTransformer, Data
from deforce import CfnRegressor, CfnClassifier
from deforce import DfoCfnRegressor, DfoCfnClassifier
```
2) What can you do with all `model` classes
```python
from deforce import CfnRegressor, CfnClassifier, DfoCfnRegressor, DfoCfnClassifier
## Use standard CFN model for regression problem
regressor = CfnRegressor(hidden_size=50, act1_name="tanh", act2_name="sigmoid", obj_name="MSE",
max_epochs=1000, batch_size=32, optimizer="SGD", optimizer_paras=None, verbose=False, seed=42)
## Use standard CFN model for classification problem
classifier = CfnClassifier(hidden_size=50, act1_name="tanh", act2_name="sigmoid", obj_name="NLLL",
max_epochs=1000, batch_size=32, optimizer="SGD", optimizer_paras=None, verbose=False, seed=42)
## Use Metaheuristic-optimized CFN model for regression problem
print(DfoCfnClassifier.SUPPORTED_OPTIMIZERS)
print(DfoCfnClassifier.SUPPORTED_REG_OBJECTIVES)
opt_paras = {"name": "WOA", "epoch": 100, "pop_size": 30}
regressor = DfoCfnRegressor(hidden_size=50, act1_name="tanh", act2_name="sigmoid",
obj_name="MSE", optimizer="OriginalWOA", optimizer_paras=opt_paras, verbose=True, seed=42)
## Use Metaheuristic-optimized CFN model for classification problem
print(DfoCfnClassifier.SUPPORTED_OPTIMIZERS)
print(DfoCfnClassifier.SUPPORTED_CLS_OBJECTIVES)
opt_paras = {"name": "WOA", "epoch": 100, "pop_size": 30}
classifier = DfoCfnClassifier(hidden_size=50, act1_name="tanh", act2_name="softmax",
obj_name="CEL", optimizer="OriginalWOA", optimizer_paras=opt_paras, verbose=True, seed=42)
```
3) After you define the `model`, do something with it
+ Use provides functions to train, predict, and evaluate model
```python
from deforce import CfnRegressor, Data
data = Data() # Assumption that you have provide this object like above
model = CfnRegressor(hidden_size=50, act1_name="tanh", act2_name="sigmoid", obj_name="MSE",
max_epochs=1000, batch_size=32, optimizer="SGD", optimizer_paras=None, verbose=False)
## Train the model
model.fit(data.X_train, data.y_train)
## Predicting a new result
y_pred = model.predict(data.X_test)
## Calculate metrics using score or scores functions.
print(model.score(data.X_test, data.y_test, method="MAE"))
print(model.scores(data.X_test, data.y_test, list_methods=["MAPE", "NNSE", "KGE", "MASE", "R2", "R", "R2S"]))
## Calculate metrics using evaluate function
print(model.evaluate(data.y_test, y_pred, list_metrics=("MSE", "RMSE", "MAPE", "NSE")))
## Save performance metrics to csv file
model.save_evaluation_metrics(data.y_test, y_pred, list_metrics=("RMSE", "MAE"), save_path="history",
filename="metrics.csv")
## Save training loss to csv file
model.save_training_loss(save_path="history", filename="loss.csv")
## Save predicted label
model.save_y_predicted(X=data.X_test, y_true=data.y_test, save_path="history", filename="y_predicted.csv")
## Save model
model.save_model(save_path="history", filename="traditional_CFN.pkl")
## Load model
trained_model = CfnRegressor.load_model(load_path="history", filename="traditional_CFN.pkl")
```
Raw data
{
"_id": null,
"home_page": "https://github.com/thieu1995/deforce",
"name": "deforce",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "Cascade Forward Neural Networks, machine learning, artificial intelligence, deep learning, neural networks, single hidden layer network, cascade forward networks, random projection, CFN, CFNN, feed-forward neural network, artificial neural network, classification, regression, supervised learning, online learning, generalization, optimization algorithms, Kernel CFN, Cross-validationGenetic algorithm (GA), Particle swarm optimization (PSO), Ant colony optimization (ACO), Differential evolution (DE), Simulated annealing, Grey wolf optimizer (GWO), Whale Optimization Algorithm (WOA), confusion matrix, recall, precision, accuracy, pearson correlation coefficient (PCC), spearman correlation coefficient (SCC), Global optimization, Convergence analysis, Search space exploration, Local search, Computational intelligence, Robust optimization, metaheuristic, metaheuristic algorithms, nature-inspired computing, nature-inspired algorithms, swarm-based computation, metaheuristic-based cascade forward neural networks, metaheuristic-optimized CFN, derivative free-based cascade forward neural networks, derivative free-optimized CFNN, gradient descent-based optimized cascade forward neural network, GD-based CFNN, Performance analysis, Intelligent optimization, Simulations",
"author": "Thieu",
"author_email": "nguyenthieu2102@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/3f/98/5e1ecc2d47440f1ed07e051070b766e819ec8d44a75eab330c4303737311/deforce-1.0.0.tar.gz",
"platform": null,
"description": "\n## deforce: Derivative-Free Algorithms for Optimizing Cascade Forward Neural Networks\n\n---\n\n[](https://github.com/thieu1995/deforce/releases)\n[](https://pypi.python.org/pypi/deforce) \n[](https://badge.fury.io/py/deforce)\n\n\n\n[](https://pepy.tech/project/deforce)\n[](https://github.com/thieu1995/deforce/actions/workflows/publish-package.yaml)\n\n[](https://deforce.readthedocs.io/en/latest/?badge=latest)\n[](https://t.me/+fRVCJGuGJg1mNDg1)\n\n[](https://git-scm.com/book/en/v2/GitHub-Contributing-to-a-Project)\n[](https://doi.org/10.5281/zenodo.10935437)\n[](https://www.gnu.org/licenses/gpl-3.0)\n\n`deforce` (DErivative Free Optimization foR Cascade forward nEural networks) is a Python library that implements \nvariants and the traditional version of Cascade Forward Neural Networks. These include Derivative Free-optimized CFN \nmodels (such as genetic algorithm, particle swarm optimization, whale optimization algorithm, teaching learning \noptimization, differential evolution, ...) and Gradient Descent-optimized CFN models (such as stochastic gradient \ndescent, Adam optimizer, Adelta optimizer, ...). It provides a comprehensive list of optimizers for training CFN \nmodels and is also compatible with the Scikit-Learn library. With deforce, you can perform searches and \nhyperparameter tuning for traditional CFN networks using the features provided by the Scikit-Learn library.\n\n* **Free software:** GNU General Public License (GPL) V3 license\n* **Provided Estimator**: `CfnRegressor`, `CfnClassifier`, `DfoCfnRegressor`, `DfoCfnClassifier`, `DfoTuneCfn`\n* **Total DFO-based CFN Regressor**: > 200 Models \n* **Total DFO-based CFN Classifier**: > 200 Models\n* **Total GD-based CFN Regressor**: 12 Models\n* **Total GD-based CFN Classifier**: 12 Models\n* **Supported performance metrics**: >= 67 (47 regressions and 20 classifications)\n* **Supported objective functions**: >= 67 (47 regressions and 20 classifications)\n* **Documentation:** https://deforce.readthedocs.io\n* **Python versions:** >= 3.8.x\n* **Dependencies:** numpy, scipy, scikit-learn, pandas, mealpy, permetrics, torch, skorch\n\n\n# Citation Request \n\nIf you want to understand how to use Derivative Free-optimized Cascade Forward Neural Network, you \nneed to read the paper titled **\"Optimization of neural-network model using a meta-heuristic algorithm for the estimation of dynamic Poisson\u2019s ratio of selected rock types\"**. \nThe paper can be accessed at the following [link](https://doi.org/10.1038%2Fs41598-023-38163-0)\n\nPlease include these citations if you plan to use this library:\n\n```bibtex\n@software{thieu_deforce_2024,\n author = {Van Thieu, Nguyen},\n title = {{deforce: Derivative-Free Algorithms for Optimizing Cascade Forward Neural Networks}},\n url = {https://github.com/thieu1995/deforce},\n doi = {10.5281/zenodo.10935437},\n year = {2024}\n}\n\n@article{van2023mealpy,\n title={MEALPY: An open-source library for latest meta-heuristic algorithms in Python},\n author={Van Thieu, Nguyen and Mirjalili, Seyedali},\n journal={Journal of Systems Architecture},\n year={2023},\n publisher={Elsevier},\n doi={10.1016/j.sysarc.2023.102871}\n}\n\n@article{van2023groundwater,\n title={Groundwater level modeling using Augmented Artificial Ecosystem Optimization},\n author={Van Thieu, Nguyen and Barma, Surajit Deb and Van Lam, To and Kisi, Ozgur and Mahesha, Amai},\n journal={Journal of Hydrology},\n volume={617},\n pages={129034},\n year={2023},\n publisher={Elsevier}\n}\n```\n\n# Installation\n\n* Install the [current PyPI release](https://pypi.python.org/pypi/deforce):\n```sh \n$ pip install deforce\n```\n\nAfter installation, check the installed version by:\n\n```sh\n$ python\n>>> import deforce\n>>> deforce.__version__\n```\n\n### Examples\n\nPlease check [documentation website](https://deforce.readthedocs.io/) and [examples folder](examples).\n\n1) `deforce` provides this useful classes\n\n```python\nfrom deforce import DataTransformer, Data\nfrom deforce import CfnRegressor, CfnClassifier\nfrom deforce import DfoCfnRegressor, DfoCfnClassifier\n```\n\n2) What can you do with all `model` classes\n\n```python\nfrom deforce import CfnRegressor, CfnClassifier, DfoCfnRegressor, DfoCfnClassifier\n\n## Use standard CFN model for regression problem\nregressor = CfnRegressor(hidden_size=50, act1_name=\"tanh\", act2_name=\"sigmoid\", obj_name=\"MSE\",\n max_epochs=1000, batch_size=32, optimizer=\"SGD\", optimizer_paras=None, verbose=False, seed=42)\n\n## Use standard CFN model for classification problem \nclassifier = CfnClassifier(hidden_size=50, act1_name=\"tanh\", act2_name=\"sigmoid\", obj_name=\"NLLL\",\n max_epochs=1000, batch_size=32, optimizer=\"SGD\", optimizer_paras=None, verbose=False, seed=42)\n\n## Use Metaheuristic-optimized CFN model for regression problem\nprint(DfoCfnClassifier.SUPPORTED_OPTIMIZERS)\nprint(DfoCfnClassifier.SUPPORTED_REG_OBJECTIVES)\n\nopt_paras = {\"name\": \"WOA\", \"epoch\": 100, \"pop_size\": 30}\nregressor = DfoCfnRegressor(hidden_size=50, act1_name=\"tanh\", act2_name=\"sigmoid\",\n obj_name=\"MSE\", optimizer=\"OriginalWOA\", optimizer_paras=opt_paras, verbose=True, seed=42)\n\n## Use Metaheuristic-optimized CFN model for classification problem\nprint(DfoCfnClassifier.SUPPORTED_OPTIMIZERS)\nprint(DfoCfnClassifier.SUPPORTED_CLS_OBJECTIVES)\n\nopt_paras = {\"name\": \"WOA\", \"epoch\": 100, \"pop_size\": 30}\nclassifier = DfoCfnClassifier(hidden_size=50, act1_name=\"tanh\", act2_name=\"softmax\",\n obj_name=\"CEL\", optimizer=\"OriginalWOA\", optimizer_paras=opt_paras, verbose=True, seed=42)\n```\n\n3) After you define the `model`, do something with it\n+ Use provides functions to train, predict, and evaluate model\n\n```python\nfrom deforce import CfnRegressor, Data\n\ndata = Data() # Assumption that you have provide this object like above\n\nmodel = CfnRegressor(hidden_size=50, act1_name=\"tanh\", act2_name=\"sigmoid\", obj_name=\"MSE\",\n max_epochs=1000, batch_size=32, optimizer=\"SGD\", optimizer_paras=None, verbose=False)\n\n## Train the model\nmodel.fit(data.X_train, data.y_train)\n\n## Predicting a new result\ny_pred = model.predict(data.X_test)\n\n## Calculate metrics using score or scores functions.\nprint(model.score(data.X_test, data.y_test, method=\"MAE\"))\nprint(model.scores(data.X_test, data.y_test, list_methods=[\"MAPE\", \"NNSE\", \"KGE\", \"MASE\", \"R2\", \"R\", \"R2S\"]))\n\n## Calculate metrics using evaluate function\nprint(model.evaluate(data.y_test, y_pred, list_metrics=(\"MSE\", \"RMSE\", \"MAPE\", \"NSE\")))\n\n## Save performance metrics to csv file\nmodel.save_evaluation_metrics(data.y_test, y_pred, list_metrics=(\"RMSE\", \"MAE\"), save_path=\"history\",\n filename=\"metrics.csv\")\n\n## Save training loss to csv file\nmodel.save_training_loss(save_path=\"history\", filename=\"loss.csv\")\n\n## Save predicted label\nmodel.save_y_predicted(X=data.X_test, y_true=data.y_test, save_path=\"history\", filename=\"y_predicted.csv\")\n\n## Save model\nmodel.save_model(save_path=\"history\", filename=\"traditional_CFN.pkl\")\n\n## Load model \ntrained_model = CfnRegressor.load_model(load_path=\"history\", filename=\"traditional_CFN.pkl\")\n```\n",
"bugtrack_url": null,
"license": "GPLv3",
"summary": "deforce: Derivative-Free Algorithms for Optimizing Cascade Forward Neural Networks",
"version": "1.0.0",
"project_urls": {
"Bug Tracker": "https://github.com/thieu1995/deforce/issues",
"Change Log": "https://github.com/thieu1995/deforce/blob/main/ChangeLog.md",
"Documentation": "https://deforce.readthedocs.io/",
"Forum": "https://t.me/+fRVCJGuGJg1mNDg1",
"Homepage": "https://github.com/thieu1995/deforce",
"Source Code": "https://github.com/thieu1995/deforce"
},
"split_keywords": [
"cascade forward neural networks",
" machine learning",
" artificial intelligence",
" deep learning",
" neural networks",
" single hidden layer network",
" cascade forward networks",
" random projection",
" cfn",
" cfnn",
" feed-forward neural network",
" artificial neural network",
" classification",
" regression",
" supervised learning",
" online learning",
" generalization",
" optimization algorithms",
" kernel cfn",
" cross-validationgenetic algorithm (ga)",
" particle swarm optimization (pso)",
" ant colony optimization (aco)",
" differential evolution (de)",
" simulated annealing",
" grey wolf optimizer (gwo)",
" whale optimization algorithm (woa)",
" confusion matrix",
" recall",
" precision",
" accuracy",
" pearson correlation coefficient (pcc)",
" spearman correlation coefficient (scc)",
" global optimization",
" convergence analysis",
" search space exploration",
" local search",
" computational intelligence",
" robust optimization",
" metaheuristic",
" metaheuristic algorithms",
" nature-inspired computing",
" nature-inspired algorithms",
" swarm-based computation",
" metaheuristic-based cascade forward neural networks",
" metaheuristic-optimized cfn",
" derivative free-based cascade forward neural networks",
" derivative free-optimized cfnn",
" gradient descent-based optimized cascade forward neural network",
" gd-based cfnn",
" performance analysis",
" intelligent optimization",
" simulations"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "77290d985602ca0b904e67ea1466d062e210aec0ac2d55ff4572443c1a25454d",
"md5": "f9ff8ac59e447f4f0404dcddbe614077",
"sha256": "f63c9edc8488458e42239242e6e1e9d23a57d3cbecb2c56a20dca8c02e6584dd"
},
"downloads": -1,
"filename": "deforce-1.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f9ff8ac59e447f4f0404dcddbe614077",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 43311,
"upload_time": "2024-04-06T15:43:34",
"upload_time_iso_8601": "2024-04-06T15:43:34.032186Z",
"url": "https://files.pythonhosted.org/packages/77/29/0d985602ca0b904e67ea1466d062e210aec0ac2d55ff4572443c1a25454d/deforce-1.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3f985e1ecc2d47440f1ed07e051070b766e819ec8d44a75eab330c4303737311",
"md5": "130363c5cbb0481ead85301de1433c69",
"sha256": "0b0b44e558187f3bea3216006e12df738dc442252fb194f236edc7b8b72a9da6"
},
"downloads": -1,
"filename": "deforce-1.0.0.tar.gz",
"has_sig": false,
"md5_digest": "130363c5cbb0481ead85301de1433c69",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 39621,
"upload_time": "2024-04-06T15:43:35",
"upload_time_iso_8601": "2024-04-06T15:43:35.869947Z",
"url": "https://files.pythonhosted.org/packages/3f/98/5e1ecc2d47440f1ed07e051070b766e819ec8d44a75eab330c4303737311/deforce-1.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-06 15:43:35",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "thieu1995",
"github_project": "deforce",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "deforce"
}
