# FeaSel-Net
*FeaSel-Net* is a python package that enables feature selection algorithms embedded
in a neural network architecture. It combines a leave-one-out cross-validation
(LOOCV) type of feature selection algorithm with recursive pruning of the input
nodes, such that only the most relevant nodes with the richest information are
kept for the subsequent optimization task. The recursive pruning is undertaken
by employing a ```FeatureSelection``` callback at certain points of the
optimization process. The precise procedure is explained in *Sequence of
Events*.
Originally developed for serving the task of finding biomarkers in biological
tissues, the algorithm is generically coded such that it is able to select
features for all kinds of classification tasks.
The package is an extension for the [keras](https://www.keras.io) and
[tensorflow](https://www.tensorflow.org/) libraries.
Please see the links for further information on their software packages and to
get a grasp of neural networks in general and the constructs used for
*FeaSel-Net*.
# Installation
It is best at the moment to install this as an external package with pip. This can be done by cloning the repository with the following commands:
```
pip install feasel-net
```
## Sequence of Events
1. Initiallizing Neural Network
The first step of the algorithm can be thought of a simple optimization
problem initiallized with the inputs and a binary mask for those inputs
with only ones as its entries. This behaviour is induced by using a newly
created layer type called ```LinearPass```.
<img src="images/init_nn.png"
alt="Initiallization"
height="300"
align="middle"/>
2. Training until trigger conditions are met
The neural network optimizes the classification results until one of the
following options happen:
- the training (or validation) loss value is beneath a certain threshold
- the training (or validation) accuracy value is above a certain threshold
Then - for the sake of consistency - it will count how many times in a row
the conditions are met. If this happens for multiple epochs, the actual
pruning event will start that consists of estimating the importance and
eliminating uninformative features.
3. Importance estimation
As soon as the callback is triggered, the evaluation of the
<img src="images/eval_nn.png"
alt="Evaluation"
height="300"
align="middle"/>
## Release Information
**0.0.1 - Initial Release**
- callback FeatureSelection
- trigger parameters: delta epochs, thresholds, ...
- different metrics for triggering
- etc.
- layer LinearPass
# ToDos
Until now, only dense layered architectures are supported. The plan is to also
include convolutional layers.
[x] DenseLayer support
[x] accuarcy and loss based evaluation
[ ] ConvLayer support
[ ] intermediate layers shall be supported
[ ] paper on algorithm
Raw data
{
"_id": null,
"home_page": "https://github.com/feliixism/feasel-net.git",
"name": "FeaSel-Net",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "feature selection,neural networks,machine learning,embedded feature selection",
"author": "Felix Fischer",
"author_email": "felix.fischer@ito.uni-stuttgart.de",
"download_url": "https://files.pythonhosted.org/packages/8c/dc/9d717af6166fba975ccd33f7bb91017aac26fa3d08754f689b043db7b6f1/FeaSel-Net-0.0.10.tar.gz",
"platform": null,
"description": "# FeaSel-Net\r\n*FeaSel-Net* is a python package that enables feature selection algorithms embedded \r\nin a neural network architecture. It combines a leave-one-out cross-validation \r\n(LOOCV) type of feature selection algorithm with recursive pruning of the input \r\nnodes, such that only the most relevant nodes with the richest information are \r\nkept for the subsequent optimization task. The recursive pruning is undertaken \r\nby employing a ```FeatureSelection``` callback at certain points of the \r\noptimization process. The precise procedure is explained in *Sequence of \r\nEvents*.\r\nOriginally developed for serving the task of finding biomarkers in biological \r\ntissues, the algorithm is generically coded such that it is able to select \r\nfeatures for all kinds of classification tasks.\r\n\r\nThe package is an extension for the [keras](https://www.keras.io) and \r\n[tensorflow](https://www.tensorflow.org/) libraries.\r\nPlease see the links for further information on their software packages and to \r\nget a grasp of neural networks in general and the constructs used for \r\n*FeaSel-Net*.\r\n\r\n# Installation\r\nIt is best at the moment to install this as an external package with pip. This can be done by cloning the repository with the following commands:\r\n```\r\npip install feasel-net\r\n```\r\n\r\n## Sequence of Events\r\n\r\n1. Initiallizing Neural Network\r\n\tThe first step of the algorithm can be thought of a simple optimization \r\n\tproblem initiallized with the inputs and a binary mask for those inputs \r\n\twith only ones as its entries. This behaviour is induced by using a newly \r\n\tcreated layer type called ```LinearPass```.\r\n\t\r\n\t<img\tsrc=\"images/init_nn.png\"\r\n\t\t\talt=\"Initiallization\"\r\n\t\t\theight=\"300\" \r\n\t\t\talign=\"middle\"/>\r\n2. Training until trigger conditions are met\r\n\tThe neural network optimizes the classification results until one of the \r\n\tfollowing options happen:\r\n\t- the training (or validation) loss value is beneath a certain threshold\r\n\t- the training (or validation) accuracy value is above a certain threshold\r\n\tThen - for the sake of consistency - it will count how many times in a row\r\n\tthe conditions are met. If this happens for multiple epochs, the actual\r\n\tpruning event will start that consists of estimating the importance and \r\n\teliminating uninformative features.\r\n3. Importance estimation\r\n\tAs soon as the callback is triggered, the evaluation of the \r\n\t\r\n\t<img \tsrc=\"images/eval_nn.png\"\r\n\t\t\talt=\"Evaluation\"\r\n\t\t\theight=\"300\"\r\n\t\t\talign=\"middle\"/>\r\n\r\n## Release Information\r\n**0.0.1 - Initial Release**\r\n- callback FeatureSelection\r\n\t- trigger parameters: delta epochs, thresholds, ...\r\n\t- different metrics for triggering\r\n\t- etc.\r\n- layer LinearPass\r\n\r\n# ToDos\r\nUntil now, only dense layered architectures are supported. The plan is to also\r\ninclude convolutional layers.\r\n\r\n[x] DenseLayer support\r\n\r\n[x] accuarcy and loss based evaluation\r\n\r\n[ ] ConvLayer support\r\n\r\n[ ] intermediate layers shall be supported\r\n\r\n[ ] paper on algorithm\r\n\r\n\r\n\r\n\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A Keras callback package for recursively pruning the most uninformative input nodes during training.",
"version": "0.0.10",
"project_urls": {
"Homepage": "https://github.com/feliixism/feasel-net.git"
},
"split_keywords": [
"feature selection",
"neural networks",
"machine learning",
"embedded feature selection"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8cdc9d717af6166fba975ccd33f7bb91017aac26fa3d08754f689b043db7b6f1",
"md5": "28d3627b5601779fd5adf19fa21487ea",
"sha256": "0f3b670eea785ad5f5da0f12475b138b852a53f4120a02e139adf5e6cd43b159"
},
"downloads": -1,
"filename": "FeaSel-Net-0.0.10.tar.gz",
"has_sig": false,
"md5_digest": "28d3627b5601779fd5adf19fa21487ea",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 71293,
"upload_time": "2024-03-13T12:28:37",
"upload_time_iso_8601": "2024-03-13T12:28:37.541001Z",
"url": "https://files.pythonhosted.org/packages/8c/dc/9d717af6166fba975ccd33f7bb91017aac26fa3d08754f689b043db7b6f1/FeaSel-Net-0.0.10.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-13 12:28:37",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "feliixism",
"github_project": "feasel-net",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "feasel-net"
}
JSON
