[](https://badge.fury.io/py/keras-bcr)
[](https://img.shields.io/pypi/dm/keras-bcr)
# keras-bcr : Batch Correlation Regularizer for TF2/Keras
The batch correlation regularization (BCR) technique adds a penalty loss
if the inputs and outputs before the skip-connection of a specific feature element are correlated.
The correlation coefficients are computed for each feature element seperatly across the current batch.
## Usage
```py
from keras_bcr import BatchCorrRegularizer
import tensorflow as tf
# The BCR layer is added before the addition of the skip-connection
def build_resnet_block(inputs, units=64, activation="gelu",
dropout=0.4, bcr_rate=0.1):
h = tf.keras.layers.Dense(units=units)(inputs)
h = h = tf.keras.layers.Activation(activation=activation)(h)
h = tf.keras.layers.Dropout(rate=dropout)(h)
h = BatchCorrRegularizer(bcr_rate=bcr_rate)([h, inputs]) # << HERE
outputs = tf.keras.layers.Add()([h, inputs])
return outputs
# An model with 3 ResNet blocks
def build_model(input_dim):
inputs = tf.keras.Input(shape=(input_dim,))
h = build_resnet_block(inputs, units=input_dim)
h = build_resnet_block(h, units=input_dim)
outputs = build_resnet_block(h, units=input_dim)
model = tf.keras.Model(inputs=inputs, outputs=outputs)
return model
INPUT_DIM = 64
model = build_model(input_dim=INPUT_DIM)
model.compile(optimizer=tf.keras.optimizers.Adam(), loss="mean_squared_error")
BATCH_SZ = 128
X_train = tf.random.normal([BATCH_SZ, INPUT_DIM])
y_train = tf.random.normal([BATCH_SZ])
history = model.fit(X_train, y_train, verbose=1, epochs=2)
```
## Appendix
### Installation
The `keras-bcr` [git repo](http://github.com/ulf1/keras-bcr) is available as [PyPi package](https://pypi.org/project/keras-bcr)
```sh
pip install keras-bcr
pip install git+ssh://git@github.com/ulf1/keras-bcr.git
```
### Install a virtual environment
```sh
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt --no-cache-dir
pip install -r requirements-dev.txt --no-cache-dir
pip install -r requirements-demo.txt --no-cache-dir
```
(If your git repo is stored in a folder with whitespaces, then don't use the subfolder `.venv`. Use an absolute path without whitespaces.)
### Python commands
* Jupyter for the examples: `jupyter lab`
* Check syntax: `flake8 --ignore=F401 --exclude=$(grep -v '^#' .gitignore | xargs | sed -e 's/ /,/g')`
* Run Unit Tests: `PYTHONPATH=. pytest`
Publish
```sh
python setup.py sdist
twine upload -r pypi dist/*
```
### Clean up
```sh
find . -type f -name "*.pyc" | xargs rm
find . -type d -name "__pycache__" | xargs rm -r
rm -r .pytest_cache
rm -r .venv
```
### Support
Please [open an issue](https://github.com/ulf1/keras-bcr/issues/new) for support.
### Contributing
Please contribute using [Github Flow](https://guides.github.com/introduction/flow/). Create a branch, add commits, and [open a pull request](https://github.com/ulf1/keras-bcr/compare/).
### Acknowledgements
The "Evidence" project was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - [433249742](https://gepris.dfg.de/gepris/projekt/433249742) (GU 798/27-1; GE 1119/11-1).
### Maintenance
- till 31.Aug.2023 (v0.2.0) the code repository was maintained within the DFG project [433249742](https://gepris.dfg.de/gepris/projekt/433249742)
- since 01.Sep.2023 (v0.3.0) the code repository is maintained by Ulf Hamster.
Raw data
{
"_id": null,
"home_page": "http://github.com/ulf1/keras-bcr",
"name": "keras-bcr",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "",
"author": "Ulf Hamster",
"author_email": "554c46@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/c6/f6/c07d2e1e3f0d147571d27406b7999a256082bbcfe02642d2afa72656a8f9/keras-bcr-0.3.0.tar.gz",
"platform": null,
"description": "[](https://badge.fury.io/py/keras-bcr)\n[](https://img.shields.io/pypi/dm/keras-bcr)\n\n\n# keras-bcr : Batch Correlation Regularizer for TF2/Keras\nThe batch correlation regularization (BCR) technique adds a penalty loss\nif the inputs and outputs before the skip-connection of a specific feature element are correlated.\nThe correlation coefficients are computed for each feature element seperatly across the current batch.\n\n## Usage\n\n```py\nfrom keras_bcr import BatchCorrRegularizer\nimport tensorflow as tf\n\n# The BCR layer is added before the addition of the skip-connection\ndef build_resnet_block(inputs, units=64, activation=\"gelu\",\n dropout=0.4, bcr_rate=0.1):\n h = tf.keras.layers.Dense(units=units)(inputs)\n h = h = tf.keras.layers.Activation(activation=activation)(h)\n h = tf.keras.layers.Dropout(rate=dropout)(h)\n h = BatchCorrRegularizer(bcr_rate=bcr_rate)([h, inputs]) # << HERE\n outputs = tf.keras.layers.Add()([h, inputs])\n return outputs\n\n# An model with 3 ResNet blocks\ndef build_model(input_dim):\n inputs = tf.keras.Input(shape=(input_dim,))\n h = build_resnet_block(inputs, units=input_dim)\n h = build_resnet_block(h, units=input_dim)\n outputs = build_resnet_block(h, units=input_dim)\n model = tf.keras.Model(inputs=inputs, outputs=outputs)\n return model\n\nINPUT_DIM = 64\nmodel = build_model(input_dim=INPUT_DIM)\nmodel.compile(optimizer=tf.keras.optimizers.Adam(), loss=\"mean_squared_error\")\n\nBATCH_SZ = 128\nX_train = tf.random.normal([BATCH_SZ, INPUT_DIM])\ny_train = tf.random.normal([BATCH_SZ])\n\nhistory = model.fit(X_train, y_train, verbose=1, epochs=2)\n```\n\n\n## Appendix\n\n### Installation\nThe `keras-bcr` [git repo](http://github.com/ulf1/keras-bcr) is available as [PyPi package](https://pypi.org/project/keras-bcr)\n\n```sh\npip install keras-bcr\npip install git+ssh://git@github.com/ulf1/keras-bcr.git\n```\n\n### Install a virtual environment\n\n```sh\npython3 -m venv .venv\nsource .venv/bin/activate\npip install --upgrade pip\npip install -r requirements.txt --no-cache-dir\npip install -r requirements-dev.txt --no-cache-dir\npip install -r requirements-demo.txt --no-cache-dir\n```\n\n(If your git repo is stored in a folder with whitespaces, then don't use the subfolder `.venv`. Use an absolute path without whitespaces.)\n\n### Python commands\n\n* Jupyter for the examples: `jupyter lab`\n* Check syntax: `flake8 --ignore=F401 --exclude=$(grep -v '^#' .gitignore | xargs | sed -e 's/ /,/g')`\n* Run Unit Tests: `PYTHONPATH=. pytest`\n\nPublish\n\n```sh\npython setup.py sdist \ntwine upload -r pypi dist/*\n```\n\n### Clean up \n\n```sh\nfind . -type f -name \"*.pyc\" | xargs rm\nfind . -type d -name \"__pycache__\" | xargs rm -r\nrm -r .pytest_cache\nrm -r .venv\n```\n\n\n### Support\nPlease [open an issue](https://github.com/ulf1/keras-bcr/issues/new) for support.\n\n\n### Contributing\nPlease contribute using [Github Flow](https://guides.github.com/introduction/flow/). Create a branch, add commits, and [open a pull request](https://github.com/ulf1/keras-bcr/compare/).\n\n\n### Acknowledgements\nThe \"Evidence\" project was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - [433249742](https://gepris.dfg.de/gepris/projekt/433249742) (GU 798/27-1; GE 1119/11-1).\n\n### Maintenance\n- till 31.Aug.2023 (v0.2.0) the code repository was maintained within the DFG project [433249742](https://gepris.dfg.de/gepris/projekt/433249742)\n- since 01.Sep.2023 (v0.3.0) the code repository is maintained by Ulf Hamster.\n\n\n\n",
"bugtrack_url": null,
"license": "Apache License 2.0",
"summary": "Batch Correlation Regularizer for TF2/Keras",
"version": "0.3.0",
"project_urls": {
"Homepage": "http://github.com/ulf1/keras-bcr"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c6f6c07d2e1e3f0d147571d27406b7999a256082bbcfe02642d2afa72656a8f9",
"md5": "3124e399404890e609994c82a57682c9",
"sha256": "52d3e7b4ddbd5b9a9c36f49c863226c5e7e295b40177b8ad2a814a86e2e071e2"
},
"downloads": -1,
"filename": "keras-bcr-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "3124e399404890e609994c82a57682c9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 8181,
"upload_time": "2023-07-10T15:50:40",
"upload_time_iso_8601": "2023-07-10T15:50:40.699226Z",
"url": "https://files.pythonhosted.org/packages/c6/f6/c07d2e1e3f0d147571d27406b7999a256082bbcfe02642d2afa72656a8f9/keras-bcr-0.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-10 15:50:40",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ulf1",
"github_project": "keras-bcr",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "keras-bcr"
}
JSON
