<div align="center">
<img alt="pidgan logo" src="https://raw.githubusercontent.com/mbarbetti/pidgan/main/.github/images/pidgan-logo-h-plain.png" width="300">
</div>
<h2 align="center">
<em>GAN-based models to flash-simulate the LHCb PID detectors</em>
</h2>
<p align="center">
<a href="https://www.tensorflow.org/versions"><img alt="TensorFlow versions" src="https://img.shields.io/badge/TensorFlow-2.8–2.16-f57000?style=flat&logo=tensorflow&logoColor=white"></a>
<a href="https://keras.io/keras_3"><img alt="Keras 3" src="https://img.shields.io/badge/Keras_3-compatible-d00000?style=flat&logo=keras&logoColor=white"></a>
<a href="https://scikit-learn.org/stable/whats_new.html"><img alt="scikit-learn versions" src="https://img.shields.io/badge/sklearn-1.0–1.5-f89939?style=flat&logo=scikit-learn&logoColor=white"></a>
<a href="https://www.python.org/downloads"><img alt="Python versions" src="https://img.shields.io/badge/python-3.7–3.11-blue?style=flat&logo=python&logoColor=white"></a>
</p>
<p align="center">
<a href="https://pypi.python.org/pypi/pidgan"><img alt="PyPI - Version" src="https://img.shields.io/pypi/v/pidgan"></a>
<a href="https://pypi.python.org/pypi/pidgan"><img alt="PyPI - Status" src="https://img.shields.io/pypi/status/pidgan"></a>
<a href="LICENSE"><img alt="GitHub - License" src="https://img.shields.io/github/license/mbarbetti/pidgan"></a>
<a href="https://zenodo.org/doi/10.5281/zenodo.10463727"><img alt="DOI" src="https://zenodo.org/badge/597088032.svg"></a>
</p>
<p align="center">
<a href="https://github.com/mbarbetti/pidgan/actions/workflows/tests.yml"><img alt="GitHub - Tests" src="https://github.com/mbarbetti/pidgan/actions/workflows/tests.yml/badge.svg?branch=main"></a>
<a href="https://codecov.io/gh/mbarbetti/pidgan"><img alt="Codecov" src="https://codecov.io/gh/mbarbetti/pidgan/branch/main/graph/badge.svg?token=ZLWDgWhnkq"></a>
</p>
<p align="center">
<a href="https://github.com/mbarbetti/pidgan/actions/workflows/style.yml"><img alt="GitHub - Style" src="https://github.com/mbarbetti/pidgan/actions/workflows/style.yml/badge.svg?branch=main"></a>
<a href="https://github.com/astral-sh/ruff"><img alt="Ruff" src="https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json" style="max-width:100%;"></a>
</p>
<!--
[](https://hub.docker.com/r/mbarbetti/pidgan)
-->
### What is PIDGAN?
PIDGAN is a Python package built upon TensorFlow 2 to provide ready-to-use implementations for several GAN algorithms (listed in [this table](#generative-adversarial-networks)). The package was originally designed to simplify the training and optimization of GAN-based models for the _Particle Identification_ (PID) system of the LHCb experiment. Today, PIDGAN is a versatile package that can be employed in a wide range of _High Energy Physics_ (HEP) applications and, in general, whenever one has anything to do with tabular data and aims to learn the conditional probability distributions of a set of target features. This package is one of the building blocks to define a _Flash Simulation_ framework of the LHCb experiment [1].
#### PIDGAN is (almost) all you need (for flash-simulation)
Standard techniques for simulations consume tons of CPU hours in reproducing _all_ the radiation-matter interactions occurring within a HEP detector when traversed by primary and secondary particles. Directly transforming generated particles into analysis-level objects allows Flash Simulation strategies to speed up significantly the simulation production, up to x1000 [1]. Such transformations can be defined by using _Generative Adversarial Networks_ (GAN) [[2][gan]] trained to take into account the kinematics of the traversing particles and the detection conditions (e.g., magnet polarity, occupancy).
GANs rely on the simultaneous (adversarial) training of two neural networks called _generator_ and _discriminator_, whose competition ends once reached the [Nash equilibrium](https://en.wikipedia.org/wiki/Nash_equilibrium). At this point, the generator can be used as simulator to generate new data according to the conditional probability distributions learned during the training [[3][cgan]]. By relying on the TensorFlow and Keras APIs, PIDGAN allows to define and train a GAN model with no more than 20 code lines.
```python
from pidgan.players.generators import Generator
from pidgan.players.discriminators import Discriminator
from pidgan.algorithms import GAN
x = ... # conditions
y = ... # targets
G = Generator(
output_dim=y.shape[1],
latent_dim=64,
output_activation="linear",
)
D = Discriminator(
output_dim=1,
output_activation="sigmoid",
)
model = GAN(generator=G, discriminator=D)
model.compile(
metrics=["accuracy"],
generator_optimizer="rmsprop",
discriminator_optimizer="rmsprop",
)
model.fit(x, y, batch_size=256, epochs=100)
```
### Installation guide
#### First steps
Before installing PIDGAN, we suggest preparing a fully operational TensorFlow installation by following the instructions described in the [dedicated guide](https://www.tensorflow.org/install). If your device is equipped with one of the NVIDIA GPU cards supported by TensorFlow (see [Hardware requirements](https://www.tensorflow.org/install/pip#hardware_requirements)), do not forget to verify the correct installation of the libraries for hardware acceleration by running:
```bash
python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"
```
If the equipped GPU card is not included in the list printed by the previous command, your device and/or Python environment may be misconfigured. Please refer to [this table](https://www.tensorflow.org/install/source#gpu) for the correct configuration of CUDA Toolkit and cuDNN requested by the different TensorFlow versions.
#### How to install
PIDGAN has a minimal list of [requirements](https://github.com/mbarbetti/pidgan/blob/main/requirements/base.txt):
- Python >= 3.7, < 3.13
- TensorFlow >= 2.8, < 2.17
- scikit-learn >= 1.0, < 1.6
- NumPy < 2.0
- Hopaas client (https://github.com/landerlini/hopaas_client)
The easiest way to install PIDGAN is via `pip`:
```bash
pip install pidgan
```
In addition, since `hopaas_client` is not available on [PyPI](https://pypi.org), you need to install it manually to unlock the complete set of PIDGAN functionalities:
```bash
pip install git+https://github.com/landerlini/hopaas_client
```
#### Optional dependencies
Standard HEP applications may need additional packages for data management, results visualization/validation, and model export. PIDGAN and any [additional requirements](https://github.com/mbarbetti/pidgan/blob/main/requirements/hep.txt) potentially useful in HEP can be installed via `pip` in one shot:
```bash
pip install pidgan[hep]
```
### Models available
The main components of PIDGAN are the [`algorithms`](https://github.com/mbarbetti/pidgan/tree/main/src/pidgan/algorithms) and [`players`](https://github.com/mbarbetti/pidgan/tree/main/src/pidgan/players) modules that provide, respectively, implementations for several GAN algorithms and the so-called adversarial neural networks (e.g., generator, discriminator). The objects exposed by the `algorithms` and `players` modules are implemented by subclassing the Keras [Model class](https://keras.io/api/models/model) and customizing the training procedure that is executed when one calls the `fit()` method. With [PIDGAN v0.2.0](https://github.com/mbarbetti/pidgan/releases/tag/v0.2.0) the package has been massively rewritten to be also compatible with the new multi-backend [Keras 3](https://keras.io/keras_3). At the moment, the custom training procedures defined for the various GAN algorithms are only implemented for the TensorFlow backend, while relying also on the Pytorch and Jax backends is planned for a future release. The following tables report the complete set of `algorithms` and `players` classes currently available, together with a snapshot of their implementation details.
#### Generative Adversarial Networks
| Algorithms* | Source | Avail | Test | Lipschitz** | Refs | Tutorial |
|:-----------:|:------:|:-----:|:----:|:-----------:|:----:|:--------:|
| GAN | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k2/GAN.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k3/GAN.py) | ✅ | ✅ | ❌ | [2][gan], [10][pre-wgan], [11][gan-tricks] | [](https://colab.research.google.com/github/mbarbetti/pidgan-notebooks/blob/main/tutorial-GAN-LHCb_RICH.ipynb) |
| BceGAN | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k2/BceGAN.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k3/BceGAN.py) | ✅ | ✅ | ❌ | [4][dcgan], [10][pre-wgan], [11][gan-tricks] | [](https://colab.research.google.com/github/mbarbetti/pidgan-notebooks/blob/main/tutorial-BceGAN-LHCb_RICH.ipynb) |
| LSGAN | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k2/LSGAN.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k3/LSGAN.py) | ✅ | ✅ | ❌ | [5][lsgan], [10][pre-wgan], [11][gan-tricks] | [](https://colab.research.google.com/github/mbarbetti/pidgan-notebooks/blob/main/tutorial-LSGAN-LHCb_RICH.ipynb) |
| WGAN | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k2/WGAN.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k3/WGAN.py) | ✅ | ✅ | ✅ | [6][wgan], [11][gan-tricks] | [](https://colab.research.google.com/github/mbarbetti/pidgan-notebooks/blob/main/tutorial-WGAN-LHCb_RICH.ipynb) |
| WGAN-GP | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k2/WGAN_GP.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k3/WGAN_GP.py) | ✅ | ✅ | ✅ | [7][wgan-gp], [11][gan-tricks] | [](https://colab.research.google.com/github/mbarbetti/pidgan-notebooks/blob/main/tutorial-WGAN_GP-LHCb_RICH.ipynb) |
| CramerGAN | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k2/CramerGAN.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k3/CramerGAN.py) | ✅ | ✅ | ✅ | [8][cramer-gan], [11][gan-tricks] | [](https://colab.research.google.com/github/mbarbetti/pidgan-notebooks/blob/main/tutorial-CramerGAN-LHCb_RICH.ipynb) |
| WGAN-ALP | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k2/WGAN_ALP.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k3/WGAN_ALP.py) | ✅ | ✅ | ✅ | [9][wgan-alp], [11][gan-tricks] | [](https://colab.research.google.com/github/mbarbetti/pidgan-notebooks/blob/main/tutorial-WGAN_ALP-LHCb_RICH.ipynb) |
| BceGAN-GP | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k2/BceGAN_GP.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k3/BceGAN_GP.py) | ✅ | ✅ | ✅ | [4][dcgan], [7][wgan-gp], [11][gan-tricks] | [](https://colab.research.google.com/github/mbarbetti/pidgan-notebooks/blob/main/tutorial-BceGAN_GP-LHCb_RICH.ipynb) |
| BceGAN-ALP | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k2/BceGAN_ALP.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k3/BceGAN_ALP.py) | ✅ | ✅ | ✅ | [4][dcgan], [9][wgan-alp], [11][gan-tricks] | [](https://colab.research.google.com/github/mbarbetti/pidgan-notebooks/blob/main/tutorial-BceGAN_ALP-LHCb_RICH.ipynb) |
*each GAN algorithm is designed to operate taking __conditions__ as input [[3][cgan]]
**the GAN training is regularized to ensure that the discriminator encodes a 1-Lipschitz function
#### Generators
| Players | Source | Avail | Test | Skip conn | Refs |
|:-------:|:------:|:-----:|:----:|:---------:|:----:|
| Generator | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/generators/k2/Generator.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/generators/k3/Generator.py) | ✅ | ✅ | ❌ | [2][gan], [3][cgan] |
| ResGenerator | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/generators/k2/ResGenerator.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/generators/k3/ResGenerator.py) | ✅ | ✅ | ✅ | [2][gan], [3][cgan], [12][skip-conn] |
#### Discriminators
| Players | Source | Avail | Test | Skip conn | Aux proc | Refs |
|:-------:|:------:|:-----:|:----:|:---------:|:--------:|:----:|
| Discriminator | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/discriminators/k2/Discriminator.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/discriminators/k3/Discriminator.py) | ✅ | ✅ | ❌ | ❌ | [2][gan], [3][cgan], [11][gan-tricks] |
| ResDiscriminator | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/discriminators/k2/ResDiscriminator.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/discriminators/k3/ResDiscriminator.py) | ✅ | ✅ | ✅ | ❌ | [2][gan], [3][cgan], [11][gan-tricks], [12][skip-conn] |
| AuxDiscriminator | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/discriminators/k2/AuxDiscriminator.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/discriminators/k3/AuxDiscriminator.py) | ✅ | ✅ | ✅ | ✅ | [2][gan], [3][cgan], [11][gan-tricks], [12][skip-conn], [13][aux-feat] |
#### Other players
| Players | Source | Avail | Test | Skip conn | Aux proc | Multiclass |
|:-------:|:------:|:-----:|:----:|:---------:|:--------:|:---------:|
| Classifier | [`src`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/classifiers/Classifier.py) | ✅ | ✅ | ❌ | ❌ | ❌ |
| ResClassifier | [`src`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/classifiers/ResClassifier.py) | ✅ | ✅ | ✅ | ❌ | ❌ |
| AuxClassifier | [`src`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/classifiers/AuxClassifier.py) | ✅ | ✅ | ✅ | ✅ | ❌ |
| MultiClassifier | [`src`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/classifiers/MultiClassifier.py) | ✅ | ✅ | ❌ | ❌ | ✅ |
| MultiResClassifier | [`src`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/classifiers/MultiResClassifier.py) | ✅ | ✅ | ✅ | ❌ | ✅ |
| AuxMultiClassifier | [`src`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/classifiers/AuxMultiClassifier.py) | ✅ | ✅ | ✅ | ✅ | ✅ |
### References
1. M. Barbetti, "The flash-simulation paradigm and its implementation based on Deep Generative Models for the LHCb experiment at CERN", PhD thesis, University of Firenze, 2024
2. I.J. Goodfellow _et al._, "Generative Adversarial Networks", [arXiv:1406.2661][gan]
3. M. Mirza, S. Osindero, "Conditional Generative Adversarial Nets", [arXiv:1411.1784][cgan]
4. A. Radford, L. Metz, S. Chintala, "Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks", [arXiv:1511.06434][dcgan]
5. X. Mao _et al._, "Least Squares Generative Adversarial Networks", [arXiv:1611.04076][lsgan]
6. M. Arjovsky, S. Chintala, L. Bottou, "Wasserstein GAN", [arXiv:1701.07875][wgan]
7. I. Gulrajani _et al._, "Improved Training of Wasserstein GANs", [arXiv:1704.00028][wgan-gp]
8. M.G. Bellemare _et al._, "The Cramer Distance as a Solution to Biased Wasserstein Gradients", [arXiv:1705.10743][cramer-gan]
9. D. Terjék, "Adversarial Lipschitz Regularization", [arXiv:1907.05681][wgan-alp]
10. M. Arjovsky, L. Bottou, "Towards Principled Methods for Training Generative Adversarial Networks", [arXiv:1701.04862][pre-wgan]
11. T. Salimans _et al._, "Improved Techniques for Training GANs", [arXiv:1606.03498][gan-tricks]
12. K. He _et al._, "Deep Residual Learning for Image Recognition", [arXiv:1512.03385][skip-conn]
13. A. Rogachev, F. Ratnikov, "GAN with an Auxiliary Regressor for the Fast Simulation of the Electromagnetic Calorimeter Response", [arXiv:2207.06329][aux-feat]
### Credits
Most of the GAN algorithms are an evolution of what provided by the [mbarbetti/tf-gen-models](https://github.com/mbarbetti/tf-gen-models) repository. The BceGAN model is freely inspired by the TensorFlow tutorial [Deep Convolutional Generative Adversarial Network](https://www.tensorflow.org/tutorials/generative/dcgan) and the Keras tutorial [Conditional GAN](https://keras.io/examples/generative/conditional_gan). The WGAN-ALP model is an adaptation of what provided by the [dterjek/adversarial_lipschitz_regularization](https://github.com/dterjek/adversarial_lipschitz_regularization) repository.
### Citing PIDGAN
To cite this repository:
```bibtex
@software{pidgan:2023abc,
author = "Matteo Barbetti and Lucio Anderlini",
title = "{PIDGAN: GAN-based models to flash-simulate the LHCb PID detectors}",
version = "v0.2.0",
url = "https://github.com/mbarbetti/pidgan",
doi = "10.5281/zenodo.10463728",
publisher = "Zenodo",
year = "2023",
}
```
In the above bibtex entry, the version number is intended to be that from [pidgan/version.py](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/version.py), while the year corresponds to the project's open-source release.
### License
PIDGAN has a GNU General Public License v3 (GPLv3), as found in the [LICENSE](https://github.com/mbarbetti/pidgan/blob/main/LICENSE) file.
[gan]: https://arxiv.org/abs/1406.2661
[dcgan]: https://arxiv.org/abs/1511.06434
[lsgan]: https://arxiv.org/abs/1611.04076
[wgan]: https://arxiv.org/abs/1701.07875
[wgan-gp]: https://arxiv.org/abs/1704.00028
[cramer-gan]: https://arxiv.org/abs/1705.10743
[wgan-alp]: https://arxiv.org/abs/1907.05681
[pre-wgan]: https://arxiv.org/abs/1701.04862
[gan-tricks]: https://arxiv.org/abs/1606.03498
[cgan]: https://arxiv.org/abs/1411.1784
[skip-conn]: https://arxiv.org/abs/1512.03385
[aux-feat]: https://arxiv.org/abs/2207.06329
Raw data
{
"_id": null,
"home_page": null,
"name": "pidgan",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.13,>=3.7",
"maintainer_email": null,
"keywords": "tensorflow, keras, machine learning, deep learning, generative models, generative adversarial nets, lhcb experiment, lamarr, flash-simulation, particle identification",
"author": null,
"author_email": "Matteo Barbetti <matteo.barbetti@cnaf.infn.it>, Lucio Anderlini <lucio.anderlini@fi.infn.it>",
"download_url": "https://files.pythonhosted.org/packages/b2/7e/d4faee1d905d0eae7365f8351a3171d5a7ec75f921c44b5224d43942163a/pidgan-0.2.0.tar.gz",
"platform": null,
"description": "<div align=\"center\">\n <img alt=\"pidgan logo\" src=\"https://raw.githubusercontent.com/mbarbetti/pidgan/main/.github/images/pidgan-logo-h-plain.png\" width=\"300\">\n</div>\n\n<h2 align=\"center\">\n <em>GAN-based models to flash-simulate the LHCb PID detectors</em>\n</h2>\n\n<p align=\"center\">\n <a href=\"https://www.tensorflow.org/versions\"><img alt=\"TensorFlow versions\" src=\"https://img.shields.io/badge/TensorFlow-2.8\u20132.16-f57000?style=flat&logo=tensorflow&logoColor=white\"></a>\n <a href=\"https://keras.io/keras_3\"><img alt=\"Keras 3\" src=\"https://img.shields.io/badge/Keras_3-compatible-d00000?style=flat&logo=keras&logoColor=white\"></a>\n <a href=\"https://scikit-learn.org/stable/whats_new.html\"><img alt=\"scikit-learn versions\" src=\"https://img.shields.io/badge/sklearn-1.0\u20131.5-f89939?style=flat&logo=scikit-learn&logoColor=white\"></a>\n <a href=\"https://www.python.org/downloads\"><img alt=\"Python versions\" src=\"https://img.shields.io/badge/python-3.7\u20133.11-blue?style=flat&logo=python&logoColor=white\"></a>\n</p>\n\n<p align=\"center\">\n <a href=\"https://pypi.python.org/pypi/pidgan\"><img alt=\"PyPI - Version\" src=\"https://img.shields.io/pypi/v/pidgan\"></a>\n <a href=\"https://pypi.python.org/pypi/pidgan\"><img alt=\"PyPI - Status\" src=\"https://img.shields.io/pypi/status/pidgan\"></a>\n <a href=\"LICENSE\"><img alt=\"GitHub - License\" src=\"https://img.shields.io/github/license/mbarbetti/pidgan\"></a>\n <a href=\"https://zenodo.org/doi/10.5281/zenodo.10463727\"><img alt=\"DOI\" src=\"https://zenodo.org/badge/597088032.svg\"></a>\n</p>\n\n<p align=\"center\">\n <a href=\"https://github.com/mbarbetti/pidgan/actions/workflows/tests.yml\"><img alt=\"GitHub - Tests\" src=\"https://github.com/mbarbetti/pidgan/actions/workflows/tests.yml/badge.svg?branch=main\"></a>\n <a href=\"https://codecov.io/gh/mbarbetti/pidgan\"><img alt=\"Codecov\" src=\"https://codecov.io/gh/mbarbetti/pidgan/branch/main/graph/badge.svg?token=ZLWDgWhnkq\"></a>\n</p>\n\n<p align=\"center\">\n <a href=\"https://github.com/mbarbetti/pidgan/actions/workflows/style.yml\"><img alt=\"GitHub - Style\" src=\"https://github.com/mbarbetti/pidgan/actions/workflows/style.yml/badge.svg?branch=main\"></a>\n <a href=\"https://github.com/astral-sh/ruff\"><img alt=\"Ruff\" src=\"https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json\" style=\"max-width:100%;\"></a>\n</p>\n\n<!--\n[](https://hub.docker.com/r/mbarbetti/pidgan)\n-->\n\n### What is PIDGAN?\n\nPIDGAN is a Python package built upon TensorFlow 2 to provide ready-to-use implementations for several GAN algorithms (listed in [this table](#generative-adversarial-networks)). The package was originally designed to simplify the training and optimization of GAN-based models for the _Particle Identification_ (PID) system of the LHCb experiment. Today, PIDGAN is a versatile package that can be employed in a wide range of _High Energy Physics_ (HEP) applications and, in general, whenever one has anything to do with tabular data and aims to learn the conditional probability distributions of a set of target features. This package is one of the building blocks to define a _Flash Simulation_ framework of the LHCb experiment [1].\n\n#### PIDGAN is (almost) all you need (for flash-simulation)\n\nStandard techniques for simulations consume tons of CPU hours in reproducing _all_ the radiation-matter interactions occurring within a HEP detector when traversed by primary and secondary particles. Directly transforming generated particles into analysis-level objects allows Flash Simulation strategies to speed up significantly the simulation production, up to x1000 [1]. Such transformations can be defined by using _Generative Adversarial Networks_ (GAN) [[2][gan]] trained to take into account the kinematics of the traversing particles and the detection conditions (e.g., magnet polarity, occupancy).\n\nGANs rely on the simultaneous (adversarial) training of two neural networks called _generator_ and _discriminator_, whose competition ends once reached the [Nash equilibrium](https://en.wikipedia.org/wiki/Nash_equilibrium). At this point, the generator can be used as simulator to generate new data according to the conditional probability distributions learned during the training [[3][cgan]]. By relying on the TensorFlow and Keras APIs, PIDGAN allows to define and train a GAN model with no more than 20 code lines.\n\n```python\nfrom pidgan.players.generators import Generator\nfrom pidgan.players.discriminators import Discriminator\nfrom pidgan.algorithms import GAN\n\nx = ... # conditions\ny = ... # targets\n\nG = Generator(\n output_dim=y.shape[1],\n latent_dim=64,\n output_activation=\"linear\",\n)\n\nD = Discriminator(\n output_dim=1,\n output_activation=\"sigmoid\",\n)\n\nmodel = GAN(generator=G, discriminator=D)\nmodel.compile(\n metrics=[\"accuracy\"],\n generator_optimizer=\"rmsprop\",\n discriminator_optimizer=\"rmsprop\",\n)\n\nmodel.fit(x, y, batch_size=256, epochs=100)\n```\n\n### Installation guide\n\n#### First steps\n\nBefore installing PIDGAN, we suggest preparing a fully operational TensorFlow installation by following the instructions described in the [dedicated guide](https://www.tensorflow.org/install). If your device is equipped with one of the NVIDIA GPU cards supported by TensorFlow (see [Hardware requirements](https://www.tensorflow.org/install/pip#hardware_requirements)), do not forget to verify the correct installation of the libraries for hardware acceleration by running:\n\n```bash\npython3 -c \"import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))\"\n```\n\nIf the equipped GPU card is not included in the list printed by the previous command, your device and/or Python environment may be misconfigured. Please refer to [this table](https://www.tensorflow.org/install/source#gpu) for the correct configuration of CUDA Toolkit and cuDNN requested by the different TensorFlow versions.\n\n#### How to install\n\nPIDGAN has a minimal list of [requirements](https://github.com/mbarbetti/pidgan/blob/main/requirements/base.txt):\n\n- Python >= 3.7, < 3.13\n- TensorFlow >= 2.8, < 2.17\n- scikit-learn >= 1.0, < 1.6\n- NumPy < 2.0\n- Hopaas client (https://github.com/landerlini/hopaas_client)\n\nThe easiest way to install PIDGAN is via `pip`:\n\n```bash\npip install pidgan\n```\n\nIn addition, since `hopaas_client` is not available on [PyPI](https://pypi.org), you need to install it manually to unlock the complete set of PIDGAN functionalities:\n\n```bash\npip install git+https://github.com/landerlini/hopaas_client\n```\n\n#### Optional dependencies\n\nStandard HEP applications may need additional packages for data management, results visualization/validation, and model export. PIDGAN and any [additional requirements](https://github.com/mbarbetti/pidgan/blob/main/requirements/hep.txt) potentially useful in HEP can be installed via `pip` in one shot:\n\n```bash\npip install pidgan[hep]\n```\n\n### Models available\n\nThe main components of PIDGAN are the [`algorithms`](https://github.com/mbarbetti/pidgan/tree/main/src/pidgan/algorithms) and [`players`](https://github.com/mbarbetti/pidgan/tree/main/src/pidgan/players) modules that provide, respectively, implementations for several GAN algorithms and the so-called adversarial neural networks (e.g., generator, discriminator). The objects exposed by the `algorithms` and `players` modules are implemented by subclassing the Keras [Model class](https://keras.io/api/models/model) and customizing the training procedure that is executed when one calls the `fit()` method. With [PIDGAN v0.2.0](https://github.com/mbarbetti/pidgan/releases/tag/v0.2.0) the package has been massively rewritten to be also compatible with the new multi-backend [Keras 3](https://keras.io/keras_3). At the moment, the custom training procedures defined for the various GAN algorithms are only implemented for the TensorFlow backend, while relying also on the Pytorch and Jax backends is planned for a future release. The following tables report the complete set of `algorithms` and `players` classes currently available, together with a snapshot of their implementation details.\n\n#### Generative Adversarial Networks\n\n| Algorithms* | Source | Avail | Test | Lipschitz** | Refs | Tutorial |\n|:-----------:|:------:|:-----:|:----:|:-----------:|:----:|:--------:|\n| GAN | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k2/GAN.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k3/GAN.py) | \u2705 | \u2705 | \u274c | [2][gan], [10][pre-wgan], [11][gan-tricks] | [](https://colab.research.google.com/github/mbarbetti/pidgan-notebooks/blob/main/tutorial-GAN-LHCb_RICH.ipynb) |\n| BceGAN | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k2/BceGAN.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k3/BceGAN.py) | \u2705 | \u2705 | \u274c | [4][dcgan], [10][pre-wgan], [11][gan-tricks] | [](https://colab.research.google.com/github/mbarbetti/pidgan-notebooks/blob/main/tutorial-BceGAN-LHCb_RICH.ipynb) |\n| LSGAN | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k2/LSGAN.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k3/LSGAN.py) | \u2705 | \u2705 | \u274c | [5][lsgan], [10][pre-wgan], [11][gan-tricks] | [](https://colab.research.google.com/github/mbarbetti/pidgan-notebooks/blob/main/tutorial-LSGAN-LHCb_RICH.ipynb) |\n| WGAN | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k2/WGAN.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k3/WGAN.py) | \u2705 | \u2705 | \u2705 | [6][wgan], [11][gan-tricks] | [](https://colab.research.google.com/github/mbarbetti/pidgan-notebooks/blob/main/tutorial-WGAN-LHCb_RICH.ipynb) |\n| WGAN-GP | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k2/WGAN_GP.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k3/WGAN_GP.py) | \u2705 | \u2705 | \u2705 | [7][wgan-gp], [11][gan-tricks] | [](https://colab.research.google.com/github/mbarbetti/pidgan-notebooks/blob/main/tutorial-WGAN_GP-LHCb_RICH.ipynb) |\n| CramerGAN | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k2/CramerGAN.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k3/CramerGAN.py) | \u2705 | \u2705 | \u2705 | [8][cramer-gan], [11][gan-tricks] | [](https://colab.research.google.com/github/mbarbetti/pidgan-notebooks/blob/main/tutorial-CramerGAN-LHCb_RICH.ipynb) |\n| WGAN-ALP | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k2/WGAN_ALP.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k3/WGAN_ALP.py) | \u2705 | \u2705 | \u2705 | [9][wgan-alp], [11][gan-tricks] | [](https://colab.research.google.com/github/mbarbetti/pidgan-notebooks/blob/main/tutorial-WGAN_ALP-LHCb_RICH.ipynb) |\n| BceGAN-GP | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k2/BceGAN_GP.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k3/BceGAN_GP.py) | \u2705 | \u2705 | \u2705 | [4][dcgan], [7][wgan-gp], [11][gan-tricks] | [](https://colab.research.google.com/github/mbarbetti/pidgan-notebooks/blob/main/tutorial-BceGAN_GP-LHCb_RICH.ipynb) |\n| BceGAN-ALP | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k2/BceGAN_ALP.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/algorithms/k3/BceGAN_ALP.py) | \u2705 | \u2705 | \u2705 | [4][dcgan], [9][wgan-alp], [11][gan-tricks] | [](https://colab.research.google.com/github/mbarbetti/pidgan-notebooks/blob/main/tutorial-BceGAN_ALP-LHCb_RICH.ipynb) |\n\n*each GAN algorithm is designed to operate taking __conditions__ as input [[3][cgan]]\n\n**the GAN training is regularized to ensure that the discriminator encodes a 1-Lipschitz function\n\n#### Generators\n\n| Players | Source | Avail | Test | Skip conn | Refs |\n|:-------:|:------:|:-----:|:----:|:---------:|:----:|\n| Generator | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/generators/k2/Generator.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/generators/k3/Generator.py) | \u2705 | \u2705 | \u274c | [2][gan], [3][cgan] |\n| ResGenerator | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/generators/k2/ResGenerator.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/generators/k3/ResGenerator.py) | \u2705 | \u2705 | \u2705 | [2][gan], [3][cgan], [12][skip-conn] |\n\n#### Discriminators\n\n| Players | Source | Avail | Test | Skip conn | Aux proc | Refs |\n|:-------:|:------:|:-----:|:----:|:---------:|:--------:|:----:|\n| Discriminator | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/discriminators/k2/Discriminator.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/discriminators/k3/Discriminator.py) | \u2705 | \u2705 | \u274c | \u274c | [2][gan], [3][cgan], [11][gan-tricks] |\n| ResDiscriminator | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/discriminators/k2/ResDiscriminator.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/discriminators/k3/ResDiscriminator.py) | \u2705 | \u2705 | \u2705 | \u274c | [2][gan], [3][cgan], [11][gan-tricks], [12][skip-conn] |\n| AuxDiscriminator | [`k2`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/discriminators/k2/AuxDiscriminator.py)/[`k3`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/discriminators/k3/AuxDiscriminator.py) | \u2705 | \u2705 | \u2705 | \u2705 | [2][gan], [3][cgan], [11][gan-tricks], [12][skip-conn], [13][aux-feat] |\n\n#### Other players\n\n| Players | Source | Avail | Test | Skip conn | Aux proc | Multiclass |\n|:-------:|:------:|:-----:|:----:|:---------:|:--------:|:---------:|\n| Classifier | [`src`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/classifiers/Classifier.py) | \u2705 | \u2705 | \u274c | \u274c | \u274c |\n| ResClassifier | [`src`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/classifiers/ResClassifier.py) | \u2705 | \u2705 | \u2705 | \u274c | \u274c |\n| AuxClassifier | [`src`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/classifiers/AuxClassifier.py) | \u2705 | \u2705 | \u2705 | \u2705 | \u274c |\n| MultiClassifier | [`src`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/classifiers/MultiClassifier.py) | \u2705 | \u2705 | \u274c | \u274c | \u2705 |\n| MultiResClassifier | [`src`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/classifiers/MultiResClassifier.py) | \u2705 | \u2705 | \u2705 | \u274c | \u2705 |\n| AuxMultiClassifier | [`src`](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/players/classifiers/AuxMultiClassifier.py) | \u2705 | \u2705 | \u2705 | \u2705 | \u2705 |\n\n### References\n\n1. M. Barbetti, \"The flash-simulation paradigm and its implementation based on Deep Generative Models for the LHCb experiment at CERN\", PhD thesis, University of Firenze, 2024\n2. I.J. Goodfellow _et al._, \"Generative Adversarial Networks\", [arXiv:1406.2661][gan]\n3. M. Mirza, S. Osindero, \"Conditional Generative Adversarial Nets\", [arXiv:1411.1784][cgan]\n4. A. Radford, L. Metz, S. Chintala, \"Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks\", [arXiv:1511.06434][dcgan]\n5. X. Mao _et al._, \"Least Squares Generative Adversarial Networks\", [arXiv:1611.04076][lsgan]\n6. M. Arjovsky, S. Chintala, L. Bottou, \"Wasserstein GAN\", [arXiv:1701.07875][wgan]\n7. I. Gulrajani _et al._, \"Improved Training of Wasserstein GANs\", [arXiv:1704.00028][wgan-gp]\n8. M.G. Bellemare _et al._, \"The Cramer Distance as a Solution to Biased Wasserstein Gradients\", [arXiv:1705.10743][cramer-gan]\n9. D. Terj\u00e9k, \"Adversarial Lipschitz Regularization\", [arXiv:1907.05681][wgan-alp]\n10. M. Arjovsky, L. Bottou, \"Towards Principled Methods for Training Generative Adversarial Networks\", [arXiv:1701.04862][pre-wgan]\n11. T. Salimans _et al._, \"Improved Techniques for Training GANs\", [arXiv:1606.03498][gan-tricks]\n12. K. He _et al._, \"Deep Residual Learning for Image Recognition\", [arXiv:1512.03385][skip-conn]\n13. A. Rogachev, F. Ratnikov, \"GAN with an Auxiliary Regressor for the Fast Simulation of the Electromagnetic Calorimeter Response\", [arXiv:2207.06329][aux-feat]\n\n### Credits\n\nMost of the GAN algorithms are an evolution of what provided by the [mbarbetti/tf-gen-models](https://github.com/mbarbetti/tf-gen-models) repository. The BceGAN model is freely inspired by the TensorFlow tutorial [Deep Convolutional Generative Adversarial Network](https://www.tensorflow.org/tutorials/generative/dcgan) and the Keras tutorial [Conditional GAN](https://keras.io/examples/generative/conditional_gan). The WGAN-ALP model is an adaptation of what provided by the [dterjek/adversarial_lipschitz_regularization](https://github.com/dterjek/adversarial_lipschitz_regularization) repository.\n\n### Citing PIDGAN\n\nTo cite this repository:\n\n```bibtex\n@software{pidgan:2023abc,\n author = \"Matteo Barbetti and Lucio Anderlini\",\n title = \"{PIDGAN: GAN-based models to flash-simulate the LHCb PID detectors}\",\n version = \"v0.2.0\",\n url = \"https://github.com/mbarbetti/pidgan\",\n doi = \"10.5281/zenodo.10463728\",\n publisher = \"Zenodo\",\n year = \"2023\",\n}\n```\n\nIn the above bibtex entry, the version number is intended to be that from [pidgan/version.py](https://github.com/mbarbetti/pidgan/blob/main/src/pidgan/version.py), while the year corresponds to the project's open-source release.\n\n### License\n\nPIDGAN has a GNU General Public License v3 (GPLv3), as found in the [LICENSE](https://github.com/mbarbetti/pidgan/blob/main/LICENSE) file.\n\n[gan]: https://arxiv.org/abs/1406.2661\n[dcgan]: https://arxiv.org/abs/1511.06434\n[lsgan]: https://arxiv.org/abs/1611.04076\n[wgan]: https://arxiv.org/abs/1701.07875\n[wgan-gp]: https://arxiv.org/abs/1704.00028\n[cramer-gan]: https://arxiv.org/abs/1705.10743\n[wgan-alp]: https://arxiv.org/abs/1907.05681\n[pre-wgan]: https://arxiv.org/abs/1701.04862\n[gan-tricks]: https://arxiv.org/abs/1606.03498\n[cgan]: https://arxiv.org/abs/1411.1784\n[skip-conn]: https://arxiv.org/abs/1512.03385\n[aux-feat]: https://arxiv.org/abs/2207.06329\n",
"bugtrack_url": null,
"license": "GPLv3 License",
"summary": "GAN-based models to flash-simulate the LHCb PID detectors",
"version": "0.2.0",
"project_urls": {
"repository": "https://github.com/mbarbetti/pidgan"
},
"split_keywords": [
"tensorflow",
" keras",
" machine learning",
" deep learning",
" generative models",
" generative adversarial nets",
" lhcb experiment",
" lamarr",
" flash-simulation",
" particle identification"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1dcfa176f723d72612e4e3c295bb950271d4646c9c189eac6aab376cac63d934",
"md5": "690acd523f11d6ae82a8be413bb5e954",
"sha256": "8116bc8425b73fd153d58f65dca93d9f8a15c61949eb75714150534d523c1fa8"
},
"downloads": -1,
"filename": "pidgan-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "690acd523f11d6ae82a8be413bb5e954",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.13,>=3.7",
"size": 98965,
"upload_time": "2024-07-01T23:17:03",
"upload_time_iso_8601": "2024-07-01T23:17:03.728338Z",
"url": "https://files.pythonhosted.org/packages/1d/cf/a176f723d72612e4e3c295bb950271d4646c9c189eac6aab376cac63d934/pidgan-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "b27ed4faee1d905d0eae7365f8351a3171d5a7ec75f921c44b5224d43942163a",
"md5": "48a2fbc2eddb2d468f0fa1f98e5e82e2",
"sha256": "5298b745349056e79196ecb463adef1938dd47449ad59b4a1925afd9ff6cdaff"
},
"downloads": -1,
"filename": "pidgan-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "48a2fbc2eddb2d468f0fa1f98e5e82e2",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.13,>=3.7",
"size": 55649,
"upload_time": "2024-07-01T23:17:08",
"upload_time_iso_8601": "2024-07-01T23:17:08.676236Z",
"url": "https://files.pythonhosted.org/packages/b2/7e/d4faee1d905d0eae7365f8351a3171d5a7ec75f921c44b5224d43942163a/pidgan-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-01 23:17:08",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "mbarbetti",
"github_project": "pidgan",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "pidgan"
}
JSON
