# Scalable Semi-Modular Inference with Variational Meta-Posteriors
<!-- badges: start -->
[](https://github.com/chriscarmona/modularbayes/blob/main/LICENSE)
[](https://doi.org/10.48550/arXiv.2204.00296)
[](https://arxiv.org/abs/2204.00296)
<!-- badges: end -->
This repo contain the implementation of variational methods described in the article *Scalable Semi-Modular Inference with Variational Meta-Posteriors*.
__Abstract:__
The Cut posterior and related Semi-Modular Inference (SMI) are Generalised Bayes methods for Modular Bayesian evidence combination. Analysis is broken up over modular sub-models of the joint posterior distribution. Model- misspecification in multi-modular models can be hard to fix by model elaboration alone and the Cut posterior and SMI offer a way round this. Information entering the analysis from misspecified modules is controlled by an influence parameter η related to the learning rate.
The article contains two substantial new methods. First, we give variational methods for approximating the Cut and SMI posteriors which are adapted to the inferential goals of evidence combination. We parameterise a family of variational posteriors using a Normalizing Flow for accurate approximation and end-to-end training. Secondly, we show that analysis of models with multiple cuts is feasible using a new Variational Meta-Posterior. This approximates a family of SMI posteriors indexed by η using a single set of variational parameters.
## Examples
We provide code to replicate all the experiments from our article in the [examples](https://github.com/chriscarmona/modularbayes/tree/main/examples) folder.
There are main two examples: 1) epidemiological model and 2) random effects model.
By executing the `run.sh` bash script in that folder, one can train all variational posteriors and produce visualizations and summaries (follow [*Installation instructions*](#installation-instructions) and examine `run.sh` before execution).
You can customize the output directories within the script and choose the experiments that you wish to run.
```bash
pip install -Ur examples/requirements.txt
chmod +x examples/run.sh
bash examples/run.sh
```
Results produced during the optimization can be monitored in [Tensorboard](https://www.tensorflow.org/tensorboard):
```bash
tensorboard --logdir=$HOME/modularbayes-output
```
## Installation instructions
1. \[Optional] Create a new virtual environment for this project (see [*Create a virtual environment*](#create-a-virtual-environment) below).
2. Install JAX. This may vary according to your CUDA version (See [JAX installation](https://github.com/google/jax#installation)).
3. Install the latest released version of `modularbayes` from [Pypi](https://pypi.org/project/modularbayes/) via:
```bash
pip install -U modularbayes
```
or you can install the latest development version from GitHub:
```bash
pip install git+https://github.com/chriscarmona/modularbayes
```
## Citation
If you find this work relevant for your scientific publication, we encourage you to add the following reference:
```bibtex
@misc{Carmona2022scalable,
title = {Scalable Semi-Modular Inference with Variational Meta-Posteriors},
year = {2022},
author = {Carmona, Chris U. and Nicholls, Geoff K.},
month = {4},
url = {http://arxiv.org/abs/2204.00296},
doi = {10.48550/arXiv.2204.00296},
arxivId = {2204.00296},
keywords = {Cut models, Generalized Bayes, Model misspecification, Scalable inference, Variational Bayes}
}
@InProceedings{Carmona2020smi,
title = {Semi-Modular Inference: enhanced learning in multi-modular models by tempering the influence of components},
author = {Carmona, Chris U. and Nicholls, Geoff K.},
booktitle = {Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020},
year = {2020},
editor = {Silvia Chiappa and Roberto Calandra},
volume = {108},
pages = {4226--4235},
series = {Proceedings of Machine Learning Research},
month = {26--28 Aug},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v108/carmona20a/carmona20a.pdf},
url = {http://proceedings.mlr.press/v108/carmona20a.html},
arxivId = {2003.06804},
}
```
### Creating a virtual environment
For OSX or Linux, you can use `venv` (see the [venv documentation](https://docs.python.org/3/library/venv.html)).
```bash
rm -rf ~/.virtualenvs/modularbayes
python3 -m venv ~/.virtualenvs/modularbayes
source ~/.virtualenvs/modularbayes/bin/activate
pip install -U pip
pip install -U setuptools wheel
```
Feel free to change the folder where the virtual environment is created by replacing `~/.virtualenvs/modularbayes` with a path of your choice in both commands.
Raw data
{
"_id": null,
"home_page": "https://github.com/chriscarmona/modularbayes",
"name": "modularbayes",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "carmona@stats.ox.ac.uk",
"keywords": "modular bayesian inference cut smi posterior probability distribution",
"author": "Chris Carmona",
"author_email": "carmona@stats.ox.ac.uk",
"download_url": "https://files.pythonhosted.org/packages/69/71/b26d86c0c7a9822f8cab845c1c30f51e3e9b57e4080af62fbcb9385613b4/modularbayes-0.1.4.tar.gz",
"platform": null,
"description": "# Scalable Semi-Modular Inference with Variational Meta-Posteriors\n\n<!-- badges: start -->\n[](https://github.com/chriscarmona/modularbayes/blob/main/LICENSE)\n[](https://doi.org/10.48550/arXiv.2204.00296)\n[](https://arxiv.org/abs/2204.00296)\n<!-- badges: end -->\n\nThis repo contain the implementation of variational methods described in the article *Scalable Semi-Modular Inference with Variational Meta-Posteriors*.\n\n__Abstract:__\n\nThe Cut posterior and related Semi-Modular Inference (SMI) are Generalised Bayes methods for Modular Bayesian evidence combination. Analysis is broken up over modular sub-models of the joint posterior distribution. Model- misspecification in multi-modular models can be hard to fix by model elaboration alone and the Cut posterior and SMI offer a way round this. Information entering the analysis from misspecified modules is controlled by an influence parameter \u03b7 related to the learning rate.\n\nThe article contains two substantial new methods. First, we give variational methods for approximating the Cut and SMI posteriors which are adapted to the inferential goals of evidence combination. We parameterise a family of variational posteriors using a Normalizing Flow for accurate approximation and end-to-end training. Secondly, we show that analysis of models with multiple cuts is feasible using a new Variational Meta-Posterior. This approximates a family of SMI posteriors indexed by \u03b7 using a single set of variational parameters.\n\n## Examples\n\nWe provide code to replicate all the experiments from our article in the [examples](https://github.com/chriscarmona/modularbayes/tree/main/examples) folder.\nThere are main two examples: 1) epidemiological model and 2) random effects model.\nBy executing the `run.sh` bash script in that folder, one can train all variational posteriors and produce visualizations and summaries (follow [*Installation instructions*](#installation-instructions) and examine `run.sh` before execution).\n\nYou can customize the output directories within the script and choose the experiments that you wish to run.\n```bash\npip install -Ur examples/requirements.txt\nchmod +x examples/run.sh\nbash examples/run.sh\n```\nResults produced during the optimization can be monitored in [Tensorboard](https://www.tensorflow.org/tensorboard):\n```bash\ntensorboard --logdir=$HOME/modularbayes-output\n```\n\n## Installation instructions\n\n1. \\[Optional] Create a new virtual environment for this project (see [*Create a virtual environment*](#create-a-virtual-environment) below).\n2. Install JAX. This may vary according to your CUDA version (See [JAX installation](https://github.com/google/jax#installation)).\n3. Install the latest released version of `modularbayes` from [Pypi](https://pypi.org/project/modularbayes/) via:\n```bash\npip install -U modularbayes\n```\nor you can install the latest development version from GitHub:\n```bash\npip install git+https://github.com/chriscarmona/modularbayes\n```\n\n## Citation\n\nIf you find this work relevant for your scientific publication, we encourage you to add the following reference:\n\n```bibtex\n@misc{Carmona2022scalable,\n title = {Scalable Semi-Modular Inference with Variational Meta-Posteriors},\n year = {2022},\n author = {Carmona, Chris U. and Nicholls, Geoff K.},\n month = {4},\n url = {http://arxiv.org/abs/2204.00296},\n doi = {10.48550/arXiv.2204.00296},\n arxivId = {2204.00296},\n keywords = {Cut models, Generalized Bayes, Model misspecification, Scalable inference, Variational Bayes}\n}\n\n@InProceedings{Carmona2020smi,\n title = {Semi-Modular Inference: enhanced learning in multi-modular models by tempering the influence of components},\n author = {Carmona, Chris U. and Nicholls, Geoff K.},\n booktitle = {Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020},\n year = {2020},\n editor = {Silvia Chiappa and Roberto Calandra},\n volume = {108},\n pages = {4226--4235},\n series = {Proceedings of Machine Learning Research},\n month = {26--28 Aug},\n publisher = {PMLR},\n pdf = {http://proceedings.mlr.press/v108/carmona20a/carmona20a.pdf},\n url = {http://proceedings.mlr.press/v108/carmona20a.html},\n arxivId = {2003.06804},\n}\n```\n\n### Creating a virtual environment\n\nFor OSX or Linux, you can use `venv` (see the [venv documentation](https://docs.python.org/3/library/venv.html)).\n\n```bash\nrm -rf ~/.virtualenvs/modularbayes\npython3 -m venv ~/.virtualenvs/modularbayes\nsource ~/.virtualenvs/modularbayes/bin/activate\npip install -U pip\npip install -U setuptools wheel\n```\n\nFeel free to change the folder where the virtual environment is created by replacing `~/.virtualenvs/modularbayes` with a path of your choice in both commands.\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Modular Bayesian Inference.",
"version": "0.1.4",
"project_urls": {
"Homepage": "https://github.com/chriscarmona/modularbayes"
},
"split_keywords": [
"modular",
"bayesian",
"inference",
"cut",
"smi",
"posterior",
"probability",
"distribution"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "09844294ba0700062433f414a789c4b401845a5cc048b89b3fbb494585abeb7c",
"md5": "b900114f8573d5e33211be43748f446f",
"sha256": "ac1b28eaa79900ba080b45634c5c9083323daec0a34465f60851f8187b9a9dcb"
},
"downloads": -1,
"filename": "modularbayes-0.1.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b900114f8573d5e33211be43748f446f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 85452,
"upload_time": "2023-06-24T18:24:31",
"upload_time_iso_8601": "2023-06-24T18:24:31.755447Z",
"url": "https://files.pythonhosted.org/packages/09/84/4294ba0700062433f414a789c4b401845a5cc048b89b3fbb494585abeb7c/modularbayes-0.1.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "6971b26d86c0c7a9822f8cab845c1c30f51e3e9b57e4080af62fbcb9385613b4",
"md5": "39dcbafc39f689e9250a8eaf57965cc2",
"sha256": "227219297562ab8ed6bf0f654f4426760f6ce28a7f42f5a77dab236f8260a92d"
},
"downloads": -1,
"filename": "modularbayes-0.1.4.tar.gz",
"has_sig": false,
"md5_digest": "39dcbafc39f689e9250a8eaf57965cc2",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 52512,
"upload_time": "2023-06-24T18:24:32",
"upload_time_iso_8601": "2023-06-24T18:24:32.934148Z",
"url": "https://files.pythonhosted.org/packages/69/71/b26d86c0c7a9822f8cab845c1c30f51e3e9b57e4080af62fbcb9385613b4/modularbayes-0.1.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-24 18:24:32",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "chriscarmona",
"github_project": "modularbayes",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "modularbayes"
}
JSON
