# lightweight-mcnnm
[](https://www.gnu.org/licenses/gpl-3.0)
[](https://pypi.org/project/lightweight-mcnnm/)
[](https://pypi.org/project/lightweight-mcnnm/)
[](https://mcnnm.readthedocs.io/en/latest/?badge=latest)
[](https://github.com/psf/black)
[](https://github.com/tobias-schnabel/mcnnm/actions/workflows/ci.yml)
[](https://codecov.io/gh/tobias-schnabel/mcnnm)
[](https://github.com/tobias-schnabel/mcnnm/actions/workflows/ci.yml)
[](https://github.com/tobias-schnabel/mcnnm/commits/)
lightweight-mcnnm is a Python package that provides a lightweight and performant implementation of the Matrix Completion with Nuclear Norm Minimization (MC-NNM) estimator for causal inference in panel data settings.
## Table of Contents
- [What is lightweight-mcnnm](#What-is-lightweight-mcnnm)
- [Features](#features)
- [Installation](#installation)
- [Documentation](#documentation)
- [Using lightweight-mcnnm](#using-lightweight-mcnnm)
- [Development](#development)
- [References](#references)
- [Acknowledgements](#acknowledgements)
- [License](#license)
[Changelog, Contributing, and Templates](#changelog-contributing-and-templates)
## What is lightweight-mcnnm
lightweight-mcnnm implements the MC-NNM estimator exactly as described in "Matrix Completion Methods for Causal Panel Data Models" by Susan Athey, Mohsen Bayati, Nikolay Doudchenko, Guido Imbens, and Khashayar Khosravi [(2021)](https://www.tandfonline.com/doi/full/10.1080/01621459.2021.1891924). This estimator provides a powerful tool for estimating causal effects in panel data settings, particularly when dealing with complex treatment patterns and potential confounders.
The implementation focuses on performance and minimal dependencies, making it suitable for use in various environments, including GPUs and cloud clusters.
## Features
- Lightweight implementation with minimal dependencies
- Utilizes JAX for improved performance and GPU compatibility
- Faithful to the original MC-NNM algorithm as described in [Athey et al. (2021)](https://www.tandfonline.com/doi/full/10.1080/01621459.2021.1891924)
- Suitable for large-scale panel data analysis
- Supports various treatment assignment mechanisms
- Includes unit-specific, time-specific, and unit-time specific covariates
- Offers flexible validation methods for parameter selection
### Comparison to Other Implementations
lightweight-mcnnm is designed to be lightweight and easy to use, with a focus on performance and minimal dependencies.
The other two main implementations of the MC-NNM estimator are [CausalTensor](https://github.com/TianyiPeng/causaltensor) and
[fect](https://yiqingxu.org/packages/fect/fect.html). Both packages implement MC-NNM as part of a broader set of causal inference methods. Both implement covariates and cross-validation differently from this package.
For a detailed comparison, see this notebook:
[](https://colab.research.google.com/github/tobias-schnabel/mcnnm/blob/main/Comparison.ipynb)
## Installation
### Requirements
lightweight-mcnnm is compatible with Python 3.10 or later and depends on JAX and NumPy. CUDA-compatible versions of Jax are not currently supported directly by lightweight-mcnnm, but you can use JAX with CUDA support by installing it separately.
### Installing from PyPI
The simplest way to install lightweight-mcnnm and its dependencies is from PyPI using pip:
```bash
pip install lightweight-mcnnm
```
To upgrade lightweight-mcnnm to the latest version, use:
```bash
pip install --upgrade lightweight-mcnnm
```
#### JIT Compilation
By default, this package uses JAX's JIT compilation for better performance in typical use cases. If you want to disable JIT compilation, you can add the following line at the top of your script:
```python
jax.config.update('jax_disable_jit', True)
```
Note that disabling JIT may impact performance depending on your specific use case. I have found leaving JIT enabled to be the best option for most use cases. An example use case where disabling JIT may be sensible is calling estimate() multiple times on datasets of different sizes, which triggers recompilation any time the input data shape changes.
## Documentation
The full documentation for lightweight-mcnnm is available at:
https://mcnnm.readthedocs.io/en/latest/
## Using lightweight-mcnnm
1. A comprehensive example is available here: [](https://colab.research.google.com/github/tobias-schnabel/mcnnm/blob/main/Example.ipynb)
2. Simple example of how to use lightweight-mcnnm:
```python
import jax.numpy as jnp
from lightweight_mcnnm import estimate, generate_data
Y, W, X, Z, V, true_params = generate_data(
nobs=50,
nperiods=10,
unit_fe=True,
time_fe=True,
X_cov=True,
Z_cov=True,
V_cov=True,
seed=2024,
noise_scale=0.1,
assignment_mechanism="staggered",
treatment_probability=0.1,
)
# Run estimation
results = estimate(
Y=Y,
Mask=W,
X=X,
Z=Z,
V=V,
Omega=None,
use_unit_fe=True,
use_time_fe=True,
lambda_L=None,
lambda_H=None,
validation_method='cv',
K=3,
n_lambda=30,
)
print(f"\nTrue effect: {true_params['treatment_effect']}, Estimated effect: {results.tau:.3f}")
print(f"Chosen lambda_L: {results.lambda_L:.4f}, lambda_H: {results.lambda_H:.4f}")
```
For more detailed usage instructions and examples, please refer to the documentation.
## Development
### Setting up the development environment
This project uses [Poetry](https://python-poetry.org/) for dependency management. To set up your development environment:
1. Ensure you have Poetry installed. If not, install it by following the instructions on the [official Poetry website](https://python-poetry.org/docs/#installation).
2. Clone the repository:
```bash
git clone https://github.com/tobias-schnabel/mcnnm.git
cd lightweight-mcnnm
```
3. Install the project dependencies:
```bash
poetry install
```
This command creates a virtual environment and installs all the necessary dependencies.
4. Activate the virtual environment:
```bash
poetry shell
```
Now you're ready to start developing!
### Testing and building the package
5. Running tests: use the following command:
```bash
poetry run pytest
```
6. Coverage: to generate a coverage report, run the following command:
```bash
poetry run coverage report
```
This will generate a coverage report showing the percentage of code covered by the tests.
6. Building the package: run the following command:
```bash
poetry build
```
This will create both wheel and source distributions in the dist/ directory.
## Development Workflow
### Pre-commit Hooks
This project uses pre-commit hooks to ensure code quality and consistency. Pre-commit hooks are scripts that run automatically every time you commit changes to your version control system. They help catch common issues before they get into the codebase. To set up:
1. Install pre-commit:
```bash
poetry add pre-commit
```
2. Install the hooks:
```bash
poetry run pre-commit install
```
3. Run the hooks on all files (recommended for the first setup):
```bash
poetry run pre-commit run --all-files
```
The configuration for the pre-commit hooks can be found in the .pre-commit-config.yaml file. The following hooks are configured:
• Trailing whitespace removal: Ensures no trailing whitespace is left in the code.
• End-of-file fixer: Ensures files end with a newline.
• YAML check: Validates YAML files.
• Flake8: Checks for Python style guide enforcement.
• Black: Ensures consistent code formatting.
• Bandit: Checks for common security issues in Python code.
• Mypy: Performs static type checking.
### Branch Protection
To maintain the integrity of the main branch, branch protection rules are enforced. These rules ensure that all changes to the main branch go through a review process and pass all required checks.
#### Protected Branch Rules
1. Require pull request reviews before merging: At least one approval from an administrator is required.
2. Require status checks to pass before merging: All CI checks must be successful before merging.
## References
This implementation is based on the method described in:
[Athey, S., Bayati, M., Doudchenko, N., Imbens, G., & Khosravi, K. (2021). Matrix Completion Methods for Causal Panel Data Models. Journal of the American Statistical Association, 116(536), 1716-1730.](https://www.tandfonline.com/doi/full/10.1080/01621459.2021.1891924)
## Acknowledgements
This project was inspired by and draws upon ideas from
[CausalTensor](https://github.com/TianyiPeng/causaltensor) and
[fect](https://yiqingxu.org/packages/fect/fect.html). I am grateful for their contributions to the field of causal inference.
## Citing lightweight-mcnnm
If you use lightweight-mcnnm in your research, please cite both the software and the original paper describing the method:
For the software:
Schnabel, T. (2023). lightweight-mcnnm: A Python package for Matrix Completion with Nuclear Norm Minimization. https://github.com/tobias-schnabel/mcnnm
For the method:
Athey, S., Bayati, M., Doudchenko, N., Imbens, G., & Khosravi, K. (2021). Matrix Completion Methods for Causal Panel Data Models. Journal of the American Statistical Association, 116(536), 1716-1730.
BibTeX entries:
@software{schnabel2024lightweightmcnnm,
author = {Schnabel, Tobias},
title = {lightweight-mcnnm: A Python package for Matrix Completion with Nuclear Norm Minimization},
year = {2024},
url = {https://github.com/tobias-schnabel/mcnnm}
}
@article{athey2021matrix,
title={Matrix completion methods for causal panel data models},
author={Athey, Susan and Bayati, Mohsen and Doudchenko, Nikolay and Imbens, Guido and Khosravi, Khashayar},
journal={Journal of the American Statistical Association},
volume={116},
number={536},
pages={1716--1730},
year={2021},
publisher={Taylor \& Francis}
}
## License
lightweight-mcnnm is released under the GNU General Public License v3.0. See the [LICENSE](./LICENSE) file for more details.
## Changelog, Contributing, and Templates
1. For a detailed changelog of each release, please see the [GitHub Releases page](https://github.com/tobias-schnabel/mcnnm/releases)
2. Please refer to [CONTRIBUTING.md](./CONTRIBUTING.md) for guidelines on how to contribute to this project.
3. For reporting issues, please use the template provided in [ISSUE_TEMPLATE.md](./.github/ISSUE_TEMPLATE.md)
4. For submitting pull requests, please use the template provided in [PULL_REQUEST_TEMPLATE.md](./.github/PULL_REQUEST_TEMPLATE.md)
Raw data
{
"_id": null,
"home_page": "https://github.com/tobias-schnabel/mcnnm",
"name": "lightweight-mcnnm",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.10",
"maintainer_email": null,
"keywords": "causal, inference, panel, matrix",
"author": "Tobias Schnabel",
"author_email": "ctobiasschnabel@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/22/8b/0a6ef027ceeabb04648ad129c525d6abaf7b4ed950e1a5a99306bd820b32/lightweight_mcnnm-1.0.2.tar.gz",
"platform": null,
"description": "# lightweight-mcnnm\n\n[](https://www.gnu.org/licenses/gpl-3.0)\n[](https://pypi.org/project/lightweight-mcnnm/)\n\n[](https://pypi.org/project/lightweight-mcnnm/)\n[](https://mcnnm.readthedocs.io/en/latest/?badge=latest)\n[](https://github.com/psf/black)\n[](https://github.com/tobias-schnabel/mcnnm/actions/workflows/ci.yml)\n[](https://codecov.io/gh/tobias-schnabel/mcnnm)\n[](https://github.com/tobias-schnabel/mcnnm/actions/workflows/ci.yml)\n[](https://github.com/tobias-schnabel/mcnnm/commits/)\n\n\n\nlightweight-mcnnm is a Python package that provides a lightweight and performant implementation of the Matrix Completion with Nuclear Norm Minimization (MC-NNM) estimator for causal inference in panel data settings.\n\n## Table of Contents\n- [What is lightweight-mcnnm](#What-is-lightweight-mcnnm)\n- [Features](#features)\n- [Installation](#installation)\n- [Documentation](#documentation)\n- [Using lightweight-mcnnm](#using-lightweight-mcnnm)\n- [Development](#development)\n- [References](#references)\n- [Acknowledgements](#acknowledgements)\n- [License](#license)\n [Changelog, Contributing, and Templates](#changelog-contributing-and-templates)\n\n\n## What is lightweight-mcnnm\n\nlightweight-mcnnm implements the MC-NNM estimator exactly as described in \"Matrix Completion Methods for Causal Panel Data Models\" by Susan Athey, Mohsen Bayati, Nikolay Doudchenko, Guido Imbens, and Khashayar Khosravi [(2021)](https://www.tandfonline.com/doi/full/10.1080/01621459.2021.1891924). This estimator provides a powerful tool for estimating causal effects in panel data settings, particularly when dealing with complex treatment patterns and potential confounders.\n\nThe implementation focuses on performance and minimal dependencies, making it suitable for use in various environments, including GPUs and cloud clusters.\n\n## Features\n\n- Lightweight implementation with minimal dependencies\n- Utilizes JAX for improved performance and GPU compatibility\n- Faithful to the original MC-NNM algorithm as described in [Athey et al. (2021)](https://www.tandfonline.com/doi/full/10.1080/01621459.2021.1891924)\n- Suitable for large-scale panel data analysis\n- Supports various treatment assignment mechanisms\n- Includes unit-specific, time-specific, and unit-time specific covariates\n- Offers flexible validation methods for parameter selection\n\n### Comparison to Other Implementations\nlightweight-mcnnm is designed to be lightweight and easy to use, with a focus on performance and minimal dependencies.\nThe other two main implementations of the MC-NNM estimator are [CausalTensor](https://github.com/TianyiPeng/causaltensor) and\n[fect](https://yiqingxu.org/packages/fect/fect.html). Both packages implement MC-NNM as part of a broader set of causal inference methods. Both implement covariates and cross-validation differently from this package.\nFor a detailed comparison, see this notebook:\n[](https://colab.research.google.com/github/tobias-schnabel/mcnnm/blob/main/Comparison.ipynb)\n## Installation\n\n### Requirements\n\nlightweight-mcnnm is compatible with Python 3.10 or later and depends on JAX and NumPy. CUDA-compatible versions of Jax are not currently supported directly by lightweight-mcnnm, but you can use JAX with CUDA support by installing it separately.\n\n### Installing from PyPI\n\nThe simplest way to install lightweight-mcnnm and its dependencies is from PyPI using pip:\n\n```bash\npip install lightweight-mcnnm\n```\n\n\nTo upgrade lightweight-mcnnm to the latest version, use:\n```bash\npip install --upgrade lightweight-mcnnm\n```\n\n#### JIT Compilation\nBy default, this package uses JAX's JIT compilation for better performance in typical use cases. If you want to disable JIT compilation, you can add the following line at the top of your script:\n\n```python\njax.config.update('jax_disable_jit', True)\n```\n\nNote that disabling JIT may impact performance depending on your specific use case. I have found leaving JIT enabled to be the best option for most use cases. An example use case where disabling JIT may be sensible is calling estimate() multiple times on datasets of different sizes, which triggers recompilation any time the input data shape changes.\n\n## Documentation\nThe full documentation for lightweight-mcnnm is available at:\nhttps://mcnnm.readthedocs.io/en/latest/\n## Using lightweight-mcnnm\n1. A comprehensive example is available here: [](https://colab.research.google.com/github/tobias-schnabel/mcnnm/blob/main/Example.ipynb)\n2. Simple example of how to use lightweight-mcnnm:\n```python\nimport jax.numpy as jnp\nfrom lightweight_mcnnm import estimate, generate_data\n\nY, W, X, Z, V, true_params = generate_data(\n nobs=50,\n nperiods=10,\n unit_fe=True,\n time_fe=True,\n X_cov=True,\n Z_cov=True,\n V_cov=True,\n seed=2024,\n noise_scale=0.1,\n assignment_mechanism=\"staggered\",\n treatment_probability=0.1,\n )\n\n# Run estimation\nresults = estimate(\n Y=Y,\n Mask=W,\n X=X,\n Z=Z,\n V=V,\n Omega=None,\n use_unit_fe=True,\n use_time_fe=True,\n lambda_L=None,\n lambda_H=None,\n validation_method='cv',\n K=3,\n n_lambda=30,\n)\n\nprint(f\"\\nTrue effect: {true_params['treatment_effect']}, Estimated effect: {results.tau:.3f}\")\nprint(f\"Chosen lambda_L: {results.lambda_L:.4f}, lambda_H: {results.lambda_H:.4f}\")\n```\nFor more detailed usage instructions and examples, please refer to the documentation.\n\n## Development\n\n### Setting up the development environment\n\nThis project uses [Poetry](https://python-poetry.org/) for dependency management. To set up your development environment:\n\n1. Ensure you have Poetry installed. If not, install it by following the instructions on the [official Poetry website](https://python-poetry.org/docs/#installation).\n\n2. Clone the repository:\n ```bash\n git clone https://github.com/tobias-schnabel/mcnnm.git\n cd lightweight-mcnnm\n ```\n3. Install the project dependencies:\n ```bash\n poetry install\n ```\n This command creates a virtual environment and installs all the necessary dependencies.\n4. Activate the virtual environment:\n ```bash\n poetry shell\n ```\nNow you're ready to start developing!\n### Testing and building the package\n5. Running tests: use the following command:\n ```bash\n poetry run pytest\n ```\n\n6. Coverage: to generate a coverage report, run the following command:\n ```bash\n poetry run coverage report\n ```\n This will generate a coverage report showing the percentage of code covered by the tests.\n6. Building the package: run the following command:\n ```bash\n poetry build\n ```\n This will create both wheel and source distributions in the dist/ directory.\n\n## Development Workflow\n### Pre-commit Hooks\nThis project uses pre-commit hooks to ensure code quality and consistency. Pre-commit hooks are scripts that run automatically every time you commit changes to your version control system. They help catch common issues before they get into the codebase. To set up:\n1. Install pre-commit:\n ```bash\n poetry add pre-commit\n ```\n2. Install the hooks:\n ```bash\n poetry run pre-commit install\n ```\n3. Run the hooks on all files (recommended for the first setup):\n ```bash\n poetry run pre-commit run --all-files\n ```\nThe configuration for the pre-commit hooks can be found in the .pre-commit-config.yaml file. The following hooks are configured:\n\n \u2022\tTrailing whitespace removal: Ensures no trailing whitespace is left in the code.\n \u2022\tEnd-of-file fixer: Ensures files end with a newline.\n \u2022\tYAML check: Validates YAML files.\n \u2022\tFlake8: Checks for Python style guide enforcement.\n \u2022\tBlack: Ensures consistent code formatting.\n \u2022\tBandit: Checks for common security issues in Python code.\n \u2022\tMypy: Performs static type checking.\n\n### Branch Protection\nTo maintain the integrity of the main branch, branch protection rules are enforced. These rules ensure that all changes to the main branch go through a review process and pass all required checks.\n#### Protected Branch Rules\n\n 1.\tRequire pull request reviews before merging: At least one approval from an administrator is required.\n 2.\tRequire status checks to pass before merging: All CI checks must be successful before merging.\n\n## References\nThis implementation is based on the method described in:\n[Athey, S., Bayati, M., Doudchenko, N., Imbens, G., & Khosravi, K. (2021). Matrix Completion Methods for Causal Panel Data Models. Journal of the American Statistical Association, 116(536), 1716-1730.](https://www.tandfonline.com/doi/full/10.1080/01621459.2021.1891924)\n\n## Acknowledgements\nThis project was inspired by and draws upon ideas from\n[CausalTensor](https://github.com/TianyiPeng/causaltensor) and\n[fect](https://yiqingxu.org/packages/fect/fect.html). I am grateful for their contributions to the field of causal inference.\n## Citing lightweight-mcnnm\n\nIf you use lightweight-mcnnm in your research, please cite both the software and the original paper describing the method:\n\nFor the software:\nSchnabel, T. (2023). lightweight-mcnnm: A Python package for Matrix Completion with Nuclear Norm Minimization. https://github.com/tobias-schnabel/mcnnm\n\nFor the method:\nAthey, S., Bayati, M., Doudchenko, N., Imbens, G., & Khosravi, K. (2021). Matrix Completion Methods for Causal Panel Data Models. Journal of the American Statistical Association, 116(536), 1716-1730.\n\nBibTeX entries:\n\n@software{schnabel2024lightweightmcnnm,\n author = {Schnabel, Tobias},\n title = {lightweight-mcnnm: A Python package for Matrix Completion with Nuclear Norm Minimization},\n year = {2024},\n url = {https://github.com/tobias-schnabel/mcnnm}\n}\n\n@article{athey2021matrix,\n title={Matrix completion methods for causal panel data models},\n author={Athey, Susan and Bayati, Mohsen and Doudchenko, Nikolay and Imbens, Guido and Khosravi, Khashayar},\n journal={Journal of the American Statistical Association},\n volume={116},\n number={536},\n pages={1716--1730},\n year={2021},\n publisher={Taylor \\& Francis}\n}\n## License\nlightweight-mcnnm is released under the GNU General Public License v3.0. See the [LICENSE](./LICENSE) file for more details.\n\n## Changelog, Contributing, and Templates\n1. For a detailed changelog of each release, please see the [GitHub Releases page](https://github.com/tobias-schnabel/mcnnm/releases)\n2. Please refer to [CONTRIBUTING.md](./CONTRIBUTING.md) for guidelines on how to contribute to this project.\n3. For reporting issues, please use the template provided in [ISSUE_TEMPLATE.md](./.github/ISSUE_TEMPLATE.md)\n4. For submitting pull requests, please use the template provided in [PULL_REQUEST_TEMPLATE.md](./.github/PULL_REQUEST_TEMPLATE.md)\n",
"bugtrack_url": null,
"license": "GNU3.0",
"summary": "Leightweight Implementation of Athey et al. (2021)'s MC-NNM estimator",
"version": "1.0.2",
"project_urls": {
"Documentation": "https://mcnnm.readthedocs.io/en/latest/",
"Homepage": "https://github.com/tobias-schnabel/mcnnm",
"Repository": "https://github.com/tobias-schnabel/mcnnm"
},
"split_keywords": [
"causal",
" inference",
" panel",
" matrix"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d790b96656b73c34596cf55e4e0139062ab7cf6eed0dcfe532e64e90f5408ce2",
"md5": "681717dc131d6110007ddef1d3b77c2c",
"sha256": "7781b871822c3e6828feff6eaf9b0ef0d16fb8e09fe72daae5d3daed8078908b"
},
"downloads": -1,
"filename": "lightweight_mcnnm-1.0.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "681717dc131d6110007ddef1d3b77c2c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.10",
"size": 58584,
"upload_time": "2024-09-01T16:21:14",
"upload_time_iso_8601": "2024-09-01T16:21:14.072987Z",
"url": "https://files.pythonhosted.org/packages/d7/90/b96656b73c34596cf55e4e0139062ab7cf6eed0dcfe532e64e90f5408ce2/lightweight_mcnnm-1.0.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "228b0a6ef027ceeabb04648ad129c525d6abaf7b4ed950e1a5a99306bd820b32",
"md5": "a2a7cf26f1cfac38df519e6ae5b3c201",
"sha256": "ecaa0122b2ce8fd2731ad8e4621e0b0c97357679865c38d78e150e4d33436ce0"
},
"downloads": -1,
"filename": "lightweight_mcnnm-1.0.2.tar.gz",
"has_sig": false,
"md5_digest": "a2a7cf26f1cfac38df519e6ae5b3c201",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.10",
"size": 43299,
"upload_time": "2024-09-01T16:21:15",
"upload_time_iso_8601": "2024-09-01T16:21:15.905118Z",
"url": "https://files.pythonhosted.org/packages/22/8b/0a6ef027ceeabb04648ad129c525d6abaf7b4ed950e1a5a99306bd820b32/lightweight_mcnnm-1.0.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-01 16:21:15",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "tobias-schnabel",
"github_project": "mcnnm",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "lightweight-mcnnm"
}
JSON
