# dagrad: Gradient-based structure learning for causal DAGs
## tl;dr
The `dagrad` Python library is built to provide researchers with an extensible, modular platform for developing and experimenting with continuous score-based structure learning methods, building upon the introduction of the [NOTEARS][notears] framework for differentiable structure learning. It also functions as an updated repository of [state-of-the-art implementations][#features] for various methods.
`dagrad` is a comprehensive suite of state-of-the-art __continuous score-based__ structure learning (aka causal discovery) methods, integrating them into a __unified__ framework. Users have the flexibility to use predefined loss functions or customize their own, along with different acyclicity constraints, regularizers, and optimizers with ease. Additionally, the library supports GPU acceleration, enabling faster computations.
## Introduction
A directed acyclic graphical model (also known as a Bayesian network) with `d` nodes represents a distribution over a random vector of size `d`. The focus of this library is on Bayesian Network Structure Learning (BNSL): Given samples $\mathbf{X}$ from a distribution, how can we estimate the underlying graph `G`?
The problem can be formulated as the following differentiable continuous optimization:
```math
\begin{array}{cl}
\min _{W \in \mathbb{R}^{d \times d}} & Q(W;\mathbf{X}) \\
\text { subject to } & h(W) = 0,
\end{array}
```
This formulation is versatile enough to encompass both linear and nonlinear models with any smooth objective (e.g. log-likelihood, least-squares, cross-entropy, etc.).
`dagrad` provides a unified framework that allows users to employ either predefined or customized loss functions $Q(W;\mathbf{X})$, acyclicity constraints $h(W)$, and choose between first-order or second-order optimizers:
```math
loss~function + acyclicity~constraint + optimizer => structure~learning
```
GPU acceleration is also supported.
## Run a simple demo
Use `setup.py` to install the dependencies (we recommend using virtualenv or conda). The simplest way to try out `dagrad` is to run the following example:
```bash
$ git clone https://github.com/Duntrain/DAGlearner.git
$ cd DAGlearner/
$ pip install -e .
$ cd dagrad
$ python -m dagrad.daglearner
```
The above runs the original NOTEARS method [[1]][notears] on a randomly generated 10-node Erdos-Renyi graph with 1000 samples. The output should look like the below:
```
{'fdr': 0.0, 'tpr': 1.0, 'fpr': 0.0, 'shd': 0, 'nnz': 10}
```
Want to try more examples? See an example in this [iPython notebook][examples].
## Features
Below is an overview of the functionalities provided by the package:
| __Method(`method`)__ | __Model(`model`)__ |__Loss(`loss_fn`)__ |__Regularizers(`reg`)__|__h(`h_fn`)__ |__Optimizer(`optimizer`)__ | __Computation Library(`compute_lib`)__ |__Device(`device`)__|
| -------- | -------- |----|---|-------------------|----| ----------| --------------|
|`'notears'`[1] | `'linear'`,<br>`'nonlinear'` |`'l2'`, `'logll'`, `'user_loss'`|`'l1'`<br> `'l2'`<br> `'mcp'`<br> `'none'`<br>`'user_reg'` |`'h_exp_sq'`<br>`'h_poly_sq'`<br>`'h_poly_abs'`<br>`'user_h'` |Adam(`'adam'`),<br>LBFGS(`'lbfgs'`) | Numpy(`'numpy'`),<br>Torch(`'torch'`), | CPU(`'cpu'`)<br>CUDA(`'cuda'`) |
| `'dagma'`[2] | `'linear'`,<br>`'nonlinear'` |`'l2'`, <br> `'logll'`, <br> `'user_loss'`|`'l1'`<br> `'l2'`<br> `'mcp'`<br> `'none'`, `'user_reg'`| `'h_logdet_sq'`<br>`'h_logdet_abs'`<br>`'user_h'` |Adam(`'adam'`) | Numpy(`'numpy'`)<br>Torch(`'torch'`) | CPU(`'cpu'`)<br>CUDA(`'cuda'`) |
| `'topo'`[3] |`'linear'`,<br>`'nonlinear'` |`'l2'`,<br> `'logll'`,<br>`'user_loss'`| `'l1'`<br> `'l2'`<br> `'mcp'`<br> `'none'`<br> `'user_reg'` |`'h_exp_topo'`<br>`'h_logdet_topo'`<br>`'h_poly_topo'`<br>`'user_h'` |Adam(`'adam'`),<br> LBFGS(`'lbfgs'`)| Numpy(`'numpy'`) for linear <br> Torch(`'torch'`) for nonlinear | CPU(`'cpu'`) |
<!-- | __Method(`method`)__ | __Model(`model`)__ |__Loss(`loss_fn`)__ |__Regularizers(`reg`)__|__h(`h_fn`)__ |__Optimizer(`optimizer`)__ | __Computation Library(`compute_lib`)__|__Device(`device`)__|
| -------- | -------- |----|---|-------------------|----| ----------| --------------|
| NOTEARS<br>(`'notears'`)[1] | NonLinear(`'nonlinear'`) |`'l2'`<br> `'logll'`<br> `'user_loss'`|`'l1'`<br> `'l2'`<br> `'mcp'`<br>`'none'`<br> `'user_reg'` |`'h_exp_sq'`<br>`'h_poly_sq'`<br>`'h_poly_abs'`<br>`'user_h'` |Adam(`'adam'`)<br>LBFGS(`lbfgs`) | Torch(`'torch'`) | CPU(`'cpu'`)<br>CUDA(`'cuda'`) |
| DAGAM<br>(`'dagma'`)[2] | NonLinear(`'nonlinear'`) |`'l2'`<br> `'logll'`<br> `'user_loss'`|`'l1'`<br> `'l2'`<br> `'mcp'`<br> `'none'`<br> `'user_reg'`| `'h_logdet_sq'`<br>`'h_logdet_abs'`<br>`'user_h'` |Adam(`'adam'`)<br>LBFGS(`lbfgs`) | Torch(`'torch'`) | CPU(`'cpu'`)<br>CUDA(`'cuda'`) |
| TOPO<br>(`'topo'`)[3] | NonLinear(`'nonlinear'`) |`'l2'`<br> `'logll'`<br> `'user_loss'`| `'l1'`<br> `'l2'`<br> `'mcp'`<br>`'none'`<br> `'user_reg'` |`'h_exp_topo'`<br>`'h_logdet_topo'`<br>`'h_poly_topo'`<br>`'user_h'` |Adam(`'adam'`)<br>LBFGS(`lbfgs`)| Torch(`'torch'`) | CPU(`'cpu'`) | -->
- For the linear (`'linear'`) model, the loss function (`loss_fn`) can be configured as logistic loss (`'logistic'`) for all three methods.
- In the linear (`'linear'`) model, the default optimizer (`'optimizer'`) for TOPO (`'topo'`) is [scikit-learn](https://scikit-learn.org/stable/) (`'sklearn'`), a state-of-the-art package for solving linear model problems.
- In the linear (`'linear'`) model, NOTEARS (`'notears'`) and DAGMA (`'dagma'`) also support computation libraries (`compute_lib`) such as Torch (`'torch'`), and can perform computations on either CPU (`'cpu'`) or GPU (`'cuda'`).
## Requirements
- Python 3.7+
- `numpy`
- `scipy`
- `scikit-learn`
- `python-igraph`
- `tqdm`
- `dagma`
- `notears`: installed from github repo
- `torch`: Used for models with GPU acceleration
<!-- ## Contents and Structure
<span style="color:red">This part need be modified once the final structure is determined.</span>
```plaintext
├── README.md # Project overview and documentation
├── src/ # Source code directory
│ ├── main.py # Main entry point of the application
│ ├── utils.py # Utility functions
│ └── module/ # Additional modules
│ ├── module1.py # Description of module1
│ └── module2.py # Description of module2
├── data/ # Directory for datasets
│ ├── raw/ # Raw, unprocessed data
│ └── processed/ # Processed data ready for analysis
├── tests/ # Test suite
│ ├── test_main.py # Tests for main.py
│ └── test_module1.py # Tests for module1
├── requirements.txt # Python dependencies
└── docs/ # Documentation files
├── index.md # Main documentation file
└── API.md # API documentation -->
## References
[1] Zheng X, Aragam B, Ravikumar P, & Xing EP [DAGs with NO TEARS: Continuous optimization for structure learning][notears] (NeurIPS 2018, Spotlight).
[2] Zheng X, Dan C, Aragam B, Ravikumar P, & Xing EP [Learning sparse nonparametric DAGs][notearsmlp] (AISTATS 2020).
[3] Bello K, Aragam B, Ravikumar P [DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterization][dagma] (NeurIPS 2022).
[4] Deng C, Bello K, Aragam B, Ravikumar P [Optimizing NOTEARS Objectives via Topological Swaps][topo] (ICML 2023).
[5] Deng C, Bello K, Ravikumar P, Aragam B [Likelihood-based differentiable structure learning][logll] (NeurIPS 2024).
[notears]: https://arxiv.org/abs/1803.01422
[notearsmlp]: https://arxiv.org/abs/1909.13189
[dagma]: https://arxiv.org/abs/2209.08037
[topo]: https://arxiv.org/abs/2305.17277
[examples]: https://github.com/Duntrain/DAGlearner/blob/master/examples/examples.ipynb
[logll]: https://arxiv.org/pdf/2410.06163?
Raw data
{
"_id": null,
"home_page": "https://github.com/Duntrain/DAGlearner",
"name": "dagrad",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "dagma, notears, topo, causal discovery, bayesian network, structure learning",
"author": "Chang Deng",
"author_email": "changdeng@uchicago.edu",
"download_url": "https://files.pythonhosted.org/packages/65/2a/78fe1a0ecf1bc223108f1eee8f76e2c075ab358cf7b79a6445b9f9266340/dagrad-0.0.1.tar.gz",
"platform": null,
"description": "# dagrad: Gradient-based structure learning for causal DAGs\n\n## tl;dr\n\nThe `dagrad` Python library is built to provide researchers with an extensible, modular platform for developing and experimenting with continuous score-based structure learning methods, building upon the introduction of the [NOTEARS][notears] framework for differentiable structure learning. It also functions as an updated repository of [state-of-the-art implementations][#features] for various methods.\n\n`dagrad` is a comprehensive suite of state-of-the-art __continuous score-based__ structure learning (aka causal discovery) methods, integrating them into a __unified__ framework. Users have the flexibility to use predefined loss functions or customize their own, along with different acyclicity constraints, regularizers, and optimizers with ease. Additionally, the library supports GPU acceleration, enabling faster computations.\n\n## Introduction \n\nA directed acyclic graphical model (also known as a Bayesian network) with `d` nodes represents a distribution over a random vector of size `d`. The focus of this library is on Bayesian Network Structure Learning (BNSL): Given samples $\\mathbf{X}$ from a distribution, how can we estimate the underlying graph `G`?\n\nThe problem can be formulated as the following differentiable continuous optimization:\n\n```math\n\\begin{array}{cl}\n\\min _{W \\in \\mathbb{R}^{d \\times d}} & Q(W;\\mathbf{X}) \\\\\n\\text { subject to } & h(W) = 0,\n\\end{array}\n```\nThis formulation is versatile enough to encompass both linear and nonlinear models with any smooth objective (e.g. log-likelihood, least-squares, cross-entropy, etc.).\n\n`dagrad` provides a unified framework that allows users to employ either predefined or customized loss functions $Q(W;\\mathbf{X})$, acyclicity constraints $h(W)$, and choose between first-order or second-order optimizers:\n```math\nloss~function + acyclicity~constraint + optimizer => structure~learning\n```\n\nGPU acceleration is also supported.\n\n\n## Run a simple demo\nUse `setup.py` to install the dependencies (we recommend using virtualenv or conda). The simplest way to try out `dagrad` is to run the following example:\n\n```bash\n$ git clone https://github.com/Duntrain/DAGlearner.git\n$ cd DAGlearner/\n$ pip install -e .\n$ cd dagrad\n$ python -m dagrad.daglearner\n```\nThe above runs the original NOTEARS method [[1]][notears] on a randomly generated 10-node Erdos-Renyi graph with 1000 samples. The output should look like the below:\n```\n{'fdr': 0.0, 'tpr': 1.0, 'fpr': 0.0, 'shd': 0, 'nnz': 10}\n```\n\nWant to try more examples? See an example in this [iPython notebook][examples].\n\n## Features\nBelow is an overview of the functionalities provided by the package:\n\n\n| __Method(`method`)__ | __Model(`model`)__ |__Loss(`loss_fn`)__ |__Regularizers(`reg`)__|__h(`h_fn`)__ |__Optimizer(`optimizer`)__ | __Computation Library(`compute_lib`)__ |__Device(`device`)__|\n| -------- | -------- |----|---|-------------------|----| ----------| --------------| \n|`'notears'`[1] | `'linear'`,<br>`'nonlinear'` |`'l2'`, `'logll'`, `'user_loss'`|`'l1'`<br> `'l2'`<br> `'mcp'`<br> `'none'`<br>`'user_reg'` |`'h_exp_sq'`<br>`'h_poly_sq'`<br>`'h_poly_abs'`<br>`'user_h'` |Adam(`'adam'`),<br>LBFGS(`'lbfgs'`) | Numpy(`'numpy'`),<br>Torch(`'torch'`), | CPU(`'cpu'`)<br>CUDA(`'cuda'`) | \n| `'dagma'`[2] | `'linear'`,<br>`'nonlinear'` |`'l2'`, <br> `'logll'`, <br> `'user_loss'`|`'l1'`<br> `'l2'`<br> `'mcp'`<br> `'none'`, `'user_reg'`| `'h_logdet_sq'`<br>`'h_logdet_abs'`<br>`'user_h'` |Adam(`'adam'`) | Numpy(`'numpy'`)<br>Torch(`'torch'`) | CPU(`'cpu'`)<br>CUDA(`'cuda'`) |\n| `'topo'`[3] |`'linear'`,<br>`'nonlinear'` |`'l2'`,<br> `'logll'`,<br>`'user_loss'`| `'l1'`<br> `'l2'`<br> `'mcp'`<br> `'none'`<br> `'user_reg'` |`'h_exp_topo'`<br>`'h_logdet_topo'`<br>`'h_poly_topo'`<br>`'user_h'` |Adam(`'adam'`),<br> LBFGS(`'lbfgs'`)| Numpy(`'numpy'`) for linear <br> Torch(`'torch'`) for nonlinear | CPU(`'cpu'`) | \n\n\n<!-- | __Method(`method`)__ | __Model(`model`)__ |__Loss(`loss_fn`)__ |__Regularizers(`reg`)__|__h(`h_fn`)__ |__Optimizer(`optimizer`)__ | __Computation Library(`compute_lib`)__|__Device(`device`)__|\n| -------- | -------- |----|---|-------------------|----| ----------| --------------| \n| NOTEARS<br>(`'notears'`)[1] | NonLinear(`'nonlinear'`) |`'l2'`<br> `'logll'`<br> `'user_loss'`|`'l1'`<br> `'l2'`<br> `'mcp'`<br>`'none'`<br> `'user_reg'` |`'h_exp_sq'`<br>`'h_poly_sq'`<br>`'h_poly_abs'`<br>`'user_h'` |Adam(`'adam'`)<br>LBFGS(`lbfgs`) | Torch(`'torch'`) | CPU(`'cpu'`)<br>CUDA(`'cuda'`) | \n| DAGAM<br>(`'dagma'`)[2] | NonLinear(`'nonlinear'`) |`'l2'`<br> `'logll'`<br> `'user_loss'`|`'l1'`<br> `'l2'`<br> `'mcp'`<br> `'none'`<br> `'user_reg'`| `'h_logdet_sq'`<br>`'h_logdet_abs'`<br>`'user_h'` |Adam(`'adam'`)<br>LBFGS(`lbfgs`) | Torch(`'torch'`) | CPU(`'cpu'`)<br>CUDA(`'cuda'`) |\n| TOPO<br>(`'topo'`)[3] | NonLinear(`'nonlinear'`) |`'l2'`<br> `'logll'`<br> `'user_loss'`| `'l1'`<br> `'l2'`<br> `'mcp'`<br>`'none'`<br> `'user_reg'` |`'h_exp_topo'`<br>`'h_logdet_topo'`<br>`'h_poly_topo'`<br>`'user_h'` |Adam(`'adam'`)<br>LBFGS(`lbfgs`)| Torch(`'torch'`) | CPU(`'cpu'`) | -->\n\n- For the linear (`'linear'`) model, the loss function (`loss_fn`) can be configured as logistic loss (`'logistic'`) for all three methods.\n- In the linear (`'linear'`) model, the default optimizer (`'optimizer'`) for TOPO (`'topo'`) is [scikit-learn](https://scikit-learn.org/stable/) (`'sklearn'`), a state-of-the-art package for solving linear model problems.\n- In the linear (`'linear'`) model, NOTEARS (`'notears'`) and DAGMA (`'dagma'`) also support computation libraries (`compute_lib`) such as Torch (`'torch'`), and can perform computations on either CPU (`'cpu'`) or GPU (`'cuda'`).\n## Requirements\n- Python 3.7+\n- `numpy`\n- `scipy`\n- `scikit-learn`\n- `python-igraph`\n- `tqdm`\n- `dagma`\n- `notears`: installed from github repo\n- `torch`: Used for models with GPU acceleration\n\n\n<!-- ## Contents and Structure\n<span style=\"color:red\">This part need be modified once the final structure is determined.</span>\n```plaintext\n\u251c\u2500\u2500 README.md # Project overview and documentation\n\u251c\u2500\u2500 src/ # Source code directory\n\u2502 \u251c\u2500\u2500 main.py # Main entry point of the application\n\u2502 \u251c\u2500\u2500 utils.py # Utility functions\n\u2502 \u2514\u2500\u2500 module/ # Additional modules\n\u2502 \u251c\u2500\u2500 module1.py # Description of module1\n\u2502 \u2514\u2500\u2500 module2.py # Description of module2\n\u251c\u2500\u2500 data/ # Directory for datasets\n\u2502 \u251c\u2500\u2500 raw/ # Raw, unprocessed data\n\u2502 \u2514\u2500\u2500 processed/ # Processed data ready for analysis\n\u251c\u2500\u2500 tests/ # Test suite\n\u2502 \u251c\u2500\u2500 test_main.py # Tests for main.py\n\u2502 \u2514\u2500\u2500 test_module1.py # Tests for module1\n\u251c\u2500\u2500 requirements.txt # Python dependencies\n\u2514\u2500\u2500 docs/ # Documentation files\n \u251c\u2500\u2500 index.md # Main documentation file\n \u2514\u2500\u2500 API.md # API documentation -->\n\n\n## References\n\n[1] Zheng X, Aragam B, Ravikumar P, & Xing EP [DAGs with NO TEARS: Continuous optimization for structure learning][notears] (NeurIPS 2018, Spotlight).\n\n[2] Zheng X, Dan C, Aragam B, Ravikumar P, & Xing EP [Learning sparse nonparametric DAGs][notearsmlp] (AISTATS 2020).\n\n[3] Bello K, Aragam B, Ravikumar P [DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterization][dagma] (NeurIPS 2022). \n\n[4] Deng C, Bello K, Aragam B, Ravikumar P [Optimizing NOTEARS Objectives via Topological Swaps][topo] (ICML 2023).\n\n[5] Deng C, Bello K, Ravikumar P, Aragam B [Likelihood-based differentiable structure learning][logll] (NeurIPS 2024).\n\n[notears]: https://arxiv.org/abs/1803.01422\n[notearsmlp]: https://arxiv.org/abs/1909.13189\n[dagma]: https://arxiv.org/abs/2209.08037\n[topo]: https://arxiv.org/abs/2305.17277\n[examples]: https://github.com/Duntrain/DAGlearner/blob/master/examples/examples.ipynb\n[logll]: https://arxiv.org/pdf/2410.06163?\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A project for DAG learning",
"version": "0.0.1",
"project_urls": {
"Documentation": "https://readthedocs.org",
"Homepage": "https://github.com/Duntrain/DAGlearner",
"Issues": "https://github.com/Duntrain/DAGlearner/issues",
"Repository": "https://github.com/Duntrain/DAGlearner"
},
"split_keywords": [
"dagma",
" notears",
" topo",
" causal discovery",
" bayesian network",
" structure learning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6a7d362e3ea172b5b63936dd7566c88a7e6fe4ff77c64556b453daa9b192f7cc",
"md5": "921d8878110d2fc541595e17747d2202",
"sha256": "e3108330628b0638c8fe78979708bc06591f6461d94dd6236b37133ee35b35d9"
},
"downloads": -1,
"filename": "dagrad-0.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "921d8878110d2fc541595e17747d2202",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 66502,
"upload_time": "2024-10-22T23:23:31",
"upload_time_iso_8601": "2024-10-22T23:23:31.451672Z",
"url": "https://files.pythonhosted.org/packages/6a/7d/362e3ea172b5b63936dd7566c88a7e6fe4ff77c64556b453daa9b192f7cc/dagrad-0.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "652a78fe1a0ecf1bc223108f1eee8f76e2c075ab358cf7b79a6445b9f9266340",
"md5": "b59589593f19bfcb56af3a70c1d10591",
"sha256": "68e3b3ea5629a1a574e869a1183241151ae1dfbce49ffdb115b41db85afc12d6"
},
"downloads": -1,
"filename": "dagrad-0.0.1.tar.gz",
"has_sig": false,
"md5_digest": "b59589593f19bfcb56af3a70c1d10591",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 56617,
"upload_time": "2024-10-22T23:23:32",
"upload_time_iso_8601": "2024-10-22T23:23:32.686712Z",
"url": "https://files.pythonhosted.org/packages/65/2a/78fe1a0ecf1bc223108f1eee8f76e2c075ab358cf7b79a6445b9f9266340/dagrad-0.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-22 23:23:32",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Duntrain",
"github_project": "DAGlearner",
"github_not_found": true,
"lcname": "dagrad"
}
JSON
