<p align="center">
<img width = "800" src="https://github.com/jeffjennings/arksia/blob/main/logo.png?raw=true"/>
</p>
<p align="center">
A pipeline for image analysis of the ALMA large program ARKS ('ALMA survey to Resolve exoKuiper belt Substructures') datasets
</p>
<p align="center">
<!-- current release -->
<a href="https://github.com/jeffjennings/arksia/releases">
<img src="https://img.shields.io/github/release/jeffjennings/arksia/all.svg">
</a>
<!-- current version on pypi -->
<a href="https://pypi.python.org/pypi/arksia">
<img src="https://img.shields.io/pypi/v/arksia.svg">
<!-- license -->
<a href="https://www.gnu.org/licenses/lgpl-3.0">
<img src="https://img.shields.io/badge/license-LGPL%20v3-blue.svg">
</a>
<br/>
<!-- tests -->
<a href="https://github.com/jeffjennings/arksia/actions/workflows/tests.yml">
<img src="https://github.com/jeffjennings/arksia/actions/workflows/tests.yml/badge.svg">
</a>
<!-- coverage -->
<a href="https://github.com/jeffjennings/arksia/blob/coverage/coverage.md">
<img src="https://github.com/jeffjennings/arksia/blob/coverage/coverage.svg">
</a>
</p>
This package is intended for members of the ARKS collaboration to conduct imaging and analysis on the large program data, and ultimately for anyone to reproduce results of this aspect of the collaboration's published results. It is designed as a pipeline to be run from the command line for rapid reproducibility or simple variations on the published analysis.
Install
-------
```pip install arksia```
Pipeline scope
--------------
The pipeline is run from the terminal using input parameter files. It has the following, modular capabilities:
- extracts a radial brightness profile from a clean image
- processes an existing rave fit to obtain a brightness profile and 1d, 2d residuals in consistent units
- runs frank to obtain a brightness profile, obtain 1d residuals, image the 2d residuals (using MPoL)
- runs frank for 1+1D vertical (disk aspect ratio) inference
- runs parametric fits to a nonparametric radial brightness profile using a range of physically motivated parametric forms (uses JAX for hardware acceleration)
- produces plots to compare clean, rave, frank brightness profiles, radial visibility profiles, images, residuals
- produces plots to compare frank fits with different hyperparameter values
- produces plots to compare nonparametric and parametric brightness profiles
- produces plots to assess frank vertical inference over grids of _h_, _alpha_, _wsmooth_
The pipeline runs from general and source-specific parameter files. It can be run in bulk (across multiple sources).
Staging input files for the pipeline
------------------------------------
1) Ensure you have the following parameter files (you may name them differently and pass them in when running the pipeline):
* 'pars_gen.json' (contains parameters to choose which of the pipeline modules run, as well as sensible choices for the pipeline parameters applicable to all sources)
* 'pars_source.json' (contains sensible choices for source-specific best-fit parameters, as well as metrics of supplied clean images)
* 'summary_disc_parameters.csv' (used to read disk geometry and stellar flux)
2) In a single directory, depending on which modular components of the pipeline you will run, store the following input files (and set this directory as the `input_dir` in `pars_gen.json`):
* if using the pipeline to extract a radial profile from a clean image and/or to compare clean results to frank and rave:
- primary beam-corrected CLEAN image ('<>.pbcor.fits'), primary beam image ('<>.pb.fits'), CLEAN model image ('<>.model.fits') for each robust value
* if using the pipeline to run frank fits:
- visibility datasets from the ARKS survey ('<>corrected.txt')
* if using the pipeline to compare rave results to clean and frank:
- rave fit array files ('<>.npy') for each robust value
Running the pipeline for a single source
----------------------------------------
The main pipeline file is `pipeline.py`. By default the pipeline runs using the parameter files `./pars_gen.json` and `./pars_source.json`, and the table `./summary_disc_parameters.csv`. For a description of the parameters, see `description_pars_gen.json` and `description_pars_source.json`, or from a terminal run `python -c 'import arksia.pipeline; arksia.pipeline.helper()'`.
The pipeline can be run from the terminal for fits/analysis of a single source with `python -m arksia.pipeline -d '<disk name>'`, where the disk name, e.g. `'HD76582'`, must match a source name in the source-specific .json parameter file (by default `./pars_source.json`).
### Setting up frank fits ###
- When performing a new `frank` fit, it's recommended to set `method` to `"LogNormal"` to perform fits in logarithmic brightness space. This is enforced in some parts of the pipeline because the logarithmic fits are in general a better choice. The exception is that when running a frank 1+1D fit to find _h_, `method` must be `"Normal"` (it will be enforced).
### Setting up parametric fits ###
- To install the needed dependencies for parametric fits, `pip install arksia[analysis]`
* If when running a parametric fit you receive the warning `WARNING:jax._src.xla_bridge:CUDA backend failed to initialize`, update CUDA with `pip install -U "jax[cuda12_pip]" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html`
- To run a parametric fit to a nonparametric `frank` profile, set the `form` parameter in `pars_gen.json` as a list of parametric functional forms among those in `description_pars_gen.json`. Functional forms in `form` will be fit sequentially.
* If during a fit the loss function (shown to the left of the progress bar) is still varying at the end of the optimization, increase `niter` or change `learn_rate` in `pars_gen.json`.
Running the pipeline for multiple/all sources
---------------------------------------------
The pipeline can be looped over multiple sources via `python bulk_pipeline_run.py`.
Obtaining key results for multiple/all sources
----------------------------------------------
Survey-wide summary results are a `.txt` file per source with all radial brightness profiles (clean, rave, frank) sampled at the same radii, and figures with a panel for each source showing the clean, rave, frank brightness profiles (one figure without uncertainties, one with). These are generated via `python bulk_pipeline_results.py`.
Raw data
{
"_id": null,
"home_page": "https://github.com/jeffjennings/arksia",
"name": "arksia",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "",
"keywords": "science,astronomy,interferometry",
"author": "Jeff Jennings, Sebastian Marino",
"author_email": "jjennings1519@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/c8/13/cd6c97dd4e211b9b8b034fa90e9272692ec44c97cffa339827c89729cf41/arksia-0.2.1.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n <img width = \"800\" src=\"https://github.com/jeffjennings/arksia/blob/main/logo.png?raw=true\"/>\n </p>\n\n<p align=\"center\">\n A pipeline for image analysis of the ALMA large program ARKS ('ALMA survey to Resolve exoKuiper belt Substructures') datasets\n</p>\n\n<p align=\"center\">\n <!-- current release -->\n <a href=\"https://github.com/jeffjennings/arksia/releases\">\n <img src=\"https://img.shields.io/github/release/jeffjennings/arksia/all.svg\">\n </a>\n\n <!-- current version on pypi -->\n <a href=\"https://pypi.python.org/pypi/arksia\">\n <img src=\"https://img.shields.io/pypi/v/arksia.svg\">\n\n <!-- license -->\n <a href=\"https://www.gnu.org/licenses/lgpl-3.0\">\n <img src=\"https://img.shields.io/badge/license-LGPL%20v3-blue.svg\"> \n </a> \n\n <br/>\n <!-- tests -->\n <a href=\"https://github.com/jeffjennings/arksia/actions/workflows/tests.yml\">\n <img src=\"https://github.com/jeffjennings/arksia/actions/workflows/tests.yml/badge.svg\">\n </a> \n\n <!-- coverage -->\n <a href=\"https://github.com/jeffjennings/arksia/blob/coverage/coverage.md\">\n <img src=\"https://github.com/jeffjennings/arksia/blob/coverage/coverage.svg\">\n </a>\n\n</p>\n\nThis package is intended for members of the ARKS collaboration to conduct imaging and analysis on the large program data, and ultimately for anyone to reproduce results of this aspect of the collaboration's published results. It is designed as a pipeline to be run from the command line for rapid reproducibility or simple variations on the published analysis. \n\nInstall\n-------\n```pip install arksia```\n\nPipeline scope\n--------------\nThe pipeline is run from the terminal using input parameter files. It has the following, modular capabilities:\n- extracts a radial brightness profile from a clean image\n- processes an existing rave fit to obtain a brightness profile and 1d, 2d residuals in consistent units\n- runs frank to obtain a brightness profile, obtain 1d residuals, image the 2d residuals (using MPoL)\n- runs frank for 1+1D vertical (disk aspect ratio) inference\n- runs parametric fits to a nonparametric radial brightness profile using a range of physically motivated parametric forms (uses JAX for hardware acceleration)\n- produces plots to compare clean, rave, frank brightness profiles, radial visibility profiles, images, residuals\n- produces plots to compare frank fits with different hyperparameter values\n- produces plots to compare nonparametric and parametric brightness profiles\n- produces plots to assess frank vertical inference over grids of _h_, _alpha_, _wsmooth_\n\nThe pipeline runs from general and source-specific parameter files. It can be run in bulk (across multiple sources).\n\nStaging input files for the pipeline\n------------------------------------\n1) Ensure you have the following parameter files (you may name them differently and pass them in when running the pipeline):\n * 'pars_gen.json' (contains parameters to choose which of the pipeline modules run, as well as sensible choices for the pipeline parameters applicable to all sources)\n * 'pars_source.json' (contains sensible choices for source-specific best-fit parameters, as well as metrics of supplied clean images)\n * 'summary_disc_parameters.csv' (used to read disk geometry and stellar flux)\n \n2) In a single directory, depending on which modular components of the pipeline you will run, store the following input files (and set this directory as the `input_dir` in `pars_gen.json`):\n * if using the pipeline to extract a radial profile from a clean image and/or to compare clean results to frank and rave:\n - primary beam-corrected CLEAN image ('<>.pbcor.fits'), primary beam image ('<>.pb.fits'), CLEAN model image ('<>.model.fits') for each robust value \n * if using the pipeline to run frank fits:\n - visibility datasets from the ARKS survey ('<>corrected.txt')\n * if using the pipeline to compare rave results to clean and frank:\n - rave fit array files ('<>.npy') for each robust value\n\nRunning the pipeline for a single source\n----------------------------------------\nThe main pipeline file is `pipeline.py`. By default the pipeline runs using the parameter files `./pars_gen.json` and `./pars_source.json`, and the table `./summary_disc_parameters.csv`. For a description of the parameters, see `description_pars_gen.json` and `description_pars_source.json`, or from a terminal run `python -c 'import arksia.pipeline; arksia.pipeline.helper()'`.\n\nThe pipeline can be run from the terminal for fits/analysis of a single source with `python -m arksia.pipeline -d '<disk name>'`, where the disk name, e.g. `'HD76582'`, must match a source name in the source-specific .json parameter file (by default `./pars_source.json`).\n\n### Setting up frank fits ###\n- When performing a new `frank` fit, it's recommended to set `method` to `\"LogNormal\"` to perform fits in logarithmic brightness space. This is enforced in some parts of the pipeline because the logarithmic fits are in general a better choice. The exception is that when running a frank 1+1D fit to find _h_, `method` must be `\"Normal\"` (it will be enforced).\n\n### Setting up parametric fits ###\n- To install the needed dependencies for parametric fits, `pip install arksia[analysis]`\n * If when running a parametric fit you receive the warning `WARNING:jax._src.xla_bridge:CUDA backend failed to initialize`, update CUDA with `pip install -U \"jax[cuda12_pip]\" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html`\n \n- To run a parametric fit to a nonparametric `frank` profile, set the `form` parameter in `pars_gen.json` as a list of parametric functional forms among those in `description_pars_gen.json`. Functional forms in `form` will be fit sequentially.\n * If during a fit the loss function (shown to the left of the progress bar) is still varying at the end of the optimization, increase `niter` or change `learn_rate` in `pars_gen.json`.\n\nRunning the pipeline for multiple/all sources\n---------------------------------------------\nThe pipeline can be looped over multiple sources via `python bulk_pipeline_run.py`. \n\nObtaining key results for multiple/all sources\n----------------------------------------------\nSurvey-wide summary results are a `.txt` file per source with all radial brightness profiles (clean, rave, frank) sampled at the same radii, and figures with a panel for each source showing the clean, rave, frank brightness profiles (one figure without uncertainties, one with). These are generated via `python bulk_pipeline_results.py`.\n",
"bugtrack_url": null,
"license": "LGPLv3",
"summary": "arksia, part of the ALMA large program ARKS",
"version": "0.2.1",
"project_urls": {
"Bug Tracker": "https://github.com/jeffjennings/arksia/issues",
"Homepage": "https://github.com/jeffjennings/arksia"
},
"split_keywords": [
"science",
"astronomy",
"interferometry"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c813cd6c97dd4e211b9b8b034fa90e9272692ec44c97cffa339827c89729cf41",
"md5": "a94b868fe06f8e715c8f0dfa9917875f",
"sha256": "cbaa9aebb8d131a316b2d823347b6086686d0f216574e73c8c714e49b6ed5298"
},
"downloads": -1,
"filename": "arksia-0.2.1.tar.gz",
"has_sig": false,
"md5_digest": "a94b868fe06f8e715c8f0dfa9917875f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 38525,
"upload_time": "2024-01-10T22:02:05",
"upload_time_iso_8601": "2024-01-10T22:02:05.321622Z",
"url": "https://files.pythonhosted.org/packages/c8/13/cd6c97dd4e211b9b8b034fa90e9272692ec44c97cffa339827c89729cf41/arksia-0.2.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-10 22:02:05",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "jeffjennings",
"github_project": "arksia",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "arksia"
}
JSON
