astroblast


Nameastroblast JSON
Version 2.0.1 PyPI version JSON
download
home_pageNone
SummarySynchtrotron peak estimator for blazars
upload_time2024-07-08 10:53:52
maintainerNone
docs_urlNone
authorNone
requires_python>=3.8
licenseMIT
keywords blazar synchrotron
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # BlaST

BlaST (**Bla**zar **S**ynchrotron **T**ool) is a tool to estimate the
synchrotron peak of blazars given its spectral energy distribution. The package is available on pip
and can simply be installed via

```
pip install astroblast
```

Running the tool is as simple as typing

```
blast sed.txt
```

where `sed.txt` is a output file generated by the *VOUBlazar* tool and might look something like the following:

```
   1  matched source  227.16230  -49.88400  99
 Frequency     nufnu     nufnu unc.  nufnu unc. start time   end time   Catalog     Reference
    Hz       erg/cm2/s     upper       lower        MJD         MJD   
---------------------------------------------------------------------------------------------------------------------------
 2.418E+17   2.185E-13   3.139E-13   1.230E-13  55000.0000  55000.0000  RASS        Boller et al. 2016, A&A, 103, 1                                                                                                                                                                         
 2.418E+17   5.085E-13   6.281E-13   3.889E-13  58150.0000  58150.1016  OUSXB       Giommi et al. 2019, Accepted for publication in A&A  
```

This gives the following output:

```
12.42 (+/- 0.71)
```

The first value is the synchrotron peak and the second the 95% prediction interval both in `log10`. It is possible to specify a different prediction interval width by the `-w` argument, followed by the width in sigmas:

```
blast sed.txt -w 1.0
```

BlaST also supports bulk estimation, e.g. estimating a whole catalogue, by providing a directory or zip file containing
the seds as well as an output file in which the results will be written to as csv:

```
blast seds.zip -o estimates.csv
```

With V2 you now also have the option to estimate the synchrotron peak flux (`erg/cm2/s`), again in `log10`.
You enable this with the `-f` flag:

```
-12.39 (+/- 0.63) @ 12.39 (+/- 0.39)
```

## Changes from v1 to v2

With v2 there has been some major changes. Some are listed in the following:

- **New data set** (courtesy of N. Krieger)  
The data set has been refined resulting in a new set independent of the previous
one. This one now also includes labels for the flux.
- **Flux estimate**  
Biggest difference is the estimation of the synchrotron peak's flux by an
independent ensemble of neural networks.
- **Save models in ONNX format**  
Previously, the tool needed pytorch installed, which is quite large. We moved to
ONNX runtime to make the installation slimmer.

## How it works

BlaST consists of an ensemble of similar neural networks power by [pytorch](https://pytorch.org/) based on the method presented in 
[Lakshminarayanan et. al. 2016][2]. The data set (see `SEDs.zip`) is based on the three catalogues [4LAC][3], [3HSP][4]
and [5BZCat][5] and divided into 5 bags. For each bag an ensemble of 5 models are trained totalling 25 independent models.

The bagging allows the reapply the ensemble on the training data as an out-of-bag estimation, i.e. excluding the
ensemble members that were trained on a training set without that specific bag. The results are shown in `estimates.csv`.

For specific details, e.g. how the data was enhanced and how the bias in the data set was handles, take a look in
`BlaST.ipynb` which is the notebook used to train the models. Note that if you want to run that notebook you'll also
need to install [PyTorch Lightning](https://www.pytorchlightning.ai/)

## Performance

The performance was evaluated using out-of-bag estimates on the training data
including a 95% prediction interval. Metrics are shown as median and 25%/75%
quantile.
Since the model for predicting the peak frequency and peak flux are independent,
metrics are shown for both.

| Metric   | Value  | Freq  | Flux  |
|----------|--------|-------|-------|
| Abs Err  | Median | 0.163 | 0.104 |
|          | 25%    | 0.075 | 0.049 |
|          | 75%    | 0.314 | 0.186 |
| PI Width | Median | 1.267 | 1.087 |
|          | 25%    | 1.009 | 0.971 |
|          | 75%    | 1.625 | 1.198 |
| Interval | Median | 1.400 | 1.088 |
| Score    | 25%    | 1.010 | 0.972 |
|          | 75%    | 1.636 | 1.199 |

### Peak Frequency Estimate

The following show the prediction histogram with the median as black line, as well as the 90% and 10% quantile shown as
dotted lines

![Prediction Histogram: Peak Frequency](img/hist_freq.png)

The next shows the prediction interval (95%) widths. The percentages above and below show the respective amount of samples
outside their intervals.

![Prediction Intervals: Peak Frequency](img/pi_freq.png)

The final one shows the prediction error and the prediction interval distribution using a moving window and smoothed using cubic splines.

![Error Distribution: Peak Frequency](img/dist_freq.png)

### Peak Flux Estimate

Same plots, but now for the peak flux estimate.

![Prediction Histogram: Peak Flux](img/hist_flux.png)
![Prediction Intervals: Peak Flux](img/pi_flux.png)
![Error Distribution: Peak Flux](img/dist_flux.png)

## Use BlaST in code

BlaST can also be imported as python package:

```python
from blast import *

sed, pos = parse_sed('sed.txt', position=True) #reads the file
bag = get_bag(pos) #Returns bag if sed was part of training
bins = bin_data(sed)

# estimate peak frequency
estimator = PeakFrequencyEstimator()
peak, err = estimator(bins, bag)
# estimate peak flux
estimator = PeakFluxEstimator()
peak, err = estimator(bins, bag)
```

## References

- [*Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles*, Lakshminarayanan et. al., 2016][2]
- [*The Fourth Catalog of Active Galactic Nuclei Detected by the Fermi Large Area Telescope*, The Fermit-LAT collaboration, 2019][3]
- [*The 3HSP catalogue of Extreme & High Synchrotron Peaked Blazars*, Chang et. al., 2019][4]
- [*The 5th edition of the Roma-BZCAT. A short presentation*, Massaro et. al., 2015][5]

[2]: https://arxiv.org/abs/1612.01474 "Lakshminarayanan et. al., 2016"
[3]: https://arxiv.org/abs/1905.10771 "The Fermit-LAT collaboration, 2019"
[4]: https://arxiv.org/abs/1909.08279 "Chang et. al., 2019"
[5]: https://www.ssdc.asi.it/bzcat/ "Massaro et. al. 2015"

            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "astroblast",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.8",
    "maintainer_email": null,
    "keywords": "blazar, synchrotron",
    "author": null,
    "author_email": "Tobias Kerscher <88444139+tkerscher@users.noreply.github.com>",
    "download_url": "https://files.pythonhosted.org/packages/33/6c/7904b020017f912ab0783fa3c32be78de25acee49896fb3f13aa7a209cc4/astroblast-2.0.1.tar.gz",
    "platform": null,
    "description": "# BlaST\r\n\r\nBlaST (**Bla**zar **S**ynchrotron **T**ool) is a tool to estimate the\r\nsynchrotron peak of blazars given its spectral energy distribution. The package is available on pip\r\nand can simply be installed via\r\n\r\n```\r\npip install astroblast\r\n```\r\n\r\nRunning the tool is as simple as typing\r\n\r\n```\r\nblast sed.txt\r\n```\r\n\r\nwhere `sed.txt` is a output file generated by the *VOUBlazar* tool and might look something like the following:\r\n\r\n```\r\n   1  matched source  227.16230  -49.88400  99\r\n Frequency     nufnu     nufnu unc.  nufnu unc. start time   end time   Catalog     Reference\r\n    Hz       erg/cm2/s     upper       lower        MJD         MJD   \r\n---------------------------------------------------------------------------------------------------------------------------\r\n 2.418E+17   2.185E-13   3.139E-13   1.230E-13  55000.0000  55000.0000  RASS        Boller et al. 2016, A&A, 103, 1                                                                                                                                                                         \r\n 2.418E+17   5.085E-13   6.281E-13   3.889E-13  58150.0000  58150.1016  OUSXB       Giommi et al. 2019, Accepted for publication in A&A  \r\n```\r\n\r\nThis gives the following output:\r\n\r\n```\r\n12.42 (+/- 0.71)\r\n```\r\n\r\nThe first value is the synchrotron peak and the second the 95% prediction interval both in `log10`. It is possible to specify a different prediction interval width by the `-w` argument, followed by the width in sigmas:\r\n\r\n```\r\nblast sed.txt -w 1.0\r\n```\r\n\r\nBlaST also supports bulk estimation, e.g. estimating a whole catalogue, by providing a directory or zip file containing\r\nthe seds as well as an output file in which the results will be written to as csv:\r\n\r\n```\r\nblast seds.zip -o estimates.csv\r\n```\r\n\r\nWith V2 you now also have the option to estimate the synchrotron peak flux (`erg/cm2/s`), again in `log10`.\r\nYou enable this with the `-f` flag:\r\n\r\n```\r\n-12.39 (+/- 0.63) @ 12.39 (+/- 0.39)\r\n```\r\n\r\n## Changes from v1 to v2\r\n\r\nWith v2 there has been some major changes. Some are listed in the following:\r\n\r\n- **New data set** (courtesy of N. Krieger)  \r\nThe data set has been refined resulting in a new set independent of the previous\r\none. This one now also includes labels for the flux.\r\n- **Flux estimate**  \r\nBiggest difference is the estimation of the synchrotron peak's flux by an\r\nindependent ensemble of neural networks.\r\n- **Save models in ONNX format**  \r\nPreviously, the tool needed pytorch installed, which is quite large. We moved to\r\nONNX runtime to make the installation slimmer.\r\n\r\n## How it works\r\n\r\nBlaST consists of an ensemble of similar neural networks power by [pytorch](https://pytorch.org/) based on the method presented in \r\n[Lakshminarayanan et. al. 2016][2]. The data set (see `SEDs.zip`) is based on the three catalogues [4LAC][3], [3HSP][4]\r\nand [5BZCat][5] and divided into 5 bags. For each bag an ensemble of 5 models are trained totalling 25 independent models.\r\n\r\nThe bagging allows the reapply the ensemble on the training data as an out-of-bag estimation, i.e. excluding the\r\nensemble members that were trained on a training set without that specific bag. The results are shown in `estimates.csv`.\r\n\r\nFor specific details, e.g. how the data was enhanced and how the bias in the data set was handles, take a look in\r\n`BlaST.ipynb` which is the notebook used to train the models. Note that if you want to run that notebook you'll also\r\nneed to install [PyTorch Lightning](https://www.pytorchlightning.ai/)\r\n\r\n## Performance\r\n\r\nThe performance was evaluated using out-of-bag estimates on the training data\r\nincluding a 95% prediction interval. Metrics are shown as median and 25%/75%\r\nquantile.\r\nSince the model for predicting the peak frequency and peak flux are independent,\r\nmetrics are shown for both.\r\n\r\n| Metric   | Value  | Freq  | Flux  |\r\n|----------|--------|-------|-------|\r\n| Abs Err  | Median | 0.163 | 0.104 |\r\n|          | 25%    | 0.075 | 0.049 |\r\n|          | 75%    | 0.314 | 0.186 |\r\n| PI Width | Median | 1.267 | 1.087 |\r\n|          | 25%    | 1.009 | 0.971 |\r\n|          | 75%    | 1.625 | 1.198 |\r\n| Interval | Median | 1.400 | 1.088 |\r\n| Score    | 25%    | 1.010 | 0.972 |\r\n|          | 75%    | 1.636 | 1.199 |\r\n\r\n### Peak Frequency Estimate\r\n\r\nThe following show the prediction histogram with the median as black line, as well as the 90% and 10% quantile shown as\r\ndotted lines\r\n\r\n![Prediction Histogram: Peak Frequency](img/hist_freq.png)\r\n\r\nThe next shows the prediction interval (95%) widths. The percentages above and below show the respective amount of samples\r\noutside their intervals.\r\n\r\n![Prediction Intervals: Peak Frequency](img/pi_freq.png)\r\n\r\nThe final one shows the prediction error and the prediction interval distribution using a moving window and smoothed using cubic splines.\r\n\r\n![Error Distribution: Peak Frequency](img/dist_freq.png)\r\n\r\n### Peak Flux Estimate\r\n\r\nSame plots, but now for the peak flux estimate.\r\n\r\n![Prediction Histogram: Peak Flux](img/hist_flux.png)\r\n![Prediction Intervals: Peak Flux](img/pi_flux.png)\r\n![Error Distribution: Peak Flux](img/dist_flux.png)\r\n\r\n## Use BlaST in code\r\n\r\nBlaST can also be imported as python package:\r\n\r\n```python\r\nfrom blast import *\r\n\r\nsed, pos = parse_sed('sed.txt', position=True) #reads the file\r\nbag = get_bag(pos) #Returns bag if sed was part of training\r\nbins = bin_data(sed)\r\n\r\n# estimate peak frequency\r\nestimator = PeakFrequencyEstimator()\r\npeak, err = estimator(bins, bag)\r\n# estimate peak flux\r\nestimator = PeakFluxEstimator()\r\npeak, err = estimator(bins, bag)\r\n```\r\n\r\n## References\r\n\r\n- [*Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles*, Lakshminarayanan et. al., 2016][2]\r\n- [*The Fourth Catalog of Active Galactic Nuclei Detected by the Fermi Large Area Telescope*, The Fermit-LAT collaboration, 2019][3]\r\n- [*The 3HSP catalogue of Extreme & High Synchrotron Peaked Blazars*, Chang et. al., 2019][4]\r\n- [*The 5th edition of the Roma-BZCAT. A short presentation*, Massaro et. al., 2015][5]\r\n\r\n[2]: https://arxiv.org/abs/1612.01474 \"Lakshminarayanan et. al., 2016\"\r\n[3]: https://arxiv.org/abs/1905.10771 \"The Fermit-LAT collaboration, 2019\"\r\n[4]: https://arxiv.org/abs/1909.08279 \"Chang et. al., 2019\"\r\n[5]: https://www.ssdc.asi.it/bzcat/ \"Massaro et. al. 2015\"\r\n",
    "bugtrack_url": null,
    "license": "MIT",
    "summary": "Synchtrotron peak estimator for blazars",
    "version": "2.0.1",
    "project_urls": {
        "Homepage": "https://github.com/tkerscher/blast"
    },
    "split_keywords": [
        "blazar",
        " synchrotron"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "ee4288a5122b10fff2c5238c8fa0843661c12000c7d71b1853c2c11bc5d5e8e4",
                "md5": "d7e6636a965d364dd758bcfa90541038",
                "sha256": "e11645f0ae6613f6fbc5c510eb961aabcdc41fde26e84c5edd83e1a4aebcf587"
            },
            "downloads": -1,
            "filename": "astroblast-2.0.1-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "d7e6636a965d364dd758bcfa90541038",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.8",
            "size": 10975113,
            "upload_time": "2024-07-08T10:53:46",
            "upload_time_iso_8601": "2024-07-08T10:53:46.738278Z",
            "url": "https://files.pythonhosted.org/packages/ee/42/88a5122b10fff2c5238c8fa0843661c12000c7d71b1853c2c11bc5d5e8e4/astroblast-2.0.1-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "336c7904b020017f912ab0783fa3c32be78de25acee49896fb3f13aa7a209cc4",
                "md5": "6ee17842da879ad963e059ec66e0ed57",
                "sha256": "7a35713c8c24ab2ade980bb502e45d032f3751d9dd2c04c3674a012475a3f509"
            },
            "downloads": -1,
            "filename": "astroblast-2.0.1.tar.gz",
            "has_sig": false,
            "md5_digest": "6ee17842da879ad963e059ec66e0ed57",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.8",
            "size": 10974362,
            "upload_time": "2024-07-08T10:53:52",
            "upload_time_iso_8601": "2024-07-08T10:53:52.733468Z",
            "url": "https://files.pythonhosted.org/packages/33/6c/7904b020017f912ab0783fa3c32be78de25acee49896fb3f13aa7a209cc4/astroblast-2.0.1.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-07-08 10:53:52",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "tkerscher",
    "github_project": "blast",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "lcname": "astroblast"
}
        
Elapsed time: 4.82382s