========
FetchSEP
========
.. image:: https://img.shields.io/pypi/v/fetchsep.svg
:target: https://pypi.python.org/pypi/fetchsep
.. image:: https://img.shields.io/travis/rickyegeland/fetchsep.svg
:target: https://travis-ci.com/rickyegeland/fetchsep
.. image:: https://readthedocs.org/projects/fetchsep/badge/?version=latest
:target: https://fetchsep.readthedocs.io/en/latest/?version=latest
:alt: Documentation Status
Identify SEP elevations above background in a time series (`idsep`) and analyze events individually (`opsep`)
* Free software: MIT license
* Documentation: https://fetchsep.readthedocs.io.
Set Up
======
By default, `idsep` and `opsep` will create the necessary output
directories in the current working directory where the command is
executed. The directories that will be created are `data`, `ouptut`,
`plots`, and `lists`. The directories will not be overwritten if they
already exist. If you would like to choose another output location,
generate a config file (`fetchsep.cfg`) with `opsep --ExportConfig`
and edit the paths described there.
A configuration file may also be placed in your home directory with
the name `.fetchsep`. The configuration file does not need to be
complete; you may specify only the values for which you wish to
override the default. The order of config value precidence
is 1. current working directory `fetchsep.cfg`, 2. home directory
`.fetchsep`, 3. fetchsep defaults.
Run
===
To run OpSEP to process individual SEP events:
| opsep --StartDate 2012-05-16 --EndDate 2012-05-22 --Experiment GOES-13 --FluxType integral --showplot
IDSEP
=====
The `idsep` code will read in a long time series and automatically identify increases above background. This is done by estimating a mean background level plus an expected level of variation (sigma). All flux less than mean + 3sigma are considered background while all points above mean + 3sigma are identified as increases.
By assigning a set of criteria, increases that are most likely due to SEP events are identified and an SEP event list is output for each energy channel.
The code also outputs a file containing every single high flux point above the mean + 3sigma threshold.
Note that in fetchsep/utils/config.py, number of sigma, the initial window used to estimate background levels, and the final sliding window used to estimate background levels can be adjusted.
| idsep_nsigma = 3
| init_win = 150 #days to average initial estimate of threshold
| sliding_win = 27 #days in sliding window to calculate final threshold
| percent_points = 0.9 #Percent of points that must be in the sliding
| #window to calculate the background; otherwise use
| #previous good value
OPSEP
=====
The `opsep` code was previously supported at https://github.com/ktindiana/operational-sep and is now transitioned to this package going forward. Please see the operational-sep repository for extensive documentation until the documentation in this repository can be updated.
`opsep` is intended to assess each individual SEP event at a time, extracting information such as start and end times, peak fluxes, and event fluence.
The code will output various csv files and a json file with accompany txt files. The JSON file is in the same format as required by the SEP Scoreboard to submit forecasts.
`opsep` creates files from observations that can be directly compared to SEP model forecasts sent to the SEP Scoreboard.
For time profile SEP models, `opsep` may be used to create the JSON files that can be submitted to the SEP Scoreboard.
Running `opsep` for your own time series
----------------------------------------
Users may input their own time series into `opsep` by specifying some
information in the config file:
| ##### DELIMETER between columns of file with time series
| user_delim = " " #any string
| ##### COLUMNS containing the fluxes you want to analyze
| user_col = arr.array('i',[1,2,3,4,5,6,7,8])
| err_col = arr.array('i',[]) #set to [] if no uncertainties. err_col only used by idsep
| ##### ENERGY BINS associated with user file and columns
| #For differential bins, use the format:
| user_energy_bins = [[Elow1,Ehigh1],[Elow2,Ehigh2],etc]
| #For integral bins, use the format:
| user_energy_bins = [[Elow1,-1],[Elow2,-1],[Elow3,-1],etc]
Automatically generate a Processed SEP Event list
-------------------------------------------------
It is possible to run both codes with a single button push to create a preliminary SEP event list.
The code:
| fetchsep_prepare_obs
will first run `idsep` on a specified data set and identify all increases above background. Output files are created that are then used to automatically run `opsep` in batch mode to analyze each quiet and elevated period. This creates a set of json another other supporting files for each SEP event and quiet time period in the time series.
Note that manual intervention is required to get a truly good event list. The automated method is not perfect at identifying individual SEP events, but it will get you 80% of the way there.
Support
-------
Do not hesitate to contact Katie Whitman at kathryn.whitman@nasa.gov for support with this code.
Credits
-------
This package was created with Cookiecutter_ and the `audreyr/cookiecutter-pypackage`_ project template.
.. _Cookiecutter: https://github.com/audreyr/cookiecutter
.. _`audreyr/cookiecutter-pypackage`: https://github.com/audreyr/cookiecutter-pypackage
History
=======
0.3.0 (2023-11-02)
------------------
* Fixed bug preventing fluence calculation for differential fluxes
* Improved plots for differential energy channels
* Added capability to move output produced by `opsep` to a target dir
* Updated filenames for plots to contain full zulu string
* Added padding to SEP start and end times in `fetchsep_prepare_obs`
workflow
* Removed unneeded asciitable dependency
0.2.2 (2023-09-22)
------------------
* Fix packaging bug that failed to include and correctly use the CCMC
json templates
0.2.1 (2023-09-22)
------------------
* Fix packaging bug that failed to include the default configuration
0.2.0 (2023-09-22)
------------------
* Add GOES-18 differential and real time integral fluxes; changes
GOES-R integral flux source to CCMC's HAPI API
* Add configuration by user config file in the configparser INI
format, replacing placement of config values in config.py. This
fixes the pip deployment issue whereby a user could not easily
update the code configuration.
* Add --ExportConfig option to opsep & idsep. Users may use this to
prepare a custom configuration.
* Fix: Create necessary output directories instead of crashing if
they don't exist
0.1.4 (2023-08-22)
------------------
* Fix bug preventing command-line scripts from executing in Python
interpreter
0.1.3 (2023-08-22)
------------------
* Update utils/read_datasets.py to add new version, v3-0-1, for GOES-R
differential files
* Fix bug in ccmc_json_handler.py that would crash for model profiles
with 0 values
* Fix deployment of command-line scripts
* Update README documentation
0.1.2 (2023-08-22)
------------------
* Fix README for PyPI distribution
* Update developer package requirements
0.1.1 (2023-08-08)
------------------
* Test bump2version procedure
0.1.0 (2023-06-01)
------------------
* First release on PyPI.
Raw data
{
"_id": null,
"home_page": "https://github.com/rickyegeland/fetchsep",
"name": "fetchsep",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "fetchsep",
"author": "Kathryn Whitman",
"author_email": "kathryn.whitman@nasa.gov",
"download_url": "https://files.pythonhosted.org/packages/92/00/6a8a07592b9ef053e60c6c85442cf46d9eae7cd88414c77410009bd0ae7a/fetchsep-0.3.0.tar.gz",
"platform": null,
"description": "========\nFetchSEP\n========\n\n\n.. image:: https://img.shields.io/pypi/v/fetchsep.svg\n :target: https://pypi.python.org/pypi/fetchsep\n\n.. image:: https://img.shields.io/travis/rickyegeland/fetchsep.svg\n :target: https://travis-ci.com/rickyegeland/fetchsep\n\n.. image:: https://readthedocs.org/projects/fetchsep/badge/?version=latest\n :target: https://fetchsep.readthedocs.io/en/latest/?version=latest\n :alt: Documentation Status\n\n\n\n\nIdentify SEP elevations above background in a time series (`idsep`) and analyze events individually (`opsep`)\n\n\n* Free software: MIT license\n* Documentation: https://fetchsep.readthedocs.io.\n\n\nSet Up\n======\n\nBy default, `idsep` and `opsep` will create the necessary output\ndirectories in the current working directory where the command is\nexecuted. The directories that will be created are `data`, `ouptut`,\n`plots`, and `lists`. The directories will not be overwritten if they\nalready exist. If you would like to choose another output location,\ngenerate a config file (`fetchsep.cfg`) with `opsep --ExportConfig`\nand edit the paths described there.\n\nA configuration file may also be placed in your home directory with\nthe name `.fetchsep`. The configuration file does not need to be\ncomplete; you may specify only the values for which you wish to\noverride the default. The order of config value precidence\nis 1. current working directory `fetchsep.cfg`, 2. home directory\n`.fetchsep`, 3. fetchsep defaults.\n\n\nRun\n===\n\nTo run OpSEP to process individual SEP events:\n\n | opsep --StartDate 2012-05-16 --EndDate 2012-05-22 --Experiment GOES-13 --FluxType integral --showplot\n\nIDSEP\n=====\n\nThe `idsep` code will read in a long time series and automatically identify increases above background. This is done by estimating a mean background level plus an expected level of variation (sigma). All flux less than mean + 3sigma are considered background while all points above mean + 3sigma are identified as increases.\n\nBy assigning a set of criteria, increases that are most likely due to SEP events are identified and an SEP event list is output for each energy channel. \n\nThe code also outputs a file containing every single high flux point above the mean + 3sigma threshold.\n\nNote that in fetchsep/utils/config.py, number of sigma, the initial window used to estimate background levels, and the final sliding window used to estimate background levels can be adjusted. \n\n | idsep_nsigma = 3\n | init_win = 150 #days to average initial estimate of threshold\n | sliding_win = 27 #days in sliding window to calculate final threshold\n | percent_points = 0.9 #Percent of points that must be in the sliding\n | #window to calculate the background; otherwise use\n | #previous good value\n\n\n\nOPSEP\n=====\n\nThe `opsep` code was previously supported at https://github.com/ktindiana/operational-sep and is now transitioned to this package going forward. Please see the operational-sep repository for extensive documentation until the documentation in this repository can be updated.\n\n`opsep` is intended to assess each individual SEP event at a time, extracting information such as start and end times, peak fluxes, and event fluence.\n\nThe code will output various csv files and a json file with accompany txt files. The JSON file is in the same format as required by the SEP Scoreboard to submit forecasts.\n\n`opsep` creates files from observations that can be directly compared to SEP model forecasts sent to the SEP Scoreboard.\n\nFor time profile SEP models, `opsep` may be used to create the JSON files that can be submitted to the SEP Scoreboard.\n\n\n\nRunning `opsep` for your own time series\n----------------------------------------\n\nUsers may input their own time series into `opsep` by specifying some\ninformation in the config file:\n\n | ##### DELIMETER between columns of file with time series\n | user_delim = \" \" #any string\n | ##### COLUMNS containing the fluxes you want to analyze\n | user_col = arr.array('i',[1,2,3,4,5,6,7,8])\n | err_col = arr.array('i',[]) #set to [] if no uncertainties. err_col only used by idsep\n | ##### ENERGY BINS associated with user file and columns\n | #For differential bins, use the format:\n | user_energy_bins = [[Elow1,Ehigh1],[Elow2,Ehigh2],etc]\n | #For integral bins, use the format:\n | user_energy_bins = [[Elow1,-1],[Elow2,-1],[Elow3,-1],etc]\n\n\n \nAutomatically generate a Processed SEP Event list\n-------------------------------------------------\n\nIt is possible to run both codes with a single button push to create a preliminary SEP event list. \nThe code:\n\n | fetchsep_prepare_obs\n\nwill first run `idsep` on a specified data set and identify all increases above background. Output files are created that are then used to automatically run `opsep` in batch mode to analyze each quiet and elevated period. This creates a set of json another other supporting files for each SEP event and quiet time period in the time series.\n\nNote that manual intervention is required to get a truly good event list. The automated method is not perfect at identifying individual SEP events, but it will get you 80% of the way there. \n\nSupport\n-------\n\nDo not hesitate to contact Katie Whitman at kathryn.whitman@nasa.gov for support with this code.\n\nCredits\n-------\n\nThis package was created with Cookiecutter_ and the `audreyr/cookiecutter-pypackage`_ project template.\n\n.. _Cookiecutter: https://github.com/audreyr/cookiecutter\n.. _`audreyr/cookiecutter-pypackage`: https://github.com/audreyr/cookiecutter-pypackage\n\n\nHistory\n=======\n\n0.3.0 (2023-11-02)\n------------------\n\n* Fixed bug preventing fluence calculation for differential fluxes\n* Improved plots for differential energy channels\n* Added capability to move output produced by `opsep` to a target dir\n* Updated filenames for plots to contain full zulu string\n* Added padding to SEP start and end times in `fetchsep_prepare_obs`\n workflow\n* Removed unneeded asciitable dependency\n\n0.2.2 (2023-09-22)\n------------------\n\n* Fix packaging bug that failed to include and correctly use the CCMC\n json templates\n\n0.2.1 (2023-09-22)\n------------------\n\n* Fix packaging bug that failed to include the default configuration\n\n0.2.0 (2023-09-22)\n------------------\n\n* Add GOES-18 differential and real time integral fluxes; changes\n GOES-R integral flux source to CCMC's HAPI API\n* Add configuration by user config file in the configparser INI\n format, replacing placement of config values in config.py. This\n fixes the pip deployment issue whereby a user could not easily\n update the code configuration.\n* Add --ExportConfig option to opsep & idsep. Users may use this to\n prepare a custom configuration.\n* Fix: Create necessary output directories instead of crashing if\n they don't exist\n\n0.1.4 (2023-08-22)\n------------------\n\n* Fix bug preventing command-line scripts from executing in Python\n interpreter\n\n0.1.3 (2023-08-22)\n------------------\n\n* Update utils/read_datasets.py to add new version, v3-0-1, for GOES-R\n differential files\n* Fix bug in ccmc_json_handler.py that would crash for model profiles\n with 0 values\n* Fix deployment of command-line scripts\n* Update README documentation\n\n0.1.2 (2023-08-22)\n------------------\n\n* Fix README for PyPI distribution\n* Update developer package requirements\n\n0.1.1 (2023-08-08)\n------------------\n\n* Test bump2version procedure\n\n0.1.0 (2023-06-01)\n------------------\n\n* First release on PyPI.\n",
"bugtrack_url": null,
"license": "MIT license",
"summary": "Identify SEP elevations above background in a time series (idsep) and analyze events individually (opsep)",
"version": "0.3.0",
"project_urls": {
"Homepage": "https://github.com/rickyegeland/fetchsep"
},
"split_keywords": [
"fetchsep"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1e8baa7a08f167490c89790c8f6f6f7e8ffc415250a68be4de6633d3ab8fe472",
"md5": "6900f562a33b5433b047a0941be83f2d",
"sha256": "1f667077f5571e498c0af6efb6f52c39bfbeb1b58d832980895ff705da81a9d0"
},
"downloads": -1,
"filename": "fetchsep-0.3.0-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "6900f562a33b5433b047a0941be83f2d",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.8",
"size": 155301,
"upload_time": "2023-11-02T16:54:42",
"upload_time_iso_8601": "2023-11-02T16:54:42.316976Z",
"url": "https://files.pythonhosted.org/packages/1e/8b/aa7a08f167490c89790c8f6f6f7e8ffc415250a68be4de6633d3ab8fe472/fetchsep-0.3.0-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "92006a8a07592b9ef053e60c6c85442cf46d9eae7cd88414c77410009bd0ae7a",
"md5": "e06e36c1ec5317cc113fa7c6a26a2bef",
"sha256": "941b342caff79480ff18111d8a2dd86abfb4222bcf12eb27b2aabbc8c5dc2c22"
},
"downloads": -1,
"filename": "fetchsep-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "e06e36c1ec5317cc113fa7c6a26a2bef",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 148304,
"upload_time": "2023-11-02T16:54:44",
"upload_time_iso_8601": "2023-11-02T16:54:44.372116Z",
"url": "https://files.pythonhosted.org/packages/92/00/6a8a07592b9ef053e60c6c85442cf46d9eae7cd88414c77410009bd0ae7a/fetchsep-0.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-02 16:54:44",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "rickyegeland",
"github_project": "fetchsep",
"travis_ci": true,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "astropy",
"specs": [
[
"==",
"5.1"
]
]
},
{
"name": "python-dateutil",
"specs": [
[
"==",
"2.8.2"
]
]
},
{
"name": "matplotlib",
"specs": [
[
"==",
"3.5.3"
]
]
},
{
"name": "numpy",
"specs": [
[
"==",
"1.23.3"
]
]
},
{
"name": "scipy",
"specs": [
[
"==",
"1.9.1"
]
]
},
{
"name": "pandas",
"specs": [
[
"==",
"1.4.4"
]
]
},
{
"name": "wget",
"specs": [
[
"==",
"3.2"
]
]
},
{
"name": "zulu",
"specs": [
[
"==",
"2.0.0"
]
]
},
{
"name": "lmfit",
"specs": [
[
"==",
"1.0.3"
]
]
},
{
"name": "cycler",
"specs": [
[
"==",
"0.10.0"
]
]
},
{
"name": "scikit-learn",
"specs": [
[
"==",
"1.1.2"
]
]
},
{
"name": "seaborn",
"specs": [
[
"==",
"0.12.0"
]
]
},
{
"name": "netCDF4",
"specs": [
[
"==",
"1.6.0"
]
]
}
],
"tox": true,
"lcname": "fetchsep"
}
JSON
