<!--- Top of README Badges (automated) --->
[](https://pypi.org/project/icecube-skywriter/) [](https://github.com/icecube/skywriter/) [](https://github.com/icecube/skywriter/blob/main/LICENSE) [](https://github.com/icecube/skywriter/) [](https://github.com/icecube/skywriter/issues?q=is%3Aissue+sort%3Aupdated-desc+is%3Aopen) [](https://github.com/icecube/skywriter/pulls?q=is%3Apr+sort%3Aupdated-desc+is%3Aopen)
<!--- End of README Badges (automated) --->
# skywriter
Upstream Tools for SkyDriver & the Skymap Scanner.
## I3 to JSON converter
The main tool provided by `skywriter` is the `i3_to_json` converter function and script. The function reads a (list of) I3 file(s), processes them using the the realtime alert followup code (`AlertEventFollowup` module) and writes the events out as JSON files.
As SkyMap Scanner is designed to process the `SplitUncleanedInIcePulses` pulse series, the converter generates it by running the `I3TriggerSplitter` module on the original DAQ (Q) frame, hence creating one or more new Physics (P) frames (one for each "subevent" in the data). If already present, the `SplitUncleanedInIcePulses` object is deleted beforehand. In case of multiple subevents, only the subevent-id corresponding matching the original P-frame is processed.
The `AlertEventFollowup` module expects the Physics I3Frame to have a given set of keys. Such set of keys is currently hardcoded in the converter code. If the keys are present in the original P-frame, the corresponding objects are copied over to the output JSON. If not, they are filled with dummy values. If you plan to use these keys in your own Skymap Scanner reconstruction module make sure they have meaningful values.
Extra `I3Particle` objects, typically corresponding to reconstruction outputs, can be "extracted" through the `--extra` option. This means that the equatorial coordinates are calculated and written out in a JSON branch.
### GCD
In order to produce lightweight payloads, the `AlertEventFollowup` code uses the so called "GCD diff" functionality (sometimes called "GCD compress"). The input GCD is compared against a given "base GCD" and only the difference is stored in the JSON-encoded GCD, along with the filename of the base GCD.
Skymap Scanner will look up a matching filename of the base GCD file in a pre-defined set of locations. In principle, it should not matter much which base GCD is used as long as Skymap Scanner can retrieve it. If you use a custom base GCD you should make sure it is made available to Skymap Scanner.
### Example usage
Assuming you have set up your environment and loaded an IceTray shell, you can run the converter as (for example):
```
python skywriter/i3_to_json.py --extra "OnlineL2_SplineMPE" /data/ana/realtime/alert_catalog_v2/input_files/Level2pass2_IC86.2011_data_Run00118435_Subrun00000000_00000144_event58198553.i3
```
this will use the default base GCD.
### Known limitations
- The converter always writes out JSON files and does not allow to simply obtain JSON-encoded string corresponding to the input event. Yet you may treat output files as ephemeral, and delete them after the scan.
- The converter supports the reading of a sequence of I3 files, but it is mostly tested with a single input file. If you encounter problems with converting multiple files, please file an issue. More in general, rather than producing sets of JSON files for archival, it could be better to do any conversion on-the-fly and track the conversion options in the user code.
- It is not possible to copy an arbitrary key/object from the input P-frame to the output P-frame. There is also no warranty that a particle extracted with the `--extra` option will have a corresponding object in the output P-frame. If you need such a feature, or you want to improve this behavior, feel free to file a pull request!
- Events in the input frame(s) are currently identified by `run_id`, `event_id`, `subevent_id`, which in turn are used to name the output JSON files. There is no warranty that these are unique in simulation events. Note that while Skymap Scanner does not care about the event being scanned having a unique id, SkyDriver currently identifies events by `run_id` and `event_id` only (although in principle, you could still scan different events with identical `run_id` and `event_id`, it is just a matter of bookkeeping and information retrieval). The best approach would be to ensure that sets of simulated events processed with SkyDriver are assigned unique `(run_id, event_id)`.
Raw data
{
"_id": null,
"home_page": "https://github.com/icecube/skywriter",
"name": "icecube-skywriter",
"maintainer": "",
"docs_url": null,
"requires_python": "<3.12,>=3.8",
"maintainer_email": "",
"keywords": "skymap scanner,skymap,HEALPix,neutrino,reconstruction,IceCube",
"author": "WIPAC Developers",
"author_email": "developers@icecube.wisc.edu",
"download_url": "https://files.pythonhosted.org/packages/7e/db/96328ef60fa9b15424cd731245b04282f6e6f8daad42c1339cdbe7eb26e3/icecube-skywriter-0.0.6.tar.gz",
"platform": null,
"description": "<!--- Top of README Badges (automated) --->\n[](https://pypi.org/project/icecube-skywriter/) [](https://github.com/icecube/skywriter/) [](https://github.com/icecube/skywriter/blob/main/LICENSE) [](https://github.com/icecube/skywriter/) [](https://github.com/icecube/skywriter/issues?q=is%3Aissue+sort%3Aupdated-desc+is%3Aopen) [](https://github.com/icecube/skywriter/pulls?q=is%3Apr+sort%3Aupdated-desc+is%3Aopen) \n<!--- End of README Badges (automated) --->\n# skywriter\nUpstream Tools for SkyDriver & the Skymap Scanner.\n\n## I3 to JSON converter\nThe main tool provided by `skywriter` is the `i3_to_json` converter function and script. The function reads a (list of) I3 file(s), processes them using the the realtime alert followup code (`AlertEventFollowup` module) and writes the events out as JSON files.\n\nAs SkyMap Scanner is designed to process the `SplitUncleanedInIcePulses` pulse series, the converter generates it by running the `I3TriggerSplitter` module on the original DAQ (Q) frame, hence creating one or more new Physics (P) frames (one for each \"subevent\" in the data). If already present, the `SplitUncleanedInIcePulses` object is deleted beforehand. In case of multiple subevents, only the subevent-id corresponding matching the original P-frame is processed.\n\nThe `AlertEventFollowup` module expects the Physics I3Frame to have a given set of keys. Such set of keys is currently hardcoded in the converter code. If the keys are present in the original P-frame, the corresponding objects are copied over to the output JSON. If not, they are filled with dummy values. If you plan to use these keys in your own Skymap Scanner reconstruction module make sure they have meaningful values.\n\nExtra `I3Particle` objects, typically corresponding to reconstruction outputs, can be \"extracted\" through the `--extra` option. This means that the equatorial coordinates are calculated and written out in a JSON branch.\n\n### GCD\nIn order to produce lightweight payloads, the `AlertEventFollowup` code uses the so called \"GCD diff\" functionality (sometimes called \"GCD compress\"). The input GCD is compared against a given \"base GCD\" and only the difference is stored in the JSON-encoded GCD, along with the filename of the base GCD.\n\nSkymap Scanner will look up a matching filename of the base GCD file in a pre-defined set of locations. In principle, it should not matter much which base GCD is used as long as Skymap Scanner can retrieve it. If you use a custom base GCD you should make sure it is made available to Skymap Scanner.\n\n\n### Example usage\nAssuming you have set up your environment and loaded an IceTray shell, you can run the converter as (for example):\n```\npython skywriter/i3_to_json.py --extra \"OnlineL2_SplineMPE\" /data/ana/realtime/alert_catalog_v2/input_files/Level2pass2_IC86.2011_data_Run00118435_Subrun00000000_00000144_event58198553.i3\n```\nthis will use the default base GCD.\n\n\n### Known limitations\n- The converter always writes out JSON files and does not allow to simply obtain JSON-encoded string corresponding to the input event. Yet you may treat output files as ephemeral, and delete them after the scan.\n- The converter supports the reading of a sequence of I3 files, but it is mostly tested with a single input file. If you encounter problems with converting multiple files, please file an issue. More in general, rather than producing sets of JSON files for archival, it could be better to do any conversion on-the-fly and track the conversion options in the user code.\n- It is not possible to copy an arbitrary key/object from the input P-frame to the output P-frame. There is also no warranty that a particle extracted with the `--extra` option will have a corresponding object in the output P-frame. If you need such a feature, or you want to improve this behavior, feel free to file a pull request!\n- Events in the input frame(s) are currently identified by `run_id`, `event_id`, `subevent_id`, which in turn are used to name the output JSON files. There is no warranty that these are unique in simulation events. Note that while Skymap Scanner does not care about the event being scanned having a unique id, SkyDriver currently identifies events by `run_id` and `event_id` only (although in principle, you could still scan different events with identical `run_id` and `event_id`, it is just a matter of bookkeeping and information retrieval). The best approach would be to ensure that sets of simulated events processed with SkyDriver are assigned unique `(run_id, event_id)`.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Upstream Tools for SkyDriver & the Skymap Scanner",
"version": "0.0.6",
"project_urls": {
"Download": "https://pypi.org/project/icecube-skywriter/",
"Homepage": "https://github.com/icecube/skywriter",
"Source": "https://github.com/icecube/skywriter",
"Tracker": "https://github.com/icecube/skywriter/issues"
},
"split_keywords": [
"skymap scanner",
"skymap",
"healpix",
"neutrino",
"reconstruction",
"icecube"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "22ebdf6333906f17a591896e6acd0a91f710afe17f2ca962a6536ae1f6ecb80f",
"md5": "3214ed8c9709dca15a350c0b278a66f3",
"sha256": "90a4fb5e88767ff11948f20b4637678a3de4b8897e6d95d2d62ae473bf69994e"
},
"downloads": -1,
"filename": "icecube_skywriter-0.0.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "3214ed8c9709dca15a350c0b278a66f3",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.12,>=3.8",
"size": 9616,
"upload_time": "2023-10-24T16:04:02",
"upload_time_iso_8601": "2023-10-24T16:04:02.973340Z",
"url": "https://files.pythonhosted.org/packages/22/eb/df6333906f17a591896e6acd0a91f710afe17f2ca962a6536ae1f6ecb80f/icecube_skywriter-0.0.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7edb96328ef60fa9b15424cd731245b04282f6e6f8daad42c1339cdbe7eb26e3",
"md5": "08555b63f6c97811f8d889f87400fca1",
"sha256": "61d8021fa738ce08e8a05207ff14410ec75ab59165be7903af35681936eced45"
},
"downloads": -1,
"filename": "icecube-skywriter-0.0.6.tar.gz",
"has_sig": false,
"md5_digest": "08555b63f6c97811f8d889f87400fca1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.12,>=3.8",
"size": 9066,
"upload_time": "2023-10-24T16:04:04",
"upload_time_iso_8601": "2023-10-24T16:04:04.612979Z",
"url": "https://files.pythonhosted.org/packages/7e/db/96328ef60fa9b15424cd731245b04282f6e6f8daad42c1339cdbe7eb26e3/icecube-skywriter-0.0.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-10-24 16:04:04",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "icecube",
"github_project": "skywriter",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "icecube-skywriter"
}
JSON
