# RFInder
**Insallation instructions**
```
pip install rfinder
```
To create a local repository, type:
```
git clone git@github.com:caracal-pipeline/RFInder.git
```
***
**Requisites**
For a successfull installation make sure to have installed the following packages.
- RFInder makes use of the most common `python` packages (e.g. `numpy`, `scipy`, `astropy`).
- The parameter file is in `yaml` format, hence [`pyaml`](https://anaconda.org/anaconda/pyyaml), and [`json`](https://anaconda.org/conda-forge/json-c) packages should be installed,
- The `logging` module is used to print out warnings.
- `.gif` file of multiple plots can be created if `ffmpeg` is installed.
- `casacore` is utilized to open casa tables.
- install it with `python_casacore`: `pip install python-casacore` or `conda install -c conda-forge python-casacore`
- `texmaker` to plot latex fancy formulae
- `dvipng`
- `python tk`
- RFInder class `rfinder_uzero.py` which can be used to solve the [u=0 problem](https://archive-gw-1.kat.ac.za/public/repository/10.48479/bhpj-nz95/index.html) makes use of `wsclean`, which must have been previously installed.
***
**Description**
This is a set of tools that have been developed in preparation of the Apertif & MeerKAT surveys.
The main function of `rfinder` is to identify the presence of RFI in an observation and visualize it according to different parameters. Two are the main functions:
- estimate the RFI present in an MS file through a sigma clipping (`rms_clip`)
- read the `FLAG` column of an MS file (`use_flags`) and summarize how RFI affects the data products of an observation.
These are the products that `rfinder` provides and summarizes in an `.html` file:
- presence of RFI per frequency channel and baseline length.
- percentage flagged visibilities due to RFI per frequency channel.
- increase in noice due to RFI per frequency channel.
- estimated noise per frequency channel, assuming natural weighting.
check out the [WiKi](https://github.com/Fil8/RFInder/wiki) for a complete illustration of `RFInder`.
***
**Usage**
RFInder takes its variables from a default parameter file and from terminal, if any are given.
From your current working directory typying `rfinder` this message will be shown:
```
------ Reading default installation parameter file ------
MSNAME & telescope missing
please edit rfinder_default.yml in your current directory
or run: rfinder -i msname -fl <num> -tel <meerkat,apertif,wsrt>
(assuming the observation is located in your current directory)
------ RFInder out ------
```
Hence, you have to set the name of the MSfile you wish to analyse. There are two ways to do this. By specifying from terminal the path to the msfile from your current directory, the field number of the source you whish to analyse, and the telescope of the observation:
```
rfinder -i msname -fl <num> -tel <meerkat,apertif,wsrt>
```
or, editing the `rfinder_default.yml` configuration file that has been copied in your current directory (workdir, in the configuration file).
This configuration file is read automatically by RFInder through the command `rfinder`. A short explanation of the parameters is given in the configuration file, and by typing `rfinder -h` (see below).
If you wish to use a different configuration file (at your own risk!!), type: `rfinder -c <path_to_configuration_file>`.
**Minimal instructions**
- Default `rfinder` will scan the MSfile in chunks of 10 minutes averaging 10 channels together. The output product will be an `html` file where the `gis` scan through the time steps to show the identified RFI/flags.
- Running `rfinder -noCh` after `rfinder` will produce a `full_report.html` file containing both the analysis over time steps and the analysis of the dataset as a whole.
- Running `rfinder -noCh -noMov` will analyse the full dataset as a whole and generate the `full_report.html` without embedded movies.
_Attention_: the option `rfinder -noCh` will end with a report successfully generated, only if it is run after `rfinder`. Otherwise run `rfinder -noCh -noMov`.
(These [tutorials](https://github.com/Fil8/RFInder/tree/master/tutorials) show the different capabilities of `rfinder`. **outdated**)
**Output products**
If `rfinder` runs correctly, you will find the following output products in your current directory:
- the folder `rfi_pol` in your current directory, or in the directory specified by the `-odir` parameter (`pol` is the stokes parameters for which you analysed RFI).
- Within, there are the `.html` reports that you wished to generate.
- The configuration file `rfinder_default.yml` contains the parameters of the last run.
- A `log` of the commands run by the program is stored in `log-rfinder.log`, in your working directory.
**Help**
`rfinder -h` will show you a (minimal) help:
```
usage: rfinder [-h] [-v] [-c CONFIG] [-w WORKING_DIR] [-odir OUTPUT_DIR]
[-i INPUT] [-fl FIELD] [-tel TELESCOPE] [-mode RFIMODE]
[-pol POLARIZATION]
[-fint [FREQUENCY_INTERVAL [FREQUENCY_INTERVAL ...]]]
[-spwAv SPW_AV] [-tStep TIME_STEP] [-sig SIGMA_CLIP]
[-baseCut BASELINE_CUT] [-noCh] [-yesCh] [-noSpw] [-yesSpw]
[-noClp] [-yesClp]
RFInder: package to visualize the flagged RFI in a dataset
version 1.0.3
install path /home/maccagni/programs/RFInder/rfinder
Filippo Maccagni <filippo.maccagni@gmial.com>
optional arguments:
-h, --help Print help message and exit
-v, --version show program's version number and exit
-c CONFIG, --config CONFIG
RFInder configuration file (YAML format)
-idir INPUT_DIR, --input_dir WORKING_DIR
select working directory (MS file assumed to be here)
-odir OUTPUT_DIR, --output_dir OUTPUT_DIR
select output directory
-i INPUT, --input INPUT
input ['MS'] file
-fl FIELD, --field FIELD
select field of MS file to analyze
-tel TELESCOPE, --telescope TELESCOPE
select telescope: meerkat, apertif, wsrt
-mode RFIMODE, --rfimode RFIMODE
select mode where to investigate RFI: use_flags or
rms_clip
-pol POLARIZATION, --polarization POLARIZATION
select stokes parameter: xx, yy, xy, yx, q (also in
CAPS)
-fint [FREQUENCY_INTERVAL [FREQUENCY_INTERVAL ...]], --frequency_interval [FREQUENCY_INTERVAL [FREQUENCY_INTERVAL ...]]
select frequency interval where to measure noise in
GHz
-spwAv SPW_AV, --spw_av SPW_AV
select number of channels to average
-tStep TIME_STEP, --time_step TIME_STEP
select time step in minutes in which divide the
analysis of the MSfile
-sig SIGMA_CLIP, --sigma_clip SIGMA_CLIP
select sigma clip for rms_clip mode to find RFI
-baseCut BASELINE_CUT, --baseline_cut BASELINE_CUT
select cut in baseline lenght [m] for differential RFI
analysis
-noCh, --no_chunks desable chunking in time
-yesCh, --yes_chunks enable chunking in time
-noSpw, --no_spw_av desable averaging in channels
-yesSpw, --yes_spw_av
enable averaging in channels
-noClp, --no_cleanup desable cleanup of intermediate products
-yesClp, --yes_cleanup
enable cleanup of intermediate products
Run a command. This can be:
rfinder
rfinder -c path_to_config_file.yml
rfinder -i <ngc1399.ms> -fl <num> -tel <meerkat/apertif/wsrt>
```
***
**License**
This project is licensed under the GNU General Public License v3.0 - see [license](https://github.com/Fil8/RFInder/blob/master/LICENSE.md) for details.
***
<p>© <sub> Filippo M. Maccagni 2018 </sub></p>
Raw data
{
"_id": null,
"home_page": "https://github.com/caracal-pipeline/rfinder",
"name": "rfinder",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4.0",
"maintainer_email": "",
"keywords": "Astronomy,Visualisation,RFI",
"author": "Filippo Marcello Maccagni",
"author_email": "filippo.maccagni@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/67/ed/0e6c7123e8f0040eb9cae927e72133cc1c5ab0282bb73e71518fa7522dd7/rfinder-1.1.0.tar.gz",
"platform": null,
"description": "# RFInder\n\n**Insallation instructions**\n\n```\npip install rfinder\n```\n\nTo create a local repository, type:\n\n```\ngit clone git@github.com:caracal-pipeline/RFInder.git\n```\n\n***\n\n**Requisites**\n\nFor a successfull installation make sure to have installed the following packages.\n\n- RFInder makes use of the most common `python` packages (e.g. `numpy`, `scipy`, `astropy`). \n- The parameter file is in `yaml` format, hence [`pyaml`](https://anaconda.org/anaconda/pyyaml), and [`json`](https://anaconda.org/conda-forge/json-c) packages should be installed,\n- The `logging` module is used to print out warnings.\n- `.gif` file of multiple plots can be created if `ffmpeg` is installed.\n- `casacore` is utilized to open casa tables.\n - install it with `python_casacore`: `pip install python-casacore` or `conda install -c conda-forge python-casacore` \n- `texmaker` to plot latex fancy formulae\n- `dvipng`\n- `python tk`\n- RFInder class `rfinder_uzero.py` which can be used to solve the [u=0 problem](https://archive-gw-1.kat.ac.za/public/repository/10.48479/bhpj-nz95/index.html) makes use of `wsclean`, which must have been previously installed.\n\n\n***\n**Description**\n\nThis is a set of tools that have been developed in preparation of the Apertif & MeerKAT surveys.\n\nThe main function of `rfinder` is to identify the presence of RFI in an observation and visualize it according to different parameters. Two are the main functions:\n\n- estimate the RFI present in an MS file through a sigma clipping (`rms_clip`)\n- read the `FLAG` column of an MS file (`use_flags`) and summarize how RFI affects the data products of an observation. \n\nThese are the products that `rfinder` provides and summarizes in an `.html` file:\n\n- presence of RFI per frequency channel and baseline length.\n- percentage flagged visibilities due to RFI per frequency channel. \n- increase in noice due to RFI per frequency channel.\n- estimated noise per frequency channel, assuming natural weighting. \n\ncheck out the [WiKi](https://github.com/Fil8/RFInder/wiki) for a complete illustration of `RFInder`.\n\n***\n**Usage**\n\nRFInder takes its variables from a default parameter file and from terminal, if any are given. \n\nFrom your current working directory typying `rfinder` this message will be shown: \n\n```\n------ Reading default installation parameter file ------\n\nMSNAME & telescope missing\n \t\tplease edit rfinder_default.yml in your current directory\n \t\tor run: rfinder -i msname -fl <num> -tel <meerkat,apertif,wsrt>\n \t\t(assuming the observation is located in your current directory)\n \n\n------ RFInder out ------\n```\n\nHence, you have to set the name of the MSfile you wish to analyse. There are two ways to do this. By specifying from terminal the path to the msfile from your current directory, the field number of the source you whish to analyse, and the telescope of the observation:\n\n```\n rfinder -i msname -fl <num> -tel <meerkat,apertif,wsrt>\n```\n\nor, editing the `rfinder_default.yml` configuration file that has been copied in your current directory (workdir, in the configuration file). \n\nThis configuration file is read automatically by RFInder through the command `rfinder`. A short explanation of the parameters is given in the configuration file, and by typing `rfinder -h` (see below).\n\nIf you wish to use a different configuration file (at your own risk!!), type: `rfinder -c <path_to_configuration_file>`.\n\n**Minimal instructions**\n\n- Default `rfinder` will scan the MSfile in chunks of 10 minutes averaging 10 channels together. The output product will be an `html` file where the `gis` scan through the time steps to show the identified RFI/flags.\n\n- Running `rfinder -noCh` after `rfinder` will produce a `full_report.html` file containing both the analysis over time steps and the analysis of the dataset as a whole.\n\n- Running `rfinder -noCh -noMov` will analyse the full dataset as a whole and generate the `full_report.html` without embedded movies.\n\n_Attention_: the option `rfinder -noCh` will end with a report successfully generated, only if it is run after `rfinder`. Otherwise run `rfinder -noCh -noMov`.\n\n(These [tutorials](https://github.com/Fil8/RFInder/tree/master/tutorials) show the different capabilities of `rfinder`. **outdated**)\n\n\n**Output products**\n\nIf `rfinder` runs correctly, you will find the following output products in your current directory: \n\n- the folder `rfi_pol` in your current directory, or in the directory specified by the `-odir` parameter (`pol` is the stokes parameters for which you analysed RFI). \n\t- Within, there are the `.html` reports that you wished to generate. \n- The configuration file `rfinder_default.yml` contains the parameters of the last run.\n- A `log` of the commands run by the program is stored in `log-rfinder.log`, in your working directory.\n\n**Help**\n\n`rfinder -h` will show you a (minimal) help:\n\n```\nusage: rfinder [-h] [-v] [-c CONFIG] [-w WORKING_DIR] [-odir OUTPUT_DIR]\n [-i INPUT] [-fl FIELD] [-tel TELESCOPE] [-mode RFIMODE]\n [-pol POLARIZATION]\n [-fint [FREQUENCY_INTERVAL [FREQUENCY_INTERVAL ...]]]\n [-spwAv SPW_AV] [-tStep TIME_STEP] [-sig SIGMA_CLIP]\n [-baseCut BASELINE_CUT] [-noCh] [-yesCh] [-noSpw] [-yesSpw]\n [-noClp] [-yesClp]\n\nRFInder: package to visualize the flagged RFI in a dataset\n\nversion 1.0.3\n\ninstall path /home/maccagni/programs/RFInder/rfinder\n\nFilippo Maccagni <filippo.maccagni@gmial.com>\n\noptional arguments:\n -h, --help Print help message and exit\n -v, --version show program's version number and exit\n -c CONFIG, --config CONFIG\n RFInder configuration file (YAML format)\n -idir INPUT_DIR, --input_dir WORKING_DIR\n select working directory (MS file assumed to be here)\n -odir OUTPUT_DIR, --output_dir OUTPUT_DIR\n select output directory\n -i INPUT, --input INPUT\n input ['MS'] file\n -fl FIELD, --field FIELD\n select field of MS file to analyze\n -tel TELESCOPE, --telescope TELESCOPE\n select telescope: meerkat, apertif, wsrt\n -mode RFIMODE, --rfimode RFIMODE\n select mode where to investigate RFI: use_flags or\n rms_clip\n -pol POLARIZATION, --polarization POLARIZATION\n select stokes parameter: xx, yy, xy, yx, q (also in\n CAPS)\n -fint [FREQUENCY_INTERVAL [FREQUENCY_INTERVAL ...]], --frequency_interval [FREQUENCY_INTERVAL [FREQUENCY_INTERVAL ...]]\n select frequency interval where to measure noise in\n GHz\n -spwAv SPW_AV, --spw_av SPW_AV\n select number of channels to average\n -tStep TIME_STEP, --time_step TIME_STEP\n select time step in minutes in which divide the\n analysis of the MSfile\n -sig SIGMA_CLIP, --sigma_clip SIGMA_CLIP\n select sigma clip for rms_clip mode to find RFI\n -baseCut BASELINE_CUT, --baseline_cut BASELINE_CUT\n select cut in baseline lenght [m] for differential RFI\n analysis\n -noCh, --no_chunks desable chunking in time\n -yesCh, --yes_chunks enable chunking in time\n -noSpw, --no_spw_av desable averaging in channels\n -yesSpw, --yes_spw_av\n enable averaging in channels\n -noClp, --no_cleanup desable cleanup of intermediate products\n -yesClp, --yes_cleanup\n enable cleanup of intermediate products\n\nRun a command. This can be: \nrfinder \nrfinder -c path_to_config_file.yml\nrfinder -i <ngc1399.ms> -fl <num> -tel <meerkat/apertif/wsrt>\n```\n\n***\n\n**License**\n\nThis project is licensed under the GNU General Public License v3.0 - see [license](https://github.com/Fil8/RFInder/blob/master/LICENSE.md) for details.\n\n\n ***\n <p>© <sub> Filippo M. Maccagni 2018 </sub></p>\n\n",
"bugtrack_url": null,
"license": "GNU GPL v3",
"summary": "Tool for identify the presence of RFI in an observation, flagging and visualisation",
"version": "1.1.0",
"project_urls": {
"Homepage": "https://github.com/caracal-pipeline/rfinder",
"Repository": "https://github.com/caracal-pipeline/rfinder"
},
"split_keywords": [
"astronomy",
"visualisation",
"rfi"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "769a3bb758e0b31179966132855eb5c288473426c4c55c1cbe60ddf81a37531f",
"md5": "915f35773ec23be940ac1675eca395cd",
"sha256": "3bdc72c6f8335f20302a80115789bb79b6481b52461235ee0661e651cab17e4f"
},
"downloads": -1,
"filename": "rfinder-1.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "915f35773ec23be940ac1675eca395cd",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4.0",
"size": 50811,
"upload_time": "2024-02-23T15:22:12",
"upload_time_iso_8601": "2024-02-23T15:22:12.158847Z",
"url": "https://files.pythonhosted.org/packages/76/9a/3bb758e0b31179966132855eb5c288473426c4c55c1cbe60ddf81a37531f/rfinder-1.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "67ed0e6c7123e8f0040eb9cae927e72133cc1c5ab0282bb73e71518fa7522dd7",
"md5": "f46d953dbcb96ae82d6836456e6edbd4",
"sha256": "c44b123a0b00103667556d2b5c1862f9061571a4ac156e27349528afe6e74049"
},
"downloads": -1,
"filename": "rfinder-1.1.0.tar.gz",
"has_sig": false,
"md5_digest": "f46d953dbcb96ae82d6836456e6edbd4",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4.0",
"size": 46703,
"upload_time": "2024-02-23T15:22:13",
"upload_time_iso_8601": "2024-02-23T15:22:13.382898Z",
"url": "https://files.pythonhosted.org/packages/67/ed/0e6c7123e8f0040eb9cae927e72133cc1c5ab0282bb73e71518fa7522dd7/rfinder-1.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-23 15:22:13",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "caracal-pipeline",
"github_project": "rfinder",
"travis_ci": true,
"coveralls": false,
"github_actions": true,
"lcname": "rfinder"
}
JSON
