# WAGASCI ANPAN for Python v0.3.10
This package contains all the Python modules used by the ANPAN
framework of the WAGASCI/T2K experiment.
## What is ANPAN?
ANPAN (Acquisition Networked Program for Accelerated Neutrinos) is a collective
name to identify all the software used to acquire data and analyze it for
the WAGASCI experiment. It is composed of many packages:
- [Pyrame](http://llr.in2p3.fr/sites/pyrame/) (C/Python2)
- [ANPAN DAQ](https://llrgit.in2p3.fr/online/wagasci) (C/Python2)
- [WagasciCalibration](https://git.t2k.org/wagasci_babymind/WagasciCalibration) (C++11)
- [WagasciMidas](https://git.t2k.org/wagasci_babymind/WagasciMidas) (C++11/JS)
- [WagasciRunControl](https://git.t2k.org/wagasci_babymind/WagasciRunControl) (Python2)
- [WagasciAnpy (this package)](https://git.t2k.org/wagasci_babymind/WagasciAnpy) (Python2/3)
## Python compatibility
- **Python2.7 or Python3.4+**
This package is fully compatible with both Python2.7 and Python3.4+. It is
tested usually with Python 3.8 and sporadically with other Python versions.
However, I strongly suggest you to use Python 3 whenever possible because
Python 2 has now reached end of life and many packages are starting to drop
support for it. I am still trying to ensure Python 2 compatibility but I might
drop it in the future for non DAQ related modules.
## Dependencies
Many functions of this package depend on ROOT
- [ROOT](https://ph-root-2.cern.ch/)
To use the analysis programs, you need the WagasciCalibration package:
- [WagasciCalibration](https://git.t2k.org/wagasci_babymind/WagasciCalibration)
To download and update the WAGASCI run database you may need the BorgBackup
program:
- [BorgBackup](https://borgbackup.readthedocs.io/en/stable/installation.html)
The ``pyrmod`` module depends on Pyrame. This module is used only by the DAQ
so no need to install Pyrame on a non DAQ machine.
### ROOT
Be sure that the version of ROOT that you installed is compatible with
your Python version. The precompiled binaries for ROOT until version 6.22.0
are usually only compatible with Python 2. From ROOT version 6.22.0 on,
ROOT should be compatible with both Python 2 and Python 3, so my suggestion
is to just update your ROOT version.
### WAGASCI calibration software (optional)
To analyze the raw data you need the WAGASCI calibration software.
For the installation instructions refer to
[this page](https://git.t2k.org/wagasci_babymind/WagasciCalibration).
## Installation (Linux)
Open a terminal and issue
```shell script
python3 -m pip install --upgrade --user wagascianpy
or (recomended)
python3 -m pip install --upgrade --user .
```
## Installation (MacOS)
Currently the support for MacOS is only partial. The SSH connection is broken
because of some issues internal to the [Python paramiko
module](http://www.paramiko.org/), so the program can be used in local mode
only. I tried to fix the issue but the problem is beyond my skills. Probably in
the following months the paramiko developers will come up with a solution. At
the time of writing, the current (broken) version of paramiko is 2.7.1. When a
new version of paramiko is released, try again to run the software and see what
happens.
I have tested the program only in MacOS Mojave and MacOS Catalina. Both showed
more or less the same problems. You can try older versions of MacOS and might
have better luck.
#### MacOS Mojave/Catalina
Because MacOS Mojave does not bundle Python3 we need to install it manually. The
easiest way is by using the package manager brew. The following commands are
taken from this [webpage](https://installpython3.com/mac/).
The first step is to install the Apple's Xcode suite. Most probably you already
have it already installed, in that case you can skip this step. In your Terminal
app, run the following command to install XCode and its command-line tools:
```shell script
$ xcode-select --install
```
Next install Homebrew by copy/pasting the following command into Terminal and
then type Enter:
```shell script
$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
$ brew update
```
To confirm Homebrew installed correctly, run this command:
```shell script
$ brew doctor
Your system is ready to brew.
```
Now we can install the latest version of Python 3. Type the following command
into Terminal and press Enter:
```shell script
$ brew install python3
```
To confirm which version of Python was installed, run the following command in
Terminal:
```shell script
$ /usr/local/bin/python3 --version
Python 3.7.7
```
Many functions of the WAGASCI database viewer use borgbackup. We can easily
install it using brew.
```shell script
$ brew cask install borgbackup
```
For other ways of installing borg refer to the
[borg website](https://borgbackup.readthedocs.io/en/stable/installation.html).
Then we can finally install the wagascianpy package
```shell script
$ python3 -m pip install --user --upgrade wagascianpy
```
If you use MacOS Catalina continue reading the next section.
#### MacOS Catalina - troubleshooting
**MacOS Catalina comes with its own version of Python3. However
[as stated in the Python website](https://www.python.org/download/mac/tcltk/)
the Apple-supplied Tcl/Tk 8.5 has serious bugs that can cause application
crashes.
So follow the same instructions as MacOS Mojave but also install tkinter:
```shell script
$ brew install tcl-tk
```
## Configuration
### Environment parameters
You can easily configure all the parameters from the GUI itself (or from the
command line if you run the programs from the shell). However, because
inserting the same values over and over again can be tedius, you can modify the
configuration file called `wagasci.ini`. This file is created in the
`${HOME}/.wagasci` folder on the first execution of the program and filled with
the default values. You can just open and edit it with any text editor. The
options names and values are in one to one correlation with the GUI fields
and CLI arguments.
Upon start, the program tries to load the configuration file from the following
directories in this order:
- Environment : `${WAGASCI_CONFIGDIR}`
- User : `${HOME}/.wagasci`
- System : `/usr/local/wagasci`
The configuration file must be called `wagasci.ini`. Both the WAGASCI database
viewer and the spill history plotter can use the same configuration file.
### Good runs
The list of good runs and bad runs is hardcoded at the beginning of the
`wagascianpy/database/wagascidb.py` file. If a run is completely bad, add it
to the `_BAD_RUNS` list. If a run is bad only for a single detector, add it
to the `_BAD_RUNS_X` where X is the name of the detector. After modifying
the bad run list, to apply it again, please rerun `wagasciviewer.py
--rebuild-wagasci-database --batch-mode`, and then `wagasciviewer.py
--apply-detector-flags --batch-mode`.
### Markers
When drawing the history plots, the trouble markers and maintenance day
markers can be drawn along with the data. The start time and stop time of
these markers is hardcoded at the beginning of the
`wagascianpy/plotting/marker.py` file.
## WAGASCI database viewer
The following subsections explain the usage of the WAGASCI database viewer GUI
program.
### Run the program
To open the GUI program, just type in a terminal:
```shell script
wagasciviewer.py
````
If you wish to run the program without a GUI in batch mode, just type in a
terminal
```shell script
wagasciviewer.py --batch-mode <other arguments>
```
### WAGASCI run repository
The word repository refers to the place where the runs are backed up. It might
be a simple repository where the runs are stored without encription (plain file
and folders), or a borg repository where all the files are encripted. The
repository can be either local (same machine) or remote (remote machine
reachable by SSH). In case of a borg repository it is necessary for borgbackup
to be installed on both the local machine and the remote machine.
* **Simple repository** : Unencripted repository
* **Borg repository** : Borg repository
* **Repository location** : For a remote repository the syntax is
`<hostname>:<path of repository>`. The SSH configuration is taken from the
user `~/.ssh/config` file by looking for `<hostname>`, so make sure that you
can connect to the remote machine by just opening the terminal and `ssh
<hostname>`. In case of local repository just insert the repository path. For
more info about how to connect to KEKCC by SSH refer to the [WAGASCI expert
shifter manual](https://ja.overleaf.com/7887288445tmqmnjknrynm)
(Troubleshooting - KEKCC (8.6)).
### WAGASCI run database
Location of the WAGASCI run database. The database is a file (usually named
`wagascidb.db`) that contains some info about each run. If the update database
checkbox is checked, the software checks if the database is synchronized with
the repository (if the database describes the current state of the repository).
* **Database location**: location of the database file. Usually it is the same
as the repository. In case you specify a different location, enter the
filename too. Leave as it is if in doubt.
* **Update database**: If left unchecked the database is read as is. If checked,
the runs that are present in both the repository and database the database are
not updated and only the runs that are present in the repository but not in
the database are added. Leave disabled if in doubt.
* **Rebuild database**: If checked each run in the repository is
inserted/overwritten into the database even if already present. Leave disabled
if in doubt.
* **WAGASCI library**: path to the directory containing the WAGASCI
calibration software libraries. It is only used to update the database and
analyze the raw data. Not needed if you only wish to visualize the database.
### Beam Summary Data repository
The word repository refers to the place where the Beam Summary Data files are
stored. The local repository can be an empty folder or a folder containing
the BSD files. If the remote repository field is left empty the program assumes
that the BSD files are already present in the local folder. If the remote
repository field is filled, the remote repository and the local repository are
synchronized using rsync.
* **Local repository** : Location of BSD files in the local machine.
* **Remote repository** : Location of the BSD files in the remote machine (the
syntax is the same as the WAGASCI run repository). If left empty only the
local repository is used.
### Beam Summary Data database
Location of the BSD database. The database is a file (usually named `bsddb.db`)
that contains some info about each BSD file. If the update or rebuild database
checkbox is checked, the software checks if the database is synchronized with
the repository (if the database describes the current state of the repository).
* **Database location**: location of the database file. Usually it is the same
as the repository. In case you specify a different location, enter the
filename too. Leave as it is if in doubt.
* **Update database**: If left unchecked the database is read as is. If checked,
the runs that are present in both the repository and database the database are
not updated and only the runs that are present in the repository but not in
the database are added. Leave disabled if in doubt.
* **Rebuild database**: If checked each run in the repository is
inserted/overwritten into the database even if already present. Leave disabled
if in doubt.
### Query
#### Getters
* **Get time interval**: Select an interval and the push the Get interval
button. In the Run list frame will appear all the runs in that period.
* **Get run interval**: Select all runs whose number is within the interval.
* **Get all**: Get all runs regardless of the interval.
#### Conditions:
* **Include overlapping runs**: If unchecked only the runs that start AND stop
inside the time interval are shown. If checked all the runs that start OR stop
inside the interval are shown.
* **Only good runs**: select only the runs marked as good (good_run_flag == 1)
#### Analyzers:
* **Download**: download the selected runs from the WAGASCI run repository into
a local directory selected by the user (both borg and simple repositories
are supported)
* **Decoder**: decode the downloaded runs using the wgDecoder program
* **ADC histogram**: plot the ADC distribution channel by channel
* **BCID histogram**: plot the BCID distribution for the whole DIF
* **Spill Number Fixer**: fix the spill number for the decoded runs using the
wgFixSpillNumber program
* **Beam Summary Data**: integrate the BSD info into the decoded runs using the
wgBeamSummaryData program
* **Temperature**: integrate the temperature info into the decoded runs using
the wgTemperature program
* **ADC calibration**: insert PEU information in the decoded data TTree using
the wgAdcCalib program
* **Start analysis**: apply all the selected analyzers above to the downloaded
runs
#### Topology
* **Topology**: select which sub-detectors to download (not used for analysis)
### Run list
List of runs that meet the conditions in the query. You can click on the run
name to show more info about that particular run.
### Run info
Show all the info about one particular run.
### GUI customization
The GUI can be partially customized using the
[PyGubu](https://github.com/alejandroautalan/pygubu) program. PyGubu is
automatically installed by pip as a dependence of this package.
To customize the GUI, locate the package installation folder and the gui.ui
file inside it, open a terminal and
```shell script
mkdir -p $HOME/.wagasci
cp gui.ui $HOME/.wagasci/
pygubu-designer $HOME/.wagasci/gui.ui
```
The program tries to load the GUI file `gui.ui` in the same way as the
configuration file as explained above.
## WAGASCI spill history plotter
The ``history_plotter.py`` script is provided by this package. It creates various
history plots for the WAGASCI experiment. It is a CLI only script and all its
features can be controlled by command line arguments. However it also tries to
read the configuration file ``wagasci.ini`` for required arguments that are not
specified in the command line.
For further information, please consult the help message of the script:
```shell script
history_plotter.py --help
```
## MIDAS history to SQLite converter
To convert midas history files into SQLite format, please use the
``mhistory_sqlite_converter.py``, script. You need to have MIDAS installed
to run the script.
For further information, please consult the help message of the script:
```shell script
mhistory_sqlite_converter.py --help
```
## Calibration scripts
Many calibration scripts are provided by this package. They all take the raw
data as input and perform some calibration analysis on it. They depend
strongly on the [WAGASCI calibration software](https://git.t2k.org/wagasci_babymind/WagasciCalibration).
The documentation about how to use them is included in the [WAGASCI expert
shifter manual](https://ja.overleaf.com/7887288445tmqmnjknrynm).
## Other Python modules
List of all the Python modules provided by this package:
- `wagascianpy.analysis`
Contains modules for analysing the WAGASCI raw data, decoded data and slow
data.
- `wagascianpy.analysis.analysis` : Ctypes wrappers around
WagasciCalibration libraries.
- `wagascianpy.analysis.analyzer` : Takes the Ctypes wrappers and organize
them in a abstract factory design patter.
- `wagascianpy.analysis.beam_summary_data` : module to integrate the BSD
information inside the decoded WAGASCI data. It can be used as a
standalone program or as a analyzer of `wagascianpy.analysis.analyzer`.
- `wagascianpy.analysis.mhistory2sqlite` : module to convert many mhistory
files containing slow devices data into a SQLite database.
- `wagascianpy.analysis.spill` : factory design patter to generate
various spill objects. A spill object contains information about the spill
number. It is useful when fixing the spill number bugs or when
integrating the BSD information.
- `wagascianpy.database`
Modules to manage the WAGASCI run database and the BSD database.
- `wagascianpy.database.bsddb` : BSD database creation and access.
- `wagascianpy.database.db_record` : Virtual database record that is
inherited by the WAGASCI run record and BSD record.
- `wagascianpy.database.my_tinydb` : Virtual database build upon the
`tinydb` external module. It is inherited by the WAGASCI run database and
BSD database.
- `wagascianpy.database.wagascidb` : WAGASCI run database creation and access.
- `wagascianpy.plotting`
Modules to generate history plots (plots where the X axis is time).
- `wagascianpy.plotting.colors` : manage plot colors
- `wagascianpy.plotting.detector` : class to store the read TTrees for each
subdetector and to iterate over them.
- `wagascianpy.plotting.graph` : class to build a generic graph
- `wagascianpy.plotting.harvest` : class to harvest the needed data from the
TTrees. It returns the X points and Y points. It is implemented using the
strategy design pattern.
- `wagascianpy.plotting.marker` : class to plot time markers as the
maintenance days, WAGASCI runs start and stop time, electronics troubles
, etc...
- `wagascianpy.plotting.plotter` : class to plot the actual TMultiGraph
- `wagascianpy.plotting.topology` : class to specify which subdetectors to
plot and which not
- `wagascianpy.program`
Modules to generate a runnable program using the analyzers of
`wagascianpy.analysis.analyzer`. Implemented using the builder design
pattern.
- `wagascianpy.program.director` : director of the builder design pattern.
- `wagascianpy.program.program` : final program that is builded.
- `wagascianpy.program.program_builder` : builder that builds the program.
- `wagascianpy.pyrmod`
Interface for the Pyrame modules. Used only for DAQ.
- `wagascianpy.pyrmod.CCC` : control the CCC board
- `wagascianpy.pyrmod.pyrmod` : wrapper around generic Pyrame module
- `wagascianpy.utils`
Miscellaneous utilities.
- `wagascianpy.utils.change_bitstream` : module to change the configuration of
the SPIROC chip for all the chips in one go.
- `wagascianpy.utils.classproperty` : implement Python property for classes
and not only for instances
- `wagascianpy.utils.configuration` : parse the configuration file and
create a global configuration class
- `wagascianpy.utils.environment` : module to read the WAGASCI
environmental variables from a file or from the shell environment.
- `wagascianpy.utils.utils` : all other utilities are gathered here.
- `wagascianpy.viewer`
Module used by the WAGASCI database viewer GUI program. They are not
supposed to be used anywhere else.
- `wagascianpy.viewer.configuration` : fill the global configuration
- `wagascianpy.viewer.downloader` : download runs from remote repository
- `wagascianpy.viewer.parse_args` : parse the CLI arguments
- `wagascianpy.viewer.show_run_info` : build the 'run info' tab in the GUI
- `wagascianpy.viewer.topology` : little class to select which sub
-detectors are enabled and which not
- `wagascianpy.viewer.utils` - GUI utilities
## Troubleshooting
- If you get an error about SSH trying to set a different locale in the KEKCC
repository try the solution explained in this
[stackoverflow post](https://stackoverflow.com/questions/29609371/how-do-not-pass-locale-through-ssh).
Raw data
{
"_id": null,
"home_page": "https://git.t2k.org/wagasci_babymind/WagasciAnpy",
"name": "wagascianpy",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "WAGASCI ANPAN for Python",
"author": "Pintaudi Giorgio",
"author_email": "giorgio.pimpa@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/7b/34/d39d804227bc67d1763ebef9056db6d455a9a918984a5f8fddb7fb73c9dc/wagascianpy-0.3.10.tar.gz",
"platform": null,
"description": "# WAGASCI ANPAN for Python v0.3.10\n\nThis package contains all the Python modules used by the ANPAN\n framework of the WAGASCI/T2K experiment.\n\n## What is ANPAN?\n\nANPAN (Acquisition Networked Program for Accelerated Neutrinos) is a collective\n name to identify all the software used to acquire data and analyze it for\n the WAGASCI experiment. It is composed of many packages:\n - [Pyrame](http://llr.in2p3.fr/sites/pyrame/) (C/Python2)\n - [ANPAN DAQ](https://llrgit.in2p3.fr/online/wagasci) (C/Python2)\n - [WagasciCalibration](https://git.t2k.org/wagasci_babymind/WagasciCalibration) (C++11)\n - [WagasciMidas](https://git.t2k.org/wagasci_babymind/WagasciMidas) (C++11/JS)\n - [WagasciRunControl](https://git.t2k.org/wagasci_babymind/WagasciRunControl) (Python2)\n - [WagasciAnpy (this package)](https://git.t2k.org/wagasci_babymind/WagasciAnpy) (Python2/3)\n\n## Python compatibility\n\n - **Python2.7 or Python3.4+**\n\n This package is fully compatible with both Python2.7 and Python3.4+. It is\n tested usually with Python 3.8 and sporadically with other Python versions.\n \n However, I strongly suggest you to use Python 3 whenever possible because\n Python 2 has now reached end of life and many packages are starting to drop \n support for it. I am still trying to ensure Python 2 compatibility but I might \n drop it in the future for non DAQ related modules.\n\n## Dependencies\n\nMany functions of this package depend on ROOT\n\n - [ROOT](https://ph-root-2.cern.ch/)\n\nTo use the analysis programs, you need the WagasciCalibration package:\n\n - [WagasciCalibration](https://git.t2k.org/wagasci_babymind/WagasciCalibration)\n\nTo download and update the WAGASCI run database you may need the BorgBackup\n program:\n\n - [BorgBackup](https://borgbackup.readthedocs.io/en/stable/installation.html)\n\nThe ``pyrmod`` module depends on Pyrame. This module is used only by the DAQ\nso no need to install Pyrame on a non DAQ machine. \n\n### ROOT\n\nBe sure that the version of ROOT that you installed is compatible with\nyour Python version. The precompiled binaries for ROOT until version 6.22.0\nare usually only compatible with Python 2. From ROOT version 6.22.0 on,\nROOT should be compatible with both Python 2 and Python 3, so my suggestion\nis to just update your ROOT version.\n\n### WAGASCI calibration software (optional)\n\nTo analyze the raw data you need the WAGASCI calibration software.\nFor the installation instructions refer to \n [this page](https://git.t2k.org/wagasci_babymind/WagasciCalibration).\n\n## Installation (Linux)\n\nOpen a terminal and issue\n```shell script\npython3 -m pip install --upgrade --user wagascianpy\nor (recomended)\npython3 -m pip install --upgrade --user .\n```\n\n## Installation (MacOS)\n\nCurrently the support for MacOS is only partial. The SSH connection is broken\nbecause of some issues internal to the [Python paramiko\nmodule](http://www.paramiko.org/), so the program can be used in local mode\nonly. I tried to fix the issue but the problem is beyond my skills. Probably in\nthe following months the paramiko developers will come up with a solution. At\nthe time of writing, the current (broken) version of paramiko is 2.7.1. When a\nnew version of paramiko is released, try again to run the software and see what\nhappens.\n\nI have tested the program only in MacOS Mojave and MacOS Catalina. Both showed\nmore or less the same problems. You can try older versions of MacOS and might\nhave better luck.\n\n#### MacOS Mojave/Catalina\n\nBecause MacOS Mojave does not bundle Python3 we need to install it manually. The\neasiest way is by using the package manager brew. The following commands are\ntaken from this [webpage](https://installpython3.com/mac/).\n\nThe first step is to install the Apple's Xcode suite. Most probably you already\nhave it already installed, in that case you can skip this step. In your Terminal\napp, run the following command to install XCode and its command-line tools:\n\n```shell script\n$ xcode-select --install\n```\n\nNext install Homebrew by copy/pasting the following command into Terminal and\nthen type Enter:\n\n```shell script\n$ ruby -e \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)\"\n$ brew update\n```\n\nTo confirm Homebrew installed correctly, run this command:\n```shell script\n$ brew doctor\nYour system is ready to brew.\n```\n\nNow we can install the latest version of Python 3. Type the following command\ninto Terminal and press Enter:\n\n```shell script\n$ brew install python3\n```\n\nTo confirm which version of Python was installed, run the following command in\nTerminal:\n\n```shell script\n$ /usr/local/bin/python3 --version\nPython 3.7.7\n```\n\nMany functions of the WAGASCI database viewer use borgbackup. We can easily\ninstall it using brew.\n\n```shell script\n$ brew cask install borgbackup\n```\n\nFor other ways of installing borg refer to the\n[borg website](https://borgbackup.readthedocs.io/en/stable/installation.html).\n\nThen we can finally install the wagascianpy package\n\n```shell script\n$ python3 -m pip install --user --upgrade wagascianpy\n```\n\nIf you use MacOS Catalina continue reading the next section.\n\n#### MacOS Catalina - troubleshooting\n\n**MacOS Catalina comes with its own version of Python3. However\n[as stated in the Python website](https://www.python.org/download/mac/tcltk/)\nthe Apple-supplied Tcl/Tk 8.5 has serious bugs that can cause application \ncrashes.\n\nSo follow the same instructions as MacOS Mojave but also install tkinter:\n```shell script\n$ brew install tcl-tk\n```\n\n## Configuration\n\n### Environment parameters\n\nYou can easily configure all the parameters from the GUI itself (or from the\ncommand line if you run the programs from the shell). However, because\ninserting the same values over and over again can be tedius, you can modify the\nconfiguration file called `wagasci.ini`. This file is created in the\n`${HOME}/.wagasci` folder on the first execution of the program and filled with\nthe default values. You can just open and edit it with any text editor. The\noptions names and values are in one to one correlation with the GUI fields\nand CLI arguments.\n\nUpon start, the program tries to load the configuration file from the following\n directories in this order:\n- Environment : `${WAGASCI_CONFIGDIR}`\n- User : `${HOME}/.wagasci`\n- System : `/usr/local/wagasci`\n\nThe configuration file must be called `wagasci.ini`. Both the WAGASCI database\nviewer and the spill history plotter can use the same configuration file.\n\n### Good runs \n\nThe list of good runs and bad runs is hardcoded at the beginning of the \n`wagascianpy/database/wagascidb.py` file. If a run is completely bad, add it \nto the `_BAD_RUNS` list. If a run is bad only for a single detector, add it \nto the `_BAD_RUNS_X` where X is the name of the detector. After modifying \nthe bad run list, to apply it again, please rerun `wagasciviewer.py \n--rebuild-wagasci-database --batch-mode`, and then `wagasciviewer.py \n--apply-detector-flags --batch-mode`.\n\n### Markers\n\nWhen drawing the history plots, the trouble markers and maintenance day \nmarkers can be drawn along with the data. The start time and stop time of \nthese markers is hardcoded at the beginning of the \n`wagascianpy/plotting/marker.py` file.\n\n## WAGASCI database viewer\n\n\n\nThe following subsections explain the usage of the WAGASCI database viewer GUI\n program.\n\n### Run the program\n\nTo open the GUI program, just type in a terminal:\n\n```shell script\nwagasciviewer.py\n````\n\nIf you wish to run the program without a GUI in batch mode, just type in a\n terminal\n \n ```shell script\nwagasciviewer.py --batch-mode <other arguments>\n```\n\n### WAGASCI run repository\n\nThe word repository refers to the place where the runs are backed up. It might\nbe a simple repository where the runs are stored without encription (plain file\nand folders), or a borg repository where all the files are encripted. The\nrepository can be either local (same machine) or remote (remote machine\nreachable by SSH). In case of a borg repository it is necessary for borgbackup\nto be installed on both the local machine and the remote machine.\n\n* **Simple repository** : Unencripted repository\n* **Borg repository** : Borg repository\n* **Repository location** : For a remote repository the syntax is\n `<hostname>:<path of repository>`. The SSH configuration is taken from the\n user `~/.ssh/config` file by looking for `<hostname>`, so make sure that you\n can connect to the remote machine by just opening the terminal and `ssh\n <hostname>`. In case of local repository just insert the repository path. For\n more info about how to connect to KEKCC by SSH refer to the [WAGASCI expert\n shifter manual](https://ja.overleaf.com/7887288445tmqmnjknrynm)\n (Troubleshooting - KEKCC (8.6)).\n\n### WAGASCI run database\n\nLocation of the WAGASCI run database. The database is a file (usually named\n`wagascidb.db`) that contains some info about each run. If the update database\ncheckbox is checked, the software checks if the database is synchronized with\nthe repository (if the database describes the current state of the repository).\n\n* **Database location**: location of the database file. Usually it is the same\n as the repository. In case you specify a different location, enter the\n filename too. Leave as it is if in doubt.\n* **Update database**: If left unchecked the database is read as is. If checked,\n the runs that are present in both the repository and database the database are\n not updated and only the runs that are present in the repository but not in\n the database are added. Leave disabled if in doubt.\n* **Rebuild database**: If checked each run in the repository is\n inserted/overwritten into the database even if already present. Leave disabled\n if in doubt.\n* **WAGASCI library**: path to the directory containing the WAGASCI\n calibration software libraries. It is only used to update the database and\n analyze the raw data. Not needed if you only wish to visualize the database.\n \n### Beam Summary Data repository\n\nThe word repository refers to the place where the Beam Summary Data files are\nstored. The local repository can be an empty folder or a folder containing\nthe BSD files. If the remote repository field is left empty the program assumes\nthat the BSD files are already present in the local folder. If the remote\nrepository field is filled, the remote repository and the local repository are\nsynchronized using rsync.\n\n\n* **Local repository** : Location of BSD files in the local machine.\n* **Remote repository** : Location of the BSD files in the remote machine (the\n syntax is the same as the WAGASCI run repository). If left empty only the\n local repository is used.\n\n### Beam Summary Data database\n\nLocation of the BSD database. The database is a file (usually named `bsddb.db`)\nthat contains some info about each BSD file. If the update or rebuild database\ncheckbox is checked, the software checks if the database is synchronized with\nthe repository (if the database describes the current state of the repository).\n\n* **Database location**: location of the database file. Usually it is the same\n as the repository. In case you specify a different location, enter the\n filename too. Leave as it is if in doubt.\n* **Update database**: If left unchecked the database is read as is. If checked,\n the runs that are present in both the repository and database the database are\n not updated and only the runs that are present in the repository but not in\n the database are added. Leave disabled if in doubt.\n* **Rebuild database**: If checked each run in the repository is\n inserted/overwritten into the database even if already present. Leave disabled\n if in doubt.\n\n### Query\n\n#### Getters\n* **Get time interval**: Select an interval and the push the Get interval\n button. In the Run list frame will appear all the runs in that period.\n* **Get run interval**: Select all runs whose number is within the interval.\n* **Get all**: Get all runs regardless of the interval.\n\n#### Conditions:\n* **Include overlapping runs**: If unchecked only the runs that start AND stop\n inside the time interval are shown. If checked all the runs that start OR stop\n inside the interval are shown.\n* **Only good runs**: select only the runs marked as good (good_run_flag == 1)\n\n#### Analyzers:\n* **Download**: download the selected runs from the WAGASCI run repository into\n a local directory selected by the user (both borg and simple repositories\n are supported)\n* **Decoder**: decode the downloaded runs using the wgDecoder program\n* **ADC histogram**: plot the ADC distribution channel by channel\n* **BCID histogram**: plot the BCID distribution for the whole DIF\n* **Spill Number Fixer**: fix the spill number for the decoded runs using the\n wgFixSpillNumber program\n* **Beam Summary Data**: integrate the BSD info into the decoded runs using the\n wgBeamSummaryData program\n* **Temperature**: integrate the temperature info into the decoded runs using\n the wgTemperature program\n* **ADC calibration**: insert PEU information in the decoded data TTree using\n the wgAdcCalib program\n* **Start analysis**: apply all the selected analyzers above to the downloaded\n runs\n\n#### Topology\n* **Topology**: select which sub-detectors to download (not used for analysis)\n\n### Run list\n\nList of runs that meet the conditions in the query. You can click on the run\nname to show more info about that particular run.\n\n### Run info\n\nShow all the info about one particular run.\n\n### GUI customization\nThe GUI can be partially customized using the\n[PyGubu](https://github.com/alejandroautalan/pygubu) program. PyGubu is\n automatically installed by pip as a dependence of this package.\n\n To customize the GUI, locate the package installation folder and the gui.ui\n file inside it, open a terminal and\n ```shell script\nmkdir -p $HOME/.wagasci\ncp gui.ui $HOME/.wagasci/\npygubu-designer $HOME/.wagasci/gui.ui\n```\n\nThe program tries to load the GUI file `gui.ui` in the same way as the\n configuration file as explained above.\n\n## WAGASCI spill history plotter\n\nThe ``history_plotter.py`` script is provided by this package. It creates various\nhistory plots for the WAGASCI experiment. It is a CLI only script and all its\nfeatures can be controlled by command line arguments. However it also tries to\nread the configuration file ``wagasci.ini`` for required arguments that are not\nspecified in the command line.\n\nFor further information, please consult the help message of the script:\n\n```shell script\nhistory_plotter.py --help\n```\n\n## MIDAS history to SQLite converter\n\nTo convert midas history files into SQLite format, please use the\n ``mhistory_sqlite_converter.py``, script. You need to have MIDAS installed\n to run the script.\n \n For further information, please consult the help message of the script:\n\n```shell script\nmhistory_sqlite_converter.py --help\n```\n\n## Calibration scripts\n\nMany calibration scripts are provided by this package. They all take the raw\ndata as input and perform some calibration analysis on it. They depend\nstrongly on the [WAGASCI calibration software](https://git.t2k.org/wagasci_babymind/WagasciCalibration).\n\nThe documentation about how to use them is included in the [WAGASCI expert\n shifter manual](https://ja.overleaf.com/7887288445tmqmnjknrynm).\n\n## Other Python modules\n\nList of all the Python modules provided by this package:\n\n- `wagascianpy.analysis`\n\n Contains modules for analysing the WAGASCI raw data, decoded data and slow\n data.\n - `wagascianpy.analysis.analysis` : Ctypes wrappers around\n WagasciCalibration libraries.\n - `wagascianpy.analysis.analyzer` : Takes the Ctypes wrappers and organize\n them in a abstract factory design patter.\n - `wagascianpy.analysis.beam_summary_data` : module to integrate the BSD\n information inside the decoded WAGASCI data. It can be used as a\n standalone program or as a analyzer of `wagascianpy.analysis.analyzer`.\n - `wagascianpy.analysis.mhistory2sqlite` : module to convert many mhistory\n files containing slow devices data into a SQLite database.\n - `wagascianpy.analysis.spill` : factory design patter to generate\n various spill objects. A spill object contains information about the spill\n number. It is useful when fixing the spill number bugs or when\n integrating the BSD information.\n\n- `wagascianpy.database`\n\n Modules to manage the WAGASCI run database and the BSD database.\n\n - `wagascianpy.database.bsddb` : BSD database creation and access.\n - `wagascianpy.database.db_record` : Virtual database record that is\n inherited by the WAGASCI run record and BSD record.\n - `wagascianpy.database.my_tinydb` : Virtual database build upon the\n `tinydb` external module. It is inherited by the WAGASCI run database and\n BSD database.\n - `wagascianpy.database.wagascidb` : WAGASCI run database creation and access.\n\n- `wagascianpy.plotting`\n\n Modules to generate history plots (plots where the X axis is time).\n\n - `wagascianpy.plotting.colors` : manage plot colors\n - `wagascianpy.plotting.detector` : class to store the read TTrees for each\n subdetector and to iterate over them.\n - `wagascianpy.plotting.graph` : class to build a generic graph\n - `wagascianpy.plotting.harvest` : class to harvest the needed data from the\n TTrees. It returns the X points and Y points. It is implemented using the\n strategy design pattern.\n - `wagascianpy.plotting.marker` : class to plot time markers as the\n maintenance days, WAGASCI runs start and stop time, electronics troubles\n , etc...\n - `wagascianpy.plotting.plotter` : class to plot the actual TMultiGraph\n - `wagascianpy.plotting.topology` : class to specify which subdetectors to\n plot and which not\n\n- `wagascianpy.program`\n\n Modules to generate a runnable program using the analyzers of\n `wagascianpy.analysis.analyzer`. Implemented using the builder design\n pattern.\n\n - `wagascianpy.program.director` : director of the builder design pattern.\n - `wagascianpy.program.program` : final program that is builded.\n - `wagascianpy.program.program_builder` : builder that builds the program.\n\n- `wagascianpy.pyrmod`\n\n Interface for the Pyrame modules. Used only for DAQ.\n \n - `wagascianpy.pyrmod.CCC` : control the CCC board\n - `wagascianpy.pyrmod.pyrmod` : wrapper around generic Pyrame module\n\n- `wagascianpy.utils`\n\n Miscellaneous utilities.\n\n - `wagascianpy.utils.change_bitstream` : module to change the configuration of\n the SPIROC chip for all the chips in one go.\n - `wagascianpy.utils.classproperty` : implement Python property for classes\n and not only for instances\n - `wagascianpy.utils.configuration` : parse the configuration file and\n create a global configuration class\n - `wagascianpy.utils.environment` : module to read the WAGASCI\n environmental variables from a file or from the shell environment.\n - `wagascianpy.utils.utils` : all other utilities are gathered here.\n\n- `wagascianpy.viewer`\n\n Module used by the WAGASCI database viewer GUI program. They are not\n supposed to be used anywhere else.\n\n - `wagascianpy.viewer.configuration` : fill the global configuration\n - `wagascianpy.viewer.downloader` : download runs from remote repository\n - `wagascianpy.viewer.parse_args` : parse the CLI arguments\n - `wagascianpy.viewer.show_run_info` : build the 'run info' tab in the GUI\n - `wagascianpy.viewer.topology` : little class to select which sub\n -detectors are enabled and which not\n - `wagascianpy.viewer.utils` - GUI utilities\n\n## Troubleshooting\n\n - If you get an error about SSH trying to set a different locale in the KEKCC\nrepository try the solution explained in this\n[stackoverflow post](https://stackoverflow.com/questions/29609371/how-do-not-pass-locale-through-ssh).\n",
"bugtrack_url": null,
"license": "GPLv3",
"summary": "WAGASCI ANPAN for Python",
"version": "0.3.10",
"project_urls": {
"Homepage": "https://git.t2k.org/wagasci_babymind/WagasciAnpy"
},
"split_keywords": [
"wagasci",
"anpan",
"for",
"python"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3303398e6c6a0cb55b06d85acc3957f5dc171d3879a77ddb88f0c58a61a7b480",
"md5": "eabc958915dc40f338ebe76cdd7a90ea",
"sha256": "2609b822b2cb5c53d25ae2c2cfa636b90f2db64da1f37c64c63f9a322f2137a3"
},
"downloads": -1,
"filename": "wagascianpy-0.3.10-py3-none-any.whl",
"has_sig": false,
"md5_digest": "eabc958915dc40f338ebe76cdd7a90ea",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 168653,
"upload_time": "2024-11-04T09:02:56",
"upload_time_iso_8601": "2024-11-04T09:02:56.974643Z",
"url": "https://files.pythonhosted.org/packages/33/03/398e6c6a0cb55b06d85acc3957f5dc171d3879a77ddb88f0c58a61a7b480/wagascianpy-0.3.10-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7b34d39d804227bc67d1763ebef9056db6d455a9a918984a5f8fddb7fb73c9dc",
"md5": "a49f6c9fee470706c873ce9498eb3eb8",
"sha256": "f415e65a1606fe772a58f30f67832bcb114f90d9b914a11893e24f559096eb56"
},
"downloads": -1,
"filename": "wagascianpy-0.3.10.tar.gz",
"has_sig": false,
"md5_digest": "a49f6c9fee470706c873ce9498eb3eb8",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 159610,
"upload_time": "2024-11-04T09:02:59",
"upload_time_iso_8601": "2024-11-04T09:02:59.823241Z",
"url": "https://files.pythonhosted.org/packages/7b/34/d39d804227bc67d1763ebef9056db6d455a9a918984a5f8fddb7fb73c9dc/wagascianpy-0.3.10.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-04 09:02:59",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "wagascianpy"
}
JSON