sif_parser
============
### A small package to read Andor Technology Multi-Channel files.
**This package was renamed from [sif_reader](https://www.github.com/fujiisoup/sif_reader).**
## Install
This package can be installed via `pip`
```bash
pip install sif_parser
```
or if you have `git` installed in your system, you can also do
```bash
pip install git+https://www.github.com/fujiisoup/sif_parser
```
## Basic usage
It provides the following methods,
### `sif_parser.np_open`
Read '.sif' file and return as a `np.ndarray` for image and an `OrderedDict` for metadata.
```python
>>> import sif_parser
>>> data, info = sif_parser.np_open('/path/to/file.sif')
>>> data
array([[[887. , 881.25, 875.65, ..., 866.05, 870. ],
[905.6 , 872.7 , 900.7 , ..., 871.4 , 866.45],
...,
[885.6 , 879.4 , 873.5 , ..., 883.6 , 877. ],
[879.4 , 873. , 880.5 , ..., 881. , 867. ]]],
dtype=float32)
>>> info
OrderedDict([('SifVersion', 65559),
('ExperimentTime', 1254330082),
('DetectorTemperature', -100.0),
...
])
```
If your calibration data is included in the file, this will be included as
`info['Calibration_data']` or `info['Calibration_data_for_frame_1']`.
#### Lazy load
If your data is very big but you are only interested in a certain part of the file, you can use `lazy` load feature.
```python
>>> data, info = sif_parser.np_open('path/to/file', lazy='memmap') # <-- it only reads the header.
>>> data.shape # <-- we know the shape
(1900, 74, 84)
>>> da = data[10] # <-- we can even index the data, BEFORE actually reading the file
>>> np.array(da) # <-- Read only the 10th frame and store it into the memory
```
We can use either `lazy='memmap'` and `lazy='dask'`.
With `lazy='memmap'`, we use [`np.memmap`](https://numpy.org/doc/stable/reference/generated/numpy. memmap.html), where we create an off-memory data that points the `sif` file.
With`lazy='dask'`, `dask.Array` will be returned.
See [`dask`](https://www.dask.org/) for the details. For this option, `dask` must be installed in your system.
### `sif_parser.xr_open('/path/to/file.sif')`:
**`xarray` must be installed to use this method.**
Read 'sif' file and return as a `xr.DataArray`.
The metadata is stored in `xr.DataArray.attrs`.
The calibration data and timestamps are stored as coordinates.
`xarray` is a very useful package to handle multi-dimensional arrays with metadata.
See [xarray project](http://xarray.pydata.org) for the details.
```python
>>> sif_parser.xr_open('testings/examples/image.sif')
<xarray.DataArray (Time: 1, height: 512, width: 512)>
array([[[887. , 881.25, 875.65, ..., 866.05, 870. ],
[905.6 , 872.7 , 900.7 , ..., 871.4 , 866.45],
[922.6 , 883.95, 899. , ..., 864.6 , 864.8 ],
...,
[880.65, 857.95, 883.55, ..., 866. , 875.55],
[885.6 , 879.4 , 873.5 , ..., 883.6 , 877. ],
[879.4 , 873. , 880.5 , ..., 881. , 867. ]]],
dtype=float32)
Coordinates:
* Time (Time) float64 0.0
Dimensions without coordinates: height, width
Attributes:
SifVersion: 65559
ExperimentTime: 1254330082
DetectorTemperature: -100.0
...
```
#### Lazy load
Lazy load is also possible for `xr_open`. To do so, just pass either `lazy='memmap'` or `lazy='dask'`.
### `sif_parser.np_spool_open('/path/to/spool_files')`:
Read from a directory the binary files and metadata generated via spooling and return a np.array.
Spooling acquisition save your data directly on disk when reading from your camera. When spooling acquisition is enabled, a directory is created in your PC and the data is written directly on the hard disk as it is being acquired (see the Andor SDK Manual for more details).
Spooling acquisition normally generates the following files by default and must be present in the directory:
- 1 file with the extension `*.sifx`. This is the header of the file containing the metadata. You could also read it by using the method `sif_parser.np_open('/path/to/my_file.sifx', ignore_corrupt=True)`.
- 1 file with the extension `*.ini`. This file contains information on the image format such as number of pixels by row (AOIWidth) number of rows and (AOIHeight),and padding bytes (AOIStride), pixel encoding, etc. (See the Andor SDK manual for more details).
- 1 or set of files with the extension `*spool.dat` containing the actual image data as binary files.
```python
>>> data, info = sif_parser.np_spool_open('/path/to/spool_files')
>>> data
array([[[2873, 2861, 2876, ..., 4016, 4185, 4086],
[2846, 2730, 2915, ..., 4101, 4136, 4290],
...,
[8269, 8247, 8554, ..., 4177, 3988, 4072],
[8332, 8224, 9474, ..., 4112, 4056, 4124]]],
dtype=uint32),
>>> info
OrderedDict([('SifVersion', 65567),
('ExperimentTime', 1688045153),
('DetectorTemperature', 0.0),
...
])
```
## Utils
### `sif_parser.utils.extract_calibration`
The `Calibration_data` entry of `info` contains coefficients of a cubic
polynomial used to calculate the wavelengths of an image.
To facilitate this `sif_parser.utils` contains the `extract_calibration`
function, which returns the wavelength of each pixel.
```python
data, info = sif_parser.np_open('path/to/file.sif')
wavelengths = sif_parser.utils.extract_calibration(info)
```
### `sif_parser.utils.parse`
Used to parse a .sif file into a 2 column numpy array as wavelengths and counts.
```python
import pandas as pd
import sif_parser
# parse the 'my_pl.sif' file
(data, info) = sif_parser.utils.parse('my_pl.sif')
# place data into a pandas Series
df = pd.Series(data[:, 1], index = data[:, 0])
```
## CLI
Installs a command line interface (CLI) named `sif_parser` that can be used to
convert .sif files to .csv.
Convert all .sif files in the current directory to .csv.
```bash
sif_parser
```
Convert all .sif files ending in `pl` in the current directly into a single .csv.
```bash
sif_parser --join *pl.sif
```
## Use as a plugin for PIL
**NOTE!! This feature was removed.**
See the issue [#7](https://github.com/fujiisoup/sif_reader/issues/7)
Previously, we have provided a plugin for PIL,
```python
from PIL import image
import sif_parser.plugin
I = Image.open('/path/to/file.sif')
```
## History
This plugin is originally developed by [soemraws](https://github.com/soemraws)
based on Marcel Leutenegger's MATLAB script.
## Current status
Andor has changed `sif` format for many times.
Although I have tested this package with as many kinds of `sif` files as I have
(the test suit is always checking the compatibility, as the badge above shows),
it might be still incompatible with your particular `sif` file.
If your file cannot be read by this script,
please raise an issue in github.
If you send me your file, I can add your file into the test suit
(I have a private repo in order to keep your sif file private).
Contribution is also very welcome.
## License of original MATLAB script
Copyright (c) 2006, Marcel Leutenegger
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution
* Neither the name of the Ecole Polytechnique Fédérale de Lausanne, Laboratoire d'Optique Biomédicale nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
Raw data
{
"_id": null,
"home_page": "https://github.com/fujiisoup/sif_parser",
"name": "sif-parser",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "imaging, Andor",
"author": "Keisuke Fujii",
"author_email": "fujii@me.kyoto-u.ac.jp",
"download_url": "https://files.pythonhosted.org/packages/44/38/73d1d454726fdf873362f5a8ab0cca1cfe927a0ce5761ffe86f69b35e0c7/sif_parser-0.3.5.tar.gz",
"platform": null,
"description": "sif_parser\r\n============\r\n\r\n\r\n\r\n### A small package to read Andor Technology Multi-Channel files.\r\n\r\n\r\n**This package was renamed from [sif_reader](https://www.github.com/fujiisoup/sif_reader).**\r\n\r\n\r\n## Install\r\n\r\nThis package can be installed via `pip`\r\n\r\n```bash\r\npip install sif_parser\r\n```\r\n\r\nor if you have `git` installed in your system, you can also do\r\n\r\n```bash\r\npip install git+https://www.github.com/fujiisoup/sif_parser\r\n```\r\n\r\n## Basic usage\r\n\r\n\r\nIt provides the following methods,\r\n\r\n### `sif_parser.np_open`\r\n\r\nRead '.sif' file and return as a `np.ndarray` for image and an `OrderedDict` for metadata.\r\n\r\n```python\r\n>>> import sif_parser\r\n>>> data, info = sif_parser.np_open('/path/to/file.sif')\r\n>>> data\r\narray([[[887. , 881.25, 875.65, ..., 866.05, 870. ],\r\n [905.6 , 872.7 , 900.7 , ..., 871.4 , 866.45],\r\n ...,\r\n [885.6 , 879.4 , 873.5 , ..., 883.6 , 877. ],\r\n [879.4 , 873. , 880.5 , ..., 881. , 867. ]]],\r\n dtype=float32)\r\n>>> info\r\nOrderedDict([('SifVersion', 65559),\r\n ('ExperimentTime', 1254330082),\r\n ('DetectorTemperature', -100.0),\r\n ...\r\n ])\r\n```\r\n\r\nIf your calibration data is included in the file, this will be included as\r\n`info['Calibration_data']` or `info['Calibration_data_for_frame_1']`.\r\n\r\n#### Lazy load\r\nIf your data is very big but you are only interested in a certain part of the file, you can use `lazy` load feature.\r\n\r\n```python\r\n>>> data, info = sif_parser.np_open('path/to/file', lazy='memmap') # <-- it only reads the header.\r\n>>> data.shape # <-- we know the shape\r\n (1900, 74, 84)\r\n>>> da = data[10] # <-- we can even index the data, BEFORE actually reading the file \r\n>>> np.array(da) # <-- Read only the 10th frame and store it into the memory\r\n```\r\n\r\nWe can use either `lazy='memmap'` and `lazy='dask'`. \r\nWith `lazy='memmap'`, we use [`np.memmap`](https://numpy.org/doc/stable/reference/generated/numpy. memmap.html), where we create an off-memory data that points the `sif` file.\r\nWith`lazy='dask'`, `dask.Array` will be returned. \r\nSee [`dask`](https://www.dask.org/) for the details. For this option, `dask` must be installed in your system.\r\n\r\n\r\n### `sif_parser.xr_open('/path/to/file.sif')`:\r\n\r\n**`xarray` must be installed to use this method.**\r\n\r\nRead 'sif' file and return as a `xr.DataArray`.\r\nThe metadata is stored in `xr.DataArray.attrs`.\r\nThe calibration data and timestamps are stored as coordinates.\r\n\r\n`xarray` is a very useful package to handle multi-dimensional arrays with metadata.\r\nSee [xarray project](http://xarray.pydata.org) for the details.\r\n\r\n```python\r\n>>> sif_parser.xr_open('testings/examples/image.sif')\r\n<xarray.DataArray (Time: 1, height: 512, width: 512)>\r\narray([[[887. , 881.25, 875.65, ..., 866.05, 870. ],\r\n [905.6 , 872.7 , 900.7 , ..., 871.4 , 866.45],\r\n [922.6 , 883.95, 899. , ..., 864.6 , 864.8 ],\r\n ...,\r\n [880.65, 857.95, 883.55, ..., 866. , 875.55],\r\n [885.6 , 879.4 , 873.5 , ..., 883.6 , 877. ],\r\n [879.4 , 873. , 880.5 , ..., 881. , 867. ]]],\r\n dtype=float32)\r\nCoordinates:\r\n * Time (Time) float64 0.0\r\nDimensions without coordinates: height, width\r\nAttributes:\r\n SifVersion: 65559\r\n ExperimentTime: 1254330082\r\n DetectorTemperature: -100.0\r\n ...\r\n```\r\n\r\n#### Lazy load\r\nLazy load is also possible for `xr_open`. To do so, just pass either `lazy='memmap'` or `lazy='dask'`.\r\n\r\n### `sif_parser.np_spool_open('/path/to/spool_files')`:\r\n\r\nRead from a directory the binary files and metadata generated via spooling and return a np.array. \r\nSpooling acquisition save your data directly on disk when reading from your camera. When spooling acquisition is enabled, a directory is created in your PC and the data is written directly on the hard disk as it is being acquired (see the Andor SDK Manual for more details).\r\n\r\nSpooling acquisition normally generates the following files by default and must be present in the directory:\r\n- 1 file with the extension `*.sifx`. This is the header of the file containing the metadata. You could also read it by using the method `sif_parser.np_open('/path/to/my_file.sifx', ignore_corrupt=True)`.\r\n- 1 file with the extension `*.ini`. This file contains information on the image format such as number of pixels by row (AOIWidth) number of rows and (AOIHeight),and padding bytes (AOIStride), pixel encoding, etc. (See the Andor SDK manual for more details).\r\n- 1 or set of files with the extension `*spool.dat` containing the actual image data as binary files.\r\n\r\n```python\r\n>>> data, info = sif_parser.np_spool_open('/path/to/spool_files')\r\n\r\n>>> data\r\narray([[[2873, 2861, 2876, ..., 4016, 4185, 4086],\r\n [2846, 2730, 2915, ..., 4101, 4136, 4290],\r\n ...,\r\n [8269, 8247, 8554, ..., 4177, 3988, 4072],\r\n [8332, 8224, 9474, ..., 4112, 4056, 4124]]], \r\n dtype=uint32),\r\n>>> info\r\nOrderedDict([('SifVersion', 65567),\r\n ('ExperimentTime', 1688045153),\r\n ('DetectorTemperature', 0.0),\r\n ...\r\n ])\r\n```\r\n## Utils\r\n\r\n### `sif_parser.utils.extract_calibration`\r\nThe `Calibration_data` entry of `info` contains coefficients of a cubic\r\npolynomial used to calculate the wavelengths of an image.\r\nTo facilitate this `sif_parser.utils` contains the `extract_calibration`\r\nfunction, which returns the wavelength of each pixel.\r\n\r\n```python\r\ndata, info = sif_parser.np_open('path/to/file.sif')\r\nwavelengths = sif_parser.utils.extract_calibration(info)\r\n```\r\n\r\n### `sif_parser.utils.parse`\r\nUsed to parse a .sif file into a 2 column numpy array as wavelengths and counts.\r\n\r\n```python\r\nimport pandas as pd\r\nimport sif_parser\r\n\r\n\r\n# parse the 'my_pl.sif' file\r\n(data, info) = sif_parser.utils.parse('my_pl.sif')\r\n\r\n# place data into a pandas Series\r\ndf = pd.Series(data[:, 1], index = data[:, 0])\r\n```\r\n\r\n## CLI\r\n\r\nInstalls a command line interface (CLI) named `sif_parser` that can be used to\r\nconvert .sif files to .csv.\r\n\r\nConvert all .sif files in the current directory to .csv.\r\n```bash\r\nsif_parser\r\n```\r\n\r\nConvert all .sif files ending in `pl` in the current directly into a single .csv.\r\n```bash\r\nsif_parser --join *pl.sif\r\n```\r\n\r\n## Use as a plugin for PIL\r\n\r\n**NOTE!! This feature was removed.**\r\nSee the issue [#7](https://github.com/fujiisoup/sif_reader/issues/7)\r\n\r\nPreviously, we have provided a plugin for PIL,\r\n\r\n```python\r\nfrom PIL import image\r\nimport sif_parser.plugin\r\n\r\nI = Image.open('/path/to/file.sif')\r\n```\r\n\r\n## History\r\n\r\nThis plugin is originally developed by [soemraws](https://github.com/soemraws)\r\nbased on Marcel Leutenegger's MATLAB script.\r\n\r\n\r\n## Current status\r\n\r\nAndor has changed `sif` format for many times.\r\nAlthough I have tested this package with as many kinds of `sif` files as I have\r\n(the test suit is always checking the compatibility, as the badge above shows),\r\nit might be still incompatible with your particular `sif` file.\r\n\r\nIf your file cannot be read by this script,\r\nplease raise an issue in github.\r\nIf you send me your file, I can add your file into the test suit\r\n(I have a private repo in order to keep your sif file private).\r\n\r\nContribution is also very welcome.\r\n\r\n\r\n## License of original MATLAB script\r\n\r\nCopyright (c) 2006, Marcel Leutenegger\r\nAll rights reserved.\r\n\r\nRedistribution and use in source and binary forms, with or without\r\nmodification, are permitted provided that the following conditions are\r\nmet:\r\n* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.\r\n* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution\r\n* Neither the name of the Ecole Polytechnique F\u00e9d\u00e9rale de Lausanne, Laboratoire d'Optique Biom\u00e9dicale nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.\r\n\r\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\"\r\nAND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\r\nIMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE\r\nARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE\r\nLIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR\r\nCONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF\r\nSUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS\r\nINTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN\r\nCONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)\r\nARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE\r\nPOSSIBILITY OF SUCH DAMAGE.\r\n",
"bugtrack_url": null,
"license": "BSD 3-clause",
"summary": "Python package to read Andor sif file.",
"version": "0.3.5",
"project_urls": {
"Homepage": "https://github.com/fujiisoup/sif_parser"
},
"split_keywords": [
"imaging",
" andor"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "443873d1d454726fdf873362f5a8ab0cca1cfe927a0ce5761ffe86f69b35e0c7",
"md5": "fbe57c8e82a664df4a76345f009a510a",
"sha256": "2d48d4df74222de1024d30574cac4a84174a204268ad47ca28d453baf75e6373"
},
"downloads": -1,
"filename": "sif_parser-0.3.5.tar.gz",
"has_sig": false,
"md5_digest": "fbe57c8e82a664df4a76345f009a510a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 18536,
"upload_time": "2024-08-14T16:36:53",
"upload_time_iso_8601": "2024-08-14T16:36:53.654063Z",
"url": "https://files.pythonhosted.org/packages/44/38/73d1d454726fdf873362f5a8ab0cca1cfe927a0ce5761ffe86f69b35e0c7/sif_parser-0.3.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-14 16:36:53",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "fujiisoup",
"github_project": "sif_parser",
"travis_ci": true,
"coveralls": false,
"github_actions": true,
"lcname": "sif-parser"
}
JSON
