# twixtools
[](https://github.com/pehses/twixtools/actions/workflows/test_action.yml)
### Purpose
twixtools provide reading and limited writing capability of Siemens MRI raw data files (.dat). In addition, it also includes the compression utility twixzip (see further below for a description).
### Requirements
The tool works under Python 3.7 with the following package installed:
* numpy ≥ 1.17.3
### Installation
Use [pip](https://pip.pypa.io/en/stable/) to install the twixtools.
```bash
pip install gt-twixtools
```
### Demo code
A jupyter notebook that demonstrates the basic functionality of the `read_twix`, `map_twix`, and `write_twix` tools can be found in `demo/recon_example.ipynb`.
## read_twix: "low-level" access to twix data
The raw data file can be parsed using the read_twix function:
```python
import twixtools
multi_twix = twixtools.read_twix(filename)
```
The function returns a list of individual measurements (of length >=1). The last measurement usually corresponds to the imaging scan, earlier measurements often include calibration data. Each measurement contains a python dict() with the following entries:
* 'mdb': measurement data divided into blocks (return type: list)
* 'hdr': dict of parsed protocol header strings (each dict element contains another dict with protocol information)
* 'hdr_str': dict of original protocol header strings (divided into different protocol types)
- note that this is the protocol information that is used for twix file writing (by `write_twix`), so make sure to make necessary adjustments here and not in ['hdr']
* ('raidfile_hdr': required for twix file writing, otherwise of little importance)
Each invididual 'mdb' in the list of mdbs consists of a data and a header (line counters and such) part, which can be accessed as follows:
```python
mdb = multi_twix[-1]['mdb'][0] # first mdb element of last measurement
mdb.data # data of first mdb (may or may not be imaging data)
mdb.mdh # full miniheader information stored as a numpy dtype object
```
Different data types can be distinguished by returning a list of active flags, or by directly checking whether the data is assumed to be from an imaging scan (and not from a calibration scan such as a phase correction scan or a noise measurement):
```python
mdb.get_active_flags() # get all active MDH flags
mdb.is_image_scan() # check if this an image scan (True or False)
```
Line Counters can be accessed as follows:
```python
mdb.cLin # returns line number
mdb.cPar # returns partition number
mdb.c<tab> # with line completion enabled, this should give you a list of all counters
```
The full minidata header (mdh) information is stored in a `mdb.mdh` special numpy dtype object. You can print a list of its entry names by printing `mdb.mdh.dtype.names`.
### Example code
```python
import numpy as np
import twixtools
# read all image data from file
def read_image_data(filename):
out = list()
for mdb in twixtools.read_twix(filename)[-1]['mdb']:
if mdb.is_image_scan():
out.append(mdb.data)
return np.asarray(out) # 3D numpy array [acquisition_counter, n_channel, n_column]
# read image data from list of mdbs and sort into 3d k-space (+ coil dim.)
def import_kspace(mdb_list)
image_mdbs = []
for mdb in mdb_list:
if mdb.is_image_scan():
image_mdbs.append(mdb)
n_line = 1 + max([mdb.cLin for mdb in image_mdbs])
n_part = 1 + max([mdb.cPar for mdb in image_mdbs])
n_channel, n_column = image_mdbs[0].data.shape
out = np.zeros([n_part, n_line, n_channel, n_column], dtype=np.complex64)
for mdb in image_mdbs:
# '+=' takes care of averaging, but careful in case of other counters (e.g. echoes)
out[mdb.cPar, mdb.cLin] += mdb.data
return out # 4D numpy array [n_part, n_line, n_channel, n_column]
```
### Extract the PMU data
The physiological data (PMU) can be extracted from the twix-file with the following command:
```python
import twixtools
multi_twix = twixtools.read_twix(filename, parse_pmu=True, parse_data=False)
```
The pmu data will be stored in `multi_twix[scan_r]['pmu']` as dict where every key corresponds to a different physiological sinal:
```python
# get the first ecg channel
ecg1 = multi_twix[-1]['pmu']['ecg1']
# get the respiratory motion
resp1 = multi_twix[-1]['pmu']['resp1']
# get the pulse plethysmograph
pulse = multi_twix[-1]['pmu']['pulse']
# get the ECG learning data
ecg1_learn = multi_twix[-1]['pmu']['learn']['ecg1']
```
The pmu data can be written into a file with:
```
twixtools.write_pmu(multi_twix[scan_nr], outfile)
```
## map_twix: "high level" access to twix data
`map_twix` is a high-level function that takes the data obtained from `read_twix` (in the form of `Mdb` objects), and maps it to multi-dimensional "k-space" arrays. These `twix_array` objects are generated for different data types (image/noise adjust/phase-correction/... scan) and can be accessed with `numpy.ndarray` array-slicing syntax.
Optional flags control additional feature and also have an impact on size and shape of the multidimensional arrays. The following flags are currently available (stored in the `flags` dict within each `twix_array` object):
* `average`: dict of bools that determines which dimensions should be averaged.
* `squeeze_ave_dims`: bool that determines whether averaged dimensions should be removed/squeezed from the array's shape.
* `remove_os`: oversampling removal. Reduces the number of columns by a factor of two.
* `regrid`: bool that controls ramp-sampling regridding (if applicable)
* `skip_empty_lead`: skips to first line & partition that is found in mdb list (e.g. if first line counter is 10, the output array starts at line counter 10).
* `zf_missing_lines`: zero-fill k-space to include lines and partitions that are higher than the maximum counter found in the mdb list, but are still within the k-space matrix according to the twix header.
For example code, please look at the `demo/recon_example.ipynb` jupyter file.
## twixzip Compression Utility
### Purpose
twixzip is a Python based command line tool for Siemens MRI raw data compression. Following compression methods can be selected via the command line:
* Oversampling removal
* Lossy floating point compression using the zfp library
* Single coil compression (scc) based on singular value decomposition (SVD)
* Geometric coil compression (gcc) based on SVD
* Optionally FID navigators can be removed
Before applying the selected compression method(s), lossless compression (gzip) is applied to the header and meta data information which is then added to a hdf5 file. All additional meta information necessary for decompression (e.g. coil compression matrices) are also stored in the hdf5 file.
### Additional Requirements
* pyzfp ≥ 0.3.1
* pytables ≥ 3.6.1
The pyzfp and pytables libraries can be installed via pip:
pip install pyzfp
pip install tables
### Usage
Executing the command twixzip</span>.py in an open terminal gives an overview of all possible arguments. Optional arguments are:
-h: help
-d: decompress data
Input and output directories & filenames are required arguments that can be selected via:
-i infile: input file
-o outfile: output file
In the compression mode the input file should be an MRI raw data file, in the decompression mode (`-d`) it should be the hdf5 file containing the compressed data. The output file is then an hdf5 file (compression mode) or an MRI raw data file (decompression mode).
Compression methods can be selected via:
--remove_fidnav: removes FID navigators
--remove_os: removes oversampling
--scc -n NCC: single coil compression (SCC) - keep NCC virtual coils
--scc -t CC_TOL: SCC - number of coils is calculated with a tolerance for the singular values
--scc_bart -n NCC: SCC using BART
--gcc -n NCC: geometric coil compression (GCC) - keep NCC virtual coils
--gcc -t CC_TOL: GCC - number of coils is calculated with a tolerance for the singular values
--gcc_bart -n NCC: GCC using the Berkeley Advanved Reconstruction Toolbox (BART) [1]
--zfp --zfp_tol ZFP_TOL: floating point compression with ZFP_TOL tolerance
--zfp --zfp_prec ZFP_PREC: floating point compression with ZFP_PREC precision (not recommended)
The optional argument `--testmode` can be used to automatically decompress the data after compression. The created decompressed MRI raw data filename contains the selected compression method. The option `--profile` can be used to profile the compression code.
[1] BART Toolbox for Computational Magnetic Resonance Imaging, DOI: 10.5281/zenodo.592960
## Acknowledgements
The protocol header parsing code originates from William Clarke's excellent pymapvbvd project (https://github.com/wexeee/pymapvbvd).
Raw data
{
"_id": null,
"home_page": "https://github.com/pehses/twixtools",
"name": "gt-twixtools",
"maintainer": "",
"docs_url": null,
"requires_python": "<4,>=3.7",
"maintainer_email": "",
"keywords": "twix,,siemens,,mri",
"author": "Philipp Ehses (modified by Martin Buehrer)",
"author_email": "philipp.ehses@dzne.de (martin.buehrer@gyrotools.com)",
"download_url": "https://files.pythonhosted.org/packages/01/c7/fedc3d9507c647fea7e775ac6917363dd4d3739bf2bee1cb52d4360c8792/gt-twixtools-1.0.1.tar.gz",
"platform": null,
"description": "# twixtools\n\n[](https://github.com/pehses/twixtools/actions/workflows/test_action.yml)\n\n### Purpose\n\ntwixtools provide reading and limited writing capability of Siemens MRI raw data files (.dat). In addition, it also includes the compression utility twixzip (see further below for a description).\n\n\n### Requirements\n\nThe tool works under Python 3.7 with the following package installed:\n\n* numpy ≥ 1.17.3\n\n\n### Installation\n\nUse [pip](https://pip.pypa.io/en/stable/) to install the twixtools.\n\n```bash\npip install gt-twixtools\n```\n\n### Demo code\n\nA jupyter notebook that demonstrates the basic functionality of the `read_twix`, `map_twix`, and `write_twix` tools can be found in `demo/recon_example.ipynb`.\n\n\n## read_twix: \"low-level\" access to twix data\n\nThe raw data file can be parsed using the read_twix function:\n\n```python\nimport twixtools\nmulti_twix = twixtools.read_twix(filename)\n```\n\nThe function returns a list of individual measurements (of length >=1). The last measurement usually corresponds to the imaging scan, earlier measurements often include calibration data. Each measurement contains a python dict() with the following entries:\n\n* 'mdb': measurement data divided into blocks (return type: list)\n* 'hdr': dict of parsed protocol header strings (each dict element contains another dict with protocol information)\n* 'hdr_str': dict of original protocol header strings (divided into different protocol types)\n - note that this is the protocol information that is used for twix file writing (by `write_twix`), so make sure to make necessary adjustments here and not in ['hdr']\n* ('raidfile_hdr': required for twix file writing, otherwise of little importance)\n\n\nEach invididual 'mdb' in the list of mdbs consists of a data and a header (line counters and such) part, which can be accessed as follows:\n\n```python\nmdb = multi_twix[-1]['mdb'][0] # first mdb element of last measurement\nmdb.data # data of first mdb (may or may not be imaging data)\nmdb.mdh # full miniheader information stored as a numpy dtype object\n ```\n\nDifferent data types can be distinguished by returning a list of active flags, or by directly checking whether the data is assumed to be from an imaging scan (and not from a calibration scan such as a phase correction scan or a noise measurement):\n\n```python\nmdb.get_active_flags() # get all active MDH flags\nmdb.is_image_scan() # check if this an image scan (True or False)\n```\n\nLine Counters can be accessed as follows:\n```python\nmdb.cLin # returns line number\nmdb.cPar # returns partition number\nmdb.c<tab> # with line completion enabled, this should give you a list of all counters\n```\n\nThe full minidata header (mdh) information is stored in a `mdb.mdh` special numpy dtype object. You can print a list of its entry names by printing `mdb.mdh.dtype.names`.\n\n\n### Example code\n```python\nimport numpy as np\nimport twixtools\n\n# read all image data from file\ndef read_image_data(filename):\n out = list()\n for mdb in twixtools.read_twix(filename)[-1]['mdb']:\n if mdb.is_image_scan():\n out.append(mdb.data)\n return np.asarray(out) # 3D numpy array [acquisition_counter, n_channel, n_column]\n\n\n# read image data from list of mdbs and sort into 3d k-space (+ coil dim.)\ndef import_kspace(mdb_list)\n image_mdbs = []\n for mdb in mdb_list:\n if mdb.is_image_scan():\n image_mdbs.append(mdb)\n\n n_line = 1 + max([mdb.cLin for mdb in image_mdbs])\n n_part = 1 + max([mdb.cPar for mdb in image_mdbs])\n n_channel, n_column = image_mdbs[0].data.shape\n\n out = np.zeros([n_part, n_line, n_channel, n_column], dtype=np.complex64)\n for mdb in image_mdbs:\n # '+=' takes care of averaging, but careful in case of other counters (e.g. echoes)\n out[mdb.cPar, mdb.cLin] += mdb.data\n\n return out # 4D numpy array [n_part, n_line, n_channel, n_column]\n```\n\n### Extract the PMU data\nThe physiological data (PMU) can be extracted from the twix-file with the following command:\n\n```python\nimport twixtools\nmulti_twix = twixtools.read_twix(filename, parse_pmu=True, parse_data=False)\n```\n\nThe pmu data will be stored in `multi_twix[scan_r]['pmu']` as dict where every key corresponds to a different physiological sinal:\n\n```python\n# get the first ecg channel\necg1 = multi_twix[-1]['pmu']['ecg1']\n\n# get the respiratory motion\nresp1 = multi_twix[-1]['pmu']['resp1']\n\n# get the pulse plethysmograph\npulse = multi_twix[-1]['pmu']['pulse']\n\n# get the ECG learning data\necg1_learn = multi_twix[-1]['pmu']['learn']['ecg1']\n```\n\nThe pmu data can be written into a file with:\n```\ntwixtools.write_pmu(multi_twix[scan_nr], outfile)\n```\n\n## map_twix: \"high level\" access to twix data\n`map_twix` is a high-level function that takes the data obtained from `read_twix` (in the form of `Mdb` objects), and maps it to multi-dimensional \"k-space\" arrays. These `twix_array` objects are generated for different data types (image/noise adjust/phase-correction/... scan) and can be accessed with `numpy.ndarray` array-slicing syntax.\n\n\nOptional flags control additional feature and also have an impact on size and shape of the multidimensional arrays. The following flags are currently available (stored in the `flags` dict within each `twix_array` object):\n * `average`: dict of bools that determines which dimensions should be averaged.\n * `squeeze_ave_dims`: bool that determines whether averaged dimensions should be removed/squeezed from the array's shape.\n * `remove_os`: oversampling removal. Reduces the number of columns by a factor of two.\n * `regrid`: bool that controls ramp-sampling regridding (if applicable)\n * `skip_empty_lead`: skips to first line & partition that is found in mdb list (e.g. if first line counter is 10, the output array starts at line counter 10).\n * `zf_missing_lines`: zero-fill k-space to include lines and partitions that are higher than the maximum counter found in the mdb list, but are still within the k-space matrix according to the twix header.\n\nFor example code, please look at the `demo/recon_example.ipynb` jupyter file.\n\n## twixzip Compression Utility\n\n### Purpose\n\ntwixzip is a Python based command line tool for Siemens MRI raw data compression. Following compression methods can be selected via the command line:\n\n* Oversampling removal\n* Lossy floating point compression using the zfp library\n* Single coil compression (scc) based on singular value decomposition (SVD)\n* Geometric coil compression (gcc) based on SVD\n* Optionally FID navigators can be removed\n\nBefore applying the selected compression method(s), lossless compression (gzip) is applied to the header and meta data information which is then added to a hdf5 file. All additional meta information necessary for decompression (e.g. coil compression matrices) are also stored in the hdf5 file.\n\n\n### Additional Requirements\n\n* pyzfp ≥ 0.3.1\n* pytables ≥ 3.6.1\n\nThe pyzfp and pytables libraries can be installed via pip:\n\n pip install pyzfp\n pip install tables\n\n\n### Usage\n\nExecuting the command twixzip</span>.py in an open terminal gives an overview of all possible arguments. Optional arguments are:\n\n -h: help \n -d: decompress data\n\nInput and output directories & filenames are required arguments that can be selected via:\n\n -i infile: input file \n -o outfile: output file\n\nIn the compression mode the input file should be an MRI raw data file, in the decompression mode (`-d`) it should be the hdf5 file containing the compressed data. The output file is then an hdf5 file (compression mode) or an MRI raw data file (decompression mode).\n\nCompression methods can be selected via:\n\n --remove_fidnav: removes FID navigators \n --remove_os: removes oversampling\n --scc -n NCC: single coil compression (SCC) - keep NCC virtual coils\n --scc -t CC_TOL: SCC - number of coils is calculated with a tolerance for the singular values\n --scc_bart -n NCC: SCC using BART \n --gcc -n NCC: geometric coil compression (GCC) - keep NCC virtual coils\n --gcc -t CC_TOL: GCC - number of coils is calculated with a tolerance for the singular values\n --gcc_bart -n NCC: GCC using the Berkeley Advanved Reconstruction Toolbox (BART) [1] \n --zfp --zfp_tol ZFP_TOL: floating point compression with ZFP_TOL tolerance\n --zfp --zfp_prec ZFP_PREC: floating point compression with ZFP_PREC precision (not recommended)\n\nThe optional argument `--testmode` can be used to automatically decompress the data after compression. The created decompressed MRI raw data filename contains the selected compression method. The option `--profile` can be used to profile the compression code.\n\n[1] BART Toolbox for Computational Magnetic Resonance Imaging, DOI: 10.5281/zenodo.592960\n\n\n## Acknowledgements\n\nThe protocol header parsing code originates from William Clarke's excellent pymapvbvd project (https://github.com/wexeee/pymapvbvd).\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "file reader for Siemens twix(.dat)-files",
"version": "1.0.1",
"project_urls": {
"Homepage": "https://github.com/pehses/twixtools"
},
"split_keywords": [
"twix",
"",
"siemens",
"",
"mri"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f433699e8ccf1cb9f271b811bd41d5c3a1c6db3e585b5148ebd7ef44ca71bec8",
"md5": "69ed8ac8a4d696124f5ac6e1a5837688",
"sha256": "2022d0fec4e19fbf0d67cba94ff9d4a182d36add9cff24a50853bfe72043a764"
},
"downloads": -1,
"filename": "gt_twixtools-1.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "69ed8ac8a4d696124f5ac6e1a5837688",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4,>=3.7",
"size": 54450,
"upload_time": "2024-01-25T07:57:48",
"upload_time_iso_8601": "2024-01-25T07:57:48.259195Z",
"url": "https://files.pythonhosted.org/packages/f4/33/699e8ccf1cb9f271b811bd41d5c3a1c6db3e585b5148ebd7ef44ca71bec8/gt_twixtools-1.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "01c7fedc3d9507c647fea7e775ac6917363dd4d3739bf2bee1cb52d4360c8792",
"md5": "b0923032030ac8706d541f2742d91e57",
"sha256": "8bffa161243979d612da8f6e0502ef9146fe81994414f86a0744e7268f7830b4"
},
"downloads": -1,
"filename": "gt-twixtools-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "b0923032030ac8706d541f2742d91e57",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4,>=3.7",
"size": 42568,
"upload_time": "2024-01-25T07:57:49",
"upload_time_iso_8601": "2024-01-25T07:57:49.500789Z",
"url": "https://files.pythonhosted.org/packages/01/c7/fedc3d9507c647fea7e775ac6917363dd4d3739bf2bee1cb52d4360c8792/gt-twixtools-1.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-25 07:57:49",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "pehses",
"github_project": "twixtools",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "gt-twixtools"
}
JSON
