# Python DataMatrix
*An intuitive, Pythonic way to work with tabular data.*
Sebastiaan Mathôt <br />
Copyright 2015-2024 <br />
<https://pydatamatrix.eu/>
[](https://github.com/open-cogsci/python-datamatrix/actions/workflows/publish-package.yaml)
[](https://github.com/open-cogsci/python-datamatrix/actions/workflows/run-unittests.yaml)
## Contents
- [About](#about)
- [Features](#features)
- [Ultra-short cheat sheet](#ultra-short-cheat-sheet)
- [Documentation](#documentation)
- [Dependencies](#dependencies)
- [Installation](#installation)
- [License](#license)
## About
`DataMatrix` is an intuitive Python library for working with column-based, time-series, and multidimensional data. It's a light-weight and easy-to-use alternative to `pandas`.
`DataMatrix` is also one of the core libraries of [OpenSesame](https://osdoc.cogsci.nl/), a graphical experiment builder for the social sciences, and [Rapunzel](https://rapunzel.cogsci.nl/), a modern code editor for numerical computing with Python and R.
## Features
- [An intuitive syntax](https://pydatamatrix.eu/basic) that makes your code easy to read
- Mix tabular data with [time series](https://pydatamatrix.eu/series) and [multidimensional data](https://pydatamatrix.eu/multidimensional) in a single data structure
- Support for [large data](https://pydatamatrix.eu/largedata) by intelligent (and automatic) offloading of data to disk when memory is running low
- Advanced [memoization (caching)](https://pydatamatrix.eu//memoization)
- Requires only the Python standard libraries (but you can use `numpy` to improve performance)
- Compatible with your favorite data-science libraries:
- `seaborn` and `matplotlib` for [plotting](https://pythontutorials.eu/numerical/plotting)
- `scipy`, `statsmodels`, and `pingouin` for [statistics](https://pythontutorials.eu/numerical/statistics)
- `mne` for analysis of electroencephalographic (EEG) and magnetoencephalographic (MEG) data
- [Convert](https://pydatamatrix.eu/convert) to and from `pandas.DataFrame`
- Looks pretty inside a Jupyter Notebook
## Ultra-short cheat sheet
```python
from datamatrix import DataMatrix, io
# Read a DataMatrix from file
dm = io.readtxt('data.csv')
# Create a new DataMatrix
dm = DataMatrix(length=5)
# The first two rows
print(dm[:2])
# Create a new column and initialize it with the Fibonacci series
dm.fibonacci = 0, 1, 1, 2, 3
# You can also specify column names as if they are dict keys
dm['fibonacci'] = 0, 1, 1, 2, 3
# Remove 0 and 3 with a simple selection
dm = (dm.fibonacci > 0) & (dm.fibonacci < 3)
# Get a list of indices that match certain criteria
print(dm[(dm.fibonacci > 0) & (dm.fibonacci < 3)])
# Select 1, 1, and 2 by matching any of the values in a set
dm = dm.fibonacci == {1, 2}
# Select all odd numbers with a lambda expression
dm = dm.fibonacci == (lambda x: x % 2)
# Change all 1s to -1
dm.fibonacci[dm.fibonacci == 1] = -1
# The first two cells from the fibonacci column
print(dm.fibonacci[:2])
# Column mean
print(dm.fibonacci[...])
# Multiply all fibonacci cells by 2
dm.fibonacci_times_two = dm.fibonacci * 2
# Loop through all rows
for row in dm:
print(row.fibonacci) # get the fibonacci cell from the row
# Loop through all columns
for colname, col in dm.columns:
for cell in col: # Loop through all cells in the column
print(cell) # do something with the cell
# Or just see which columns exist
print(dm.column_names)
```
## Documentation
The basic documentation (including function and module references) is hosted on <https://pydatamatrix.eu/>. Additional tutorials can be found in the data-science course on <https://pythontutorials.eu/>.
## Dependencies
`DataMatrix` requires only the Python standard library. That is, you can use it without installing any additional Python packages (although the pip and conda packages install some of the optional dependencies by default). Python 3.7 and higher are supported.
The following packages are required for extra functionality:
- `numpy` and `scipy` for using the `FloatColumn`, `IntColumn`, `SeriesColumn`, `MultiDimensionalColumn` objects
- `pandas` for conversion to and from `pandas.DataFrame`
- `mne` for conversion to and from `mne.Epochs` and `mne.TFR`
- `fastnumbers` for improved performance
- `prettytable` for creating a text representation of a DataMatrix (e.g. to print it out)
- `openpyxl` for reading and writing `.xlsx` files
- `json_tricks` for hashing, serialization to and from `json`, and memoization (caching)
- `tomlkit` for reading configuration from `pyproject.toml`
- `psutil` for dynamic loading of large data
## Installation
### PyPi
~~~
pip install datamatrix
~~~
*Historical note:* The DataMatrix project used to correspond to another package of the same name, which was discontinued in 2010. If you want to install this package, you can do still do so by providing an explicit version (0.9 is the latest version of this package), as shown below. With thanks to [dennogumi.org](https://www.dennogumi.org/) for handing over this project's entry on PyPi, thus avoiding much unnecessary confusion!
~~~
# Doesn't install datamatrix but a previous package by the same name!
pip install datamatrix==0.9
~~~
### Anaconda
~~~
conda install datamatrix -c conda-forge
~~~
### Ubuntu
~~~
sudo add-apt-repository ppa:smathot/cogscinl # for stable releases
sudo add-apt-repository ppa:smathot/rapunzel # for development releases
sudo apt-get update
sudo apt install python3-datamatrix
~~~
## License
`python-datamatrix` is licensed under the [GNU General Public License
v3](http://www.gnu.org/licenses/gpl-3.0.en.html).
Raw data
{
"_id": null,
"home_page": null,
"name": "datamatrix",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "data analysis, scientific software, time series",
"author": null,
"author_email": "Sebastiaan Math\u00f4t <s.mathot@cogsci.nl>",
"download_url": "https://files.pythonhosted.org/packages/eb/a8/2ecee3c8bf9189c78585e370ed1345e6b9b7b26901bacc537a62ae85882f/datamatrix-1.0.15.tar.gz",
"platform": null,
"description": "# Python DataMatrix\n\n*An intuitive, Pythonic way to work with tabular data.*\n\nSebastiaan Math\u00f4t <br />\nCopyright 2015-2024 <br />\n<https://pydatamatrix.eu/>\n\n\n[](https://github.com/open-cogsci/python-datamatrix/actions/workflows/publish-package.yaml)\n\n[](https://github.com/open-cogsci/python-datamatrix/actions/workflows/run-unittests.yaml)\n\n\n## Contents\n\n- [About](#about)\n- [Features](#features)\n- [Ultra-short cheat sheet](#ultra-short-cheat-sheet)\n- [Documentation](#documentation)\n- [Dependencies](#dependencies)\n- [Installation](#installation)\n- [License](#license)\n\n\n## About\n\n`DataMatrix` is an intuitive Python library for working with column-based, time-series, and multidimensional data. It's a light-weight and easy-to-use alternative to `pandas`.\n\n`DataMatrix` is also one of the core libraries of [OpenSesame](https://osdoc.cogsci.nl/), a graphical experiment builder for the social sciences, and [Rapunzel](https://rapunzel.cogsci.nl/), a modern code editor for numerical computing with Python and R.\n\n\n## Features\n\n- [An intuitive syntax](https://pydatamatrix.eu/basic) that makes your code easy to read\n- Mix tabular data with [time series](https://pydatamatrix.eu/series) and [multidimensional data](https://pydatamatrix.eu/multidimensional) in a single data structure\n- Support for [large data](https://pydatamatrix.eu/largedata) by intelligent (and automatic) offloading of data to disk when memory is running low\n- Advanced [memoization (caching)](https://pydatamatrix.eu//memoization)\n- Requires only the Python standard libraries (but you can use `numpy` to improve performance)\n- Compatible with your favorite data-science libraries:\n - `seaborn` and `matplotlib` for [plotting](https://pythontutorials.eu/numerical/plotting)\n - `scipy`, `statsmodels`, and `pingouin` for [statistics](https://pythontutorials.eu/numerical/statistics)\n - `mne` for analysis of electroencephalographic (EEG) and magnetoencephalographic (MEG) data\n - [Convert](https://pydatamatrix.eu/convert) to and from `pandas.DataFrame`\n - Looks pretty inside a Jupyter Notebook\n\n\n## Ultra-short cheat sheet\n\n```python\nfrom datamatrix import DataMatrix, io\n# Read a DataMatrix from file\ndm = io.readtxt('data.csv')\n# Create a new DataMatrix\ndm = DataMatrix(length=5)\n# The first two rows\nprint(dm[:2])\n# Create a new column and initialize it with the Fibonacci series\ndm.fibonacci = 0, 1, 1, 2, 3\n# You can also specify column names as if they are dict keys\ndm['fibonacci'] = 0, 1, 1, 2, 3\n# Remove 0 and 3 with a simple selection\ndm = (dm.fibonacci > 0) & (dm.fibonacci < 3)\n# Get a list of indices that match certain criteria\nprint(dm[(dm.fibonacci > 0) & (dm.fibonacci < 3)])\n# Select 1, 1, and 2 by matching any of the values in a set\ndm = dm.fibonacci == {1, 2}\n# Select all odd numbers with a lambda expression\ndm = dm.fibonacci == (lambda x: x % 2)\n# Change all 1s to -1\ndm.fibonacci[dm.fibonacci == 1] = -1\n# The first two cells from the fibonacci column\nprint(dm.fibonacci[:2])\n# Column mean\nprint(dm.fibonacci[...])\n# Multiply all fibonacci cells by 2\ndm.fibonacci_times_two = dm.fibonacci * 2\n# Loop through all rows\nfor row in dm:\n print(row.fibonacci) # get the fibonacci cell from the row\n# Loop through all columns\nfor colname, col in dm.columns:\n for cell in col: # Loop through all cells in the column\n print(cell) # do something with the cell\n# Or just see which columns exist\nprint(dm.column_names)\n```\n\n\n## Documentation\n\nThe basic documentation (including function and module references) is hosted on <https://pydatamatrix.eu/>. Additional tutorials can be found in the data-science course on <https://pythontutorials.eu/>.\n\n\n## Dependencies\n\n`DataMatrix` requires only the Python standard library. That is, you can use it without installing any additional Python packages (although the pip and conda packages install some of the optional dependencies by default). Python 3.7 and higher are supported.\n\nThe following packages are required for extra functionality:\n\n- `numpy` and `scipy` for using the `FloatColumn`, `IntColumn`, `SeriesColumn`, `MultiDimensionalColumn` objects\n- `pandas` for conversion to and from `pandas.DataFrame`\n- `mne` for conversion to and from `mne.Epochs` and `mne.TFR`\n- `fastnumbers` for improved performance\n- `prettytable` for creating a text representation of a DataMatrix (e.g. to print it out)\n- `openpyxl` for reading and writing `.xlsx` files\n- `json_tricks` for hashing, serialization to and from `json`, and memoization (caching)\n- `tomlkit` for reading configuration from `pyproject.toml`\n- `psutil` for dynamic loading of large data\n\n\n## Installation\n\n\n### PyPi\n\n~~~\npip install datamatrix\n~~~\n\n\n*Historical note:* The DataMatrix project used to correspond to another package of the same name, which was discontinued in 2010. If you want to install this package, you can do still do so by providing an explicit version (0.9 is the latest version of this package), as shown below. With thanks to [dennogumi.org](https://www.dennogumi.org/) for handing over this project's entry on PyPi, thus avoiding much unnecessary confusion!\n\n~~~\n# Doesn't install datamatrix but a previous package by the same name!\npip install datamatrix==0.9\n~~~\n\n\n### Anaconda\n\n~~~\nconda install datamatrix -c conda-forge\n~~~\n\n\n### Ubuntu\n\n~~~\nsudo add-apt-repository ppa:smathot/cogscinl # for stable releases\nsudo add-apt-repository ppa:smathot/rapunzel # for development releases\nsudo apt-get update\nsudo apt install python3-datamatrix\n~~~\n\n\n## License\n\n`python-datamatrix` is licensed under the [GNU General Public License\nv3](http://www.gnu.org/licenses/gpl-3.0.en.html).\n\n",
"bugtrack_url": null,
"license": null,
"summary": "This file is part of datamatrix.",
"version": "1.0.15",
"project_urls": {
"Documentation": "https://pydatamatrix.eu",
"Source": "https://github.com/open-cogsci/datamatrix"
},
"split_keywords": [
"data analysis",
" scientific software",
" time series"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0ad4d2621f6434e112ca44f50141797e65ef060c6d990bbbfcdb6a35fe2b3e60",
"md5": "e013ad97283227733799e9a9a29a115b",
"sha256": "9c9613ef2fefc462525095367ca64d43ea5395fe7a0741b228c78c99a48be5f3"
},
"downloads": -1,
"filename": "datamatrix-1.0.15-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e013ad97283227733799e9a9a29a115b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 108193,
"upload_time": "2024-12-17T10:12:19",
"upload_time_iso_8601": "2024-12-17T10:12:19.177575Z",
"url": "https://files.pythonhosted.org/packages/0a/d4/d2621f6434e112ca44f50141797e65ef060c6d990bbbfcdb6a35fe2b3e60/datamatrix-1.0.15-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "eba82ecee3c8bf9189c78585e370ed1345e6b9b7b26901bacc537a62ae85882f",
"md5": "27e04ae3c986f8c9ceaba0eeccd3dba1",
"sha256": "771359e33b8bea0e8fe544b78015338bbe300541905c57c98992bd58aa21c3d7"
},
"downloads": -1,
"filename": "datamatrix-1.0.15.tar.gz",
"has_sig": false,
"md5_digest": "27e04ae3c986f8c9ceaba0eeccd3dba1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 78540,
"upload_time": "2024-12-17T10:12:20",
"upload_time_iso_8601": "2024-12-17T10:12:20.474458Z",
"url": "https://files.pythonhosted.org/packages/eb/a8/2ecee3c8bf9189c78585e370ed1345e6b9b7b26901bacc537a62ae85882f/datamatrix-1.0.15.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-17 10:12:20",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "open-cogsci",
"github_project": "datamatrix",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "datamatrix"
}
JSON
