dask-ms


Namedask-ms JSON
Version 0.2.23 PyPI version JSON
download
home_pageNone
Summaryxarray Dataset from CASA Tables
upload_time2024-11-11 11:00:52
maintainerNone
docs_urlNone
authorSimon Perkins
requires_python<3.13,>=3.10
licenseBSD3
keywords
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            ================================
xarray Datasets from CASA Tables
================================

.. image:: https://img.shields.io/pypi/v/dask-ms.svg
        :target: https://pypi.python.org/pypi/dask-ms

.. image:: https://github.com/ratt-ru/dask-ms/actions/workflows/ci.yml/badge.svg
        :target: https://github.com/ratt-ru/dask-ms/actions/workflows/ci.yml

.. image:: https://readthedocs.org/projects/dask-ms/badge/?version=latest
        :target: https://dask-ms.readthedocs.io/en/latest/?badge=latest
        :alt: Documentation Status

Constructs xarray_ ``Datasets`` from CASA Tables via python-casacore_.
The ``Variables`` contained in the ``Dataset`` are dask_ arrays backed by
deferred calls to :code:`casacore.tables.table.getcol`.

Supports writing ``Variables`` back to the respective column in the Table.

The intention behind this package is to support the Measurement Set as
a data source and sink for the purposes of writing parallel, distributed
Radio Astronomy algorithms.

Installation
============

To install with xarray_ support:

.. code-block:: bash

  $ pip install dask-ms[xarray]

Without xarray_ similar, but reduced Dataset functionality is replicated
in dask-ms itself. Expert users may wish to use this option to reduce
python package dependencies.

.. code-block:: bash

  $ pip install dask-ms


Documentation
=============

https://dask-ms.readthedocs.io

Gitter Page
===========

https://gitter.im/dask-ms/community

Example Usage
=============


.. code-block:: python

    import dask.array as da
    from daskms import xds_from_table, xds_to_table

    # Create xarray datasets from Measurement Set "WSRT.MS"
    ds = xds_from_table("WSRT.MS")
    # Set the flag Variable on first Dataset to it's inverse
    ds[0]['flag'] = (ds[0].flag.dims, da.logical_not(ds[0].flag))
    # Write the flag column back to the Measurement Set
    xds_to_table(ds, "WSRT.MS", "FLAG").compute()

    print ds

  [<xarray.Dataset>
   Dimensions:         (chan: 64, corr: 4, row: 6552, uvw: 3)
   Coordinates:
       ROWID           (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>
   Dimensions without coordinates: chan, corr, row, uvw
   Data variables:
       IMAGING_WEIGHT  (row, chan) float32 dask.array<shape=(6552, 64), chunksize=(6552, 64)>
       ANTENNA1        (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>
       STATE_ID        (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>
       EXPOSURE        (row) float64 dask.array<shape=(6552,), chunksize=(6552,)>
       MODEL_DATA      (row, chan, corr) complex64 dask.array<shape=(6552, 64, 4), chunksize=(6552, 64, 4)>
       FLAG_ROW        (row) bool dask.array<shape=(6552,), chunksize=(6552,)>
       CORRECTED_DATA  (row, chan, corr) complex64 dask.array<shape=(6552, 64, 4), chunksize=(6552, 64, 4)>
       PROCESSOR_ID    (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>
       WEIGHT          (row, corr) float32 dask.array<shape=(6552, 4), chunksize=(6552, 4)>
       FLAG            (row, chan, corr) bool dask.array<shape=(6552, 64, 4), chunksize=(6552, 64, 4)>
       TIME            (row) float64 dask.array<shape=(6552,), chunksize=(6552,)>
       SIGMA           (row, corr) float32 dask.array<shape=(6552, 4), chunksize=(6552, 4)>
       SCAN_NUMBER     (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>
       INTERVAL        (row) float64 dask.array<shape=(6552,), chunksize=(6552,)>
       OBSERVATION_ID  (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>
       TIME_CENTROID   (row) float64 dask.array<shape=(6552,), chunksize=(6552,)>
       ARRAY_ID        (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>
       ANTENNA2        (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>
       DATA            (row, chan, corr) complex64 dask.array<shape=(6552, 64, 4), chunksize=(6552, 64, 4)>
       FEED1           (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>
       FEED2           (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>
       UVW             (row, uvw) float64 dask.array<shape=(6552, 3), chunksize=(6552, 3)>
   Attributes:
       FIELD_ID:      0
       DATA_DESC_ID:  0]

-----------
Limitations
-----------

1. Many Measurement Sets columns are defined as variably shaped,
   but the actual data is fixed.
   dask-ms_ will infer the shape of the
   data from the first row and must be consistent
   with that of other rows.
   For example, this may be issue where multiple Spectral Windows
   are present in the Measurement Set with differing channels
   per SPW.

   dask-ms_ works around this by partitioning the
   Measurement Set into multiple datasets.
   The first row's shape is used to infer the shape of the partition.
   Thus, in the case of multiple Spectral Window's, we can partition
   the Measurement Set by DATA_DESC_ID to create a dataset for
   each Spectral Window.

.. _dask: https://dask.pydata.org
.. _dask-ms: https://github.com/ska-sa/dask-ms
.. _xarray: https://xarray.pydata.org
.. _python-casacore: https://github.com/casacore/python-casacore


            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "dask-ms",
    "maintainer": null,
    "docs_url": null,
    "requires_python": "<3.13,>=3.10",
    "maintainer_email": null,
    "keywords": null,
    "author": "Simon Perkins",
    "author_email": "simon.perkins@gmail.com",
    "download_url": "https://files.pythonhosted.org/packages/c7/d7/d31f53f47dc9ced103dcfef8481aa000c498e532d17807e278d4e67be9ae/dask_ms-0.2.23.tar.gz",
    "platform": null,
    "description": "================================\nxarray Datasets from CASA Tables\n================================\n\n.. image:: https://img.shields.io/pypi/v/dask-ms.svg\n        :target: https://pypi.python.org/pypi/dask-ms\n\n.. image:: https://github.com/ratt-ru/dask-ms/actions/workflows/ci.yml/badge.svg\n        :target: https://github.com/ratt-ru/dask-ms/actions/workflows/ci.yml\n\n.. image:: https://readthedocs.org/projects/dask-ms/badge/?version=latest\n        :target: https://dask-ms.readthedocs.io/en/latest/?badge=latest\n        :alt: Documentation Status\n\nConstructs xarray_ ``Datasets`` from CASA Tables via python-casacore_.\nThe ``Variables`` contained in the ``Dataset`` are dask_ arrays backed by\ndeferred calls to :code:`casacore.tables.table.getcol`.\n\nSupports writing ``Variables`` back to the respective column in the Table.\n\nThe intention behind this package is to support the Measurement Set as\na data source and sink for the purposes of writing parallel, distributed\nRadio Astronomy algorithms.\n\nInstallation\n============\n\nTo install with xarray_ support:\n\n.. code-block:: bash\n\n  $ pip install dask-ms[xarray]\n\nWithout xarray_ similar, but reduced Dataset functionality is replicated\nin dask-ms itself. Expert users may wish to use this option to reduce\npython package dependencies.\n\n.. code-block:: bash\n\n  $ pip install dask-ms\n\n\nDocumentation\n=============\n\nhttps://dask-ms.readthedocs.io\n\nGitter Page\n===========\n\nhttps://gitter.im/dask-ms/community\n\nExample Usage\n=============\n\n\n.. code-block:: python\n\n    import dask.array as da\n    from daskms import xds_from_table, xds_to_table\n\n    # Create xarray datasets from Measurement Set \"WSRT.MS\"\n    ds = xds_from_table(\"WSRT.MS\")\n    # Set the flag Variable on first Dataset to it's inverse\n    ds[0]['flag'] = (ds[0].flag.dims, da.logical_not(ds[0].flag))\n    # Write the flag column back to the Measurement Set\n    xds_to_table(ds, \"WSRT.MS\", \"FLAG\").compute()\n\n    print ds\n\n  [<xarray.Dataset>\n   Dimensions:         (chan: 64, corr: 4, row: 6552, uvw: 3)\n   Coordinates:\n       ROWID           (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>\n   Dimensions without coordinates: chan, corr, row, uvw\n   Data variables:\n       IMAGING_WEIGHT  (row, chan) float32 dask.array<shape=(6552, 64), chunksize=(6552, 64)>\n       ANTENNA1        (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>\n       STATE_ID        (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>\n       EXPOSURE        (row) float64 dask.array<shape=(6552,), chunksize=(6552,)>\n       MODEL_DATA      (row, chan, corr) complex64 dask.array<shape=(6552, 64, 4), chunksize=(6552, 64, 4)>\n       FLAG_ROW        (row) bool dask.array<shape=(6552,), chunksize=(6552,)>\n       CORRECTED_DATA  (row, chan, corr) complex64 dask.array<shape=(6552, 64, 4), chunksize=(6552, 64, 4)>\n       PROCESSOR_ID    (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>\n       WEIGHT          (row, corr) float32 dask.array<shape=(6552, 4), chunksize=(6552, 4)>\n       FLAG            (row, chan, corr) bool dask.array<shape=(6552, 64, 4), chunksize=(6552, 64, 4)>\n       TIME            (row) float64 dask.array<shape=(6552,), chunksize=(6552,)>\n       SIGMA           (row, corr) float32 dask.array<shape=(6552, 4), chunksize=(6552, 4)>\n       SCAN_NUMBER     (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>\n       INTERVAL        (row) float64 dask.array<shape=(6552,), chunksize=(6552,)>\n       OBSERVATION_ID  (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>\n       TIME_CENTROID   (row) float64 dask.array<shape=(6552,), chunksize=(6552,)>\n       ARRAY_ID        (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>\n       ANTENNA2        (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>\n       DATA            (row, chan, corr) complex64 dask.array<shape=(6552, 64, 4), chunksize=(6552, 64, 4)>\n       FEED1           (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>\n       FEED2           (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>\n       UVW             (row, uvw) float64 dask.array<shape=(6552, 3), chunksize=(6552, 3)>\n   Attributes:\n       FIELD_ID:      0\n       DATA_DESC_ID:  0]\n\n-----------\nLimitations\n-----------\n\n1. Many Measurement Sets columns are defined as variably shaped,\n   but the actual data is fixed.\n   dask-ms_ will infer the shape of the\n   data from the first row and must be consistent\n   with that of other rows.\n   For example, this may be issue where multiple Spectral Windows\n   are present in the Measurement Set with differing channels\n   per SPW.\n\n   dask-ms_ works around this by partitioning the\n   Measurement Set into multiple datasets.\n   The first row's shape is used to infer the shape of the partition.\n   Thus, in the case of multiple Spectral Window's, we can partition\n   the Measurement Set by DATA_DESC_ID to create a dataset for\n   each Spectral Window.\n\n.. _dask: https://dask.pydata.org\n.. _dask-ms: https://github.com/ska-sa/dask-ms\n.. _xarray: https://xarray.pydata.org\n.. _python-casacore: https://github.com/casacore/python-casacore\n\n",
    "bugtrack_url": null,
    "license": "BSD3",
    "summary": "xarray Dataset from CASA Tables",
    "version": "0.2.23",
    "project_urls": null,
    "split_keywords": [],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "e870ccbfdaef8d0ee2861e0b8a08ed83d403b5de1e801d14e65af22a7a912e58",
                "md5": "e8fd54f8f9357ef6e652db5d3411185b",
                "sha256": "f4c42c1b379ba579cbb0c9b8676d16d0f47ee0d0ed1921925ec373182db316e4"
            },
            "downloads": -1,
            "filename": "dask_ms-0.2.23-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "e8fd54f8f9357ef6e652db5d3411185b",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": "<3.13,>=3.10",
            "size": 157878,
            "upload_time": "2024-11-11T11:00:50",
            "upload_time_iso_8601": "2024-11-11T11:00:50.897020Z",
            "url": "https://files.pythonhosted.org/packages/e8/70/ccbfdaef8d0ee2861e0b8a08ed83d403b5de1e801d14e65af22a7a912e58/dask_ms-0.2.23-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "c7d7d31f53f47dc9ced103dcfef8481aa000c498e532d17807e278d4e67be9ae",
                "md5": "a1d0feab901717752565967f605b1763",
                "sha256": "54450f7cc0c16a464d8aa79e0e5bdaa49f6494155096ce35f9bbe628b585987b"
            },
            "downloads": -1,
            "filename": "dask_ms-0.2.23.tar.gz",
            "has_sig": false,
            "md5_digest": "a1d0feab901717752565967f605b1763",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": "<3.13,>=3.10",
            "size": 122555,
            "upload_time": "2024-11-11T11:00:52",
            "upload_time_iso_8601": "2024-11-11T11:00:52.265212Z",
            "url": "https://files.pythonhosted.org/packages/c7/d7/d31f53f47dc9ced103dcfef8481aa000c498e532d17807e278d4e67be9ae/dask_ms-0.2.23.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-11-11 11:00:52",
    "github": false,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "lcname": "dask-ms"
}
        
Elapsed time: 0.31913s