hypershell


Namehypershell JSON
Version 2.6.5 PyPI version JSON
download
home_pagehttps://hypershell.org
SummaryA cross-platform, high-throughput computing utility for processing shell commands over a distributed, asynchronous queue.
upload_time2025-02-16 04:03:19
maintainerNone
docs_urlNone
authorGeoffrey Lentner
requires_python<4.0,>=3.9
licenseApache-2.0
keywords distributed-computing command-line-tool shell-scripting high-performance-computing high-throughput-computing
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            HyperShell v2: Distributed Task Execution for HPC
=================================================

.. image:: https://img.shields.io/badge/license-Apache-blue.svg?style=flat
    :target: https://www.apache.org/licenses/LICENSE-2.0
    :alt: License

.. image:: https://img.shields.io/github/v/release/hypershell/hypershell?sort=semver
    :target: https://github.com/hypershell/hypershell/releases
    :alt: Github Release

.. image:: https://img.shields.io/badge/Python-3.9+-blue.svg
    :target: https://www.python.org/downloads
    :alt: Python Versions

.. image:: https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg
    :target: https://www.contributor-covenant.org/version/2/1/code_of_conduct/
    :alt: Code of Conduct

.. image:: https://readthedocs.org/projects/hypershell/badge/?version=latest
    :target: https://hypershell.readthedocs.io/en/latest/?badge=latest
    :alt: Documentation Status

|

*HyperShell* is an elegant, cross-platform, high-throughput computing utility for
processing shell commands over a distributed, asynchronous queue. It is a highly
scalable workflow automation tool for *many-task* scenarios.

Built on Python and tested on Linux, macOS, and Windows.

Several tools offer similar functionality but not all together in a single tool with
the user ergonomics we provide. Novel design elements include but are not limited to

* **Cross-platform:** run on any platform where Python runs. In fact, the server and
  client can run on different platforms in the same cluster.
* **Client-server:** workloads do not need to be monolithic. Run the server as a
  stand-alone service with SQLite or Postgres as a persistent database and dynamically
  scale clients as needed.
* **Staggered launch:** At the largest scales (1000s of nodes, 100k+ of workers),
  the launch process can be challenging. Come up gradually to balance the workload.
* **Database in-the-loop:** run in-memory for quick, ad-hoc workloads. Otherwise,
  include a database for persistence, recovery when restarting, and search.


Documentation
-------------

Documentation is available at `hypershell.readthedocs.io <https://hypershell.readthedocs.io>`_.
For basic usage information on the command line use: ``hs --help``. For a more
comprehensive usage guide on the command line you can view the manual page with 
``man hs``.


Contributions
-------------

Contributions are welcome. If you find bugs or have questions, open an *Issue* here.
We've added a Code of Conduct recently, adapted from the
`Contributor Covenant <https://www.contributor-covenant.org/>`_, version 2.0.


Citation
--------

If *HyperShell* has helped in your research please consider citing us.

.. code-block:: bibtex

    @inproceedings{lentner_2022,
        author = {Lentner, Geoffrey and Gorenstein, Lev},
        title = {HyperShell v2: Distributed Task Execution for HPC},
        year = {2022},
        isbn = {9781450391610},
        publisher = {Association for Computing Machinery},
        url = {https://doi.org/10.1145/3491418.3535138},
        doi = {10.1145/3491418.3535138},
        booktitle = {Practice and Experience in Advanced Research Computing},
        articleno = {80},
        numpages = {3},
        series = {PEARC '22}
    }

            

Raw data

            {
    "_id": null,
    "home_page": "https://hypershell.org",
    "name": "hypershell",
    "maintainer": null,
    "docs_url": null,
    "requires_python": "<4.0,>=3.9",
    "maintainer_email": null,
    "keywords": "distributed-computing, command-line-tool, shell-scripting, high-performance-computing, high-throughput-computing",
    "author": "Geoffrey Lentner",
    "author_email": "glentner@purdue.edu",
    "download_url": "https://files.pythonhosted.org/packages/5c/85/a05b8ec6ef97e98b56c3fad9c36470953c256af86d2f23b131a378d9d5cf/hypershell-2.6.5.tar.gz",
    "platform": null,
    "description": "HyperShell v2: Distributed Task Execution for HPC\n=================================================\n\n.. image:: https://img.shields.io/badge/license-Apache-blue.svg?style=flat\n    :target: https://www.apache.org/licenses/LICENSE-2.0\n    :alt: License\n\n.. image:: https://img.shields.io/github/v/release/hypershell/hypershell?sort=semver\n    :target: https://github.com/hypershell/hypershell/releases\n    :alt: Github Release\n\n.. image:: https://img.shields.io/badge/Python-3.9+-blue.svg\n    :target: https://www.python.org/downloads\n    :alt: Python Versions\n\n.. image:: https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg\n    :target: https://www.contributor-covenant.org/version/2/1/code_of_conduct/\n    :alt: Code of Conduct\n\n.. image:: https://readthedocs.org/projects/hypershell/badge/?version=latest\n    :target: https://hypershell.readthedocs.io/en/latest/?badge=latest\n    :alt: Documentation Status\n\n|\n\n*HyperShell* is an elegant, cross-platform, high-throughput computing utility for\nprocessing shell commands over a distributed, asynchronous queue. It is a highly\nscalable workflow automation tool for *many-task* scenarios.\n\nBuilt on Python and tested on Linux, macOS, and Windows.\n\nSeveral tools offer similar functionality but not all together in a single tool with\nthe user ergonomics we provide. Novel design elements include but are not limited to\n\n* **Cross-platform:** run on any platform where Python runs. In fact, the server and\n  client can run on different platforms in the same cluster.\n* **Client-server:** workloads do not need to be monolithic. Run the server as a\n  stand-alone service with SQLite or Postgres as a persistent database and dynamically\n  scale clients as needed.\n* **Staggered launch:** At the largest scales (1000s of nodes, 100k+ of workers),\n  the launch process can be challenging. Come up gradually to balance the workload.\n* **Database in-the-loop:** run in-memory for quick, ad-hoc workloads. Otherwise,\n  include a database for persistence, recovery when restarting, and search.\n\n\nDocumentation\n-------------\n\nDocumentation is available at `hypershell.readthedocs.io <https://hypershell.readthedocs.io>`_.\nFor basic usage information on the command line use: ``hs --help``. For a more\ncomprehensive usage guide on the command line you can view the manual page with \n``man hs``.\n\n\nContributions\n-------------\n\nContributions are welcome. If you find bugs or have questions, open an *Issue* here.\nWe've added a Code of Conduct recently, adapted from the\n`Contributor Covenant <https://www.contributor-covenant.org/>`_, version 2.0.\n\n\nCitation\n--------\n\nIf *HyperShell* has helped in your research please consider citing us.\n\n.. code-block:: bibtex\n\n    @inproceedings{lentner_2022,\n        author = {Lentner, Geoffrey and Gorenstein, Lev},\n        title = {HyperShell v2: Distributed Task Execution for HPC},\n        year = {2022},\n        isbn = {9781450391610},\n        publisher = {Association for Computing Machinery},\n        url = {https://doi.org/10.1145/3491418.3535138},\n        doi = {10.1145/3491418.3535138},\n        booktitle = {Practice and Experience in Advanced Research Computing},\n        articleno = {80},\n        numpages = {3},\n        series = {PEARC '22}\n    }\n",
    "bugtrack_url": null,
    "license": "Apache-2.0",
    "summary": "A cross-platform, high-throughput computing utility for processing shell commands over a distributed, asynchronous queue.",
    "version": "2.6.5",
    "project_urls": {
        "Documentation": "https://hypershell.readthedocs.io",
        "Homepage": "https://hypershell.org",
        "Repository": "https://github.com/hypershell/hypershell"
    },
    "split_keywords": [
        "distributed-computing",
        " command-line-tool",
        " shell-scripting",
        " high-performance-computing",
        " high-throughput-computing"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "57a04a1db2e290dce8fb8c3e62be2f4250b07aa0ce72e0215e07fb94cb915fc3",
                "md5": "24c49b20c491f7b5c763e97a6d6f8209",
                "sha256": "3107c5719c942df75676bc0cbc7e65fdb44a42f37f059f9ae5fc67fb7d38daee"
            },
            "downloads": -1,
            "filename": "hypershell-2.6.5-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "24c49b20c491f7b5c763e97a6d6f8209",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": "<4.0,>=3.9",
            "size": 98848,
            "upload_time": "2025-02-16T04:03:16",
            "upload_time_iso_8601": "2025-02-16T04:03:16.802641Z",
            "url": "https://files.pythonhosted.org/packages/57/a0/4a1db2e290dce8fb8c3e62be2f4250b07aa0ce72e0215e07fb94cb915fc3/hypershell-2.6.5-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "5c85a05b8ec6ef97e98b56c3fad9c36470953c256af86d2f23b131a378d9d5cf",
                "md5": "fc7cabce4b3a277dd142ace8736ea864",
                "sha256": "fb3bbbbdd5be3ab44d84bc60ade7b091a9376df29bbf5e6f83f777be9b10529a"
            },
            "downloads": -1,
            "filename": "hypershell-2.6.5.tar.gz",
            "has_sig": false,
            "md5_digest": "fc7cabce4b3a277dd142ace8736ea864",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": "<4.0,>=3.9",
            "size": 78868,
            "upload_time": "2025-02-16T04:03:19",
            "upload_time_iso_8601": "2025-02-16T04:03:19.108967Z",
            "url": "https://files.pythonhosted.org/packages/5c/85/a05b8ec6ef97e98b56c3fad9c36470953c256af86d2f23b131a378d9d5cf/hypershell-2.6.5.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-02-16 04:03:19",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "hypershell",
    "github_project": "hypershell",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "lcname": "hypershell"
}
        
Elapsed time: 0.41066s