Pegaflow


NamePegaflow JSON
Version 5.0.3 PyPI version JSON
download
home_pagehttps://github.com/polyactis/pegaflow
SummaryAn easy-to-use Python API for Pegasus 5.0
upload_time2021-06-18 09:03:41
maintainer
docs_urlNone
authorYu S. Huang
requires_python>=3.5
licenseApache2
keywords scientific workflows
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            - [1. Pegasus Workflow Management System Python3 API](#1-pegasus-workflow-management-system-python3-api)
- [2. Prerequisites](#2-prerequisites)
- [3. Installation](#3-installation)
- [4. Examples](#4-examples)

# 1. Pegasus Workflow Management System Python3 API

Pegaflow, https://github.com/polyactis/pegaflow, is an easy-to-use package of the Python3 APIs for Pegasus WMS (http://pegasus.isi.edu/). It is compatible with Pegasus 5.0.0. Pegasus allows a developer to connect dependent computing jobs into a DAG (Directed Acyclic Graph) and run jobs according to the dependency.

[Workflow.py](pegaflow/Workflow.py) is the key difference from the official Pegasus Python APIs. Inheriting [Workflow.py](pegaflow/Workflow.py), users can write Pegasus workflows in an Object-Oriented way. It significantly reduces the amount of coding in writing a Pegasus workflow.

Pegasus jobs do NOT support UNIX pipes while many UNIX programs can only output to stdout. A shell wrapper, [pegaflow/tools/pipe2File.sh](pegaflow/tools/pipe2File.sh), is offered to redirect the output (stdout) of a program to a file. [pegaflow/tools/](pegaflow/tools/) contains a few other useful shell scripts.

- **Workflow.py**
- The monitoring API
- The Stampede database API
- The Pegasus statistics API
- The Pegasus plots API
- Miscellaneous Pegasus utilities
- The Pegasus service, including the ensemble manager and dashboard

Part of this package's source code is copied from https://github.com/pegasus-isi/pegasus, version 5.0.0,

# 2. Prerequisites

Pegasus and HTCondor (Condor) are only required on computers on which you intend to submit and run workflows.

On computers where only DAG yml files are outputted, no need to install Pegasus and Condor.

- Pegasus https://github.com/pegasus-isi/pegasus
- HTCondor https://research.cs.wisc.edu/htcondor/, the underlying job scheduler.
- Linux command bc and gzip are needed if [pegaflow/tools/pipe2File.sh](pegaflow/tools/pipe2File.sh) is to be used.

# 3. Installation

Install pegaflow:

```python
pip3 install --upgrade pegaflow
```

# 4. Examples

Check [pegaflow/example/](pegaflow/example/) for examples.

- [pegaflow/example/WordCountFiles.py](pegaflow/example/WordCountFiles.py) is an example that inherits [Workflow.py](pegaflow/Workflow.py). Users should be familiar with Object-Oriented programming.
- [pegaflow/example/WCFiles_Function.py](pegaflow/example/WCFiles_Function.py) is a procedural-programming example.



# 20210616 Upgraded to be compatible with Pegasus 5.0.0.

# 20200326 An easy-to-use Python3 package for Pegasus-wms API

Pegaflow is a package of the Python3 APIs for Pegasus WMS (http://pegasus.isi.edu/), with a helper class Workflow.py.



            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/polyactis/pegaflow",
    "name": "Pegaflow",
    "maintainer": "",
    "docs_url": null,
    "requires_python": ">=3.5",
    "maintainer_email": "",
    "keywords": "scientific workflows",
    "author": "Yu S. Huang",
    "author_email": "polyactis@gmail.com",
    "download_url": "https://files.pythonhosted.org/packages/5b/2d/a8019969672972cf9eb62105a3e0a13f99bb2d011d85670bd398e1be78c6/Pegaflow-5.0.3.tar.gz",
    "platform": "",
    "description": "- [1. Pegasus Workflow Management System Python3 API](#1-pegasus-workflow-management-system-python3-api)\n- [2. Prerequisites](#2-prerequisites)\n- [3. Installation](#3-installation)\n- [4. Examples](#4-examples)\n\n# 1. Pegasus Workflow Management System Python3 API\n\nPegaflow, https://github.com/polyactis/pegaflow, is an easy-to-use package of the Python3 APIs for Pegasus WMS (http://pegasus.isi.edu/). It is compatible with Pegasus 5.0.0. Pegasus allows a developer to connect dependent computing jobs into a DAG (Directed Acyclic Graph) and run jobs according to the dependency.\n\n[Workflow.py](pegaflow/Workflow.py) is the key difference from the official Pegasus Python APIs. Inheriting [Workflow.py](pegaflow/Workflow.py), users can write Pegasus workflows in an Object-Oriented way. It significantly reduces the amount of coding in writing a Pegasus workflow.\n\nPegasus jobs do NOT support UNIX pipes while many UNIX programs can only output to stdout. A shell wrapper, [pegaflow/tools/pipe2File.sh](pegaflow/tools/pipe2File.sh), is offered to redirect the output (stdout) of a program to a file. [pegaflow/tools/](pegaflow/tools/) contains a few other useful shell scripts.\n\n- **Workflow.py**\n- The monitoring API\n- The Stampede database API\n- The Pegasus statistics API\n- The Pegasus plots API\n- Miscellaneous Pegasus utilities\n- The Pegasus service, including the ensemble manager and dashboard\n\nPart of this package's source code is copied from https://github.com/pegasus-isi/pegasus, version 5.0.0,\n\n# 2. Prerequisites\n\nPegasus and HTCondor (Condor) are only required on computers on which you intend to submit and run workflows.\n\nOn computers where only DAG yml files are outputted, no need to install Pegasus and Condor.\n\n- Pegasus https://github.com/pegasus-isi/pegasus\n- HTCondor https://research.cs.wisc.edu/htcondor/, the underlying job scheduler.\n- Linux command bc and gzip are needed if [pegaflow/tools/pipe2File.sh](pegaflow/tools/pipe2File.sh) is to be used.\n\n# 3. Installation\n\nInstall pegaflow:\n\n```python\npip3 install --upgrade pegaflow\n```\n\n# 4. Examples\n\nCheck [pegaflow/example/](pegaflow/example/) for examples.\n\n- [pegaflow/example/WordCountFiles.py](pegaflow/example/WordCountFiles.py) is an example that inherits [Workflow.py](pegaflow/Workflow.py). Users should be familiar with Object-Oriented programming.\n- [pegaflow/example/WCFiles_Function.py](pegaflow/example/WCFiles_Function.py) is a procedural-programming example.\n\n\n\n# 20210616 Upgraded to be compatible with Pegasus 5.0.0.\n\n# 20200326 An easy-to-use Python3 package for Pegasus-wms API\n\nPegaflow is a package of the Python3 APIs for Pegasus WMS (http://pegasus.isi.edu/), with a helper class Workflow.py.\n\n\n",
    "bugtrack_url": null,
    "license": "Apache2",
    "summary": "An easy-to-use Python API for Pegasus 5.0",
    "version": "5.0.3",
    "split_keywords": [
        "scientific",
        "workflows"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "5a85a384367f0ecc474e7b18c25932c0",
                "sha256": "c585268f2665162986c5f348c242e6890078bb566e6085c30980d3923128bf81"
            },
            "downloads": -1,
            "filename": "Pegaflow-5.0.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "5a85a384367f0ecc474e7b18c25932c0",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.5",
            "size": 613410,
            "upload_time": "2021-06-18T09:03:39",
            "upload_time_iso_8601": "2021-06-18T09:03:39.289919Z",
            "url": "https://files.pythonhosted.org/packages/86/2a/0bd2b13e20fe866e62d623bc9778f9b782083565ccc4859ab3a5eecd0147/Pegaflow-5.0.3-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "b57d539181085ca3cacffb466d935a55",
                "sha256": "2cdc7b97c34cff73ae127a0582eea41a056893cf13d501e0280d15c2a57d39dc"
            },
            "downloads": -1,
            "filename": "Pegaflow-5.0.3.tar.gz",
            "has_sig": false,
            "md5_digest": "b57d539181085ca3cacffb466d935a55",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.5",
            "size": 495986,
            "upload_time": "2021-06-18T09:03:41",
            "upload_time_iso_8601": "2021-06-18T09:03:41.748668Z",
            "url": "https://files.pythonhosted.org/packages/5b/2d/a8019969672972cf9eb62105a3e0a13f99bb2d011d85670bd398e1be78c6/Pegaflow-5.0.3.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2021-06-18 09:03:41",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "github_user": "polyactis",
    "github_project": "pegaflow",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "lcname": "pegaflow"
}
        
Elapsed time: 0.36161s