IceProd
=======
.. image:: https://zenodo.org/badge/58235078.svg
:target: https://zenodo.org/badge/latestdoi/58235078
IceProd is a Python framework for distributed management of batch jobs.
It runs as a layer on top of other batch middleware, such as HTCondor,
and can pool together resources from different batch systems.
The primary purpose is to coordinate and administer many large sets of
jobs at once, keeping a history of the entire job lifecycle.
See also: Aartsen, Mark G., et al. "The IceProd framework: Distributed data processing for the IceCube neutrino observatory." Journal of parallel and distributed computing 75 (2015): 198-211.
**Note:**
For IceCube users with CVMFS access, IceProd is already installed.
To load the environment execute::
/cvmfs/icecube.wisc.edu/iceprod/latest/env-shell.sh
or::
eval `/cvmfs/icecube.wisc.edu/iceprod/latest/setup.sh`
depending on whether you want to get a new shell or load the variables
into the current shell.
Installation
------------
**Platforms**:
IceProd should run on any Unix-like platform, although only
Linux has been extensively tested and can be recommented for production
deployment.
**Prerequisites**:
Listed here are any packages outside pip:
* Python 3.7+
* MongoDB 3.6+ (for the REST API)
* nginx (for ssl offloading and better security)
* globus (for data transfer)
**Installation**:
From the latest release:
Get the tarball link from https://github.com/WIPACrepo/iceprod/releases/latest
Then install like::
pip install https://github.com/WIPACrepo/iceprod/archive/v2.0.0.tar.gz
**Installing from master**:
If you must install the dev version from master, do::
pip install --upgrade git+git://github.com/WIPACrepo/iceprod.git#egg=iceprod
Raw data
{
"_id": null,
"home_page": "https://github.com/WIPACrepo/iceprod",
"name": "iceprod",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.13,>=3.11",
"maintainer_email": null,
"keywords": "batch, workload",
"author": "WIPAC Developers",
"author_email": "developers@icecube.wisc.edu",
"download_url": "https://files.pythonhosted.org/packages/3d/00/bdad7baabf9182a18268102b6e1e5f7f952393305bc4015f4a3a75241b73/iceprod-3.0.36.tar.gz",
"platform": null,
"description": "IceProd\n=======\n\n.. image:: https://zenodo.org/badge/58235078.svg\n :target: https://zenodo.org/badge/latestdoi/58235078\n\nIceProd is a Python framework for distributed management of batch jobs.\nIt runs as a layer on top of other batch middleware, such as HTCondor,\nand can pool together resources from different batch systems.\nThe primary purpose is to coordinate and administer many large sets of\njobs at once, keeping a history of the entire job lifecycle.\n\nSee also: Aartsen, Mark G., et al. \"The IceProd framework: Distributed data processing for the IceCube neutrino observatory.\" Journal of parallel and distributed computing 75 (2015): 198-211.\n\n**Note:**\n\nFor IceCube users with CVMFS access, IceProd is already installed.\nTo load the environment execute::\n\n /cvmfs/icecube.wisc.edu/iceprod/latest/env-shell.sh\n\nor::\n\n eval `/cvmfs/icecube.wisc.edu/iceprod/latest/setup.sh`\n\ndepending on whether you want to get a new shell or load the variables\ninto the current shell.\n\nInstallation\n------------\n\n**Platforms**:\n\nIceProd should run on any Unix-like platform, although only\nLinux has been extensively tested and can be recommented for production\ndeployment.\n\n**Prerequisites**:\n\nListed here are any packages outside pip:\n\n* Python 3.7+\n* MongoDB 3.6+ (for the REST API)\n* nginx (for ssl offloading and better security)\n* globus (for data transfer)\n\n**Installation**:\n\nFrom the latest release:\n\nGet the tarball link from https://github.com/WIPACrepo/iceprod/releases/latest\n\nThen install like::\n\n pip install https://github.com/WIPACrepo/iceprod/archive/v2.0.0.tar.gz\n\n**Installing from master**:\n\nIf you must install the dev version from master, do::\n\n pip install --upgrade git+git://github.com/WIPACrepo/iceprod.git#egg=iceprod\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "IceCube dataset management system",
"version": "3.0.36",
"project_urls": {
"Download": "https://pypi.org/project/iceprod/",
"Homepage": "https://github.com/WIPACrepo/iceprod",
"Source": "https://github.com/WIPACrepo/iceprod",
"Tracker": "https://github.com/WIPACrepo/iceprod/issues"
},
"split_keywords": [
"batch",
" workload"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0a6bb2e3deebf8eafa377a54e90614b2e8a314dd0165654908ecb6b65007cf55",
"md5": "e57cd71646c22f52c7bffea1cd6bf813",
"sha256": "669744d198f38108044fc11a0b0c4f1580b9d72ded6fc2000415141b0627db36"
},
"downloads": -1,
"filename": "iceprod-3.0.36-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e57cd71646c22f52c7bffea1cd6bf813",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.13,>=3.11",
"size": 199508,
"upload_time": "2024-11-22T22:34:42",
"upload_time_iso_8601": "2024-11-22T22:34:42.523633Z",
"url": "https://files.pythonhosted.org/packages/0a/6b/b2e3deebf8eafa377a54e90614b2e8a314dd0165654908ecb6b65007cf55/iceprod-3.0.36-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3d00bdad7baabf9182a18268102b6e1e5f7f952393305bc4015f4a3a75241b73",
"md5": "37c1b7400c4c2dd23f35872388f9f3ac",
"sha256": "553903243f0efd6097620052c688f5dbded26a098f0e8296b175dd2c15c30fcf"
},
"downloads": -1,
"filename": "iceprod-3.0.36.tar.gz",
"has_sig": false,
"md5_digest": "37c1b7400c4c2dd23f35872388f9f3ac",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.13,>=3.11",
"size": 155598,
"upload_time": "2024-11-22T22:34:43",
"upload_time_iso_8601": "2024-11-22T22:34:43.957868Z",
"url": "https://files.pythonhosted.org/packages/3d/00/bdad7baabf9182a18268102b6e1e5f7f952393305bc4015f4a3a75241b73/iceprod-3.0.36.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-22 22:34:43",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "WIPACrepo",
"github_project": "iceprod",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "anyio",
"specs": [
[
"==",
"4.6.2.post1"
]
]
},
{
"name": "asyncache",
"specs": [
[
"==",
"0.3.1"
]
]
},
{
"name": "attrs",
"specs": [
[
"==",
"24.2.0"
]
]
},
{
"name": "boto3",
"specs": [
[
"==",
"1.35.68"
]
]
},
{
"name": "botocore",
"specs": [
[
"==",
"1.35.68"
]
]
},
{
"name": "cachetools",
"specs": [
[
"==",
"5.5.0"
]
]
},
{
"name": "certifi",
"specs": [
[
"==",
"2024.8.30"
]
]
},
{
"name": "cffi",
"specs": [
[
"==",
"1.17.1"
]
]
},
{
"name": "charset-normalizer",
"specs": [
[
"==",
"3.4.0"
]
]
},
{
"name": "cryptography",
"specs": [
[
"==",
"43.0.3"
]
]
},
{
"name": "dnspython",
"specs": [
[
"==",
"2.7.0"
]
]
},
{
"name": "exceptiongroup",
"specs": [
[
"==",
"1.2.2"
]
]
},
{
"name": "h11",
"specs": [
[
"==",
"0.14.0"
]
]
},
{
"name": "htcondor",
"specs": [
[
"==",
"24.1.1"
]
]
},
{
"name": "httpcore",
"specs": [
[
"==",
"1.0.7"
]
]
},
{
"name": "httpx",
"specs": [
[
"==",
"0.27.2"
]
]
},
{
"name": "idna",
"specs": [
[
"==",
"3.10"
]
]
},
{
"name": "jmespath",
"specs": [
[
"==",
"1.0.1"
]
]
},
{
"name": "jsonschema",
"specs": [
[
"==",
"4.23.0"
]
]
},
{
"name": "jsonschema-specifications",
"specs": [
[
"==",
"2024.10.1"
]
]
},
{
"name": "ldap3",
"specs": [
[
"==",
"2.9.1"
]
]
},
{
"name": "motor",
"specs": [
[
"==",
"3.6.0"
]
]
},
{
"name": "psutil",
"specs": [
[
"==",
"6.1.0"
]
]
},
{
"name": "pyasn1",
"specs": [
[
"==",
"0.6.1"
]
]
},
{
"name": "pycparser",
"specs": [
[
"==",
"2.22"
]
]
},
{
"name": "pyjwt",
"specs": [
[
"==",
"2.10.0"
]
]
},
{
"name": "pymongo",
"specs": [
[
"==",
"4.9.2"
]
]
},
{
"name": "pyopenssl",
"specs": [
[
"==",
"24.2.1"
]
]
},
{
"name": "python-dateutil",
"specs": [
[
"==",
"2.9.0.post0"
]
]
},
{
"name": "pyyaml",
"specs": [
[
"==",
"6.0.2"
]
]
},
{
"name": "qrcode",
"specs": [
[
"==",
"8.0"
]
]
},
{
"name": "referencing",
"specs": [
[
"==",
"0.35.1"
]
]
},
{
"name": "requests",
"specs": [
[
"==",
"2.32.3"
]
]
},
{
"name": "requests-futures",
"specs": [
[
"==",
"1.0.2"
]
]
},
{
"name": "requests-toolbelt",
"specs": [
[
"==",
"1.0.0"
]
]
},
{
"name": "rpds-py",
"specs": [
[
"==",
"0.21.0"
]
]
},
{
"name": "s3transfer",
"specs": [
[
"==",
"0.10.4"
]
]
},
{
"name": "setproctitle",
"specs": [
[
"==",
"1.3.4"
]
]
},
{
"name": "six",
"specs": [
[
"==",
"1.16.0"
]
]
},
{
"name": "sniffio",
"specs": [
[
"==",
"1.3.1"
]
]
},
{
"name": "statsd",
"specs": [
[
"==",
"4.0.1"
]
]
},
{
"name": "tornado",
"specs": [
[
"==",
"6.4.2"
]
]
},
{
"name": "typing-extensions",
"specs": [
[
"==",
"4.12.2"
]
]
},
{
"name": "unidecode",
"specs": [
[
"==",
"1.3.8"
]
]
},
{
"name": "urllib3",
"specs": [
[
"==",
"2.2.3"
]
]
},
{
"name": "wipac-dev-tools",
"specs": [
[
"==",
"1.13.0"
]
]
},
{
"name": "wipac-rest-tools",
"specs": [
[
"==",
"1.8.2"
]
]
}
],
"lcname": "iceprod"
}
JSON
