# glue-utils
[](LICENSE)
## Description
Reusable utilities for working with Glue PySpark jobs
## Installation
### As a runtime (or production) dependency...
```sh
pip install glue-utils
```
### For development...
This library does not include `pyspark` and `aws-glue-libs` as
dependencies as they are already pre-installed in Glue's runtime
environment.
To help in developing your Glue jobs locally in your IDE, it is helpful
to install `pyspark` and `aws-glue-libs`. Unfortunately, `aws-glue-libs`
is not available through PyPI so we can only install it from its git
repository.
```sh
pip install pyspark==3.3.0
pip install git+https://github.com/awslabs/aws-glue-libs.git@master
```
To make your local environment as close to Glue's runtime as possible,
use the versions specified in [this document](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-python-libraries.html#glue-modules-provided).
## Documentation
For more details on what you can use this library for, check out the
[project wiki](https://github.com/dashmug/glue-utils/wiki).
Raw data
{
"_id": null,
"home_page": "https://github.com/dashmug/glue-utils",
"name": "glue-utils",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10,<4.0",
"maintainer_email": "",
"keywords": "aws,glue,pyspark,spark,etl,data,data-engineering",
"author": "Noel Llevares",
"author_email": "dashmug@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/10/dc/55690c3f75ade5338af010b9baaaecbe1c12d4d56172059726010961e232/glue_utils-0.1.1.tar.gz",
"platform": null,
"description": "# glue-utils\n\n[](LICENSE)\n\n## Description\n\nReusable utilities for working with Glue PySpark jobs\n\n## Installation\n\n### As a runtime (or production) dependency...\n\n```sh\npip install glue-utils\n```\n\n### For development...\n\nThis library does not include `pyspark` and `aws-glue-libs` as \ndependencies as they are already pre-installed in Glue's runtime \nenvironment.\n\nTo help in developing your Glue jobs locally in your IDE, it is helpful \nto install `pyspark` and `aws-glue-libs`. Unfortunately, `aws-glue-libs` \nis not available through PyPI so we can only install it from its git \nrepository.\n\n```sh\npip install pyspark==3.3.0\npip install git+https://github.com/awslabs/aws-glue-libs.git@master\n```\n\nTo make your local environment as close to Glue's runtime as possible, \nuse the versions specified in [this document](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-python-libraries.html#glue-modules-provided).\n\n## Documentation\n\nFor more details on what you can use this library for, check out the \n[project wiki](https://github.com/dashmug/glue-utils/wiki).",
"bugtrack_url": null,
"license": "MIT",
"summary": "Reusable utilities for working with Glue PySpark jobs",
"version": "0.1.1",
"project_urls": {
"Documentation": "https://github.com/dashmug/glue-utils/wiki",
"Homepage": "https://github.com/dashmug/glue-utils",
"Repository": "https://github.com/dashmug/glue-utils/issues"
},
"split_keywords": [
"aws",
"glue",
"pyspark",
"spark",
"etl",
"data",
"data-engineering"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "eafe3aacadd79cd0d540e8934e571fe5f168a29161a325e9fcd88bfcb50e135b",
"md5": "fad89e2968e2689382d10011523ddbc9",
"sha256": "ae76b9f98c9d3d76bd1616bb7636f004f13855f51264929353d136c2ff7b5db3"
},
"downloads": -1,
"filename": "glue_utils-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "fad89e2968e2689382d10011523ddbc9",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10,<4.0",
"size": 3756,
"upload_time": "2024-03-07T09:37:36",
"upload_time_iso_8601": "2024-03-07T09:37:36.015889Z",
"url": "https://files.pythonhosted.org/packages/ea/fe/3aacadd79cd0d540e8934e571fe5f168a29161a325e9fcd88bfcb50e135b/glue_utils-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "10dc55690c3f75ade5338af010b9baaaecbe1c12d4d56172059726010961e232",
"md5": "f7b84f91f759b8e47f5cc3ccf447c2ff",
"sha256": "0bac170c586676c7a1493d7504749390309ee81291e0c5d5ee6e2ed93d364663"
},
"downloads": -1,
"filename": "glue_utils-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "f7b84f91f759b8e47f5cc3ccf447c2ff",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10,<4.0",
"size": 4212,
"upload_time": "2024-03-07T09:37:37",
"upload_time_iso_8601": "2024-03-07T09:37:37.727152Z",
"url": "https://files.pythonhosted.org/packages/10/dc/55690c3f75ade5338af010b9baaaecbe1c12d4d56172059726010961e232/glue_utils-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-07 09:37:37",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "dashmug",
"github_project": "glue-utils",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "glue-utils"
}
JSON
