# glue-utils
[](LICENSE)
## Description
Reusable utilities for working with Glue PySpark jobs
## Installation
### As a runtime (or production) dependency...
```sh
pip install glue-utils
```
### For development...
This library does not include `pyspark` and `aws-glue-libs` as
dependencies as they are already pre-installed in Glue's runtime
environment.
To help in developing your Glue jobs locally in your IDE, it is helpful
to install `pyspark` and `aws-glue-libs`. Unfortunately, `aws-glue-libs`
is not available through PyPI so we can only install it from its git
repository.
```sh
pip install pyspark==3.3.0
pip install git+https://github.com/awslabs/aws-glue-libs.git@master
```
To make your local environment as close to Glue's runtime as possible,
use the versions specified in [this document](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-python-libraries.html#glue-modules-provided).
## Documentation
For more details on what you can use this library for, check out the
[project wiki](https://github.com/dashmug/glue-utils/wiki).
Raw data
{
"_id": null,
"home_page": "https://github.com/dashmug/glue-utils",
"name": "glue-utils",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.10",
"maintainer_email": null,
"keywords": "aws, glue, pyspark, spark, etl, data, data-engineering",
"author": "Noel Llevares",
"author_email": "dashmug@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/dd/3e/5a5a07da24c33799044f49ef96bf055ff51edbeefaf553d7385faacdfad8/glue_utils-0.2.0.tar.gz",
"platform": null,
"description": "# glue-utils\n\n[](LICENSE)\n\n## Description\n\nReusable utilities for working with Glue PySpark jobs\n\n## Installation\n\n### As a runtime (or production) dependency...\n\n```sh\npip install glue-utils\n```\n\n### For development...\n\nThis library does not include `pyspark` and `aws-glue-libs` as \ndependencies as they are already pre-installed in Glue's runtime \nenvironment.\n\nTo help in developing your Glue jobs locally in your IDE, it is helpful \nto install `pyspark` and `aws-glue-libs`. Unfortunately, `aws-glue-libs` \nis not available through PyPI so we can only install it from its git \nrepository.\n\n```sh\npip install pyspark==3.3.0\npip install git+https://github.com/awslabs/aws-glue-libs.git@master\n```\n\nTo make your local environment as close to Glue's runtime as possible, \nuse the versions specified in [this document](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-python-libraries.html#glue-modules-provided).\n\n## Documentation\n\nFor more details on what you can use this library for, check out the \n[project wiki](https://github.com/dashmug/glue-utils/wiki).",
"bugtrack_url": null,
"license": "MIT",
"summary": "Reusable utilities for working with Glue PySpark jobs",
"version": "0.2.0",
"project_urls": {
"Documentation": "https://github.com/dashmug/glue-utils/wiki",
"Homepage": "https://github.com/dashmug/glue-utils",
"Repository": "https://github.com/dashmug/glue-utils/issues"
},
"split_keywords": [
"aws",
" glue",
" pyspark",
" spark",
" etl",
" data",
" data-engineering"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d3aaf8717fc7cfad7af3256d8110d7c009eed8b3727b51ba55fb87de6e979095",
"md5": "8b816d48a33ce9cca6c5eb3e546b4dee",
"sha256": "0394a91b036aa9a0a03581474d19a46daa60e3caed0766884c6bf694aeecb51a"
},
"downloads": -1,
"filename": "glue_utils-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8b816d48a33ce9cca6c5eb3e546b4dee",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.10",
"size": 4680,
"upload_time": "2024-05-02T06:56:16",
"upload_time_iso_8601": "2024-05-02T06:56:16.214122Z",
"url": "https://files.pythonhosted.org/packages/d3/aa/f8717fc7cfad7af3256d8110d7c009eed8b3727b51ba55fb87de6e979095/glue_utils-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "dd3e5a5a07da24c33799044f49ef96bf055ff51edbeefaf553d7385faacdfad8",
"md5": "1dadcfa2020af215f60a84087e72635c",
"sha256": "4ddfd5670e458973f26bd7315c1e011f73f843bda1d3053f93763ed1f956b8a6"
},
"downloads": -1,
"filename": "glue_utils-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "1dadcfa2020af215f60a84087e72635c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.10",
"size": 4681,
"upload_time": "2024-05-02T06:56:17",
"upload_time_iso_8601": "2024-05-02T06:56:17.453159Z",
"url": "https://files.pythonhosted.org/packages/dd/3e/5a5a07da24c33799044f49ef96bf055ff51edbeefaf553d7385faacdfad8/glue_utils-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-02 06:56:17",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "dashmug",
"github_project": "glue-utils",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "glue-utils"
}
JSON
