# LogNub
[](https://github.com/ChethanUK/lognub/actions/workflows/build_test.yml) [](https://github.com/ChethanUK/lognub/actions/workflows/code_quality_lint_checkers.yml) [](https://github.com/ChethanUK/lognub/actions/workflows/codeql-analysis.yml) [](https://codecov.io/gh/ChethanUK/lognub)
Loguru utility package
## TODO
1. Move logwrap [on top of loguru] extension out as a seperate package.
1. Add Test containers for [amundsen](https://www.amundsen.io/amundsen/), etc..
## Getting Started
1. Setup [SDKMAN](#setup-sdkman)
1. Setup [Java](#setup-java)
1. Setup [Apache Spark](#setup-apache-spark)
1. Install [Poetry](#poetry)
1. Install Pre-commit and [follow instruction in here](PreCommit.MD)
1. Run [tests locally](#running-tests-locally)
### Setup SDKMAN
SDKMAN is a tool for managing parallel Versions of multiple Software Development Kits on any Unix based
system. It provides a convenient command line interface for installing, switching, removing and listing
Candidates. SDKMAN! installs smoothly on Mac OSX, Linux, WSL, Cygwin, etc... Support Bash and ZSH shells. See
documentation on the [SDKMAN! website](https://sdkman.io).
Open your favourite terminal and enter the following:
```bash
$ curl -s https://get.sdkman.io | bash
If the environment needs tweaking for SDKMAN to be installed,
the installer will prompt you accordingly and ask you to restart.
Next, open a new terminal or enter:
$ source "$HOME/.sdkman/bin/sdkman-init.sh"
Lastly, run the following code snippet to ensure that installation succeeded:
$ sdk version
```
### Setup Java
Install Java Now open favourite terminal and enter the following:
```bash
List the AdoptOpenJDK OpenJDK versions
$ sdk list java
Install the Java 8:
$ sdk install java 8.0.292.hs-adpt
Set Java 8 as default Java Version:
$ sdk default java 8.0.292.hs-adpt
OR
To install For Java 11
$ sdk install java 11.0.10.hs-adpt
```
### Setup Apache Spark
Install Java Now open favourite terminal and enter the following:
```bash
List the Apache Spark versions:
$ sdk list spark
To install For Spark 3
$ sdk install spark 3.0.2
To install For Spark 3.1
$ sdk install spark 3.0.2
```
## Install PyEnv and Python 3.8
Either install pyenv via brew or github:
```bash
brew install pyenv
Then setup in zshrc:
echo 'eval "$(pyenv init --path)"' >> ~/.zprofile
echo 'eval "$(pyenv init -)"' >> ~/.zshrc
```
Then Install Python 3.8:
```bash
pyenv install 3.8.11
```
cd allocator project directory:
```bash
# Checkout git repo of iAllocator
cd dse-iAllocator
cd customer/allocator
# Now set default python version as 3.8.11
pyenv local 3.8.11
# Verify python version
python -V
Python 3.8.11
```
## Create VirtualENV
Create local venv inside allocator project:
```bash
cd dse-iAllocator [Repo directory]
cd customer/allocator
# Create virtualenv locally
python -m venv .venv
# Verify activate file exists
ls -G .venv/bin
# Activate virtual env python
source .venv/bin/activate
```
Verify virtual env and python is working and .venv is activated:
```bash
which python
# should end with {$ROOT_DIR}.venv/bin/python
which pip
# should end with {$ROOT_DIR}.venv/bin/pip
```
### Poetry
Poetry [Commands](https://python-poetry.org/docs/cli/#search) - Python package management tool
Install Poetry:
Install poetry using brew
```bash
brew install poetry
```
OR
Follow instructions for Linux: [here](https://python-poetry.org/docs/#osx--linux--bashonwindows-install-instructions)
```bash
# For osx / linux / bash on windows install:
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
```
Install the dep packages:
NOTE: Make sure you are connected to OpenVPN[since some are internal packages - catalog_client]
```bash
Install psycopg2 binary via PIP:
$ pip install psycopg2-binary==2.9.1
Install rest of packages via Poetry:
$ poetry install
# --tree: List the dependencies as a tree.
# --latest (-l): Show the latest version.
# --outdated (-o): Show the latest version but only for packages that are outdated.
poetry show -o
To update any package:
#$ poetry update pandas
```
## Running Tests Locally
Take a look at tests in `tests/dataquality` and `tests/jobs`
```bash
$ poetry run pytest
Ran 95 tests in 96.95s
```
Thats it, ENV is setup
NOTE: Loguru Wrap Package extracted from different internal package
NOTE: It's just curated stuff, Created for personal usage.
Raw data
{
"_id": null,
"home_page": "https://github.com/ChethanUK/",
"name": "lognub",
"maintainer": "ChethanUK",
"docs_url": null,
"requires_python": ">=3.7.2,<4.0",
"maintainer_email": "chethanuk@outlook.com",
"keywords": "Loguru,data-ops,data-engineering,best-practices",
"author": "ChethanUK",
"author_email": "chethanuk@outlook.com",
"download_url": "https://files.pythonhosted.org/packages/57/af/9f27b2488a4a925ed0eec017f21dcf36143ebdedbad79afaa5ec522dc897/lognub-0.2.1.tar.gz",
"platform": null,
"description": "# LogNub\n\n[](https://github.com/ChethanUK/lognub/actions/workflows/build_test.yml) [](https://github.com/ChethanUK/lognub/actions/workflows/code_quality_lint_checkers.yml) [](https://github.com/ChethanUK/lognub/actions/workflows/codeql-analysis.yml) [](https://codecov.io/gh/ChethanUK/lognub)\n\nLoguru utility package\n\n## TODO\n\n1. Move logwrap [on top of loguru] extension out as a seperate package.\n1. Add Test containers for [amundsen](https://www.amundsen.io/amundsen/), etc..\n\n## Getting Started\n\n1. Setup [SDKMAN](#setup-sdkman)\n1. Setup [Java](#setup-java)\n1. Setup [Apache Spark](#setup-apache-spark)\n1. Install [Poetry](#poetry)\n1. Install Pre-commit and [follow instruction in here](PreCommit.MD)\n1. Run [tests locally](#running-tests-locally)\n\n### Setup SDKMAN\n\nSDKMAN is a tool for managing parallel Versions of multiple Software Development Kits on any Unix based\nsystem. It provides a convenient command line interface for installing, switching, removing and listing\nCandidates. SDKMAN! installs smoothly on Mac OSX, Linux, WSL, Cygwin, etc... Support Bash and ZSH shells. See\ndocumentation on the [SDKMAN! website](https://sdkman.io).\n\nOpen your favourite terminal and enter the following:\n\n```bash\n$ curl -s https://get.sdkman.io | bash\nIf the environment needs tweaking for SDKMAN to be installed,\nthe installer will prompt you accordingly and ask you to restart.\n\nNext, open a new terminal or enter:\n\n$ source \"$HOME/.sdkman/bin/sdkman-init.sh\"\n\nLastly, run the following code snippet to ensure that installation succeeded:\n\n$ sdk version\n```\n\n### Setup Java\n\nInstall Java Now open favourite terminal and enter the following:\n\n```bash\nList the AdoptOpenJDK OpenJDK versions\n$ sdk list java\n\nInstall the Java 8:\n$ sdk install java 8.0.292.hs-adpt\n\nSet Java 8 as default Java Version:\n$ sdk default java 8.0.292.hs-adpt\n\nOR \n\nTo install For Java 11\n$ sdk install java 11.0.10.hs-adpt\n```\n\n### Setup Apache Spark\n\nInstall Java Now open favourite terminal and enter the following:\n\n```bash\nList the Apache Spark versions:\n$ sdk list spark\n\nTo install For Spark 3\n$ sdk install spark 3.0.2\n\nTo install For Spark 3.1\n$ sdk install spark 3.0.2\n```\n\n## Install PyEnv and Python 3.8\n\nEither install pyenv via brew or github:\n```bash\nbrew install pyenv\n\nThen setup in zshrc:\necho 'eval \"$(pyenv init --path)\"' >> ~/.zprofile\n\necho 'eval \"$(pyenv init -)\"' >> ~/.zshrc\n```\n\nThen Install Python 3.8:\n\n```bash\npyenv install 3.8.11\n```\n\ncd allocator project directory:\n```bash\n# Checkout git repo of iAllocator\ncd dse-iAllocator \ncd customer/allocator\n# Now set default python version as 3.8.11\npyenv local 3.8.11\n# Verify python version\npython -V\nPython 3.8.11\n```\n\n## Create VirtualENV\n\nCreate local venv inside allocator project:\n```bash\ncd dse-iAllocator [Repo directory] \ncd customer/allocator\n# Create virtualenv locally \npython -m venv .venv\n# Verify activate file exists \nls -G .venv/bin\n# Activate virtual env python\nsource .venv/bin/activate\n```\n\nVerify virtual env and python is working and .venv is activated:\n\n```bash\nwhich python\n# should end with {$ROOT_DIR}.venv/bin/python\nwhich pip\n# should end with {$ROOT_DIR}.venv/bin/pip\n```\n\n### Poetry\n\nPoetry [Commands](https://python-poetry.org/docs/cli/#search) - Python package management tool\n\nInstall Poetry:\n\nInstall poetry using brew\n```bash\nbrew install poetry\n```\n\nOR\n\nFollow instructions for Linux: [here](https://python-poetry.org/docs/#osx--linux--bashonwindows-install-instructions)\n\n```bash\n# For osx / linux / bash on windows install:\ncurl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -\n```\n\nInstall the dep packages:\n\nNOTE: Make sure you are connected to OpenVPN[since some are internal packages - catalog_client] \n```bash\nInstall psycopg2 binary via PIP:\n$ pip install psycopg2-binary==2.9.1\n\nInstall rest of packages via Poetry:\n\n$ poetry install\n\n# --tree: List the dependencies as a tree.\n# --latest (-l): Show the latest version.\n# --outdated (-o): Show the latest version but only for packages that are outdated.\npoetry show -o\n\nTo update any package:\n#$ poetry update pandas\n```\n\n## Running Tests Locally\n\nTake a look at tests in `tests/dataquality` and `tests/jobs`\n\n```bash\n$ poetry run pytest\nRan 95 tests in 96.95s\n```\n\nThats it, ENV is setup\n\nNOTE: Loguru Wrap Package extracted from different internal package\nNOTE: It's just curated stuff, Created for personal usage.\n\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Dumb Log Utlity for personal use",
"version": "0.2.1",
"project_urls": {
"Documentation": "https://github.com/ChethanUK/lognub#readme",
"Homepage": "https://github.com/ChethanUK/",
"Repository": "https://github.com/ChethanUK/lognub"
},
"split_keywords": [
"loguru",
"data-ops",
"data-engineering",
"best-practices"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a7ea702ff14e71d4e894edbffbe3763ccf2609cca80cdd34d387a09554aadb74",
"md5": "5286ca8af06ca9ccd63fb0e894fb8797",
"sha256": "3d11c2f4810216b6a92ed3e39209739cb6ae3da74dc45c3f35a0bcc715bf1d97"
},
"downloads": -1,
"filename": "lognub-0.2.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "5286ca8af06ca9ccd63fb0e894fb8797",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7.2,<4.0",
"size": 14170,
"upload_time": "2023-08-09T12:38:48",
"upload_time_iso_8601": "2023-08-09T12:38:48.813846Z",
"url": "https://files.pythonhosted.org/packages/a7/ea/702ff14e71d4e894edbffbe3763ccf2609cca80cdd34d387a09554aadb74/lognub-0.2.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "57af9f27b2488a4a925ed0eec017f21dcf36143ebdedbad79afaa5ec522dc897",
"md5": "d54bb5788cd2146f38f85e247e1ef98c",
"sha256": "26c3c08ced0fcf51f1879a19647e9e4199a1649f722ccfcce0e0541f15dd4a5a"
},
"downloads": -1,
"filename": "lognub-0.2.1.tar.gz",
"has_sig": false,
"md5_digest": "d54bb5788cd2146f38f85e247e1ef98c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7.2,<4.0",
"size": 13020,
"upload_time": "2023-08-09T12:38:50",
"upload_time_iso_8601": "2023-08-09T12:38:50.546224Z",
"url": "https://files.pythonhosted.org/packages/57/af/9f27b2488a4a925ed0eec017f21dcf36143ebdedbad79afaa5ec522dc897/lognub-0.2.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-08-09 12:38:50",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ChethanUK",
"github_project": "lognub#readme",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"lcname": "lognub"
}
JSON
