# ML & data helper code!
Jakub Langr (c) 2021
This is a CLI utility for speeding up basic Data Engineering/Data Science tasks
```
usage: dt [-h] [-d] [-q] [-v]
{cai,config,ec2,gcp,gh,hist,load,monitor,nf,py,run,s3,sftp,sg,ssh,td,viz}
...
ML & data helper code!
optional arguments:
-h, --help show this help message and exit
-d, --debug full application debug mode
-q, --quiet suppress all console output
-v, --version show program's version number and exit
sub-commands:
{cai,config,ec2,gcp,gh,hist,load,monitor,nf,py,run,s3,sftp,sg,ssh,td,viz}
cai Ops with Codex AI from OpenAI.
config Operations with config.
ec2 AWS EC2 helper commands.
gcp GCP VM Instances helper commands.These are currently
in alpha and some may not work
gh All Github related operations
hist Operations on CLI hist
load Appends defaults: from zshrc.txt to ~/.bashrc or
~/.zshrc or tmux conf
monitor Monitor lack of GPU activity. When there is none, runs
-job
nf Displays the number of files
py (WIP/Pre-Alpha) Execute a Python command across all
files in current dir.
run Operations with snippets
s3 Operations with s3 buckets
sftp Operations to easily work with remote servers /
devices
sg AWS Security Groups helper functions
ssh Operations on the SSH config
td Manage TODOs using Google Keep
viz Basic viz using streamlit for image comparisons
Usage: dt command [args] [kwargs]
```
The screen above can be called using:
```
dt -h
```
For example, to run to list the running ec2 isntances run
```
dt ec2 ls --profile <profile>
```
## Installation
```
$ pip install data-toolkit
```
## Tips and examples
Personally I find most helpful the following commands:
```
dt ec2 ls
```
Shows running EC2 instances with a range of parameters depending on the width of your terminal.
```
dt sg ls
```
Shows an overview of all security groups in the current region.
```
dt sg show
```
Shows detail of either all or one security group in the current region.
```
dt ssh update
```
Updates the SSH config with with the IP addresses of the running EC2 instances (matches them by name/tag).
When I'm travelling I like to use the following commands:
```
dt sg update -n <sg_name> -m "22:jakub 4g"
```
Which updates the `<sg_name>` security group to allow SSH from the current IP address and port 22 and sets the description to "jakub 4g".
**Hope this helps!**
## Development
This project includes a number of helpers in the `Makefile` to streamline common development tasks.
### Environment Setup
The following demonstrates setting up and working with a development environment:
```
### create a virtualenv for development
$ make virtualenv
$ source env/bin/activate
### run dt cli application
$ dt --help
### run pytest / coverage
$ make test
```
Raw data
{
"_id": null,
"home_page": "https://github.com/jakublangr/data-toolkit/",
"name": "data-toolkit",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "",
"author": "Jakub Langr",
"author_email": "james.langr@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/22/3e/cf697f06d6af8cef8e52bfadef800355766f41a750663424df9ee71f152f/data-toolkit-1.4.10.tar.gz",
"platform": null,
"description": "# ML & data helper code!\nJakub Langr (c) 2021\n\nThis is a CLI utility for speeding up basic Data Engineering/Data Science tasks\n\n```\nusage: dt [-h] [-d] [-q] [-v]\n {cai,config,ec2,gcp,gh,hist,load,monitor,nf,py,run,s3,sftp,sg,ssh,td,viz}\n ...\n\nML & data helper code!\n\noptional arguments:\n -h, --help show this help message and exit\n -d, --debug full application debug mode\n -q, --quiet suppress all console output\n -v, --version show program's version number and exit\n\nsub-commands:\n {cai,config,ec2,gcp,gh,hist,load,monitor,nf,py,run,s3,sftp,sg,ssh,td,viz}\n cai Ops with Codex AI from OpenAI.\n config Operations with config.\n ec2 AWS EC2 helper commands.\n gcp GCP VM Instances helper commands.These are currently\n in alpha and some may not work\n gh All Github related operations\n hist Operations on CLI hist\n load Appends defaults: from zshrc.txt to ~/.bashrc or\n ~/.zshrc or tmux conf\n monitor Monitor lack of GPU activity. When there is none, runs\n -job\n nf Displays the number of files\n py (WIP/Pre-Alpha) Execute a Python command across all\n files in current dir.\n run Operations with snippets\n s3 Operations with s3 buckets\n sftp Operations to easily work with remote servers /\n devices\n sg AWS Security Groups helper functions\n ssh Operations on the SSH config\n td Manage TODOs using Google Keep\n viz Basic viz using streamlit for image comparisons\n\nUsage: dt command [args] [kwargs]\n```\nThe screen above can be called using:\n```\ndt -h\n``` \n\nFor example, to run to list the running ec2 isntances run \n\n```\ndt ec2 ls --profile <profile>\n```\n\n## Installation\n\n```\n$ pip install data-toolkit\n```\n\n## Tips and examples\n\nPersonally I find most helpful the following commands:\n```\ndt ec2 ls\n```\nShows running EC2 instances with a range of parameters depending on the width of your terminal.\n```\ndt sg ls\n```\nShows an overview of all security groups in the current region.\n```\ndt sg show\n```\nShows detail of either all or one security group in the current region.\n```\ndt ssh update\n```\nUpdates the SSH config with with the IP addresses of the running EC2 instances (matches them by name/tag).\n\nWhen I'm travelling I like to use the following commands:\n```\ndt sg update -n <sg_name> -m \"22:jakub 4g\"\n```\nWhich updates the `<sg_name>` security group to allow SSH from the current IP address and port 22 and sets the description to \"jakub 4g\".\n\n**Hope this helps!**\n## Development\n\nThis project includes a number of helpers in the `Makefile` to streamline common development tasks.\n\n### Environment Setup\n\nThe following demonstrates setting up and working with a development environment:\n\n```\n### create a virtualenv for development\n\n$ make virtualenv\n\n$ source env/bin/activate\n\n\n### run dt cli application\n\n$ dt --help\n\n\n### run pytest / coverage\n\n$ make test\n```\n\n\n",
"bugtrack_url": null,
"license": "(c) Jakub Langr",
"summary": "ML & data helper code!",
"version": "1.4.10",
"project_urls": {
"Homepage": "https://github.com/jakublangr/data-toolkit/"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a57fd2d49b3b6b5cb598d6414b2f10764650e8bc79f8778da04e6647b61f865d",
"md5": "6c2387c125ccaaf06ab910a9eab97731",
"sha256": "dfc647683f882a987514f46d584b5a8dec8e78ddcfefdfb7bc78077314535a19"
},
"downloads": -1,
"filename": "data_toolkit-1.4.10-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6c2387c125ccaaf06ab910a9eab97731",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 63011,
"upload_time": "2023-07-12T16:28:51",
"upload_time_iso_8601": "2023-07-12T16:28:51.511896Z",
"url": "https://files.pythonhosted.org/packages/a5/7f/d2d49b3b6b5cb598d6414b2f10764650e8bc79f8778da04e6647b61f865d/data_toolkit-1.4.10-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "223ecf697f06d6af8cef8e52bfadef800355766f41a750663424df9ee71f152f",
"md5": "a07d8dcb3ff19ea1d3ac46c4ddcff7d1",
"sha256": "0d09d656b4ef433a1e49294707e6fd6b03fea27074e0414285c48c94a722bef6"
},
"downloads": -1,
"filename": "data-toolkit-1.4.10.tar.gz",
"has_sig": false,
"md5_digest": "a07d8dcb3ff19ea1d3ac46c4ddcff7d1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 50870,
"upload_time": "2023-07-12T16:28:53",
"upload_time_iso_8601": "2023-07-12T16:28:53.172994Z",
"url": "https://files.pythonhosted.org/packages/22/3e/cf697f06d6af8cef8e52bfadef800355766f41a750663424df9ee71f152f/data-toolkit-1.4.10.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-12 16:28:53",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "jakublangr",
"github_project": "data-toolkit",
"github_not_found": true,
"lcname": "data-toolkit"
}
JSON
