# Flytekit Custom Plugins
This repository contains multiple custom Flytekit plugins for demonstrating user-defined tasks.
## Table of Contents
- [Introduction](#introduction)
- [Installation](#installation)
- [Development Installation](#development-installation)
- [Project Structure](#project-structure)
- [Usage](#usage)
- [Development Workflow](#development-workflow)
- [Testing](#testing)
## Introduction
This project demonstrates how to create custom plugins for Flytekit using Python. Flytekit is a Python SDK for defining, deploying, and managing data and ML workflows on Flyte. The custom plugins in this repository extend Flytekit's capabilities by adding new tasks.
## Installation
To install the package and its dependencies, use the following command:
```bash
pip install .
```
### Development Installation
```bash
pip install .[dev]
```
## Project Structure
flytekit-custom-plugins/
├── README.md
├── pyproject.toml
├── src/
│ ├── __init__.py
│ ├── plugin_one/
│ │ ├── __init__.py
│ │ ├── custom_task_one.py
│ ├── plugin_two/
│ │ ├── __init__.py
│ │ ├── custom_task_two.py
└── tests/
├── __init__.py
├── test_plugin_one.py
├── test_plugin_two.py
### File overview
1. README.md: Project documentation.
2. pyproject.toml: Configuration file for project metadata, dependencies, and build system.
3. src/: Source code directory.
4. plugin_one/: Contains the first custom plugin.
5. custom_task_one.py: Implementation of the first custom task.
6. plugin_two/: Contains the second custom plugin.
7. custom_task_two.py: Implementation of the second custom task.
8. tests/: Test suite directory.
9. test_plugin_one.py: Tests for the first custom plugin.
10. test_plugin_two.py: Tests for the second custom plugin.
## Usage
```python
from plugin_one.custom_task_one import MyCustomTaskOne
from plugin_two.custom_task_two import MyCustomTaskTwo
task_one = MyCustomTaskOne(name="example_one", some_parameter="value")
result_one = task_one.execute()
print(result_one)
task_two = MyCustomTaskTwo(name="example_two", another_parameter="value")
result_two = task_two.execute()
print(result_two)
```
## Development Workflow
```bash
python -m venv venv
source venv/bin/activate # On Windows, use `venv\Scripts\activate`
pip install .[dev]
```
## Adding a New Plugin
```bash
mkdir src/plugin_three
touch src/plugin_three/__init__.py
```
Implement a new plugin.
Register the pluging in the toml file.
## Testing
Ensure to write the tests for the plugins implmenented in the tests folder.
Run pytest in the console.
Raw data
{
"_id": null,
"home_page": "https://gitlab.com/flyte2/flyte-pipeline.git",
"name": "TestFlyte",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "Time Series Insight, TSI, TSI SDK",
"author": "Vaibhav",
"author_email": "vaibhav.baraskar@ielektron.com",
"download_url": "https://files.pythonhosted.org/packages/88/4b/6cb7d77ba3e24b0d088f784d6ddef7fe92ac05b1ac4a05062a2894612dd6/TestFlyte-1.0.2.tar.gz",
"platform": null,
"description": "# Flytekit Custom Plugins\n\nThis repository contains multiple custom Flytekit plugins for demonstrating user-defined tasks.\n\n## Table of Contents\n\n- [Introduction](#introduction)\n- [Installation](#installation)\n- [Development Installation](#development-installation)\n- [Project Structure](#project-structure)\n- [Usage](#usage)\n- [Development Workflow](#development-workflow)\n- [Testing](#testing)\n\n## Introduction\n\nThis project demonstrates how to create custom plugins for Flytekit using Python. Flytekit is a Python SDK for defining, deploying, and managing data and ML workflows on Flyte. The custom plugins in this repository extend Flytekit's capabilities by adding new tasks.\n\n## Installation\n\nTo install the package and its dependencies, use the following command:\n\n```bash\npip install .\n```\n### Development Installation\n\n```bash\npip install .[dev]\n```\n## Project Structure\n\nflytekit-custom-plugins/\n\u251c\u2500\u2500 README.md\n\u251c\u2500\u2500 pyproject.toml\n\u251c\u2500\u2500 src/\n\u2502 \u251c\u2500\u2500 __init__.py\n\u2502 \u251c\u2500\u2500 plugin_one/\n\u2502 \u2502 \u251c\u2500\u2500 __init__.py\n\u2502 \u2502 \u251c\u2500\u2500 custom_task_one.py\n\u2502 \u251c\u2500\u2500 plugin_two/\n\u2502 \u2502 \u251c\u2500\u2500 __init__.py\n\u2502 \u2502 \u251c\u2500\u2500 custom_task_two.py\n\u2514\u2500\u2500 tests/\n \u251c\u2500\u2500 __init__.py\n \u251c\u2500\u2500 test_plugin_one.py\n \u251c\u2500\u2500 test_plugin_two.py\n\n### File overview\n1. README.md: Project documentation.\n2. pyproject.toml: Configuration file for project metadata, dependencies, and build system.\n3. src/: Source code directory.\n4. plugin_one/: Contains the first custom plugin.\n5. custom_task_one.py: Implementation of the first custom task.\n6. plugin_two/: Contains the second custom plugin.\n7. custom_task_two.py: Implementation of the second custom task.\n8. tests/: Test suite directory.\n9. test_plugin_one.py: Tests for the first custom plugin.\n10. test_plugin_two.py: Tests for the second custom plugin.\n\n## Usage\n\n```python\nfrom plugin_one.custom_task_one import MyCustomTaskOne\nfrom plugin_two.custom_task_two import MyCustomTaskTwo\n\ntask_one = MyCustomTaskOne(name=\"example_one\", some_parameter=\"value\")\nresult_one = task_one.execute()\nprint(result_one)\n\ntask_two = MyCustomTaskTwo(name=\"example_two\", another_parameter=\"value\")\nresult_two = task_two.execute()\nprint(result_two)\n```\n\n## Development Workflow\n\n```bash\npython -m venv venv\nsource venv/bin/activate # On Windows, use `venv\\Scripts\\activate`\n\npip install .[dev]\n```\n\n## Adding a New Plugin\n\n```bash\nmkdir src/plugin_three\ntouch src/plugin_three/__init__.py\n```\n\nImplement a new plugin.\n\nRegister the pluging in the toml file.\n\n## Testing\n\nEnsure to write the tests for the plugins implmenented in the tests folder.\n\nRun pytest in the console.\n\n\n\n\n\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": null,
"version": "1.0.2",
"project_urls": {
"Homepage": "https://gitlab.com/flyte2/flyte-pipeline.git"
},
"split_keywords": [
"time series insight",
" tsi",
" tsi sdk"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "884b6cb7d77ba3e24b0d088f784d6ddef7fe92ac05b1ac4a05062a2894612dd6",
"md5": "9e8fe46de8e04460a0ecf767e4a41a8f",
"sha256": "5de2ba99e223ff8d25d5a62018273f4d76f011819810ce335b8bec18aa366b6c"
},
"downloads": -1,
"filename": "TestFlyte-1.0.2.tar.gz",
"has_sig": false,
"md5_digest": "9e8fe46de8e04460a0ecf767e4a41a8f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 3390,
"upload_time": "2024-08-02T11:25:21",
"upload_time_iso_8601": "2024-08-02T11:25:21.082911Z",
"url": "https://files.pythonhosted.org/packages/88/4b/6cb7d77ba3e24b0d088f784d6ddef7fe92ac05b1ac4a05062a2894612dd6/TestFlyte-1.0.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-02 11:25:21",
"github": false,
"gitlab": true,
"bitbucket": false,
"codeberg": false,
"gitlab_user": "flyte2",
"gitlab_project": "flyte-pipeline",
"lcname": "testflyte"
}
JSON