# django_mlops
!!! Docs in progress. Check back in a couple of days. !!!
Ignore the ML Ops salesmen and get back to keeping your ML system simple, flexible and impactful with this handy plugin module for Django.
This module is for Architects, Data Scientists and Data Engineers looking to save time in the long term by standardizing and simplifying their tech stack, not getting sucked in to vendor lock-in or tech stack complexity when they hit they realise the other packages don't have what they need.
# Motivation
[Read the Docs](https://eddyojb88.github.io/django_mlops/) for more info on this
# Features
- Define your pipeline processes (DAGs) simply
- Stakeholder facing visualization of your pipeline DAG in concept before and during development
- Register and execute your pipelines using either a standalone script or use Django website or via a speedy Django-ninja API
- Task monitoring
- Visualise pipeline process outputs and failures for each task
- Display data science experiment results for comparison
- Serve models with DRF or Django Ninja (FastAPI for Django)
- Minimal learning if you already know Django
- Using this with Django means you are not bogged down worrying about security, scalabili
- Scalable with Django Celery or Kuberenetes or any other extension you want to use
- Django is a mature framework with lots of security features if you want to serve models
- No vendor lockin
## Features to come
- Documentation page
- Implement better authentication and authorization options (at the moment you have to import the views in to urls and add login_required
- Async capability: allow user to use the dependency tree in the graph in order to wait for relevant tasks that have been offloaded to complete
- Make stakeholder only facing dashboard to display only tasks that succeeded
# Quick Start
Docs are here: [Read the Docs](https://eddyojb88.github.io/django_mlops/).
```
docker compose -f docker-compose-local.yml up
```
This runs both a Django container and a RabbitMQ container for the async task example with celery.
Next, connect in to the Django docker container.
There are a series of examples to showcase functionality, mostly without async. If you are not yet interested in async, you can skip the next part but if you are then within the ```example_project``` directory, run:
```
celery -A example_app worker --loglevel=info
```
This starts the celery instance for the example async task.
Next, run the django server with:
```
python manage.py runserver 0.0.0.0:8000
```
With the development server now running, you can view the list of trigger examples at:
```
http://localhost:8005/example/
```
In order to understand how this is being run, you can view the associated scripts in the ```example_app``` directory,
with ```pipeline_simple.py``` being the simplest example to view how a pipeline is registered. To view how the pipeline is called, go to ```views.py``` and the ```trigger_pipeline_simple``` function.
Once the task is complete, you can view the pipeline summary and associated info at the following page:
```
http://localhost:8005/django_mlops/task-runs-viz/
```
If wanting to conceptualize a task for stakeholders before or during development, you can view the pipeline in concept by going to:
```
http://localhost:8005/django_mlops/conceptual-dag-viz/
<img width="1057" alt="Screenshot 2024-02-27 at 11 45 02" src="https://github.com/eddyojb88/django_mlops/assets/22086433/36e80d55-4968-40e1-bf73-9eaef5247a8f">
Raw data
{
"_id": null,
"home_page": null,
"name": "django_mlops",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "django,mlops",
"author": null,
"author_email": "Your Name <eburmicz@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/21/20/f8f3d45624f2eeed8b3c8451ade7f35130d3b001dbcfeef6de824d5b9a75/django_mlops-0.1.2.tar.gz",
"platform": null,
"description": "# django_mlops\n\n!!! Docs in progress. Check back in a couple of days. !!!\n\nIgnore the ML Ops salesmen and get back to keeping your ML system simple, flexible and impactful with this handy plugin module for Django.\n\nThis module is for Architects, Data Scientists and Data Engineers looking to save time in the long term by standardizing and simplifying their tech stack, not getting sucked in to vendor lock-in or tech stack complexity when they hit they realise the other packages don't have what they need.\n\n\n\n# Motivation\n\n[Read the Docs](https://eddyojb88.github.io/django_mlops/) for more info on this\n\n\n# Features\n\n- Define your pipeline processes (DAGs) simply\n- Stakeholder facing visualization of your pipeline DAG in concept before and during development\n- Register and execute your pipelines using either a standalone script or use Django website or via a speedy Django-ninja API\n- Task monitoring\n- Visualise pipeline process outputs and failures for each task\n- Display data science experiment results for comparison\n- Serve models with DRF or Django Ninja (FastAPI for Django)\n- Minimal learning if you already know Django\n- Using this with Django means you are not bogged down worrying about security, scalabili\n- Scalable with Django Celery or Kuberenetes or any other extension you want to use\n- Django is a mature framework with lots of security features if you want to serve models \n- No vendor lockin\n\n ## Features to come\n - Documentation page\n - Implement better authentication and authorization options (at the moment you have to import the views in to urls and add login_required\n - Async capability: allow user to use the dependency tree in the graph in order to wait for relevant tasks that have been offloaded to complete\n - Make stakeholder only facing dashboard to display only tasks that succeeded\n\n# Quick Start\n\nDocs are here: [Read the Docs](https://eddyojb88.github.io/django_mlops/).\n\n```\ndocker compose -f docker-compose-local.yml up\n```\n\nThis runs both a Django container and a RabbitMQ container for the async task example with celery.\n\nNext, connect in to the Django docker container.\n\nThere are a series of examples to showcase functionality, mostly without async. If you are not yet interested in async, you can skip the next part but if you are then within the ```example_project``` directory, run:\n\n```\ncelery -A example_app worker --loglevel=info\n```\n\nThis starts the celery instance for the example async task.\n\nNext, run the django server with:\n\n```\npython manage.py runserver 0.0.0.0:8000\n```\n\nWith the development server now running, you can view the list of trigger examples at:\n\n```\nhttp://localhost:8005/example/\n```\n\nIn order to understand how this is being run, you can view the associated scripts in the ```example_app``` directory,\nwith ```pipeline_simple.py``` being the simplest example to view how a pipeline is registered. To view how the pipeline is called, go to ```views.py``` and the ```trigger_pipeline_simple``` function.\n\nOnce the task is complete, you can view the pipeline summary and associated info at the following page:\n\n```\nhttp://localhost:8005/django_mlops/task-runs-viz/\n```\n\nIf wanting to conceptualize a task for stakeholders before or during development, you can view the pipeline in concept by going to:\n\n```\nhttp://localhost:8005/django_mlops/conceptual-dag-viz/\n\n<img width=\"1057\" alt=\"Screenshot 2024-02-27 at 11 45 02\" src=\"https://github.com/eddyojb88/django_mlops/assets/22086433/36e80d55-4968-40e1-bf73-9eaef5247a8f\">",
"bugtrack_url": null,
"license": null,
"summary": "A Django app for MLOps: eliminate unnecessary duplication of tools and project complexity",
"version": "0.1.2",
"project_urls": {
"Homepage": "https://github.com/eddyojb88/django_mlops"
},
"split_keywords": [
"django",
"mlops"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "87dc69ad57ccc7b31e3beb8dbedd804e0b3e14f2a2d92ced49ca4311bec059c7",
"md5": "e3cb229a3831853011d985b103344f4f",
"sha256": "63a13b37b88afbce87c7fbd18bd9754c33a5b589b8c139aba8397fb229e3ab68"
},
"downloads": -1,
"filename": "django_mlops-0.1.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e3cb229a3831853011d985b103344f4f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 60706,
"upload_time": "2024-02-29T23:06:34",
"upload_time_iso_8601": "2024-02-29T23:06:34.381143Z",
"url": "https://files.pythonhosted.org/packages/87/dc/69ad57ccc7b31e3beb8dbedd804e0b3e14f2a2d92ced49ca4311bec059c7/django_mlops-0.1.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "2120f8f3d45624f2eeed8b3c8451ade7f35130d3b001dbcfeef6de824d5b9a75",
"md5": "744ddf8522ccbcc57abdf708ed042f83",
"sha256": "d062c4be8c4c70d5322904f9e1aafcccf287104c83c6c606800a461acbc7d2b9"
},
"downloads": -1,
"filename": "django_mlops-0.1.2.tar.gz",
"has_sig": false,
"md5_digest": "744ddf8522ccbcc57abdf708ed042f83",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 215651,
"upload_time": "2024-02-29T23:06:37",
"upload_time_iso_8601": "2024-02-29T23:06:37.495886Z",
"url": "https://files.pythonhosted.org/packages/21/20/f8f3d45624f2eeed8b3c8451ade7f35130d3b001dbcfeef6de824d5b9a75/django_mlops-0.1.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-29 23:06:37",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "eddyojb88",
"github_project": "django_mlops",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "Django",
"specs": []
},
{
"name": "django-htmx",
"specs": []
},
{
"name": "celery",
"specs": []
},
{
"name": "scikit-learn",
"specs": []
},
{
"name": "xgboost",
"specs": []
},
{
"name": "coverage",
"specs": []
}
],
"lcname": "django_mlops"
}
JSON
