# HEflow: A Privacy-Preserving Machine Learning Lifecycle Platform
HEflow is a platform to streamline privacy-preserving machine learning
development, including tracking experiments, packaging code into reproducible
runs, and sharing and deploying encrypted models. Built on top of
[MLflow](https://github.com/mlflow/mlflow),
[Seldon MLServer](https://github.com/SeldonIO/MLServer) and
[OpenMined TenSEAL](https://github.com/OpenMined/TenSEAL), HEflow offers a set
of lightweight homomorphic encryption APIs that can be used with any existing
machine learning application or library (scikit-learn, Keras, TensorFlow,
PyTorch, etc), wherever you currently run ML code (e.g. in notebooks, standalone
applications, or the cloud).
[](https://mybinder.org/v2/gh/inaccel/heflow/master)
[](https://badge.fury.io/py/heflow)
## Homomorphic Encryption (HE)
Homomorphic encryption differs from typical encryption methods in that it allows
computation to be performed directly on encrypted data without requiring access
to a secret key. The result of such a computation remains in encrypted form, and
can at a later point be revealed by the owner of the secret key. This
ground-breaking technology has enabled industry and government to provide
never-before enabled capabilities for outsourced computation securely.
Homomorphic encryption workflows, for privacy-preserving machine learning,
involve three entities:
1. an ML model owner,
2. a cloud server that performs model inference on HE encrypted data using the
pre-computed ML model, and
3. a user who sends confidential data to the cloud for model inference.
In all cases, the cloud should learn nothing about the underlying encrypted
data.
## Privacy-Preserving Machine Learning Operations (PPMLOps)
PPMLOps is a set of processes and automated steps to manage code, data, and
encrypted models. This section describes a typical PPMLOps workflow.
*PPMLOps using HEflow*
1. **Experiments**
Data scientists develop, train and tune the model on the production data,
then they encrypt ① and register it with the Encrypted Model Registry.
Model quality is evaluated by testing on held-out production data. This
pipeline can be triggered by code changes or by automated retraining jobs.
2. **Encrypted Model Registry**
Autologging saves a record of the training and evaluation process, which
includes model metrics, parameters, tags, and the encrypted model itself.
When training and hyperparameter tuning are complete, the data scientist
registers the final encrypted model artifact in the Encrypted Model Registry
for the production environment. This records a link between the encrypted
model and the code used to generate it.
2. **Encrypted Model Serving**
A continuous deployment (CD) process takes new encrypted models and deploys
② them for low-latency online serving (APIs). Options include cloud
provider serving endpoints, or custom serving applications.
2. **Invocations**
The serving system loads the Production encrypted model version from the
Encrypted Model Registry. For each request, it scores the encrypted data
③, and returns encrypted predictions ④.
## What is HEflow?
HEflow is an open source platform developed by InAccel to help manage the
complete privacy-preserving machine learning lifecycle with enterprise
reliability, security and scale. It tackles four primary functions:
### :hammer_and_wrench: Encrypted Model development
Accelerate and simplify privacy-preserving machine learning lifecycle management
with a standardized framework for developing production-ready PPML models. With
HEflow, you can bootstrap PPML projects, perform rapid iteration with ease and
ship high-quality encrypted models to production at scale.
### :clipboard: Experiment tracking
Run experiments with any ML library, framework or language, and automatically
keep track of parameters, metrics, code and encrypted models from each
experiment. By using HEflow, you can securely share, manage and compare
experiment results along with corresponding artifacts and code versions.
### :jigsaw: Encrypted Model management
Use one central place to discover and share PPML models, collaborate on moving
them from experimentation to online testing and production, integrate with
approval and governance workflows and CI/CD pipelines, and monitor PPML
deployments and their performance. HEflow facilitates sharing of expertise and
knowledge, and helps you stay in control.
### :package: Encrypted Model deployment
Quickly deploy production encrypted models for batch inference or as
[gRPC](https://github.com/grpc/grpc) homomorphic encryption APIs using built-in
integration with Docker containers or
[KServe](https://github.com/kserve/kserve). With HEflow, you can operationalize
and monitor production encrypted models to scale based on the business needs.
## Installing
Install HEflow from PyPI via `pip install heflow`
## Official HEflow Docker Image
The official HEflow Docker image is available on Docker Hub at
https://hub.docker.com/r/inaccel/heflow.
```sh
# Pull the latest version
docker pull inaccel/heflow
```
Raw data
{
"_id": null,
"home_page": "https://github.com/inaccel/heflow",
"name": "heflow",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "",
"author": "InAccel",
"author_email": "info@inaccel.com",
"download_url": "https://files.pythonhosted.org/packages/db/36/235fda4785716de0cfc9ebef246894b8aa40bfa8a1287132f87298226b21/heflow-1.5.0.tar.gz",
"platform": "Linux",
"description": "# HEflow: A Privacy-Preserving Machine Learning Lifecycle Platform\n\nHEflow is a platform to streamline privacy-preserving machine learning\ndevelopment, including tracking experiments, packaging code into reproducible\nruns, and sharing and deploying encrypted models. Built on top of\n[MLflow](https://github.com/mlflow/mlflow),\n[Seldon MLServer](https://github.com/SeldonIO/MLServer) and\n[OpenMined TenSEAL](https://github.com/OpenMined/TenSEAL), HEflow offers a set\nof lightweight homomorphic encryption APIs that can be used with any existing\nmachine learning application or library (scikit-learn, Keras, TensorFlow,\nPyTorch, etc), wherever you currently run ML code (e.g. in notebooks, standalone\napplications, or the cloud).\n\n[](https://mybinder.org/v2/gh/inaccel/heflow/master)\n[](https://badge.fury.io/py/heflow)\n\n## Homomorphic Encryption (HE)\n\nHomomorphic encryption differs from typical encryption methods in that it allows\ncomputation to be performed directly on encrypted data without requiring access\nto a secret key. The result of such a computation remains in encrypted form, and\ncan at a later point be revealed by the owner of the secret key. This\nground-breaking technology has enabled industry and government to provide\nnever-before enabled capabilities for outsourced computation securely.\n\nHomomorphic encryption workflows, for privacy-preserving machine learning,\ninvolve three entities:\n\n1. an ML model owner,\n\n2. a cloud server that performs model inference on HE encrypted data using the\npre-computed ML model, and\n\n3. a user who sends confidential data to the cloud for model inference.\n\nIn all cases, the cloud should learn nothing about the underlying encrypted\ndata.\n\n## Privacy-Preserving Machine Learning Operations (PPMLOps)\n\nPPMLOps is a set of processes and automated steps to manage code, data, and\nencrypted models. This section describes a typical PPMLOps workflow.\n\n\n*PPMLOps using HEflow*\n\n1. **Experiments**\n\n\tData scientists develop, train and tune the model on the production data,\n\tthen they encrypt ① and register it with the Encrypted Model Registry.\n\tModel quality is evaluated by testing on held-out production data. This\n\tpipeline can be triggered by code changes or by automated retraining jobs.\n\n2. **Encrypted Model Registry**\n\n\tAutologging saves a record of the training and evaluation process, which\n\tincludes model metrics, parameters, tags, and the encrypted model itself.\n\tWhen training and hyperparameter tuning are complete, the data scientist\n\tregisters the final encrypted model artifact in the Encrypted Model Registry\n\tfor the production environment. This records a link between the encrypted\n\tmodel and the code used to generate it.\n\n2. **Encrypted Model Serving**\n\n\tA continuous deployment (CD) process takes new encrypted models and deploys\n\t② them for low-latency online serving (APIs). Options include cloud\n\tprovider serving endpoints, or custom serving applications.\n\n2. **Invocations**\n\n\tThe serving system loads the Production encrypted model version from the\n\tEncrypted Model Registry. For each request, it scores the encrypted data\n\t③, and returns encrypted predictions ④.\n\n## What is HEflow?\n\nHEflow is an open source platform developed by InAccel to help manage the\ncomplete privacy-preserving machine learning lifecycle with enterprise\nreliability, security and scale. It tackles four primary functions:\n\n### :hammer_and_wrench: Encrypted Model development\n\nAccelerate and simplify privacy-preserving machine learning lifecycle management\nwith a standardized framework for developing production-ready PPML models. With\nHEflow, you can bootstrap PPML projects, perform rapid iteration with ease and\nship high-quality encrypted models to production at scale.\n\n### :clipboard: Experiment tracking\n\nRun experiments with any ML library, framework or language, and automatically\nkeep track of parameters, metrics, code and encrypted models from each\nexperiment. By using HEflow, you can securely share, manage and compare\nexperiment results along with corresponding artifacts and code versions.\n\n### :jigsaw: Encrypted Model management\n\nUse one central place to discover and share PPML models, collaborate on moving\nthem from experimentation to online testing and production, integrate with\napproval and governance workflows and CI/CD pipelines, and monitor PPML\ndeployments and their performance. HEflow facilitates sharing of expertise and\nknowledge, and helps you stay in control.\n\n### :package: Encrypted Model deployment\n\nQuickly deploy production encrypted models for batch inference or as\n[gRPC](https://github.com/grpc/grpc) homomorphic encryption APIs using built-in\nintegration with Docker containers or\n[KServe](https://github.com/kserve/kserve). With HEflow, you can operationalize\nand monitor production encrypted models to scale based on the business needs.\n\n## Installing\n\nInstall HEflow from PyPI via `pip install heflow`\n\n## Official HEflow Docker Image\n\nThe official HEflow Docker image is available on Docker Hub at\nhttps://hub.docker.com/r/inaccel/heflow.\n\n```sh\n# Pull the latest version\ndocker pull inaccel/heflow\n```\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "HEflow: A platform for the privacy-preserving machine learning lifecycle",
"version": "1.5.0",
"project_urls": {
"Homepage": "https://github.com/inaccel/heflow"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f5ff4c9ffde5ba3395b25a90803eb974e1b28aa9938f5d880812082835158241",
"md5": "e39b0bbad1ff64becec851cb17e07e3c",
"sha256": "2af6fa16b43b1381a6a5482c4ee88391e22f971a019cb847009495f0eae5dabd"
},
"downloads": -1,
"filename": "heflow-1.5.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e39b0bbad1ff64becec851cb17e07e3c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 17248,
"upload_time": "2023-08-18T09:58:02",
"upload_time_iso_8601": "2023-08-18T09:58:02.507321Z",
"url": "https://files.pythonhosted.org/packages/f5/ff/4c9ffde5ba3395b25a90803eb974e1b28aa9938f5d880812082835158241/heflow-1.5.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "db36235fda4785716de0cfc9ebef246894b8aa40bfa8a1287132f87298226b21",
"md5": "7447f41d1169c5d7b60535b4665a360b",
"sha256": "47974b9ae5bd575896698330e926e54fe7de4d48ec15640af8d3e14c021ea48a"
},
"downloads": -1,
"filename": "heflow-1.5.0.tar.gz",
"has_sig": false,
"md5_digest": "7447f41d1169c5d7b60535b4665a360b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 245511,
"upload_time": "2023-08-18T09:58:03",
"upload_time_iso_8601": "2023-08-18T09:58:03.731471Z",
"url": "https://files.pythonhosted.org/packages/db/36/235fda4785716de0cfc9ebef246894b8aa40bfa8a1287132f87298226b21/heflow-1.5.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-08-18 09:58:03",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "inaccel",
"github_project": "heflow",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "mlflow",
"specs": [
[
"==",
"2.6.0"
]
]
},
{
"name": "tenseal",
"specs": [
[
"==",
"0.3.14"
]
]
},
{
"name": "tqdm",
"specs": []
}
],
"lcname": "heflow"
}
JSON
