# [Merlin Systems](https://github.com/NVIDIA-Merlin/systems)
[](https://pypi.python.org/pypi/merlin-systems/)
[](https://nvidia-merlin.github.io/systems/stable/README.html)
Merlin Systems provides tools for combining recommendation models with other elements of production recommender systems like feature stores, nearest neighbor search, and exploration strategies into end-to-end recommendation pipelines that can be served with [Triton Inference Server](https://github.com/triton-inference-server/server).
## Quickstart
Merlin Systems uses the Merlin Operator DAG API, the same API used in [NVTabular](https://github.com/NVIDIA-Merlin/NVTabular) for feature engineering, to create serving ensembles. To combine a feature engineering workflow and a Tensorflow model into an inference pipeline:
```python
import tensorflow as tf
from merlin.systems.dag import Ensemble
from merlin.systems.dag.ops import PredictTensorflow, TransformWorkflow
from nvtabular.workflow import Workflow
# Load saved NVTabular workflow and TensorFlow model
workflow = Workflow.load(nvtabular_workflow_path)
model = tf.keras.models.load_model(tf_model_path)
# Remove target/label columns from feature processing workflowk
workflow = workflow.remove_inputs([<target_columns>])
# Define ensemble pipeline
pipeline = (
workflow.input_schema.column_names >>
TransformWorkflow(workflow) >>
PredictTensorflow(model)
)
# Export artifacts to disk
ensemble = Ensemble(pipeline, workflow.input_schema)
ensemble.export(export_path)
```
After you export your ensemble, you reference the directory to run an instance of Triton Inference Server to host your ensemble.
```shell
tritonserver --model-repository=/export_path/
```
Refer to the [Merlin Example Notebooks](https://github.com/NVIDIA-Merlin/Merlin/tree/main/examples/ranking) for exploring notebooks that demonstrate
how to train and evaluate a ranking model with Merlin Models and then how to serve it as an ensemble on [Triton Inference Server](https://github.com/triton-inference-server/server).
For training models with XGBoost and Implicit, and then serving with Systems, you can visit these [examples](https://github.com/NVIDIA-Merlin/Merlin/tree/main/examples/traditional-ml).
## Building a Four-Stage Recommender Pipeline
Merlin Systems can also build more complex serving pipelines that integrate multiple models and external tools (like feature stores and nearest neighbor search):
```python
# Load artifacts for the pipeline
retrieval_model = tf.keras.models.load_model(retrieval_model_path)
ranking_model = tf.keras.models.load_model(ranking_model_path)
feature_store = feast.FeatureStore(feast_repo_path)
# Define the fields expected in requests
request_schema = Schema([
ColumnSchema("user_id", dtype=np.int32),
])
# Fetch user features, use them to a compute user vector with retrieval model,
# and find candidate items closest to the user vector with nearest neighbor search
user_features = request_schema.column_names >> QueryFeast.from_feature_view(
store=feature_store, view="user_features", column="user_id"
)
retrieval = (
user_features
>> PredictTensorflow(retrieval_model_path)
>> QueryFaiss(faiss_index_path, topk=100)
)
# Filter out candidate items that have already interacted with
# in the current session and fetch item features for the rest
filtering = retrieval["candidate_ids"] >> FilterCandidates(
filter_out=user_features["movie_ids"]
)
item_features = filtering >> QueryFeast.from_feature_view(
store=feature_store, view="movie_features", column="filtered_ids",
)
# Join user and item features for the candidates and use them to predict relevance scores
combined_features = item_features >> UnrollFeatures(
"movie_id", user_features, unrolled_prefix="user"
)
ranking = combined_features >> PredictTensorflow(ranking_model_path)
# Sort candidate items by relevance score with some randomized exploration
ordering = combined_features["movie_id"] >> SoftmaxSampling(
relevance_col=ranking["output"], topk=10, temperature=20.0
)
# Create and export the ensemble
ensemble = Ensemble(ordering, request_schema)
ensemble.export("./ensemble")
```
Refer to the [Example Notebooks](https://github.com/NVIDIA-Merlin/Merlin/tree/main/examples/Building-and-deploying-multi-stage-RecSys) for exploring
`building-and-deploying-multi-stage-RecSys` notebooks with Merlin Models and Systems.
## Installation
Merlin Systems requires Triton Inference Server and Tensorflow. The simplest setup is to use the [Merlin Tensorflow Inference Docker container](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/merlin/containers/merlin-tensorflow-inference), which has both pre-installed.
### Installing Merlin Systems Using Pip
You can install Merlin Systems with `pip`:
```shell
pip install merlin-systems
```
### Installing Merlin Systems from Source
Merlin Systems can be installed from source by cloning the GitHub repository and running `setup.py`
```shell
git clone https://github.com/NVIDIA-Merlin/systems.git
cd systems && python setup.py develop
```
### Running Merlin Systems from Docker
Merlin Systems is installed on multiple Docker containers that are available from the NVIDIA GPU Cloud (NGC) catalog.
The following table lists the containers that include Triton Inference Server for use with Merlin.
| Container Name | Container Location | Functionality |
| ------------------- | -------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- |
| `merlin-hugectr` | <https://catalog.ngc.nvidia.com/orgs/nvidia/teams/merlin/containers/merlin-hugectr> | Merlin frameworks, HugeCTR, and Triton Inference Server |
| `merlin-tensorflow` | <https://catalog.ngc.nvidia.com/orgs/nvidia/teams/merlin/containers/merlin-tensorflow> | Merlin frameworks selected for only Tensorflow support and Triton Inference Server |
If you want to add support for GPU-accelerated workflows, you will first need to install the [NVIDIA Container Toolkit](https://github.com/NVIDIA/nvidia-docker) to provide GPU support for Docker. You can use the NGC links referenced in the table above to obtain more information about how to launch and run these containers.
## Feedback and Support
To report bugs or get help, please [open an issue](https://github.com/NVIDIA-Merlin/Systems/issues/new/choose).
Raw data
{
"_id": null,
"home_page": "https://github.com/NVIDIA-Merlin/systems",
"name": "merlin-systems",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "",
"author": "NVIDIA Corporation",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/09/36/70192450f63e7fd30913a8098a47e371cac15560654148787f161c30d150/merlin-systems-23.8.0.tar.gz",
"platform": null,
"description": "# [Merlin Systems](https://github.com/NVIDIA-Merlin/systems)\n\n\n[](https://pypi.python.org/pypi/merlin-systems/)\n\n[](https://nvidia-merlin.github.io/systems/stable/README.html)\n\nMerlin Systems provides tools for combining recommendation models with other elements of production recommender systems like feature stores, nearest neighbor search, and exploration strategies into end-to-end recommendation pipelines that can be served with [Triton Inference Server](https://github.com/triton-inference-server/server).\n\n## Quickstart\n\nMerlin Systems uses the Merlin Operator DAG API, the same API used in [NVTabular](https://github.com/NVIDIA-Merlin/NVTabular) for feature engineering, to create serving ensembles. To combine a feature engineering workflow and a Tensorflow model into an inference pipeline:\n\n```python\nimport tensorflow as tf\n\nfrom merlin.systems.dag import Ensemble\nfrom merlin.systems.dag.ops import PredictTensorflow, TransformWorkflow\nfrom nvtabular.workflow import Workflow\n\n# Load saved NVTabular workflow and TensorFlow model\nworkflow = Workflow.load(nvtabular_workflow_path)\nmodel = tf.keras.models.load_model(tf_model_path)\n\n# Remove target/label columns from feature processing workflowk\nworkflow = workflow.remove_inputs([<target_columns>])\n\n# Define ensemble pipeline\npipeline = (\n\tworkflow.input_schema.column_names >>\n\tTransformWorkflow(workflow) >>\n\tPredictTensorflow(model)\n)\n\n# Export artifacts to disk\nensemble = Ensemble(pipeline, workflow.input_schema)\nensemble.export(export_path)\n```\n\nAfter you export your ensemble, you reference the directory to run an instance of Triton Inference Server to host your ensemble.\n\n```shell\ntritonserver --model-repository=/export_path/\n```\n\nRefer to the [Merlin Example Notebooks](https://github.com/NVIDIA-Merlin/Merlin/tree/main/examples/ranking) for exploring notebooks that demonstrate\nhow to train and evaluate a ranking model with Merlin Models and then how to serve it as an ensemble on [Triton Inference Server](https://github.com/triton-inference-server/server).\n\nFor training models with XGBoost and Implicit, and then serving with Systems, you can visit these [examples](https://github.com/NVIDIA-Merlin/Merlin/tree/main/examples/traditional-ml).\n\n## Building a Four-Stage Recommender Pipeline\n\nMerlin Systems can also build more complex serving pipelines that integrate multiple models and external tools (like feature stores and nearest neighbor search):\n\n```python\n# Load artifacts for the pipeline\nretrieval_model = tf.keras.models.load_model(retrieval_model_path)\nranking_model = tf.keras.models.load_model(ranking_model_path)\nfeature_store = feast.FeatureStore(feast_repo_path)\n\n# Define the fields expected in requests\nrequest_schema = Schema([\n ColumnSchema(\"user_id\", dtype=np.int32),\n])\n\n# Fetch user features, use them to a compute user vector with retrieval model,\n# and find candidate items closest to the user vector with nearest neighbor search\nuser_features = request_schema.column_names >> QueryFeast.from_feature_view(\n store=feature_store, view=\"user_features\", column=\"user_id\"\n)\n\nretrieval = (\n user_features\n >> PredictTensorflow(retrieval_model_path)\n >> QueryFaiss(faiss_index_path, topk=100)\n)\n\n# Filter out candidate items that have already interacted with\n# in the current session and fetch item features for the rest\nfiltering = retrieval[\"candidate_ids\"] >> FilterCandidates(\n filter_out=user_features[\"movie_ids\"]\n)\n\nitem_features = filtering >> QueryFeast.from_feature_view(\n store=feature_store, view=\"movie_features\", column=\"filtered_ids\",\n)\n\n# Join user and item features for the candidates and use them to predict relevance scores\ncombined_features = item_features >> UnrollFeatures(\n \"movie_id\", user_features, unrolled_prefix=\"user\"\n)\n\nranking = combined_features >> PredictTensorflow(ranking_model_path)\n\n# Sort candidate items by relevance score with some randomized exploration\nordering = combined_features[\"movie_id\"] >> SoftmaxSampling(\n relevance_col=ranking[\"output\"], topk=10, temperature=20.0\n)\n\n# Create and export the ensemble\nensemble = Ensemble(ordering, request_schema)\nensemble.export(\"./ensemble\")\n```\n\nRefer to the [Example Notebooks](https://github.com/NVIDIA-Merlin/Merlin/tree/main/examples/Building-and-deploying-multi-stage-RecSys) for exploring\n`building-and-deploying-multi-stage-RecSys` notebooks with Merlin Models and Systems.\n\n## Installation\n\nMerlin Systems requires Triton Inference Server and Tensorflow. The simplest setup is to use the [Merlin Tensorflow Inference Docker container](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/merlin/containers/merlin-tensorflow-inference), which has both pre-installed.\n\n### Installing Merlin Systems Using Pip\n\nYou can install Merlin Systems with `pip`:\n\n```shell\npip install merlin-systems\n```\n\n### Installing Merlin Systems from Source\n\nMerlin Systems can be installed from source by cloning the GitHub repository and running `setup.py`\n\n```shell\ngit clone https://github.com/NVIDIA-Merlin/systems.git\ncd systems && python setup.py develop\n```\n\n### Running Merlin Systems from Docker\n\nMerlin Systems is installed on multiple Docker containers that are available from the NVIDIA GPU Cloud (NGC) catalog.\nThe following table lists the containers that include Triton Inference Server for use with Merlin.\n\n| Container Name | Container Location | Functionality |\n| ------------------- | -------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- |\n| `merlin-hugectr` | <https://catalog.ngc.nvidia.com/orgs/nvidia/teams/merlin/containers/merlin-hugectr> | Merlin frameworks, HugeCTR, and Triton Inference Server |\n| `merlin-tensorflow` | <https://catalog.ngc.nvidia.com/orgs/nvidia/teams/merlin/containers/merlin-tensorflow> | Merlin frameworks selected for only Tensorflow support and Triton Inference Server |\n\nIf you want to add support for GPU-accelerated workflows, you will first need to install the [NVIDIA Container Toolkit](https://github.com/NVIDIA/nvidia-docker) to provide GPU support for Docker. You can use the NGC links referenced in the table above to obtain more information about how to launch and run these containers.\n\n## Feedback and Support\n\nTo report bugs or get help, please [open an issue](https://github.com/NVIDIA-Merlin/Systems/issues/new/choose).\n\n\n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "",
"version": "23.8.0",
"project_urls": {
"Homepage": "https://github.com/NVIDIA-Merlin/systems"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "093670192450f63e7fd30913a8098a47e371cac15560654148787f161c30d150",
"md5": "11a6181fa0711a7268139bfc57aa4971",
"sha256": "8fdec782c36b5103b7a72611d7925b453372f2137612f881da2aff104dd50ca1"
},
"downloads": -1,
"filename": "merlin-systems-23.8.0.tar.gz",
"has_sig": false,
"md5_digest": "11a6181fa0711a7268139bfc57aa4971",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 72059,
"upload_time": "2023-08-29T16:28:11",
"upload_time_iso_8601": "2023-08-29T16:28:11.942535Z",
"url": "https://files.pythonhosted.org/packages/09/36/70192450f63e7fd30913a8098a47e371cac15560654148787f161c30d150/merlin-systems-23.8.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-08-29 16:28:11",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "NVIDIA-Merlin",
"github_project": "systems",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "merlin-systems"
}
JSON
