## Merlin Models
[](https://pypi.python.org/pypi/merlin-models/)
[](https://nvidia-merlin.github.io/models/)
The Merlin Models library provides standard models for recommender systems with an aim for high-quality implementations
that range from classic machine learning models to highly-advanced deep learning models.
The goal of this library is to make it easy for users in the industry to train and deploy recommender models with the best
practices that are already baked into the library. The library simplifies how users in the industry can train standard models against their dataset and put high-performance, GPU-accelerated models into production. The library also enables researchers to build custom
models by incorporating standard components of deep learning recommender models and then researchers can benchmark the new models on
example offline
datasets.
In our initial releases, Merlin Models features a TensorFlow API. The PyTorch API is initiated, but incomplete. We have PyTorch support for transformer-based, session-based recommender systems in the [Transformer4Rec](https://github.com/NVIDIA-Merlin/Transformers4Rec/) library.
### Benefits of Merlin Models
**[RecSys model implementations](https://nvidia-merlin.github.io/models/stable/models_overview.html)** - The library provides a high-level API for classic and state-of-the-art deep learning architectures for recommender models.
These models include both retrieval (e.g. Matrix Factorization, Two tower, YouTube DNN, ..) and ranking (e.g. DLRM, DCN-v2, DeepFM, ...) models.
**Building blocks** - Within Merlin Models, recommender models are built on reusable building blocks.
The design makes it easy to combine the blocks to define new architectures.
The library provides model definition blocks (MLP layers, factorization layers, input blocks, negative samplers, loss functions), training models (data loaders from Parquet files), and evaluation (e.g. ranking metrics).
**Integration with Merlin platform** - Merlin Models is deeply integrated with the other Merlin components.
For example, models depend on NVTabular for pre-processing and integrate easily with Merlin Systems for inference.
The thoughtfully-designed integration makes it straightforward to build performant end-to-end RecSys pipelines.
**[Merlin Models DataLoaders](https://nvidia-merlin.github.io/models/stable/api.html#loader-utility-functions)** - Merlin provides seamless integration with common deep learning frameworks, such as TensorFlow, PyTorch, and HugeCTR.
When training deep learning recommender system models, data loading can be a bottleneck.
To address the challenge, Merlin has custom, highly-optimized dataloaders to accelerate existing TensorFlow and PyTorch training pipelines.
The Merlin dataloaders can lead to a speedup that is nine times faster than the same training pipeline used with the GPU.
With the Merlin dataloaders, you can:
- Remove bottlenecks from data loading by processing large chunks of data at a time instead of item by item.
- Process datasets that don't fit within the GPU or CPU memory by streaming from the disk.
- Prepare batches asynchronously into the GPU to avoid CPU-to-GPU communication.
- Integrate easily into existing TensorFlow or PyTorch training pipelines by using a similar API.
To learn about the core features of Merlin Models, see the [Models Overview](https://nvidia-merlin.github.io/models/stable/models_overview.html) page.
### Installation
#### Installing Merlin Models Using Pip
Merlin Models can be installed with `pip` by running the following command:
```shell
pip install merlin-models
```
> Installing Merlin Models with `pip` does not install some additional GPU dependencies, such as the CUDA Toolkit.
> When you run Merlin Models in one of our Docker containers, the dependencies are already installed.
#### Docker Containers that include Merlin Models
Merlin Models is included in the Merlin Containers.
Refer to the [Merlin Containers](https://nvidia-merlin.github.io/Merlin/stable/containers.html) documentation page for information about the Merlin container names, URLs to the container images on the NVIDIA GPU Cloud catalog, and key Merlin components.
#### Installing Merlin Models from Source
Merlin Models can be installed from source by running the following commands:
```shell
git clone https://github.com/NVIDIA-Merlin/models
cd models && pip install -e .
```
### Getting Started
Merlin Models makes it straightforward to define architectures that adapt to different input features.
This adaptability is provided by building on a core feature of the NVTabular library.
When you use NVTabular for feature engineering, NVTabular creates a schema that identifies the input features.
You can see the `Schema` object in action by looking at the [From ETL to Training RecSys models - NVTabular and Merlin Models integrated example](https://nvidia-merlin.github.io/models/stable/examples/02-Merlin-Models-and-NVTabular-integration.html) example notebook.
You can easily build popular RecSys architectures like [DLRM](http://arxiv.org/abs/1906.00091), as shown in the following code sample.
After you define the model, you can train and evaluate it with a typical Keras model.
```python
import merlin.models.tf as mm
from merlin.io.dataset import Dataset
train = Dataset(PATH_TO_TRAIN_DATA)
valid = Dataset(PATH_TO_VALID_DATA)
model = mm.DLRMModel(
train.schema, # 1
embedding_dim=64,
bottom_block=mm.MLPBlock([128, 64]), # 2
top_block=mm.MLPBlock([128, 64, 32]),
prediction_tasks=mm.BinaryClassificationTask(train.schema) # 3
)
model.compile(optimizer="adagrad", run_eagerly=False)
model.fit(train, validation_data=valid, batch_size=1024)
eval_metrics = model.evaluate(valid, batch_size=1024, return_dict=True)
```
1. To build the internal input layer, the model identifies them from the schema object.
The schema identifies the continuous features and categorical features, for which embedding tables are created.
2. To define the body of the architecture, MLP layers are used with configurable dimensions.
3. The head of the architecture is created from the chosen task, `BinaryClassificationTask` in this example.
The target binary feature is also inferred from the schema (i.e., tagged as 'TARGET').
You can find more details and information about a low-level API in our overview of the
[Deep Learning Recommender Model](https://nvidia-merlin.github.io/models/stable/models_overview.html#deep-learning-recommender-model).
### Notebook Examples and Tutorials
View the example notebooks in the [documentation](https://nvidia-merlin.github.io/models/stable/examples/) to help you become familiar with Merlin Models.
The same notebooks are available in the `examples` directory from the [Merlin Models](https://github.com/NVIDIA-Merlin/models) GitHub repository.
### Feedback and Support
If you'd like to contribute to the library directly, see the [CONTRIBUTING.md](CONTRIBUTING.md) file.
We're particularly interested in contributions or feature requests for our feature engineering and preprocessing operations.
To further advance our Merlin Roadmap, we encourage you to share all the details regarding your recommender system pipeline in this [survey](https://developer.nvidia.com/merlin-devzone-survey).
Raw data
{
"_id": null,
"home_page": "https://github.com/NVIDIA-Merlin/models",
"name": "merlin-models",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "",
"author": "NVIDIA Corporation",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/af/49/ee899e9e3538a8861e125743ac28c76a2fea84ad9a7a54ab8915610050bb/merlin-models-23.12.0.tar.gz",
"platform": null,
"description": "## Merlin Models\n\n[](https://pypi.python.org/pypi/merlin-models/)\n\n[](https://nvidia-merlin.github.io/models/)\n\nThe Merlin Models library provides standard models for recommender systems with an aim for high-quality implementations\nthat range from classic machine learning models to highly-advanced deep learning models.\n\nThe goal of this library is to make it easy for users in the industry to train and deploy recommender models with the best\npractices that are already baked into the library. The library simplifies how users in the industry can train standard models against their dataset and put high-performance, GPU-accelerated models into production. The library also enables researchers to build custom\nmodels by incorporating standard components of deep learning recommender models and then researchers can benchmark the new models on\nexample offline\ndatasets.\n\nIn our initial releases, Merlin Models features a TensorFlow API. The PyTorch API is initiated, but incomplete. We have PyTorch support for transformer-based, session-based recommender systems in the [Transformer4Rec](https://github.com/NVIDIA-Merlin/Transformers4Rec/) library.\n\n### Benefits of Merlin Models\n\n**[RecSys model implementations](https://nvidia-merlin.github.io/models/stable/models_overview.html)** - The library provides a high-level API for classic and state-of-the-art deep learning architectures for recommender models.\nThese models include both retrieval (e.g. Matrix Factorization, Two tower, YouTube DNN, ..) and ranking (e.g. DLRM, DCN-v2, DeepFM, ...) models.\n\n**Building blocks** - Within Merlin Models, recommender models are built on reusable building blocks.\nThe design makes it easy to combine the blocks to define new architectures.\nThe library provides model definition blocks (MLP layers, factorization layers, input blocks, negative samplers, loss functions), training models (data loaders from Parquet files), and evaluation (e.g. ranking metrics).\n\n**Integration with Merlin platform** - Merlin Models is deeply integrated with the other Merlin components.\nFor example, models depend on NVTabular for pre-processing and integrate easily with Merlin Systems for inference.\nThe thoughtfully-designed integration makes it straightforward to build performant end-to-end RecSys pipelines.\n\n**[Merlin Models DataLoaders](https://nvidia-merlin.github.io/models/stable/api.html#loader-utility-functions)** - Merlin provides seamless integration with common deep learning frameworks, such as TensorFlow, PyTorch, and HugeCTR.\nWhen training deep learning recommender system models, data loading can be a bottleneck.\nTo address the challenge, Merlin has custom, highly-optimized dataloaders to accelerate existing TensorFlow and PyTorch training pipelines.\nThe Merlin dataloaders can lead to a speedup that is nine times faster than the same training pipeline used with the GPU.\n\nWith the Merlin dataloaders, you can:\n\n- Remove bottlenecks from data loading by processing large chunks of data at a time instead of item by item.\n- Process datasets that don't fit within the GPU or CPU memory by streaming from the disk.\n- Prepare batches asynchronously into the GPU to avoid CPU-to-GPU communication.\n- Integrate easily into existing TensorFlow or PyTorch training pipelines by using a similar API.\n\nTo learn about the core features of Merlin Models, see the [Models Overview](https://nvidia-merlin.github.io/models/stable/models_overview.html) page.\n\n### Installation\n\n#### Installing Merlin Models Using Pip\n\nMerlin Models can be installed with `pip` by running the following command:\n\n```shell\npip install merlin-models\n```\n\n> Installing Merlin Models with `pip` does not install some additional GPU dependencies, such as the CUDA Toolkit.\n> When you run Merlin Models in one of our Docker containers, the dependencies are already installed.\n\n#### Docker Containers that include Merlin Models\n\nMerlin Models is included in the Merlin Containers.\n\nRefer to the [Merlin Containers](https://nvidia-merlin.github.io/Merlin/stable/containers.html) documentation page for information about the Merlin container names, URLs to the container images on the NVIDIA GPU Cloud catalog, and key Merlin components.\n\n#### Installing Merlin Models from Source\n\nMerlin Models can be installed from source by running the following commands:\n\n```shell\ngit clone https://github.com/NVIDIA-Merlin/models\ncd models && pip install -e .\n```\n\n### Getting Started\n\nMerlin Models makes it straightforward to define architectures that adapt to different input features.\nThis adaptability is provided by building on a core feature of the NVTabular library.\nWhen you use NVTabular for feature engineering, NVTabular creates a schema that identifies the input features.\nYou can see the `Schema` object in action by looking at the [From ETL to Training RecSys models - NVTabular and Merlin Models integrated example](https://nvidia-merlin.github.io/models/stable/examples/02-Merlin-Models-and-NVTabular-integration.html) example notebook.\n\nYou can easily build popular RecSys architectures like [DLRM](http://arxiv.org/abs/1906.00091), as shown in the following code sample.\nAfter you define the model, you can train and evaluate it with a typical Keras model.\n\n```python\nimport merlin.models.tf as mm\nfrom merlin.io.dataset import Dataset\n\ntrain = Dataset(PATH_TO_TRAIN_DATA)\nvalid = Dataset(PATH_TO_VALID_DATA)\n\nmodel = mm.DLRMModel(\n train.schema, # 1\n embedding_dim=64,\n bottom_block=mm.MLPBlock([128, 64]), # 2\n top_block=mm.MLPBlock([128, 64, 32]),\n prediction_tasks=mm.BinaryClassificationTask(train.schema) # 3\n)\n\nmodel.compile(optimizer=\"adagrad\", run_eagerly=False)\nmodel.fit(train, validation_data=valid, batch_size=1024)\neval_metrics = model.evaluate(valid, batch_size=1024, return_dict=True)\n```\n\n1. To build the internal input layer, the model identifies them from the schema object.\n The schema identifies the continuous features and categorical features, for which embedding tables are created.\n2. To define the body of the architecture, MLP layers are used with configurable dimensions.\n3. The head of the architecture is created from the chosen task, `BinaryClassificationTask` in this example.\n The target binary feature is also inferred from the schema (i.e., tagged as 'TARGET').\n\nYou can find more details and information about a low-level API in our overview of the\n[Deep Learning Recommender Model](https://nvidia-merlin.github.io/models/stable/models_overview.html#deep-learning-recommender-model).\n\n### Notebook Examples and Tutorials\n\nView the example notebooks in the [documentation](https://nvidia-merlin.github.io/models/stable/examples/) to help you become familiar with Merlin Models.\n\nThe same notebooks are available in the `examples` directory from the [Merlin Models](https://github.com/NVIDIA-Merlin/models) GitHub repository.\n\n### Feedback and Support\n\nIf you'd like to contribute to the library directly, see the [CONTRIBUTING.md](CONTRIBUTING.md) file.\nWe're particularly interested in contributions or feature requests for our feature engineering and preprocessing operations.\nTo further advance our Merlin Roadmap, we encourage you to share all the details regarding your recommender system pipeline in this [survey](https://developer.nvidia.com/merlin-devzone-survey).\n\n\n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "Merlin recommender system models",
"version": "23.12.0",
"project_urls": {
"Homepage": "https://github.com/NVIDIA-Merlin/models"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "af49ee899e9e3538a8861e125743ac28c76a2fea84ad9a7a54ab8915610050bb",
"md5": "53ddbeae6e0c9d64852d15a07f6e51cc",
"sha256": "1d0619d406e557fd693199330971f096b260ad0a0798bf51d7c549d658317579"
},
"downloads": -1,
"filename": "merlin-models-23.12.0.tar.gz",
"has_sig": false,
"md5_digest": "53ddbeae6e0c9d64852d15a07f6e51cc",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 485961,
"upload_time": "2024-01-09T12:46:40",
"upload_time_iso_8601": "2024-01-09T12:46:40.195715Z",
"url": "https://files.pythonhosted.org/packages/af/49/ee899e9e3538a8861e125743ac28c76a2fea84ad9a7a54ab8915610050bb/merlin-models-23.12.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-09 12:46:40",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "NVIDIA-Merlin",
"github_project": "models",
"github_fetch_exception": true,
"lcname": "merlin-models"
}
JSON
