# MLL - Metric Learning Layers
MLL is a simple [PyTorch](https://pytorch.org/) package that includes the most common metric learning layers.
MLL only includes layers that are not dependent on negative sample mining and therefore drop in replacements for
the final linear layer used in classification problems.
All layers aim to achieve greater inter-class variance and minimizing intra-class variance.
Moreover, all MLL-layers can be used in conjunction with soft-targets (e.g. with [Mixup](https://arxiv.org/abs/1710.09412)).
The basis of all these layers is the scaled cosine similarity $$y = xW * s$$ between
the $d$-dimensional input vectors (features) $x \in \mathbb{R}^{1 \times d}$ and the
$c$ class weights (prototypes, embeddings) $W \in \mathbb{R}^{d \times c}$
where $||x|| = 1$ and $||W_{*, j}|| = 1 \,\, \forall j= 1\dots c$ and $s \in \mathbb{R}^+$.
## Supported Layers
We currently support the following layers:
* [x] NormalizedLinear and ScaledNormalizedLinear
* [x] [CosFace](https://arxiv.org/abs/1801.09414)
* [x] [ArcFace](https://arxiv.org/abs/1801.07698)
* [x] [AdaCos and FixedAdaCos](https://arxiv.org/abs/1905.00292)
* [x] [DeepNCM](https://openreview.net/forum?id=rkPLZ4JPM)
You can use multiple sub-centers for all layers except for DeepNCM. If you do not specify a scale,
MLL will use the heuristic from AdaCos $s = \sqrt{2} * \log{c-1}$.
## Install MLL
Simply run:
```
pip install metric-learning-layers
```
## Example
```py
import torch
import metric_learning_layers as mll
rnd_batch = torch.randn(32, 128)
rnd_labels = torch.randint(low=0, high=10, size=(32, ))
arcface = mll.ArcFace(in_features=128,
out_features=10,
num_sub_centers=1,
scale=None, # defaults to AdaCos heuristic
trainable_scale=False
)
af_out = arcface(rnd_batch, rnd_labels) # ArcFace requires labels (used to apply the margin)
# af_out: torch.Size([32, 10])
adacos = mll.AdaCos(in_features=128,
out_features=10,
num_sub_centers=1
)
ac_out = arcface(rnd_batch) # AdaCos does not require labels
# af_out: torch.Size([32, 10])
```
Raw data
{
"_id": null,
"home_page": "https://github.com/romue404/metric_learning_layers",
"name": "Metric-Learning-Layers",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "metric learning,artificial intelligence,pytorch,classification,separability,large margin",
"author": "Robert M\u00fcller",
"author_email": "robert.mueller1990@googlemail.com",
"download_url": "https://files.pythonhosted.org/packages/b1/49/cb8bb937322c6cf78de1ac49437e7094c07fd4cffc0315d22c82847d530f/Metric%20Learning%20Layers-0.1.6.tar.gz",
"platform": null,
"description": "# MLL - Metric Learning Layers\nMLL is a simple [PyTorch](https://pytorch.org/) package that includes the most common metric learning layers.\nMLL only includes layers that are not dependent on negative sample mining and therefore drop in replacements for \nthe final linear layer used in classification problems.\nAll layers aim to achieve greater inter-class variance and minimizing intra-class variance. \nMoreover, all MLL-layers can be used in conjunction with soft-targets (e.g. with [Mixup](https://arxiv.org/abs/1710.09412)).\n\nThe basis of all these layers is the scaled cosine similarity $$y = xW * s$$ between \nthe $d$-dimensional input vectors (features) $x \\in \\mathbb{R}^{1 \\times d}$ and the \n$c$ class weights (prototypes, embeddings) $W \\in \\mathbb{R}^{d \\times c}$\nwhere $||x|| = 1$ and $||W_{*, j}|| = 1 \\,\\, \\forall j= 1\\dots c$ and $s \\in \\mathbb{R}^+$.\n\n## Supported Layers\nWe currently support the following layers:\n* [x] NormalizedLinear and ScaledNormalizedLinear\n* [x] [CosFace](https://arxiv.org/abs/1801.09414)\n* [x] [ArcFace](https://arxiv.org/abs/1801.07698)\n* [x] [AdaCos and FixedAdaCos](https://arxiv.org/abs/1905.00292)\n* [x] [DeepNCM](https://openreview.net/forum?id=rkPLZ4JPM)\n\nYou can use multiple sub-centers for all layers except for DeepNCM. If you do not specify a scale, \nMLL will use the heuristic from AdaCos $s = \\sqrt{2} * \\log{c-1}$.\n\n## Install MLL\nSimply run:\n```\npip install metric-learning-layers\n```\n\n## Example\n```py\nimport torch\nimport metric_learning_layers as mll\n\n\nrnd_batch = torch.randn(32, 128)\nrnd_labels = torch.randint(low=0, high=10, size=(32, ))\n\narcface = mll.ArcFace(in_features=128, \n out_features=10, \n num_sub_centers=1, \n scale=None, # defaults to AdaCos heuristic\n trainable_scale=False\n )\n\naf_out = arcface(rnd_batch, rnd_labels) # ArcFace requires labels (used to apply the margin)\n# af_out: torch.Size([32, 10])\n\nadacos = mll.AdaCos(in_features=128, \n out_features=10, \n num_sub_centers=1 \n )\n\nac_out = arcface(rnd_batch) # AdaCos does not require labels\n# af_out: torch.Size([32, 10])\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A simple PyTorch package that includes the most common metric learning layers.",
"version": "0.1.6",
"split_keywords": [
"metric learning",
"artificial intelligence",
"pytorch",
"classification",
"separability",
"large margin"
],
"urls": [
{
"comment_text": "",
"digests": {
"md5": "b036d51358e27c3af6c0ab67b225dbbb",
"sha256": "d197165a4e4aaf7173a7be8b660dfe94ac5e6be62f436055c43125283ebb4cd0"
},
"downloads": -1,
"filename": "Metric_Learning_Layers-0.1.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b036d51358e27c3af6c0ab67b225dbbb",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 6545,
"upload_time": "2022-12-01T15:27:52",
"upload_time_iso_8601": "2022-12-01T15:27:52.536120Z",
"url": "https://files.pythonhosted.org/packages/89/38/e4488ec216f01fd0d8afe930d93c2d4cb0e50e79834faf80a43cd04e574a/Metric_Learning_Layers-0.1.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"md5": "62cf11487717b9da420e47698bbe6657",
"sha256": "e5055427b4670ca3600bfa05f3257d4c5e3a2a89fbd85bf10545c59cf245301a"
},
"downloads": -1,
"filename": "Metric Learning Layers-0.1.6.tar.gz",
"has_sig": false,
"md5_digest": "62cf11487717b9da420e47698bbe6657",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 5677,
"upload_time": "2022-12-01T15:27:54",
"upload_time_iso_8601": "2022-12-01T15:27:54.768898Z",
"url": "https://files.pythonhosted.org/packages/b1/49/cb8bb937322c6cf78de1ac49437e7094c07fd4cffc0315d22c82847d530f/Metric%20Learning%20Layers-0.1.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-12-01 15:27:54",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "romue404",
"github_project": "metric_learning_layers",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "metric-learning-layers"
}
JSON
