# The Autoembedder
[](https://github.com/chrislemke/autoembedder/actions/workflows/deploy-package.yml)
[](https://www.codacy.com/gh/chrislemke/autoembedder/dashboard?utm_source=github.com&utm_medium=referral&utm_content=chrislemke/autoembedder&utm_campaign=Badge_Grade)
[](https://pypi.org/project/autoembedder/)
[](https://python.org)
[](https://chrislemke.github.io/autoembedder/)
[](https://github.com/chrislemke/autoembedder/blob/main/LICENSE)
[](https://pypistats.org/packages/autoembedder)
[](http://mypy-lang.org/)
[](https://github.com/psf/black)
[](https://pycqa.github.io/isort/)
[](https://github.com/pre-commit/pre-commit)
## Introduction
The Autoembedder is an autoencoder with additional embedding layers for the categorical columns. Its usage is flexible, and hyperparameters like the number of layers can be easily adjusted and tuned. The data provided for training can be either a path to a [Dask](https://docs.dask.org/en/stable/dataframe.html) or [Pandas](https://pandas.pydata.org/) DataFrame stored in the Parquet format or the DataFrame object directly.
## Installation
If you are using [Poetry](https://python-poetry.org/), you can install the package with the following command:
```bash
poetry add autoembedder
```
If you are using [pip](https://pypi.org/project/pip/), you can install the package with the following command:
```bash
pip install autoembedder
```
## Installing dependencies
With [Poetry](https://python-poetry.org/):
```bash
poetry install
```
With [pip](https://pypi.org/project/pip/):
```bash
pip install -r requirements.txt
```
## Usage
### 0. Some imports
```python
from autoembedder import Autoembedder, dataloader, fit
```
### 1. Create dataloaders
First, we create two [`dataloaders`](https://chrislemke.github.io/autoembedder/autoembedder.data/#autoembedder.data.Dataset.__init__). One for training, and the other for validation data. As `source` they either accept a path to a Parquet file, to a folder of Parquet files or a [Pandas](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html)/[Dask](https://docs.dask.org/en/stable/dataframe.html) DataFrame.
```python
train_dl = dataloader(train_df)
valid_dl = dataloader(vaild_df)
```
### 2. Set parameters
Now, we need to set the parameters. They are going to be used for handling the data and training the model. In this example, only parameters for the training are set. [Here](https://github.com/chrislemke/autoembedder#parameters) you find a list of all possible parameters. This should do it:
```python
parameters = {
"hidden_layers": [[25, 20], [20, 10]],
"epochs": 10,
"lr": 0.0001,
"verbose": 1,
}
```
### 3. Initialize the autoembedder
Then, we need to initialize the [autoembedder](https://chrislemke.github.io/autoembedder/autoembedder.model/#autoembedder.model.Autoembedder). In this example, we are not using any categorical features. So we can skip the `embedding_sizes` argument.
```python
model = Autoembedder(parameters, num_cont_features=train_df.shape[1])
```
### 4. Train the model
Everything is set up. Now we can [fit](https://chrislemke.github.io/autoembedder/autoembedder.learner/#autoembedder.learner.fit) the model.
```python
fit(parameters, model, train_dl, valid_dl)
```
## Example
Check out [this Jupyter notebook](https://github.com/chrislemke/autoembedder/blob/main/example.ipynb) for an applied example using the [Credit Card Fraud Detection](https://www.kaggle.com/datasets/mlg-ulb/creditcardfraud) from Kaggle.
## Parameters
This is a list of all parameters that can be passed to the Autoembedder for training. When using the training script the `_` needs to be replaced with `-` and the parameters need to be passed as arguments. For boolean values please have a look at the `Comment` column for understanding how to pass them.
## Run the training script
You can also simply use the training script::
```bash
python3 training.py \
--epochs 20 \
--train-input-path "path/to/your/train_data" \
--test-input-path "path/to/your/test_data" \
--hidden-layers "[[12, 6], [6, 3]]"
```
for help just run:
```bash
python3 training.py --help
```
| Argument | Type | Required | Default value | Comment |
| -------------------- | ----- | -------- | ----------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| batch_size | int | False | 32 | |
| drop_last | bool | False | True | --drop-last / --no-drop-last |
| pin_memory | bool | False | True | --pin-memory / --no-pin-memory |
| num_workers | int | False | 0 | 0 means that the data will be loaded in the main process |
| use_mps | bool | False | False | --use-mps / --no-use-mps |
| model_title | str | False | autoembedder_{`datetime`}.bin | |
| model_save_path | str | False | | |
| n_save_checkpoints | int | False | | |
| lr | float | False | 0.001 | |
| amsgrad | bool | False | False | --amsgrad / --no-amsgrad |
| epochs | int | True | |
| dropout_rate | float | False | 0 | Dropout rate for the dropout layers in the encoder and decoder. |
| layer_bias | bool | False | True | --layer-bias / --no-layer-bias | |
| weight_decay | float | False | False | |
| l1_lambda | float | False | 0 | |
| xavier_init | bool | False | False | --xavier-init / --no-xavier-init |
| activation | str | False | tanh | Activation function; either `tanh`, `relu`, `leaky_relu` or `elu` |
| tensorboard_log_path | str | False | | |
| trim_eval_errors | bool | False | False |--trim-eval-errors / --no-trim-eval-errors; Removes the max and min loss when calculating the `mean loss diff` and `median loss diff`. This can be useful if some rows create very high losses. |
| verbose | int | False | 0 | Set this to `1` if you want to see the model summary and the validation and evaluation results. set this to `2` if you want to see the training progress bar. `0` means no output. |
| target | str | False | | The target column. If not set no evaluation will be performed. |
| train_input_path | str | True | | |
| test_input_path | str | True | |
| eval_input_path | str | False | | Path to the evaluation data. If no path is provided no evaluation will be performed. | |
| hidden_layers | str | True | | Contains a string representation of a list of list of integers which represents the hidden layer structure. E.g.: `"[[64, 32], [32, 16], [16, 8]]"` activation |
| cat_columns | str | False | "[]" | Contains a string representation of a list of list of categorical columns (strings). The columns which use the same encoder should be together in a list. E.g.: `"[['a', 'b'], ['c']]"`. |
| drop-cat-columns | bool | False | |--drop-cat-columns / --no-drop-cat-columns
## Why additional embedding layers?
The additional embedding layers automatically embed all columns with the Pandas `category` data type. If categorical columns have another data type, they will not be embedded and will be handled like continuous columns. Simply encoding the categorical values (e.g., with the usage of a label encoder) decreases the quality of the outcome.
Raw data
{
"_id": null,
"home_page": "https://chrislemke.github.io/autoembedder/",
"name": "autoembedder",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<3.11",
"maintainer_email": "",
"keywords": "autoencoder,embeddings,model,pytorch,neural network,machine learning,data science",
"author": "Christopher Lemke",
"author_email": "chris@syhbl.mozmail.com",
"download_url": "https://files.pythonhosted.org/packages/4b/5e/e948c6630c401f8e0943dfec163839fe64771c657507ba7cbaed708ff06d/autoembedder-0.2.5.tar.gz",
"platform": null,
"description": "\n# The Autoembedder\n[](https://github.com/chrislemke/autoembedder/actions/workflows/deploy-package.yml)\n[](https://www.codacy.com/gh/chrislemke/autoembedder/dashboard?utm_source=github.com&utm_medium=referral&utm_content=chrislemke/autoembedder&utm_campaign=Badge_Grade)\n[](https://pypi.org/project/autoembedder/)\n[](https://python.org)\n[](https://chrislemke.github.io/autoembedder/)\n[](https://github.com/chrislemke/autoembedder/blob/main/LICENSE)\n[](https://pypistats.org/packages/autoembedder)\n[](http://mypy-lang.org/)\n[](https://github.com/psf/black)\n[](https://pycqa.github.io/isort/)\n[](https://github.com/pre-commit/pre-commit)\n## Introduction\nThe Autoembedder is an autoencoder with additional embedding layers for the categorical columns. Its usage is flexible, and hyperparameters like the number of layers can be easily adjusted and tuned. The data provided for training can be either a path to a [Dask](https://docs.dask.org/en/stable/dataframe.html) or [Pandas](https://pandas.pydata.org/) DataFrame stored in the Parquet format or the DataFrame object directly.\n\n## Installation\nIf you are using [Poetry](https://python-poetry.org/), you can install the package with the following command:\n```bash\npoetry add autoembedder\n```\nIf you are using [pip](https://pypi.org/project/pip/), you can install the package with the following command:\n```bash\npip install autoembedder\n```\n\n\n## Installing dependencies\nWith [Poetry](https://python-poetry.org/):\n```bash\npoetry install\n```\nWith [pip](https://pypi.org/project/pip/):\n```bash\npip install -r requirements.txt\n```\n## Usage\n### 0. Some imports\n```python\nfrom autoembedder import Autoembedder, dataloader, fit\n```\n### 1. Create dataloaders\nFirst, we create two [`dataloaders`](https://chrislemke.github.io/autoembedder/autoembedder.data/#autoembedder.data.Dataset.__init__). One for training, and the other for validation data. As `source` they either accept a path to a Parquet file, to a folder of Parquet files or a [Pandas](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html)/[Dask](https://docs.dask.org/en/stable/dataframe.html) DataFrame.\n```python\ntrain_dl = dataloader(train_df)\nvalid_dl = dataloader(vaild_df)\n```\n\n### 2. Set parameters\nNow, we need to set the parameters. They are going to be used for handling the data and training the model. In this example, only parameters for the training are set. [Here](https://github.com/chrislemke/autoembedder#parameters) you find a list of all possible parameters. This should do it:\n```python\nparameters = {\n \"hidden_layers\": [[25, 20], [20, 10]],\n \"epochs\": 10,\n \"lr\": 0.0001,\n \"verbose\": 1,\n}\n```\n\n### 3. Initialize the autoembedder\nThen, we need to initialize the [autoembedder](https://chrislemke.github.io/autoembedder/autoembedder.model/#autoembedder.model.Autoembedder). In this example, we are not using any categorical features. So we can skip the `embedding_sizes` argument.\n```python\nmodel = Autoembedder(parameters, num_cont_features=train_df.shape[1])\n```\n\n### 4. Train the model\nEverything is set up. Now we can [fit](https://chrislemke.github.io/autoembedder/autoembedder.learner/#autoembedder.learner.fit) the model.\n```python\nfit(parameters, model, train_dl, valid_dl)\n```\n\n## Example\nCheck out [this Jupyter notebook](https://github.com/chrislemke/autoembedder/blob/main/example.ipynb) for an applied example using the [Credit Card Fraud Detection](https://www.kaggle.com/datasets/mlg-ulb/creditcardfraud) from Kaggle.\n\n## Parameters\nThis is a list of all parameters that can be passed to the Autoembedder for training. When using the training script the `_` needs to be replaced with `-` and the parameters need to be passed as arguments. For boolean values please have a look at the `Comment` column for understanding how to pass them.\n\n## Run the training script\nYou can also simply use the training script::\n```bash\npython3 training.py \\\n--epochs 20 \\\n--train-input-path \"path/to/your/train_data\" \\\n--test-input-path \"path/to/your/test_data\" \\\n--hidden-layers \"[[12, 6], [6, 3]]\"\n```\n\nfor help just run:\n```bash\npython3 training.py --help\n```\n\n| Argument | Type | Required | Default value | Comment |\n| -------------------- | ----- | -------- | ----------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| batch_size | int | False | 32 | |\n| drop_last | bool | False | True | --drop-last / --no-drop-last |\n| pin_memory | bool | False | True | --pin-memory / --no-pin-memory |\n| num_workers | int | False | 0 | 0 means that the data will be loaded in the main process |\n| use_mps | bool | False | False | --use-mps / --no-use-mps |\n| model_title | str | False | autoembedder_{`datetime`}.bin | |\n| model_save_path | str | False | | |\n| n_save_checkpoints | int | False | | |\n| lr | float | False | 0.001 | |\n| amsgrad | bool | False | False | --amsgrad / --no-amsgrad |\n| epochs | int | True | |\n| dropout_rate | float | False | 0 | Dropout rate for the dropout layers in the encoder and decoder. |\n| layer_bias | bool | False | True | --layer-bias / --no-layer-bias | |\n| weight_decay | float | False | False | |\n| l1_lambda | float | False | 0 | |\n| xavier_init | bool | False | False | --xavier-init / --no-xavier-init |\n| activation | str | False | tanh | Activation function; either `tanh`, `relu`, `leaky_relu` or `elu` |\n| tensorboard_log_path | str | False | | |\n| trim_eval_errors | bool | False | False |--trim-eval-errors / --no-trim-eval-errors; Removes the max and min loss when calculating the `mean loss diff` and `median loss diff`. This can be useful if some rows create very high losses. |\n| verbose | int | False | 0 | Set this to `1` if you want to see the model summary and the validation and evaluation results. set this to `2` if you want to see the training progress bar. `0` means no output. |\n| target | str | False | | The target column. If not set no evaluation will be performed. |\n| train_input_path | str | True | | |\n| test_input_path | str | True | |\n| eval_input_path | str | False | | Path to the evaluation data. If no path is provided no evaluation will be performed. | |\n| hidden_layers | str | True | | Contains a string representation of a list of list of integers which represents the hidden layer structure. E.g.: `\"[[64, 32], [32, 16], [16, 8]]\"` activation |\n| cat_columns | str | False | \"[]\" | Contains a string representation of a list of list of categorical columns (strings). The columns which use the same encoder should be together in a list. E.g.: `\"[['a', 'b'], ['c']]\"`. |\n| drop-cat-columns | bool | False | |--drop-cat-columns / --no-drop-cat-columns\n\n\n## Why additional embedding layers?\nThe additional embedding layers automatically embed all columns with the Pandas `category` data type. If categorical columns have another data type, they will not be embedded and will be handled like continuous columns. Simply encoding the categorical values (e.g., with the usage of a label encoder) decreases the quality of the outcome.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "PyTorch autoencoder with additional embeddings layer for categorical data.",
"version": "0.2.5",
"split_keywords": [
"autoencoder",
"embeddings",
"model",
"pytorch",
"neural network",
"machine learning",
"data science"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8b6214781dd9f731e01e4b0cb46c18eca16b450467d4bd78b03ba60dd67e5f11",
"md5": "696b47361f35f483d0f97ca6c2608b63",
"sha256": "ca75b38438d4d888bac225b1ea1ce5b0920a7c9b290bdfefedc514d681a3889e"
},
"downloads": -1,
"filename": "autoembedder-0.2.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "696b47361f35f483d0f97ca6c2608b63",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<3.11",
"size": 17685,
"upload_time": "2023-02-07T08:36:09",
"upload_time_iso_8601": "2023-02-07T08:36:09.380087Z",
"url": "https://files.pythonhosted.org/packages/8b/62/14781dd9f731e01e4b0cb46c18eca16b450467d4bd78b03ba60dd67e5f11/autoembedder-0.2.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4b5ee948c6630c401f8e0943dfec163839fe64771c657507ba7cbaed708ff06d",
"md5": "55aa2467c5e7e0d68d55df3cf7e5d6de",
"sha256": "2dcf96f43bd1fe36d16266e0b543580f89a8a3ae47221f7a393233389cd367de"
},
"downloads": -1,
"filename": "autoembedder-0.2.5.tar.gz",
"has_sig": false,
"md5_digest": "55aa2467c5e7e0d68d55df3cf7e5d6de",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<3.11",
"size": 18425,
"upload_time": "2023-02-07T08:36:11",
"upload_time_iso_8601": "2023-02-07T08:36:11.191864Z",
"url": "https://files.pythonhosted.org/packages/4b/5e/e948c6630c401f8e0943dfec163839fe64771c657507ba7cbaed708ff06d/autoembedder-0.2.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-02-07 08:36:11",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "autoembedder"
}
JSON