# adversarial-validation
[](https://pypi.org/project/advertion/)
[](https://www.python.org/)
[](https://codecov.io/gh/ilias-ant/adversarial-validation)
[](https://github.com/psf/black)
[](https://github.com/ilias-ant/adversarial-validation/actions/workflows/ci.yml)
[](https://advertion.readthedocs.io/en/latest/?badge=latest)
[](https://www.python.org/dev/peps/pep-0427/)
A tiny framework to perform adversarial validation of your training and test data.
<img src="https://raw.githubusercontent.com/ilias-ant/adversarial-validation/main/static/logo.png" width="95%" text="figjam">
**What is adversarial validation?**
A common workflow in machine learning projects (especially in Kaggle competitions) is:
1. train your ML model in a training dataset.
2. tune and validate your ML model in a validation dataset (typically is a discrete fraction of the training dataset).
3. finally, assess the actual generalization ability of your ML model in a “held-out” test dataset.
This strategy is widely accepted, but it heavily relies on the assumption that the training and test datasets are drawn
from the same underlying distribution. This is often referred to as the “*identically distributed*” property in the
literature.
This package helps you easily assert whether the "*identically distributed*" property holds true for your training and
test datasets or equivalently whether your validation dataset is a good proxy for your model's performance on the unseen
test instances.
If you are a person of details, feel free to take a deep dive to the following companion article:
[adversarial validation: can i trust my validation dataset?](https://ilias-ant.github.io/blog/adversarial-validation/)
## Install
The recommended installation is via `pip`:
```bash
pip install advertion
```
(*advertion stands for **adver**sarial valida**tion***)
## Usage
```python
from advertion import validate
train = pd.read_csv("...")
test = pd.read_csv("...")
validate(
trainset=train,
testset=test,
target="label",
)
# // {
# // "datasets_follow_same_distribution": True,
# // 'mean_roc_auc': 0.5021320833333334,
# // "adversarial_features': ['id'],
# // }
```
## How to contribute
If you wish to contribute, [this](CONTRIBUTING.md) is a great place to start!
## License
Distributed under the [Apache License 2.0](LICENSE).
Raw data
{
"_id": null,
"home_page": "https://pypi.org/project/advertion",
"name": "advertion",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8.6,<3.12",
"maintainer_email": "",
"keywords": "adversarial-validation,data-drift,machine-learning",
"author": "Ilias Antonopoulos",
"author_email": "ilias.antonopoulos@yahoo.gr",
"download_url": "https://files.pythonhosted.org/packages/1e/f5/dcb45811c91d9d150fa09a20e67bdf70932d4d40bb51cf78a00788b64efb/advertion-0.1.1.tar.gz",
"platform": null,
"description": "# adversarial-validation\n\n[](https://pypi.org/project/advertion/) \n[](https://www.python.org/)\n[](https://codecov.io/gh/ilias-ant/adversarial-validation)\n[](https://github.com/psf/black) \n[](https://github.com/ilias-ant/adversarial-validation/actions/workflows/ci.yml)\n[](https://advertion.readthedocs.io/en/latest/?badge=latest)\n[](https://www.python.org/dev/peps/pep-0427/)\n\nA tiny framework to perform adversarial validation of your training and test data.\n\n<img src=\"https://raw.githubusercontent.com/ilias-ant/adversarial-validation/main/static/logo.png\" width=\"95%\" text=\"figjam\">\n\n**What is adversarial validation?**\nA common workflow in machine learning projects (especially in Kaggle competitions) is:\n\n1. train your ML model in a training dataset.\n2. tune and validate your ML model in a validation dataset (typically is a discrete fraction of the training dataset).\n3. finally, assess the actual generalization ability of your ML model in a \u201cheld-out\u201d test dataset.\n\nThis strategy is widely accepted, but it heavily relies on the assumption that the training and test datasets are drawn \nfrom the same underlying distribution. This is often referred to as the \u201c*identically distributed*\u201d property in the \nliterature.\n\nThis package helps you easily assert whether the \"*identically distributed*\" property holds true for your training and \ntest datasets or equivalently whether your validation dataset is a good proxy for your model's performance on the unseen \ntest instances.\n\nIf you are a person of details, feel free to take a deep dive to the following companion article:\n\n[adversarial validation: can i trust my validation dataset?](https://ilias-ant.github.io/blog/adversarial-validation/)\n\n## Install\n\nThe recommended installation is via `pip`:\n\n```bash\npip install advertion\n```\n\n(*advertion stands for **adver**sarial valida**tion***)\n\n## Usage\n\n```python\nfrom advertion import validate\n\ntrain = pd.read_csv(\"...\")\ntest = pd.read_csv(\"...\")\n\nvalidate(\n trainset=train,\n testset=test,\n target=\"label\",\n)\n\n# // {\n# // \"datasets_follow_same_distribution\": True,\n# // 'mean_roc_auc': 0.5021320833333334,\n# // \"adversarial_features': ['id'],\n# // }\n```\n\n## How to contribute\n\nIf you wish to contribute, [this](CONTRIBUTING.md) is a great place to start!\n\n## License\n\nDistributed under the [Apache License 2.0](LICENSE).\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "A tiny framework to perform adversarial validation of your training and test data.",
"version": "0.1.1",
"project_urls": {
"Bug Tracker": "https://github.com/ilias-ant/adversarial-validation/issues",
"Homepage": "https://pypi.org/project/advertion",
"Repository": "https://github.com/ilias-ant/adversarial-validation"
},
"split_keywords": [
"adversarial-validation",
"data-drift",
"machine-learning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e846a85e0832b71910df75fb8544c163758e6d88cf67c83e83b90522b43319c6",
"md5": "6ad324583b672b09113113c182f21c9d",
"sha256": "a3f8e7d24b750c0d9974477a95765f42d01212364a6d46eba58b1adf9997e433"
},
"downloads": -1,
"filename": "advertion-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6ad324583b672b09113113c182f21c9d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8.6,<3.12",
"size": 6310,
"upload_time": "2023-07-22T15:53:15",
"upload_time_iso_8601": "2023-07-22T15:53:15.272365Z",
"url": "https://files.pythonhosted.org/packages/e8/46/a85e0832b71910df75fb8544c163758e6d88cf67c83e83b90522b43319c6/advertion-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "1ef5dcb45811c91d9d150fa09a20e67bdf70932d4d40bb51cf78a00788b64efb",
"md5": "40a6451a0d387a0a20dfd5add0f98321",
"sha256": "a6322d1e60ae9c17e461cd3bdcc521ce942e7de7719429b47174fa197ec97682"
},
"downloads": -1,
"filename": "advertion-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "40a6451a0d387a0a20dfd5add0f98321",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8.6,<3.12",
"size": 5568,
"upload_time": "2023-07-22T15:53:16",
"upload_time_iso_8601": "2023-07-22T15:53:16.883948Z",
"url": "https://files.pythonhosted.org/packages/1e/f5/dcb45811c91d9d150fa09a20e67bdf70932d4d40bb51cf78a00788b64efb/advertion-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-22 15:53:16",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ilias-ant",
"github_project": "adversarial-validation",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"lcname": "advertion"
}
JSON
