# TinyShift
**TinyShift** is a small experimental Python library designed to detect **data drifts** and **performance drops** in machine learning models over time. The main goal of the project is to provide quick and tiny monitoring tools to help identify when data or model performance unexpectedly change.
For more robust solutions, I highly recommend [Nannyml.](https://github.com/NannyML/nannyml)
## Technologies Used
- **Python 3.x**
- **Scikit-learn**
- **Pandas**
- **NumPy**
- **Plotly**
- **Scipy**
## Installation
To install **TinyShift** in your development environment, use **pip**:
```bash
pip install tinyshift
```
If you prefer to clone the repository and install manually:
```bash
git clone https://github.com/HeyLucasLeao/tinyshift.git
cd tinyshift
pip install .
```
> **Note:** If you want to enable plotting capabilities, you need to install the extras using Poetry:
```bash
poetry install --all-extras
```
## Usage
Below are basic examples of how to use TinyShift's features.
### 1. Data Drift Detection
To detect data drift, simply score in a new dataset to compare with the reference data. The DataDriftDetector will calculate metrics to identify significant differences.
```python
from tinyshift.detector import CategoricalDriftDetector
df = pd.DataFrame("examples.csv")
df_reference = df[(df["datetime"] < '2024-07-01')].copy()
df_analysis = df[(df["datetime"] >= '2024-07-01')].copy()
detector = CategoricalDriftDetector(df_reference, 'discrete_1', "datetime", "W", drift_limit='mad')
analysis_score = detector.score(df_analysis, "discrete_1", "datetime")
print(analysis_score)
```
### 2. Performance Tracker
To track model performance over time, use the PerformanceMonitor, which will compare model accuracy on both old and new data.
```python
from tinyshift.tracker import PerformanceTracker
df_reference = pd.read_csv('refence.csv')
df_analysis = pd.read_csv('analysis.csv')
model = load_model('model.pkl')
df_analysis['prediction'] = model.predict(df_analysis["feature_0"])
tracker = PerformanceTracker(df_reference, 'target', 'prediction', 'datetime', "W")
analysis_score = tracker.score(df_analysis, 'target', 'prediction', 'datetime')
print(analysis_score)
```
### 3. Visualization
TinyShift also provides graphs to visualize the magnitude of drift and performance changes over time.
```python
tracker.plot.scatter(analysis_score, fig_type="png")
tracker.plot.bar(analysis_score, fig_type="png")
```
## Project Structure
The basic structure of the project is as follows:
```
tinyshift
├── LICENSE
├── README.md
├── example.ipynb
├── pyproject.toml
└── tinyshift
├── base
│ ├── __init__.py
│ └── model.py
├── detector
│ ├── __init__.py
│ ├── categorical.py
│ └── continuous.py
├── plot
│ ├── __init__.py
│ └── plot.py
└── tracker
├── __init__.py
└── performance.py
```
### License
This project is licensed under the MIT License - see the LICENSE file for more details.
Raw data
{
"_id": null,
"home_page": null,
"name": "tinyshift",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.10",
"maintainer_email": null,
"keywords": "mlops, toolbox, machine-learning",
"author": "Lucas Le\u00e3o",
"author_email": "heylucasleao@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/09/1b/4d24e94628c6aed698ebca498a691969d33e7def16118806f31c6cbd775a/tinyshift-0.0.5.tar.gz",
"platform": null,
"description": "# TinyShift\n\n**TinyShift** is a small experimental Python library designed to detect **data drifts** and **performance drops** in machine learning models over time. The main goal of the project is to provide quick and tiny monitoring tools to help identify when data or model performance unexpectedly change.\nFor more robust solutions, I highly recommend [Nannyml.](https://github.com/NannyML/nannyml)\n\n## Technologies Used\n\n- **Python 3.x**\n- **Scikit-learn**\n- **Pandas**\n- **NumPy**\n- **Plotly**\n- **Scipy**\n\n## Installation\n\nTo install **TinyShift** in your development environment, use **pip**:\n\n\n```bash\npip install tinyshift\n```\nIf you prefer to clone the repository and install manually:\n```bash\ngit clone https://github.com/HeyLucasLeao/tinyshift.git\ncd tinyshift \npip install .\n```\n\n> **Note:** If you want to enable plotting capabilities, you need to install the extras using Poetry:\n\n```bash\npoetry install --all-extras\n```\n\n## Usage\nBelow are basic examples of how to use TinyShift's features.\n### 1. Data Drift Detection\nTo detect data drift, simply score in a new dataset to compare with the reference data. The DataDriftDetector will calculate metrics to identify significant differences.\n\n```python\nfrom tinyshift.detector import CategoricalDriftDetector\n\ndf = pd.DataFrame(\"examples.csv\")\ndf_reference = df[(df[\"datetime\"] < '2024-07-01')].copy()\ndf_analysis = df[(df[\"datetime\"] >= '2024-07-01')].copy()\n\ndetector = CategoricalDriftDetector(df_reference, 'discrete_1', \"datetime\", \"W\", drift_limit='mad')\n\nanalysis_score = detector.score(df_analysis, \"discrete_1\", \"datetime\")\n\nprint(analysis_score)\n```\n\n### 2. Performance Tracker\nTo track model performance over time, use the PerformanceMonitor, which will compare model accuracy on both old and new data.\n```python\nfrom tinyshift.tracker import PerformanceTracker\n\ndf_reference = pd.read_csv('refence.csv')\ndf_analysis = pd.read_csv('analysis.csv')\nmodel = load_model('model.pkl') \ndf_analysis['prediction'] = model.predict(df_analysis[\"feature_0\"])\n\ntracker = PerformanceTracker(df_reference, 'target', 'prediction', 'datetime', \"W\")\n\nanalysis_score = tracker.score(df_analysis, 'target', 'prediction', 'datetime')\n\nprint(analysis_score)\n```\n\n### 3. Visualization\nTinyShift also provides graphs to visualize the magnitude of drift and performance changes over time.\n```python\ntracker.plot.scatter(analysis_score, fig_type=\"png\")\n\ntracker.plot.bar(analysis_score, fig_type=\"png\")\n```\n\n## Project Structure\nThe basic structure of the project is as follows:\n```\ntinyshift\n\u251c\u2500\u2500 LICENSE\n\u251c\u2500\u2500 README.md\n\u251c\u2500\u2500 example.ipynb\n\u251c\u2500\u2500 pyproject.toml\n\u2514\u2500\u2500 tinyshift\n \u251c\u2500\u2500 base\n \u2502\u00a0\u00a0 \u251c\u2500\u2500 __init__.py\n \u2502\u00a0\u00a0 \u2514\u2500\u2500 model.py\n \u251c\u2500\u2500 detector\n \u2502\u00a0\u00a0 \u251c\u2500\u2500 __init__.py\n \u2502\u00a0\u00a0 \u251c\u2500\u2500 categorical.py\n \u2502\u00a0\u00a0 \u2514\u2500\u2500 continuous.py\n \u251c\u2500\u2500 plot\n \u2502\u00a0\u00a0 \u251c\u2500\u2500 __init__.py\n \u2502\u00a0\u00a0 \u2514\u2500\u2500 plot.py\n \u2514\u2500\u2500 tracker\n \u251c\u2500\u2500 __init__.py\n \u2514\u2500\u2500 performance.py \n```\n\n### License\nThis project is licensed under the MIT License - see the LICENSE file for more details.\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A small toolbox for mlops",
"version": "0.0.5",
"project_urls": null,
"split_keywords": [
"mlops",
" toolbox",
" machine-learning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "cdf81343159ffa3c22f6331619ca5917af8e89e983bb8ab460e183fbb8d7d86b",
"md5": "8c22b843f8059a7907fd9b8f8b9fe8a0",
"sha256": "68defaeadd782bb0aeb1bb0452ed3e68de06f29333c057ef7d167fb584be0bfb"
},
"downloads": -1,
"filename": "tinyshift-0.0.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8c22b843f8059a7907fd9b8f8b9fe8a0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.10",
"size": 785875,
"upload_time": "2025-01-11T20:51:36",
"upload_time_iso_8601": "2025-01-11T20:51:36.892885Z",
"url": "https://files.pythonhosted.org/packages/cd/f8/1343159ffa3c22f6331619ca5917af8e89e983bb8ab460e183fbb8d7d86b/tinyshift-0.0.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "091b4d24e94628c6aed698ebca498a691969d33e7def16118806f31c6cbd775a",
"md5": "fc2b1406353e0b6375e6c18f015dae46",
"sha256": "1ecc8f056640d12bfeb6bd72d1861ba6cdf16bd04d363b6890f6f311ad293df1"
},
"downloads": -1,
"filename": "tinyshift-0.0.5.tar.gz",
"has_sig": false,
"md5_digest": "fc2b1406353e0b6375e6c18f015dae46",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.10",
"size": 783048,
"upload_time": "2025-01-11T20:51:40",
"upload_time_iso_8601": "2025-01-11T20:51:40.217394Z",
"url": "https://files.pythonhosted.org/packages/09/1b/4d24e94628c6aed698ebca498a691969d33e7def16118806f31c6cbd775a/tinyshift-0.0.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-11 20:51:40",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "tinyshift"
}
JSON