<p align="left">
<img width=15% src="https://dai.lids.mit.edu/wp-content/uploads/2018/06/Logo_DAI_highres.png" alt=“DAI-Lab” />
<i>An open source project from Data to AI Lab at MIT.</i>
</p>
<p align="left">
<img width=20% src="https://dai.lids.mit.edu/wp-content/uploads/2018/08/orion.png" alt=“Orion” />
</p>
[](https://pypi.org/search/?c=Development+Status+%3A%3A+2+-+Pre-Alpha)
[](https://badge.fury.io/py/orion-ml)
[](https://pypi.python.org/pypi/orion-ml)
[](https://github.com/sintel-dev/Orion/actions?query=workflow%3A%22Run+Tests%22+branch%3Amaster)
[](https://pepy.tech/project/orion-ml)
[](https://mybinder.org/v2/gh/sintel-dev/Orion/master?filepath=tutorials)
# Orion
A machine learning library for unsupervised time series anomaly detection.
| Important Links | |
| --------------------------------------------- | -------------------------------------------------------------------- |
| :computer: **[Website]** | Check out the Sintel Website for more information about the project. |
| :book: **[Documentation]** | Quickstarts, User and Development Guides, and API Reference. |
| :star: **[Tutorials]** | Checkout our notebooks |
| :octocat: **[Repository]** | The link to the Github Repository of this library. |
| :scroll: **[License]** | The repository is published under the MIT License. |
| [![][Slack Logo] **Community**][Community] | Join our Slack Workspace for announcements and discussions. |
[Website]: https://sintel.dev/
[Documentation]: https://sintel-dev.github.io/Orion
[Tutorials]: https://github.com/sintel-dev/Orion/tree/master/tutorials
[Repository]: https://github.com/sintel-dev/Orion
[License]: https://github.com/sintel-dev/Orion/blob/master/LICENSE
[Community]: https://join.slack.com/t/sintel-space/shared_invite/zt-q147oimb-4HcphcxPfDAM0O9_4PaUtw
[Slack Logo]: https://github.com/sintel-dev/Orion/blob/master/docs/images/slack.png
# Overview
Orion is a machine learning library built for *unsupervised time series anomaly detection*. With a given time series data, we provide a number of “verified” ML pipelines (a.k.a Orion pipelines) that identify rare patterns and flag them for expert review.
The library makes use of a number of **automated machine learning** tools developed under [Data to AI Lab at MIT](https://dai.lids.mit.edu/).
Read about using an Orion pipeline on NYC taxi dataset in a blog series:
[Part 1: Learn about unsupervised time series anomaly detection](https://t.co/yIFVM1oRwQ?amp=1) | [Part 2: Learn how we use GANs to solving the problem? ](https://link.medium.com/cGsBD0Fevbb) | [Part 3: How does one evaluate anomaly detection pipelines?](https://link.medium.com/FqCrFXMevbb)
:--------------------------------------:|:---------------------------------------------:|:--------------------------------------------:
 |  | 
**Notebooks:** Discover *Orion* through colab by launching our [notebooks](https://drive.google.com/drive/folders/1FAcCEiE1JDsqaMjGcmiw5a5XuGh13c9Q?usp=sharing)!
# Quickstart
## Install with pip
The easiest and recommended way to install **Orion** is using [pip](https://pip.pypa.io/en/stable/):
```bash
pip install orion-ml
```
This will pull and install the latest stable release from [PyPi](https://pypi.org/).
In the following example we show how to use one of the **Orion Pipelines**.
## Fit an Orion pipeline
We will load a demo data for this example:
```python3
from orion.data import load_signal
train_data = load_signal('S-1-train')
train_data.head()
```
which should show a signal with `timestamp` and `value`.
```
timestamp value
0 1222819200 -0.366359
1 1222840800 -0.394108
2 1222862400 0.403625
3 1222884000 -0.362759
4 1222905600 -0.370746
```
In this example we use `aer` pipeline and set some hyperparameters (in this case training epochs as 5).
```python3
from orion import Orion
hyperparameters = {
'orion.primitives.aer.AER#1': {
'epochs': 5,
'verbose': True
}
}
orion = Orion(
pipeline='aer',
hyperparameters=hyperparameters
)
orion.fit(train_data)
```
## Detect anomalies using the fitted pipeline
Once it is fitted, we are ready to use it to detect anomalies in our incoming time series:
```python3
new_data = load_signal('S-1-new')
anomalies = orion.detect(new_data)
```
> :warning: Depending on your system and the exact versions that you might have installed some *WARNINGS* may be printed. These can be safely ignored as they do not interfere with the proper behavior of the pipeline.
The output of the previous command will be a ``pandas.DataFrame`` containing a table of detected anomalies:
```
start end severity
0 1402012800 1403870400 0.122539
```
# Leaderboard
In every release, we run Orion benchmark. We maintain an up-to-date leaderboard with the current scoring of the verified pipelines according to the benchmarking procedure.
We run the benchmark on **12** datasets with their known grounth truth. We record the score of the pipelines on each datasets. To compute the leaderboard table, we showcase the number of wins each pipeline has over the ARIMA pipeline.
| Pipeline | Outperforms ARIMA |
|---------------------------|--------------------|
| AER | 11 |
| TadGAN | 7 |
| LSTM Dynamic Thresholding | 8 |
| LSTM Autoencoder | 7 |
| Dense Autoencoder | 7 |
| VAE | 6 |
| LNN | 7 |
| Matrix Profile | 5 |
| [GANF](https://arxiv.org/pdf/2202.07857.pdf) | 5 |
| [Azure](https://azure.microsoft.com/en-us/products/cognitive-services/anomaly-detector/) | 0 |
You can find the scores of each pipeline on every signal recorded in the [details Google Sheets document](https://docs.google.com/spreadsheets/d/1HaYDjY-BEXEObbi65fwG0om5d8kbRarhpK4mvOZVmqU/edit?usp=sharing). The summarized results can also be browsed in the following [summary Google Sheets document](https://docs.google.com/spreadsheets/d/1ZPUwYH8LhDovVeuJhKYGXYny7472HXVCzhX6D6PObmg/edit?usp=sharing).
# Resources
Additional resources that might be of interest:
* Learn about [benchmarking pipelines](BENCHMARK.md).
* Read about [pipeline evaluation](orion/evaluation/README.md).
* Find out more about [TadGAN](https://arxiv.org/pdf/2009.07769v3.pdf).
# Citation
If you use **AER** for your research, please consider citing the following paper:
Lawrence Wong, Dongyu Liu, Laure Berti-Equille, Sarah Alnegheimish, Kalyan Veeramachaneni. [AER: Auto-Encoder with Regression for Time Series Anomaly Detection](https://arxiv.org/pdf/2212.13558.pdf).
```
@inproceedings{wong2022aer,
title={AER: Auto-Encoder with Regression for Time Series Anomaly Detection},
author={Wong, Lawrence and Liu, Dongyu and Berti-Equille, Laure and Alnegheimish, Sarah and Veeramachaneni, Kalyan},
booktitle={2022 IEEE International Conference on Big Data (IEEE BigData)},
pages={1152-1161},
doi={10.1109/BigData55660.2022.10020857},
organization={IEEE},
year={2022}
}
```
If you use **TadGAN** for your research, please consider citing the following paper:
Alexander Geiger, Dongyu Liu, Sarah Alnegheimish, Alfredo Cuesta-Infante, Kalyan Veeramachaneni. [TadGAN - Time Series Anomaly Detection Using Generative Adversarial Networks](https://arxiv.org/pdf/2009.07769v3.pdf).
```
@inproceedings{geiger2020tadgan,
title={TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks},
author={Geiger, Alexander and Liu, Dongyu and Alnegheimish, Sarah and Cuesta-Infante, Alfredo and Veeramachaneni, Kalyan},
booktitle={2020 IEEE International Conference on Big Data (IEEE BigData)},
pages={33-43},
doi={10.1109/BigData50022.2020.9378139},
organization={IEEE},
year={2020}
}
```
If you use **Orion** which is part of the **Sintel** ecosystem for your research, please consider citing the following paper:
Sarah Alnegheimish, Dongyu Liu, Carles Sala, Laure Berti-Equille, Kalyan Veeramachaneni. [Sintel: A Machine Learning Framework to Extract Insights from Signals](https://dl.acm.org/doi/pdf/10.1145/3514221.3517910).
```
@inproceedings{alnegheimish2022sintel,
title={Sintel: A Machine Learning Framework to Extract Insights from Signals},
author={Alnegheimish, Sarah and Liu, Dongyu and Sala, Carles and Berti-Equille, Laure and Veeramachaneni, Kalyan},
booktitle={Proceedings of the 2022 International Conference on Management of Data},
pages={1855–1865},
numpages={11},
publisher={Association for Computing Machinery},
doi={10.1145/3514221.3517910},
series={SIGMOD '22},
year={2022}
}
```
History
=======
## 0.6.0 - 2024-2-13
Support for python 3.10 and 3.11
### Issues resolved
* Update ``test_core`` file – [Issue #507](https://github.com/signals-dev/Orion/issues/507) by @sarahmish
* update ARIMA primitive and pipeline – [Issue #503](https://github.com/signals-dev/Orion/issues/503) by @sarahmish
* Update Dependency – [Issue #497](https://github.com/signals-dev/Orion/issues/497) & [Issue #499](https://github.com/signals-dev/Orion/issues/499) by @sarahmish
* Update Python for Dependency Test – [Issue #484](https://github.com/signals-dev/Orion/issues/484) by @sarahmish
* Add python 3.10 and 3.11 & drop 3.6 and 3.7 – [Issue #477](https://github.com/signals-dev/Orion/issues/477) by @sarahmish
* LNN Pipeline – [Issue #475](https://github.com/signals-dev/Orion/issues/475) by @sarahmish
## 0.5.2 - 2023-10-19
Support for python 3.9 and new Matrix Profile pipeline
### Issues resolved
* Send pipeline names in benchmark arguments – [Issue #466](https://github.com/signals-dev/Orion/issues/466) by @sarahmish
* Add continuation parameter for benchmark – [Issue #464](https://github.com/signals-dev/Orion/issues/464) by @sarahmish
* Fix references in documentation – [Issue #453](https://github.com/signals-dev/Orion/issues/453) by @sarahmish
* Update documentation – [Issue #448](https://github.com/signals-dev/Orion/issues/448) by @sarahmish
* Add matrix profiling method – [Issue #446](https://github.com/signals-dev/Orion/issues/446) by @sarahmish
* Support python 3.9 – [Issue #408](https://github.com/signals-dev/Orion/issues/408) by @sarahmish
## 0.5.1 - 2023-08-16
This version introduces a new dataset to the benchmark.
### Issues resolved
* Add UCR dataset to the benchmark – [Issue #443](https://github.com/signals-dev/Orion/issues/443) by @sarahmish
* docker image build failed – [Issue #439](https://github.com/signals-dev/Orion/issues/439) by @sarahmish
* Edit interval settings in ``azure`` pipeline – [Issue #436](https://github.com/signals-dev/Orion/issues/436) by @sarahmish
## 0.5.0 - 2023-05-23
This version uses ``ml-stars`` package instead of ``mlprimitives``.
### Issues resolved
* Migrate to ml-stars – [Issue #418](https://github.com/signals-dev/Orion/issues/418) by @sarahmish
* Updating ``best_cost`` in ``find_anomalies`` primitive – [Issue #403](https://github.com/signals-dev/Orion/issues/403) by @sarahmish
* Retire ``lstm_dynamic_threshold_gpu`` and ``lstm_autoencoder_gpu`` pipeline maintenance – [Issue #373](https://github.com/signals-dev/Orion/issues/373) by @sarahmish
* Typo in xlsxwriter dependency specification – [Issue #394](https://github.com/signals-dev/Orion/issues/394) by @sarahmish
* ``orion.evaluate`` uses fails when fitting – [Issue #384](https://github.com/signals-dev/Orion/issues/384) by @sarahmish
* AER pipeline with visualization option – [Issue #379](https://github.com/signals-dev/Orion/issues/379) by @sarahmish
## 0.4.1 - 2023-01-31
### Issues resolved
* Move VAE from sandbox to verified – [Issue #377](https://github.com/signals-dev/Orion/issues/377) by @sarahmish
* Pin ``opencv`` – [Issue #372](https://github.com/signals-dev/Orion/issues/372) by @sarahmish
* Pin ``scikit-learn`` – [Issue #367](https://github.com/signals-dev/Orion/issues/367) by @sarahmish
* Fix VAE documentation – [Issue #360](https://github.com/signals-dev/Orion/issues/360) by @sarahmish
## 0.4.0 - 2022-11-08
This version introduces several new enhancements:
* Support to python 3.8
* Migrating to Tensorflow 2.0
* New pipeline, namely ``VAE``, a Variational AutoEncoder model.
### Issues resolved
* Add python 3.8 – [Issue #342](https://github.com/signals-dev/Orion/issues/342) by @sarahmish
* VAE (Variational Autoencoders) pipeline implementation – [Issue #349](https://github.com/signals-dev/Orion/issues/349) by @dyuliu
* Add masking option for ``regression_errors`` – [Issue #352](https://github.com/signals-dev/Orion/issues/352) by @dyuliu
* Changes in TadGAN for tensorflow 2.0 – [Issue #161](https://github.com/signals-dev/Orion/issues/161) by @lcwong0928
* Add an automatic dependency checker – [Issue #320](https://github.com/signals-dev/Orion/issues/320) by @sarahmish
* TadGAN ``batch_size`` cannot be changed – [Issue #313](https://github.com/signals-dev/Orion/issues/313) by @sarahmish
## 0.3.2 - 2022-07-04
This version fixes some of the issues in ``aer``, ``ae``, and ``tadgan`` pipelines.
### Issues resolved
* Fix AER model predict error after loading – [Issue #304](https://github.com/signals-dev/Orion/issues/304) by @lcwong0928
* Update AE to work with any `window_size` – [Issue #300](https://github.com/signals-dev/Orion/issues/300) by @sarahmish
* Updated tadgan_viz.json – [Issue #292](https://github.com/signals-dev/Orion/issues/292) by @Hramir
## 0.3.1 - 2022-04-26
This version introduce a new pipeline, namely ``AER``, an AutoEncoder Regressor model.
### Issues resolved
* Add AER Model - [Issue #286](https://github.com/signals-dev/Orion/issues/286) by @lcwong0928
## 0.3.0 - 2022-03-31
This version deprecates the support of ``OrionDBExplorer``, which has been migrated to
[sintel](https://github.com/signals-dev/Orion). As a result, ``Orion`` no longer requires
mongoDB as a dependency.
### Issues resolved
* Update dependency - [Issue #283](https://github.com/signals-dev/Orion/issues/283) by @sarahmish
* General housekeeping - [Issue #278](https://github.com/signals-dev/Orion/issues/278) by @sarahmish
* Fix tutorial testing issue - [Issue #276](https://github.com/signals-dev/Orion/issues/276) by @sarahmish
* Migrate OrionExplorer to Sintel - [Issue #275](https://github.com/signals-dev/Orion/issues/275) by @dyuliu
* LSTM viz JSON pipeline added - [Issue #271](https://github.com/signals-dev/Orion/issues/271) by @Hramir
## 0.2.1 - 2022-02-18
This version introduces improvements and more testing.
### Issues resolved
* Adjusting builds for TadGAN - [Issue #261](https://github.com/signals-dev/Orion/issues/261) by @sarahmish
* Testing tutorials, dependencies, and OS - [Issue #251](https://github.com/signals-dev/Orion/issues/251) by @sarahmish
## 0.2.0 - 2021-10-11
This version supports multivariate timeseries as input. In addition to minor improvements
and maintenance.
### Issues resolved
* `setuptools` no longer supports `lib2to3` breaking `mongoengine` - [Issue #252](https://github.com/signals-dev/Orion/issues/252) by @sarahmish
* Supporting multivariate input - [Issue #248](https://github.com/signals-dev/Orion/issues/248) by @sarahmish
* TadGAN pipeline with visualization option - [Issue #240](https://github.com/signals-dev/Orion/issues/240) by @sarahmish
* Support saving absolute path for add_signals and add_signal when using dbExplorer - [Issue #202](https://github.com/signals-dev/Orion/issues/202) by @sarahmish
* dynamic scalability of TadGAN primitive based on `window_size` - [Issue #87](https://github.com/signals-dev/Orion/issues/87) by @sarahmish
## 0.1.7 - 2021-05-04
This version adds new features to the benchmark function where users can now save pipelines, view results as they are being calculated, and allow a single evaluation to be compared multiple times.
### Issues resolved
* Dask issues in benchmark function & improvements - [Issue #225](https://github.com/signals-dev/Orion/issues/225) by @sarahmish
* Numerical overflow when using contextual metrics - [Issue #212](https://github.com/signals-dev/Orion/issues/212) by @kronerte
## 0.1.6 - 2021-03-08
This version introduces two new pipelines: LSTM AE and Dense AE.
In addition to minor improvements, a bit of code refactoring took place to introduce
a new primtive: ``reconstruction_errors``.
### Issues resolved
* Comparison of DTW library performance - [Issue #205](https://github.com/signals-dev/Orion/issues/205) by @sarahmish
* Not able to pickle dump tadgan pipeline - [Issue #200](https://github.com/signals-dev/Orion/issues/200) by @sarahmish
* New pipeline LSTM and Dense autoencoders - [Issue #194](https://github.com/signals-dev/Orion/issues/194) by @sarahmish
* Readme - [Issue #192](https://github.com/signals-dev/Orion/issues/192) by @pvk-developer
* Unable to launch cli - [Issue #186](https://github.com/signals-dev/Orion/issues/186) by @sarahmish
* bullet points not formatted correctly in index.rst - [Issue #178](https://github.com/signals-dev/Orion/issues/178) by @micahjsmith
* Update notebooks - [Issue #176](https://github.com/signals-dev/Orion/issues/176) by @sarahmish
* Inaccuracy in README.md file in orion/evaluation/ - [Issue #157](https://github.com/signals-dev/Orion/issues/157) by @sarahmish
* Dockerfile -- docker does not find orion primitives automatically - [Issue #155](https://github.com/signals-dev/Orion/issues/155) by @sarahmish
* Primitive documentation - [Issue #151](https://github.com/signals-dev/Orion/issues/151) by @sarahmish
* Variable name inconsistency in tadgan - [Issue #150](https://github.com/signals-dev/Orion/issues/150) by @sarahmish
* Sync leaderboard tables between `BENCHMARK.md` and the docs - [Issue #148](https://github.com/signals-dev/Orion/issues/148) by @sarahmish
## 0.1.5 - 2020-12-25
This version includes the new style of documentation and a revamp of the `README.md`. In addition to some minor improvements
in the benchmark code and primitives. This release includes the transfer of `tadgan` pipeline to `verified`.
### Issues resolved
* Link with google colab - [Issue #144](https://github.com/signals-dev/Orion/issues/144) by @sarahmish
* Add `timeseries_anomalies` unittests - [Issue #136](https://github.com/signals-dev/Orion/issues/136) by @sarahmish
* Update `find_sequences` in converting series to arrays - [Issue #135](https://github.com/signals-dev/Orion/issues/135) by @sarahmish
* Definition of error/critic smooth window in score anomalies primitive - [Issue #132](https://github.com/signals-dev/Orion/issues/132) by @sarahmish
* Train-test split in benchmark enhancement - [Issue #130](https://github.com/signals-dev/Orion/issues/130) by @sarahmish
## 0.1.4 - 2020-10-16
Minor enhancements to benchmark
* Load ground truth before try-catch - [Issue #124](https://github.com/signals-dev/Orion/issues/124) by @sarahmish
* Converting timestamp to datetime in Azure primitive - [Issue #123](https://github.com/signals-dev/Orion/issues/123) by @sarahmish
* Benchmark exceptions - [Issue #120](https://github.com/signals-dev/Orion/issues/120) by @sarahmish
## 0.1.3 - 2020-09-29
New benchmark and Azure primitive.
* Implement a benchmarking function new feature - [Issue #94](https://github.com/signals-dev/Orion/issues/94) by @sarahmish
* Add azure anomaly detection as primitive new feature - [Issue #97](https://github.com/signals-dev/Orion/issues/97) by @sarahmish
* Critic and reconstruction error combination - [Issue #99](https://github.com/signals-dev/Orion/issues/99) by @sarahmish
* Fixed threshold for `find_anomalies` - [Issue #101](https://github.com/signals-dev/Orion/issues/101) by @sarahmish
* Add an option to have window size and window step size as percentages of error size - [Issue #102](https://github.com/signals-dev/Orion/issues/102) by @sarahmish
* Organize pipelines into verified and sandbox - [Issue #105](https://github.com/signals-dev/Orion/issues/105) by @sarahmish
* Ground truth parameter name enhancement - [Issue #114](https://github.com/signals-dev/Orion/issues/114) by @sarahmish
* Add benchmark dataset list and parameters to s3 bucket enhancement - [Issue #118](https://github.com/signals-dev/Orion/issues/118) by @sarahmish
## 0.1.2 - 2020-07-03
New Evaluation sub-package and refactor TadGAN.
* Two bugs when saving signalrun if there is no event detected - [Issue #92](https://github.com/signals-dev/Orion/issues/92) by @dyuliu
* File encoding/decoding issues about `README.md` and `HISTORY.md` - [Issue #88](https://github.com/signals-dev/Orion/issues/88) by @dyuliu
* Fix bottle neck of `score_anomaly` in Cyclegan primitive - [Issue #86](https://github.com/signals-dev/Orion/issues/86) by @dyuliu
* Adjust `epoch` meaning in Cyclegan primitive - [Issue #85](https://github.com/signals-dev/Orion/issues/85) by @sarahmish
* Rename evaluation to benchmark and metrics to evaluation - [Issue #83](https://github.com/signals-dev/Orion/issues/83) by @sarahmish
* Scoring function for intervals of size one - [Issue #76](https://github.com/signals-dev/Orion/issues/76) by @sarahmish
## 0.1.1 - 2020-05-11
New class and function based interfaces.
* Implement the Orion Class - [Issue #79](https://github.com/D3-AI/Orion/issues/79) by @csala
* Implement new functional interface - [Issue #80](https://github.com/D3-AI/Orion/issues/80) by @csala
## 0.1.0 - 2020-04-23
First Orion release to PyPI: https://pypi.org/project/orion-ml/
Raw data
{
"_id": null,
"home_page": "https://github.com/sintel-dev/Orion",
"name": "orion-ml",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<3.12",
"maintainer_email": "",
"keywords": "orion",
"author": "MIT Data To AI Lab",
"author_email": "dailabmit@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/79/a8/ac9a38714732c09fe0483ff0097afb980a5bbdb6649e5a04cd2de33ab247/orion-ml-0.6.0.tar.gz",
"platform": null,
"description": "<p align=\"left\">\n<img width=15% src=\"https://dai.lids.mit.edu/wp-content/uploads/2018/06/Logo_DAI_highres.png\" alt=\u201cDAI-Lab\u201d />\n<i>An open source project from Data to AI Lab at MIT.</i>\n</p>\n\n<p align=\"left\">\n<img width=20% src=\"https://dai.lids.mit.edu/wp-content/uploads/2018/08/orion.png\" alt=\u201cOrion\u201d />\n</p>\n\n[](https://pypi.org/search/?c=Development+Status+%3A%3A+2+-+Pre-Alpha)\n[](https://badge.fury.io/py/orion-ml) \n[](https://pypi.python.org/pypi/orion-ml)\n[](https://github.com/sintel-dev/Orion/actions?query=workflow%3A%22Run+Tests%22+branch%3Amaster)\n[](https://pepy.tech/project/orion-ml)\n[](https://mybinder.org/v2/gh/sintel-dev/Orion/master?filepath=tutorials)\n\n# Orion\n\nA machine learning library for unsupervised time series anomaly detection.\n\n| Important Links | |\n| --------------------------------------------- | -------------------------------------------------------------------- |\n| :computer: **[Website]** | Check out the Sintel Website for more information about the project. |\n| :book: **[Documentation]** | Quickstarts, User and Development Guides, and API Reference. |\n| :star: **[Tutorials]** | Checkout our notebooks |\n| :octocat: **[Repository]** | The link to the Github Repository of this library. |\n| :scroll: **[License]** | The repository is published under the MIT License. |\n| [![][Slack Logo] **Community**][Community] | Join our Slack Workspace for announcements and discussions. |\n\n[Website]: https://sintel.dev/\n[Documentation]: https://sintel-dev.github.io/Orion\n[Tutorials]: https://github.com/sintel-dev/Orion/tree/master/tutorials\n[Repository]: https://github.com/sintel-dev/Orion\n[License]: https://github.com/sintel-dev/Orion/blob/master/LICENSE\n[Community]: https://join.slack.com/t/sintel-space/shared_invite/zt-q147oimb-4HcphcxPfDAM0O9_4PaUtw\n[Slack Logo]: https://github.com/sintel-dev/Orion/blob/master/docs/images/slack.png\n\n# Overview\n\nOrion is a machine learning library built for *unsupervised time series anomaly detection*. With a given time series data, we provide a number of \u201cverified\u201d ML pipelines (a.k.a Orion pipelines) that identify rare patterns and flag them for expert review.\n\nThe library makes use of a number of **automated machine learning** tools developed under [Data to AI Lab at MIT](https://dai.lids.mit.edu/).\n\nRead about using an Orion pipeline on NYC taxi dataset in a blog series:\n\n[Part 1: Learn about unsupervised time series anomaly detection](https://t.co/yIFVM1oRwQ?amp=1) | [Part 2: Learn how we use GANs to solving the problem? ](https://link.medium.com/cGsBD0Fevbb) | [Part 3: How does one evaluate anomaly detection pipelines?](https://link.medium.com/FqCrFXMevbb)\n:--------------------------------------:|:---------------------------------------------:|:--------------------------------------------:\n |  | \n\n**Notebooks:** Discover *Orion* through colab by launching our [notebooks](https://drive.google.com/drive/folders/1FAcCEiE1JDsqaMjGcmiw5a5XuGh13c9Q?usp=sharing)!\n\n# Quickstart\n\n## Install with pip\n\nThe easiest and recommended way to install **Orion** is using [pip](https://pip.pypa.io/en/stable/):\n\n```bash\npip install orion-ml\n```\n\nThis will pull and install the latest stable release from [PyPi](https://pypi.org/).\n\n\nIn the following example we show how to use one of the **Orion Pipelines**.\n\n## Fit an Orion pipeline\n\nWe will load a demo data for this example:\n\n```python3\nfrom orion.data import load_signal\n\ntrain_data = load_signal('S-1-train')\ntrain_data.head()\n```\n\nwhich should show a signal with `timestamp` and `value`.\n```\n timestamp value\n0 1222819200 -0.366359\n1 1222840800 -0.394108\n2 1222862400 0.403625\n3 1222884000 -0.362759\n4 1222905600 -0.370746\n```\n\nIn this example we use `aer` pipeline and set some hyperparameters (in this case training epochs as 5).\n\n```python3\nfrom orion import Orion\n\nhyperparameters = {\n 'orion.primitives.aer.AER#1': {\n 'epochs': 5,\n 'verbose': True\n }\n}\n\norion = Orion(\n pipeline='aer',\n hyperparameters=hyperparameters\n)\n\norion.fit(train_data)\n```\n\n## Detect anomalies using the fitted pipeline\nOnce it is fitted, we are ready to use it to detect anomalies in our incoming time series:\n\n```python3\nnew_data = load_signal('S-1-new')\nanomalies = orion.detect(new_data)\n```\n> :warning: Depending on your system and the exact versions that you might have installed some *WARNINGS* may be printed. These can be safely ignored as they do not interfere with the proper behavior of the pipeline.\n\nThe output of the previous command will be a ``pandas.DataFrame`` containing a table of detected anomalies:\n\n```\n start end severity\n0 1402012800 1403870400 0.122539\n```\n\n# Leaderboard\nIn every release, we run Orion benchmark. We maintain an up-to-date leaderboard with the current scoring of the verified pipelines according to the benchmarking procedure.\n\nWe run the benchmark on **12** datasets with their known grounth truth. We record the score of the pipelines on each datasets. To compute the leaderboard table, we showcase the number of wins each pipeline has over the ARIMA pipeline.\n\n| Pipeline | Outperforms ARIMA |\n|---------------------------|--------------------|\n| AER | 11 |\n| TadGAN | 7 |\n| LSTM Dynamic Thresholding | 8 |\n| LSTM Autoencoder | 7 |\n| Dense Autoencoder | 7 |\n| VAE | 6 |\n| LNN | 7 |\n| Matrix Profile | 5 |\n| [GANF](https://arxiv.org/pdf/2202.07857.pdf) | 5 |\n| [Azure](https://azure.microsoft.com/en-us/products/cognitive-services/anomaly-detector/) | 0 |\n\n\nYou can find the scores of each pipeline on every signal recorded in the [details Google Sheets document](https://docs.google.com/spreadsheets/d/1HaYDjY-BEXEObbi65fwG0om5d8kbRarhpK4mvOZVmqU/edit?usp=sharing). The summarized results can also be browsed in the following [summary Google Sheets document](https://docs.google.com/spreadsheets/d/1ZPUwYH8LhDovVeuJhKYGXYny7472HXVCzhX6D6PObmg/edit?usp=sharing).\n\n# Resources\n\nAdditional resources that might be of interest:\n* Learn about [benchmarking pipelines](BENCHMARK.md).\n* Read about [pipeline evaluation](orion/evaluation/README.md).\n* Find out more about [TadGAN](https://arxiv.org/pdf/2009.07769v3.pdf).\n\n# Citation\n\nIf you use **AER** for your research, please consider citing the following paper:\n\nLawrence Wong, Dongyu Liu, Laure Berti-Equille, Sarah Alnegheimish, Kalyan Veeramachaneni. [AER: Auto-Encoder with Regression for Time Series Anomaly Detection](https://arxiv.org/pdf/2212.13558.pdf).\n\n```\n@inproceedings{wong2022aer,\n title={AER: Auto-Encoder with Regression for Time Series Anomaly Detection},\n author={Wong, Lawrence and Liu, Dongyu and Berti-Equille, Laure and Alnegheimish, Sarah and Veeramachaneni, Kalyan},\n booktitle={2022 IEEE International Conference on Big Data (IEEE BigData)},\n pages={1152-1161},\n doi={10.1109/BigData55660.2022.10020857},\n organization={IEEE},\n year={2022}\n}\n```\n\nIf you use **TadGAN** for your research, please consider citing the following paper:\n\nAlexander Geiger, Dongyu Liu, Sarah Alnegheimish, Alfredo Cuesta-Infante, Kalyan Veeramachaneni. [TadGAN - Time Series Anomaly Detection Using Generative Adversarial Networks](https://arxiv.org/pdf/2009.07769v3.pdf).\n\n```\n@inproceedings{geiger2020tadgan,\n title={TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks},\n author={Geiger, Alexander and Liu, Dongyu and Alnegheimish, Sarah and Cuesta-Infante, Alfredo and Veeramachaneni, Kalyan},\n booktitle={2020 IEEE International Conference on Big Data (IEEE BigData)},\n pages={33-43},\n doi={10.1109/BigData50022.2020.9378139},\n organization={IEEE},\n year={2020}\n}\n```\n\nIf you use **Orion** which is part of the **Sintel** ecosystem for your research, please consider citing the following paper:\n\nSarah Alnegheimish, Dongyu Liu, Carles Sala, Laure Berti-Equille, Kalyan Veeramachaneni. [Sintel: A Machine Learning Framework to Extract Insights from Signals](https://dl.acm.org/doi/pdf/10.1145/3514221.3517910).\n```\n@inproceedings{alnegheimish2022sintel,\n title={Sintel: A Machine Learning Framework to Extract Insights from Signals},\n author={Alnegheimish, Sarah and Liu, Dongyu and Sala, Carles and Berti-Equille, Laure and Veeramachaneni, Kalyan}, \n booktitle={Proceedings of the 2022 International Conference on Management of Data},\n pages={1855\u20131865},\n numpages={11},\n publisher={Association for Computing Machinery},\n doi={10.1145/3514221.3517910},\n series={SIGMOD '22},\n year={2022}\n}\n```\n\n\nHistory\n=======\n\n## 0.6.0 - 2024-2-13\n\nSupport for python 3.10 and 3.11\n\n### Issues resolved\n\n* Update ``test_core`` file \u2013 [Issue #507](https://github.com/signals-dev/Orion/issues/507) by @sarahmish\n* update ARIMA primitive and pipeline \u2013 [Issue #503](https://github.com/signals-dev/Orion/issues/503) by @sarahmish\n* Update Dependency \u2013 [Issue #497](https://github.com/signals-dev/Orion/issues/497) & [Issue #499](https://github.com/signals-dev/Orion/issues/499) by @sarahmish\n* Update Python for Dependency Test \u2013 [Issue #484](https://github.com/signals-dev/Orion/issues/484) by @sarahmish\n* Add python 3.10 and 3.11 & drop 3.6 and 3.7 \u2013 [Issue #477](https://github.com/signals-dev/Orion/issues/477) by @sarahmish\n* LNN Pipeline \u2013 [Issue #475](https://github.com/signals-dev/Orion/issues/475) by @sarahmish\n\n\n## 0.5.2 - 2023-10-19\n\nSupport for python 3.9 and new Matrix Profile pipeline\n\n### Issues resolved\n\n* Send pipeline names in benchmark arguments \u2013 [Issue #466](https://github.com/signals-dev/Orion/issues/466) by @sarahmish\n* Add continuation parameter for benchmark \u2013 [Issue #464](https://github.com/signals-dev/Orion/issues/464) by @sarahmish\n* Fix references in documentation \u2013 [Issue #453](https://github.com/signals-dev/Orion/issues/453) by @sarahmish\n* Update documentation \u2013 [Issue #448](https://github.com/signals-dev/Orion/issues/448) by @sarahmish\n* Add matrix profiling method \u2013 [Issue #446](https://github.com/signals-dev/Orion/issues/446) by @sarahmish\n* Support python 3.9 \u2013 [Issue #408](https://github.com/signals-dev/Orion/issues/408) by @sarahmish\n\n\n## 0.5.1 - 2023-08-16\n\nThis version introduces a new dataset to the benchmark.\n\n### Issues resolved\n\n* Add UCR dataset to the benchmark \u2013 [Issue #443](https://github.com/signals-dev/Orion/issues/443) by @sarahmish\n* docker image build failed \u2013 [Issue #439](https://github.com/signals-dev/Orion/issues/439) by @sarahmish\n* Edit interval settings in ``azure`` pipeline \u2013 [Issue #436](https://github.com/signals-dev/Orion/issues/436) by @sarahmish\n\n\n## 0.5.0 - 2023-05-23\n\nThis version uses ``ml-stars`` package instead of ``mlprimitives``.\n\n### Issues resolved\n\n* Migrate to ml-stars \u2013 [Issue #418](https://github.com/signals-dev/Orion/issues/418) by @sarahmish\n* Updating ``best_cost`` in ``find_anomalies`` primitive \u2013 [Issue #403](https://github.com/signals-dev/Orion/issues/403) by @sarahmish\n* Retire ``lstm_dynamic_threshold_gpu`` and ``lstm_autoencoder_gpu`` pipeline maintenance \u2013 [Issue #373](https://github.com/signals-dev/Orion/issues/373) by @sarahmish\n* Typo in xlsxwriter dependency specification \u2013 [Issue #394](https://github.com/signals-dev/Orion/issues/394) by @sarahmish\n* ``orion.evaluate`` uses fails when fitting \u2013 [Issue #384](https://github.com/signals-dev/Orion/issues/384) by @sarahmish\n* AER pipeline with visualization option \u2013 [Issue #379](https://github.com/signals-dev/Orion/issues/379) by @sarahmish\n\n\n## 0.4.1 - 2023-01-31\n\n### Issues resolved\n\n* Move VAE from sandbox to verified \u2013 [Issue #377](https://github.com/signals-dev/Orion/issues/377) by @sarahmish\n* Pin ``opencv`` \u2013 [Issue #372](https://github.com/signals-dev/Orion/issues/372) by @sarahmish\n* Pin ``scikit-learn`` \u2013 [Issue #367](https://github.com/signals-dev/Orion/issues/367) by @sarahmish\n* Fix VAE documentation \u2013 [Issue #360](https://github.com/signals-dev/Orion/issues/360) by @sarahmish\n\n\n## 0.4.0 - 2022-11-08\n\nThis version introduces several new enhancements:\n\n* Support to python 3.8\n* Migrating to Tensorflow 2.0\n* New pipeline, namely ``VAE``, a Variational AutoEncoder model.\n\n### Issues resolved\n\n* Add python 3.8 \u2013 [Issue #342](https://github.com/signals-dev/Orion/issues/342) by @sarahmish\n* VAE (Variational Autoencoders) pipeline implementation \u2013 [Issue #349](https://github.com/signals-dev/Orion/issues/349) by @dyuliu\n* Add masking option for ``regression_errors`` \u2013 [Issue #352](https://github.com/signals-dev/Orion/issues/352) by @dyuliu\n* Changes in TadGAN for tensorflow 2.0 \u2013 [Issue #161](https://github.com/signals-dev/Orion/issues/161) by @lcwong0928\n* Add an automatic dependency checker \u2013 [Issue #320](https://github.com/signals-dev/Orion/issues/320) by @sarahmish\n* TadGAN ``batch_size`` cannot be changed \u2013 [Issue #313](https://github.com/signals-dev/Orion/issues/313) by @sarahmish\n\n\n## 0.3.2 - 2022-07-04\n\nThis version fixes some of the issues in ``aer``, ``ae``, and ``tadgan`` pipelines.\n\n### Issues resolved\n\n* Fix AER model predict error after loading \u2013 [Issue #304](https://github.com/signals-dev/Orion/issues/304) by @lcwong0928\n* Update AE to work with any `window_size` \u2013 [Issue #300](https://github.com/signals-dev/Orion/issues/300) by @sarahmish\n* Updated tadgan_viz.json \u2013 [Issue #292](https://github.com/signals-dev/Orion/issues/292) by @Hramir\n\n\n## 0.3.1 - 2022-04-26\n\nThis version introduce a new pipeline, namely ``AER``, an AutoEncoder Regressor model.\n\n### Issues resolved\n* Add AER Model - [Issue #286](https://github.com/signals-dev/Orion/issues/286) by @lcwong0928\n\n\n## 0.3.0 - 2022-03-31\n\nThis version deprecates the support of ``OrionDBExplorer``, which has been migrated to\n[sintel](https://github.com/signals-dev/Orion). As a result, ``Orion`` no longer requires\nmongoDB as a dependency.\n\n### Issues resolved\n* Update dependency - [Issue #283](https://github.com/signals-dev/Orion/issues/283) by @sarahmish\n* General housekeeping - [Issue #278](https://github.com/signals-dev/Orion/issues/278) by @sarahmish\n* Fix tutorial testing issue - [Issue #276](https://github.com/signals-dev/Orion/issues/276) by @sarahmish\n* Migrate OrionExplorer to Sintel - [Issue #275](https://github.com/signals-dev/Orion/issues/275) by @dyuliu\n* LSTM viz JSON pipeline added - [Issue #271](https://github.com/signals-dev/Orion/issues/271) by @Hramir\n\n\n## 0.2.1 - 2022-02-18\n\nThis version introduces improvements and more testing.\n\n### Issues resolved\n* Adjusting builds for TadGAN - [Issue #261](https://github.com/signals-dev/Orion/issues/261) by @sarahmish\n* Testing tutorials, dependencies, and OS - [Issue #251](https://github.com/signals-dev/Orion/issues/251) by @sarahmish\n\n\n## 0.2.0 - 2021-10-11\n\nThis version supports multivariate timeseries as input. In addition to minor improvements\nand maintenance.\n\n### Issues resolved\n* `setuptools` no longer supports `lib2to3` breaking `mongoengine` - [Issue #252](https://github.com/signals-dev/Orion/issues/252) by @sarahmish\n* Supporting multivariate input - [Issue #248](https://github.com/signals-dev/Orion/issues/248) by @sarahmish\n* TadGAN pipeline with visualization option - [Issue #240](https://github.com/signals-dev/Orion/issues/240) by @sarahmish\n* Support saving absolute path for add_signals and add_signal when using dbExplorer - [Issue #202](https://github.com/signals-dev/Orion/issues/202) by @sarahmish\n* dynamic scalability of TadGAN primitive based on `window_size` - [Issue #87](https://github.com/signals-dev/Orion/issues/87) by @sarahmish\n\n\n## 0.1.7 - 2021-05-04\n\nThis version adds new features to the benchmark function where users can now save pipelines, view results as they are being calculated, and allow a single evaluation to be compared multiple times.\n\n### Issues resolved\n* Dask issues in benchmark function & improvements - [Issue #225](https://github.com/signals-dev/Orion/issues/225) by @sarahmish\n* Numerical overflow when using contextual metrics - [Issue #212](https://github.com/signals-dev/Orion/issues/212) by @kronerte\n\n\n## 0.1.6 - 2021-03-08\n\nThis version introduces two new pipelines: LSTM AE and Dense AE.\nIn addition to minor improvements, a bit of code refactoring took place to introduce\na new primtive: ``reconstruction_errors``.\n\n### Issues resolved\n* Comparison of DTW library performance - [Issue #205](https://github.com/signals-dev/Orion/issues/205) by @sarahmish\n* Not able to pickle dump tadgan pipeline - [Issue #200](https://github.com/signals-dev/Orion/issues/200) by @sarahmish\n* New pipeline LSTM and Dense autoencoders - [Issue #194](https://github.com/signals-dev/Orion/issues/194) by @sarahmish\n* Readme - [Issue #192](https://github.com/signals-dev/Orion/issues/192) by @pvk-developer\n* Unable to launch cli - [Issue #186](https://github.com/signals-dev/Orion/issues/186) by @sarahmish\n* bullet points not formatted correctly in index.rst - [Issue #178](https://github.com/signals-dev/Orion/issues/178) by @micahjsmith\n* Update notebooks - [Issue #176](https://github.com/signals-dev/Orion/issues/176) by @sarahmish\n* Inaccuracy in README.md file in orion/evaluation/ - [Issue #157](https://github.com/signals-dev/Orion/issues/157) by @sarahmish\n* Dockerfile -- docker does not find orion primitives automatically - [Issue #155](https://github.com/signals-dev/Orion/issues/155) by @sarahmish\n* Primitive documentation - [Issue #151](https://github.com/signals-dev/Orion/issues/151) by @sarahmish\n* Variable name inconsistency in tadgan - [Issue #150](https://github.com/signals-dev/Orion/issues/150) by @sarahmish\n* Sync leaderboard tables between `BENCHMARK.md` and the docs - [Issue #148](https://github.com/signals-dev/Orion/issues/148) by @sarahmish\n\n\n## 0.1.5 - 2020-12-25\n\nThis version includes the new style of documentation and a revamp of the `README.md`. In addition to some minor improvements\nin the benchmark code and primitives. This release includes the transfer of `tadgan` pipeline to `verified`.\n\n### Issues resolved\n* Link with google colab - [Issue #144](https://github.com/signals-dev/Orion/issues/144) by @sarahmish\n* Add `timeseries_anomalies` unittests - [Issue #136](https://github.com/signals-dev/Orion/issues/136) by @sarahmish\n* Update `find_sequences` in converting series to arrays - [Issue #135](https://github.com/signals-dev/Orion/issues/135) by @sarahmish\n* Definition of error/critic smooth window in score anomalies primitive - [Issue #132](https://github.com/signals-dev/Orion/issues/132) by @sarahmish\n* Train-test split in benchmark enhancement - [Issue #130](https://github.com/signals-dev/Orion/issues/130) by @sarahmish\n\n\n## 0.1.4 - 2020-10-16\n\nMinor enhancements to benchmark\n\n* Load ground truth before try-catch - [Issue #124](https://github.com/signals-dev/Orion/issues/124) by @sarahmish\n* Converting timestamp to datetime in Azure primitive - [Issue #123](https://github.com/signals-dev/Orion/issues/123) by @sarahmish\n* Benchmark exceptions - [Issue #120](https://github.com/signals-dev/Orion/issues/120) by @sarahmish\n\n\n## 0.1.3 - 2020-09-29\n\nNew benchmark and Azure primitive.\n\n* Implement a benchmarking function new feature - [Issue #94](https://github.com/signals-dev/Orion/issues/94) by @sarahmish\n* Add azure anomaly detection as primitive new feature - [Issue #97](https://github.com/signals-dev/Orion/issues/97) by @sarahmish\n* Critic and reconstruction error combination - [Issue #99](https://github.com/signals-dev/Orion/issues/99) by @sarahmish\n* Fixed threshold for `find_anomalies` - [Issue #101](https://github.com/signals-dev/Orion/issues/101) by @sarahmish\n* Add an option to have window size and window step size as percentages of error size - [Issue #102](https://github.com/signals-dev/Orion/issues/102) by @sarahmish\n* Organize pipelines into verified and sandbox - [Issue #105](https://github.com/signals-dev/Orion/issues/105) by @sarahmish\n* Ground truth parameter name enhancement - [Issue #114](https://github.com/signals-dev/Orion/issues/114) by @sarahmish\n* Add benchmark dataset list and parameters to s3 bucket enhancement - [Issue #118](https://github.com/signals-dev/Orion/issues/118) by @sarahmish\n\n## 0.1.2 - 2020-07-03\n\nNew Evaluation sub-package and refactor TadGAN.\n\n* Two bugs when saving signalrun if there is no event detected\u00a0- [Issue #92](https://github.com/signals-dev/Orion/issues/92) by\u00a0@dyuliu\u00a0\n* File encoding/decoding issues about `README.md` and `HISTORY.md`\u00a0- [Issue #88](https://github.com/signals-dev/Orion/issues/88) by\u00a0@dyuliu\n* Fix bottle neck of `score_anomaly` in Cyclegan primitive - [Issue #86](https://github.com/signals-dev/Orion/issues/86) by @dyuliu\n* Adjust `epoch` meaning in Cyclegan primitive\u00a0- [Issue #85](https://github.com/signals-dev/Orion/issues/85) by @sarahmish\n* Rename evaluation to benchmark and metrics to evaluation - [Issue #83](https://github.com/signals-dev/Orion/issues/83) by @sarahmish\n* Scoring function for intervals of size one\u00a0- [Issue #76](https://github.com/signals-dev/Orion/issues/76) by @sarahmish\n\n## 0.1.1 - 2020-05-11\n\nNew class and function based interfaces.\n\n* Implement the Orion Class - [Issue #79](https://github.com/D3-AI/Orion/issues/79) by @csala\n* Implement new functional interface - [Issue #80](https://github.com/D3-AI/Orion/issues/80) by @csala\n\n## 0.1.0 - 2020-04-23\n\nFirst Orion release to PyPI: https://pypi.org/project/orion-ml/\n",
"bugtrack_url": null,
"license": "MIT license",
"summary": "Orion is a machine learning library built for unsupervised time series anomaly detection.",
"version": "0.6.0",
"project_urls": {
"Homepage": "https://github.com/sintel-dev/Orion"
},
"split_keywords": [
"orion"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "18489323edee10715c86c7c0634d78a21dc923ab0a7f65425f2a4f8635816239",
"md5": "26a7af051d10ee882588ed1d1362faea",
"sha256": "f7dabdae64cfda275196fbf06c72ff638ab9cee7110bfa56529b79aa0267fc00"
},
"downloads": -1,
"filename": "orion_ml-0.6.0-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "26a7af051d10ee882588ed1d1362faea",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.8,<3.12",
"size": 131113,
"upload_time": "2024-02-13T21:31:14",
"upload_time_iso_8601": "2024-02-13T21:31:14.884082Z",
"url": "https://files.pythonhosted.org/packages/18/48/9323edee10715c86c7c0634d78a21dc923ab0a7f65425f2a4f8635816239/orion_ml-0.6.0-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "79a8ac9a38714732c09fe0483ff0097afb980a5bbdb6649e5a04cd2de33ab247",
"md5": "c464e9c1f442b7766d01e7a15b97db33",
"sha256": "2338f601e059a3afde959ca1120ad6351ec5a1740ae22740a1ebca41ca583263"
},
"downloads": -1,
"filename": "orion-ml-0.6.0.tar.gz",
"has_sig": false,
"md5_digest": "c464e9c1f442b7766d01e7a15b97db33",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<3.12",
"size": 1046920,
"upload_time": "2024-02-13T21:31:17",
"upload_time_iso_8601": "2024-02-13T21:31:17.450070Z",
"url": "https://files.pythonhosted.org/packages/79/a8/ac9a38714732c09fe0483ff0097afb980a5bbdb6649e5a04cd2de33ab247/orion-ml-0.6.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-13 21:31:17",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "sintel-dev",
"github_project": "Orion",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "orion-ml"
}
JSON
