<div align="center">
<img src="/docs/OSF-EIMTC-Logo.png" alt="OSF-EIMTC-Logo" width="500px" >
</div>
<br/><br/>
[<img alt="Ariel University Home" src="https://img.shields.io/badge/Ariel%20University-blue?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAFQAAABUAQMAAAAmpYKCAAAABGdBTUEAALGPC/xhBQAAAAFzUkdCAK7OHOkAAAAJcEhZcwAAEnQAABJ0Ad5mH3gAAAAGUExURQsUK/7//yyPHFQAAADqSURBVCjPbdCxkYRADETRT2FgshmQwmVASpfBTmiEQggyMSj6jEFCKs573u9qYJIkA4Ax3Bgk6QDYQJJOgL36K+kCMFglCeCARZIacFbPkrQB1x3eAaWwWnGMUBqhFJZV+wilEUrh4jRCaYRSuPoZoTRCKVz9jFAKK4WrnxGKsKEIv3yPMBRhQxF++R5hKMKGIvx2H9Hdw909/HYf0d3D5pdlz+4+oruHbzeGf731jxXh21a9ug9Y3CfM7qt6cgvGcGMIb8WEd/i6DVb3Ub24T5jdF0xuVY/hxhDeILxX/7oNftwHfNwnfP4ALPV3tucoteYAAAAASUVORK5CYII=" />](https://www.ariel.ac.il/)
[<img alt="ACIC Home" src="https://img.shields.io/badge/Ariel%20Cyber%20Innovation%20Center%20-blue?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAG8AAABaBAMAAABEXBVNAAAAGFBMVEXj9/kKFTis2uCBv8ZXsLN73eA+Rl4Rpq9f4ewTAAAAB3RSTlMB/UmY4eXFFy+TWQAABOpJREFUWMOtmEFv2zoMxw3lofe0QM56sl97zSQlvg6xDN8XG7kuKbAPUMvQ1x8pybbsyE42PJ261T+R/JOiqCZJdKVSwTqKozwehRBH8Vl8ws/w4z5ZXdft0npd5V62y2sV/HcF/LnCEfD0V3xtt28r4PvyvrDlfs3gji7vufQ7kl7XtMMo2NKeywbdr7d0SdHbigCgTzzMb9vtJ30g+S0exBtdTTKQr/H/dvsRIUWwzuIgBPU+xXJJ+ghSc7f03quwi4CbXrP8HjQnL2xM9RcPgsFmuoQwmoafzPNktyOV6WYgZ5UpaBjNLI0YQKqMbuouWLrTp9RoJJfAVxdg11RzbcCk2VvwFss/guhofacNTyvUZ8jYNP+7G0aomzuDYJKm5mvYPFocpjtUkXQUrGpXwY3pYmkEZyttdZiWTootUXH46R9TXBZAQweQ+WwSVauyrBX8MzNlHPz6QFnfbcqkzQ2srGiErJu6SJIP08TBNhtAKBHDrWElCzgFsgFnc9NUS+B3X3OZFlXLUyBKRZ3DRQrZiHKms+DGgSzXKEqJPlqPparMAqhtIi34UbDclbNyBUgkTxv9GMxAB2tRnfqsCAkFFwcNM60HN6DCiTJYY0KZ6eoFUCBI7IGUqqD3FReCKvj5PIJEBLbofcVpzrIoGFryfWwCnib952x0BISdC9dvyvHTva3dVZDkugLHGPvPdHI8g9aFAaxioIGasZkJDkdny+kBWHFSNc350IQxTi2WVRsFmbEWq3YJbDDGzR2opRYsFYfQ4g+nWg9WrnKSmaq2pYCqAVi4ozdYtODs8tgoxXzlyKC1BZ468OXu1vEFSyZFvg/SqC34PoJE8GmwAUgD0BzwWI1djsj+7vMgfBAFtcQSDMDK1iRNiJUZ7ojzAmhbx9iQM839aXYLz11UHBDtO44JA9iyrBOC9xd+czldoiAUBg0vHQQNtMfCdi2lLuaUB8XKxqRiJw/uR+yRGqYMmIsldATwtf0YQM7S6d2xGcGhK9seQMCEHj+G0g2anMv/UKpZNWk8sBGLdscMxQ/ynxA2nQgvhueLd058QMI9WArqZLE5B0VNojc5jKv2QJqvNNaO/fn/6Tt3GB5TeBtAd0/v76vWXgD9fESUCYQhsmzqWsqL+XEfZJG5SnWiwuk12NrcEqpRWDsSt59rClkMtPlQB2xttS0Zpbr65I6kHYfmnrZ0GI9IfmKXsq7rpqztbef8Jrnhcq6pPVND3SAYzHuK9/1n7msHZUnHEJMcTntZYIFLWP0UjF1gZrKQVsVrn/7MaO3EIYTQUd8c+/pUGqznzTABQjTF/NJKKTbhqUmYAJ2nQzNmjM7n+eur2zAwCWOcvfGu0YF88raCsZaPPYtLa/Bl9TXjXnNgsmW9s+fURojPmZVXCXGvRDiVBbt4R52kZP39iPviCA2C8xRHB7Ss+frLsn9DvPk+jWTJ/SVP1qQJH6YwBHWMQbeqsEqtwfVXOX7x5p8tmhHg7CiAD7n1B5v9ZO8HEbgvXIBPGLQFsvPXkek5sn1o0H1060nHJcf1p+worD8+acEfPcjnUb499cSNPCTDD8m3B0UzfbjzyV8O9s+B6NuOB7vckifX5joKhBHTZ0H3Rwjuud3znM0ckOy4DZx+LkyL4B8ptn/EefIvuJ7c/TEH5Pvfcdg+P1nyP67fYb4noeL/R5wAAAAASUVORK5CYII=" />](https://www.ariel-cyber.co.il/)
**O**pen-**S**ource **F**ramework for **E**ncrypted **I**nternet and **M**alicious **T**raffic **C**lassification.
## Framework publish to PyPi
The framework is been uploaded to the pypi for easy use of the framework.
## Ariel University
This framework was developed by Ariel Universtiy, you may read more about the framework in our paper:
[Open-Source Framework for Encrypted Internet and Malicious Traffic Classification](https://arxiv.org/abs/2206.10144)
## Examples
For usage examples please see the `./examples/` directory for various applications of extracting features from a set of PCAP files or evaluating state-of-the-art deep-learning models.
## Installation
Using pip:
```
python3 -m pip install OSF-EIMTC
```
Uninstall with:
```
pythom3 -m pip uninstall OSF-EIMTC
```
Install tensorflow for deep-learning models:
see the official website: https://www.tensorflow.org/install.
Generally, you can install tensorflow with pip:
```
python3 -m pip install tensorflow
```
But check the offical instructions for more customized options such as for GPU utilization.
For TLS features, TShark is required. While it is possible to install and configure it for Windows usage, the framework is yet to officialy support the system.
```
sudo apt install tshark
```
### Dependencies
- Platform
- OS: Linux (Tested with WSL Ubuntu 18.04).
- Version: Python 3.6.6+ (3.7+ is recommended)
- Python modules
- NFStream>=6.5.1
- pandas>=1.1.5
- numpy>=1.19.5
- scapy>=2.4.5 (for parsing higher level protocols such as DNS)
- unittest (for tests)
- sklearn>=1.1.0
There are some plugins that might require some additional dependancies.
- Optional Python modules
- pyasn (for ASN info, requires two additional DBs)
- functools (for caching functions results)
- click (for CLI)
- For Deep-Learning Models
- tensorflow>=2.4.0.
- 3rd party tools (that need to be installed)
- TShark 3.4.8+ (for TLS features). Try install with `sudo apt update` and then `sudo apt install tshark`, if you cannot get the 3.0+ version try to install wireshark instead via `sudo apt install wireshark`.
- 3rd party data files (provided in `./tools` directory)
- pyasn.db (ASN mapping database) - updated one can be downloaded via https://github.com/hadiasghari/pyasn.
- ip2asn.tsv (ASN context info database) - for updated data download from https://iptoasn.com/.
## Run Tests
```
python3 -m unittest discover EIMTC
```
## Project Structure
Under `./src`
```
└── EIMTC/ # Project's root directory.
├── datasets/ # Links and information for PCAP datasets.
├── metrics/ # Evaluation metrics for models.
├── models/ # State-of-the-art DL models.
├── preprocessing/
├── selection/
├── plugins/ # NFStream plugins.
├── stats/
├── temp/ # Temporary files generated by the system.
├── tests/ # System tests.
│ └── pcaps/ # PCAP files for tests.
├── thrid_party/ # 3rd-party tools used by the system.
└── tools/ # 3rd-party tools used by the system.
```
## Citation
```
article{BADER2024271,
title = {OSF-EIMTC: An open-source framework for standardized encrypted internet traffic classification},
author = {Ofek Bader and Adi Lichy and Amit Dvir and Ran Dubin and Chen Hajaj},
journal = {Computer Communications},
volume = {213},
pages = {271-284},
year = {2024}
}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/ArielCyber/OSF-EIMTC/",
"name": "OSF-EIMTC",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "nfstream, pcap, network, deep-learning, extraction",
"author": "Ariel University",
"author_email": "chenha@ariel.ac.il",
"download_url": "https://files.pythonhosted.org/packages/c1/51/9397dc62619ec17f6dc5d2496b0f96e3776ed2430e2df81fcd0ad5955625/OSF_EIMTC-0.1.60.tar.gz",
"platform": null,
"description": "<div align=\"center\">\n<img src=\"/docs/OSF-EIMTC-Logo.png\" alt=\"OSF-EIMTC-Logo\" width=\"500px\" >\n</div>\n<br/><br/>\n\n\n\n[<img alt=\"Ariel University Home\" src=\"https://img.shields.io/badge/Ariel%20University-blue?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAFQAAABUAQMAAAAmpYKCAAAABGdBTUEAALGPC/xhBQAAAAFzUkdCAK7OHOkAAAAJcEhZcwAAEnQAABJ0Ad5mH3gAAAAGUExURQsUK/7//yyPHFQAAADqSURBVCjPbdCxkYRADETRT2FgshmQwmVASpfBTmiEQggyMSj6jEFCKs573u9qYJIkA4Ax3Bgk6QDYQJJOgL36K+kCMFglCeCARZIacFbPkrQB1x3eAaWwWnGMUBqhFJZV+wilEUrh4jRCaYRSuPoZoTRCKVz9jFAKK4WrnxGKsKEIv3yPMBRhQxF++R5hKMKGIvx2H9Hdw909/HYf0d3D5pdlz+4+oruHbzeGf731jxXh21a9ug9Y3CfM7qt6cgvGcGMIb8WEd/i6DVb3Ub24T5jdF0xuVY/hxhDeILxX/7oNftwHfNwnfP4ALPV3tucoteYAAAAASUVORK5CYII=\" />](https://www.ariel.ac.il/)\n[<img alt=\"ACIC Home\" src=\"https://img.shields.io/badge/Ariel%20Cyber%20Innovation%20Center%20-blue?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAG8AAABaBAMAAABEXBVNAAAAGFBMVEXj9/kKFTis2uCBv8ZXsLN73eA+Rl4Rpq9f4ewTAAAAB3RSTlMB/UmY4eXFFy+TWQAABOpJREFUWMOtmEFv2zoMxw3lofe0QM56sl97zSQlvg6xDN8XG7kuKbAPUMvQ1x8pybbsyE42PJ261T+R/JOiqCZJdKVSwTqKozwehRBH8Vl8ws/w4z5ZXdft0npd5V62y2sV/HcF/LnCEfD0V3xtt28r4PvyvrDlfs3gji7vufQ7kl7XtMMo2NKeywbdr7d0SdHbigCgTzzMb9vtJ30g+S0exBtdTTKQr/H/dvsRIUWwzuIgBPU+xXJJ+ghSc7f03quwi4CbXrP8HjQnL2xM9RcPgsFmuoQwmoafzPNktyOV6WYgZ5UpaBjNLI0YQKqMbuouWLrTp9RoJJfAVxdg11RzbcCk2VvwFss/guhofacNTyvUZ8jYNP+7G0aomzuDYJKm5mvYPFocpjtUkXQUrGpXwY3pYmkEZyttdZiWTootUXH46R9TXBZAQweQ+WwSVauyrBX8MzNlHPz6QFnfbcqkzQ2srGiErJu6SJIP08TBNhtAKBHDrWElCzgFsgFnc9NUS+B3X3OZFlXLUyBKRZ3DRQrZiHKms+DGgSzXKEqJPlqPparMAqhtIi34UbDclbNyBUgkTxv9GMxAB2tRnfqsCAkFFwcNM60HN6DCiTJYY0KZ6eoFUCBI7IGUqqD3FReCKvj5PIJEBLbofcVpzrIoGFryfWwCnib952x0BISdC9dvyvHTva3dVZDkugLHGPvPdHI8g9aFAaxioIGasZkJDkdny+kBWHFSNc350IQxTi2WVRsFmbEWq3YJbDDGzR2opRYsFYfQ4g+nWg9WrnKSmaq2pYCqAVi4ozdYtODs8tgoxXzlyKC1BZ468OXu1vEFSyZFvg/SqC34PoJE8GmwAUgD0BzwWI1djsj+7vMgfBAFtcQSDMDK1iRNiJUZ7ojzAmhbx9iQM839aXYLz11UHBDtO44JA9iyrBOC9xd+czldoiAUBg0vHQQNtMfCdi2lLuaUB8XKxqRiJw/uR+yRGqYMmIsldATwtf0YQM7S6d2xGcGhK9seQMCEHj+G0g2anMv/UKpZNWk8sBGLdscMxQ/ynxA2nQgvhueLd058QMI9WArqZLE5B0VNojc5jKv2QJqvNNaO/fn/6Tt3GB5TeBtAd0/v76vWXgD9fESUCYQhsmzqWsqL+XEfZJG5SnWiwuk12NrcEqpRWDsSt59rClkMtPlQB2xttS0Zpbr65I6kHYfmnrZ0GI9IfmKXsq7rpqztbef8Jrnhcq6pPVND3SAYzHuK9/1n7msHZUnHEJMcTntZYIFLWP0UjF1gZrKQVsVrn/7MaO3EIYTQUd8c+/pUGqznzTABQjTF/NJKKTbhqUmYAJ2nQzNmjM7n+eur2zAwCWOcvfGu0YF88raCsZaPPYtLa/Bl9TXjXnNgsmW9s+fURojPmZVXCXGvRDiVBbt4R52kZP39iPviCA2C8xRHB7Ss+frLsn9DvPk+jWTJ/SVP1qQJH6YwBHWMQbeqsEqtwfVXOX7x5p8tmhHg7CiAD7n1B5v9ZO8HEbgvXIBPGLQFsvPXkek5sn1o0H1060nHJcf1p+worD8+acEfPcjnUb499cSNPCTDD8m3B0UzfbjzyV8O9s+B6NuOB7vckifX5joKhBHTZ0H3Rwjuud3znM0ckOy4DZx+LkyL4B8ptn/EefIvuJ7c/TEH5Pvfcdg+P1nyP67fYb4noeL/R5wAAAAASUVORK5CYII=\" />](https://www.ariel-cyber.co.il/)\n\n**O**pen-**S**ource **F**ramework for **E**ncrypted **I**nternet and **M**alicious **T**raffic **C**lassification.\n\n## Framework publish to PyPi\nThe framework is been uploaded to the pypi for easy use of the framework.\n\n## Ariel University\nThis framework was developed by Ariel Universtiy, you may read more about the framework in our paper:\n[Open-Source Framework for Encrypted Internet and Malicious Traffic Classification](https://arxiv.org/abs/2206.10144)\n\n## Examples\nFor usage examples please see the `./examples/` directory for various applications of extracting features from a set of PCAP files or evaluating state-of-the-art deep-learning models.\n\n## Installation\nUsing pip:\n```\npython3 -m pip install OSF-EIMTC\n```\n\nUninstall with:\n```\npythom3 -m pip uninstall OSF-EIMTC\n```\n\nInstall tensorflow for deep-learning models:\nsee the official website: https://www.tensorflow.org/install.\nGenerally, you can install tensorflow with pip:\n```\npython3 -m pip install tensorflow\n```\nBut check the offical instructions for more customized options such as for GPU utilization.\n\nFor TLS features, TShark is required. While it is possible to install and configure it for Windows usage, the framework is yet to officialy support the system.\n```\nsudo apt install tshark\n```\n### Dependencies\n- Platform\n - OS: Linux (Tested with WSL Ubuntu 18.04). \n - Version: Python 3.6.6+ (3.7+ is recommended)\n\n- Python modules\n - NFStream>=6.5.1\n - pandas>=1.1.5\n - numpy>=1.19.5\n - scapy>=2.4.5 (for parsing higher level protocols such as DNS)\n - unittest (for tests)\n - sklearn>=1.1.0\n\nThere are some plugins that might require some additional dependancies.\n- Optional Python modules\n - pyasn (for ASN info, requires two additional DBs)\n - functools (for caching functions results)\n - click (for CLI)\n\n- For Deep-Learning Models\n - tensorflow>=2.4.0.\n\n- 3rd party tools (that need to be installed)\n - TShark 3.4.8+ (for TLS features). Try install with `sudo apt update` and then `sudo apt install tshark`, if you cannot get the 3.0+ version try to install wireshark instead via `sudo apt install wireshark`.\n\n- 3rd party data files (provided in `./tools` directory)\n - pyasn.db (ASN mapping database) - updated one can be downloaded via https://github.com/hadiasghari/pyasn.\n - ip2asn.tsv (ASN context info database) - for updated data download from https://iptoasn.com/.\n\n\n\n## Run Tests\n```\npython3 -m unittest discover EIMTC\n```\n\n## Project Structure\nUnder `./src`\n```\n\u2514\u2500\u2500 EIMTC/ # Project's root directory.\n \u251c\u2500\u2500 datasets/ # Links and information for PCAP datasets.\n \u251c\u2500\u2500 metrics/ # Evaluation metrics for models.\n \u251c\u2500\u2500 models/ # State-of-the-art DL models.\n \u251c\u2500\u2500 preprocessing/ \n \u251c\u2500\u2500 selection/ \n \u251c\u2500\u2500 plugins/ # NFStream plugins.\n \u251c\u2500\u2500 stats/ \n \u251c\u2500\u2500 temp/ # Temporary files generated by the system.\n \u251c\u2500\u2500 tests/ # System tests.\n \u2502 \u2514\u2500\u2500 pcaps/ # PCAP files for tests.\n \u251c\u2500\u2500 thrid_party/ # 3rd-party tools used by the system.\n \u2514\u2500\u2500 tools/ # 3rd-party tools used by the system.\n```\n\n## Citation\n```\narticle{BADER2024271,\ntitle = {OSF-EIMTC: An open-source framework for standardized encrypted internet traffic classification},\nauthor = {Ofek Bader and Adi Lichy and Amit Dvir and Ran Dubin and Chen Hajaj},\njournal = {Computer Communications},\nvolume = {213},\npages = {271-284},\nyear = {2024}\n}\n```",
"bugtrack_url": null,
"license": "MIT",
"summary": "A Framework for Encrypted Internet and Malicious Traffic Classification.",
"version": "0.1.60",
"project_urls": {
"Homepage": "https://github.com/ArielCyber/OSF-EIMTC/"
},
"split_keywords": [
"nfstream",
" pcap",
" network",
" deep-learning",
" extraction"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c1519397dc62619ec17f6dc5d2496b0f96e3776ed2430e2df81fcd0ad5955625",
"md5": "eb99ee2bcaefa492c0819b9585986fab",
"sha256": "fc48a4ca2321fe21bfe1c361248ae15f66e5bc0ef49a23617300a166e58e463d"
},
"downloads": -1,
"filename": "OSF_EIMTC-0.1.60.tar.gz",
"has_sig": false,
"md5_digest": "eb99ee2bcaefa492c0819b9585986fab",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 9676088,
"upload_time": "2024-07-09T17:43:53",
"upload_time_iso_8601": "2024-07-09T17:43:53.836504Z",
"url": "https://files.pythonhosted.org/packages/c1/51/9397dc62619ec17f6dc5d2496b0f96e3776ed2430e2df81fcd0ad5955625/OSF_EIMTC-0.1.60.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-09 17:43:53",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ArielCyber",
"github_project": "OSF-EIMTC",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "osf-eimtc"
}
JSON
