# CoT-KG Network Intrusion Detection using Knowledge Graph and GraphSAGE
This project implements a network intrusion detection system using Chain of Thought (CoT), knowledge graphs and GraphSAGE model on the CICIDS2017 dataset. The Chain of Thought approach is used to enhance the knowledge graph construction and improve the interpretability of the detection process.
## Key Features
- Chain of Thought (CoT) enhanced knowledge graph construction
- GraphSAGE-based network intrusion detection
- Interpretable AI techniques for explaining detection results
- Integration of domain knowledge with machine learning
## Data Download and Preparation
The CICIDS2017 dataset is used in this project. Follow these steps to download and prepare the data:
1. Clone this repository:
```bash
git clone https://github.com/chenxingqiang/cotkg-network-intrusion-detection.git
cd cotkg-network-intrusion-detection
```
2. Install the required dependencies:
```bash
pip install -r requirements.txt
```
3. Run the data download script:
```bash
python src/download_data.py
```
This script will:
- Download the MachineLearningCSV.zip file from the CICIDS2017 dataset
- Check the integrity of the downloaded file
- Extract the contents to the `data/raw/` directory
Note: The download might take some time as the file is about 224MB.
4. After running the script, the data will be available in the `data/raw/MachineLearningCVE/` directory.
## Installation
To install the required dependencies, run:
```bash
pip install -r requirements.txt
```
## Usage
After preparing the data, you can run the main script to train and evaluate the model:
```bash
python src/main.py
```
This script will:
- Load and preprocess the data
- Perform feature engineering
- Construct the knowledge graph
- Train the GraphSAGE model
- Evaluate the model
- Generate explanations for the predictions
## Note
The raw data files are large and are not included in the git repository. They will be downloaded when you run the `download_data.py` script. If you need to share the project, others can use the same script to download the data.
## Author
Chen Xingqiang
Hanghzou Turing AI Co.,Ltd.
Email: <chen.xingqiang@iechor.com>
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
## Acknowledgments
- CICIDS2017 dataset: <https://www.unb.ca/cic/datasets/ids-2017.html>
Raw data
{
"_id": null,
"home_page": "https://github.com/chenxingqiang/cotkg-network-intrusion-detection",
"name": "cotkg-network-intrusion-detection",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": null,
"author": "Chen Xingqiang",
"author_email": "chen.xingqiang@iechor.com",
"download_url": "https://files.pythonhosted.org/packages/27/11/e79d3bef3fe83e1e7089685f18e57656f62df016b6dae7d55155787d93d8/cotkg_network_intrusion_detection-0.1.1.tar.gz",
"platform": null,
"description": "\n# CoT-KG Network Intrusion Detection using Knowledge Graph and GraphSAGE\n\nThis project implements a network intrusion detection system using Chain of Thought (CoT), knowledge graphs and GraphSAGE model on the CICIDS2017 dataset. The Chain of Thought approach is used to enhance the knowledge graph construction and improve the interpretability of the detection process.\n\n## Key Features\n\n- Chain of Thought (CoT) enhanced knowledge graph construction\n- GraphSAGE-based network intrusion detection\n- Interpretable AI techniques for explaining detection results\n- Integration of domain knowledge with machine learning\n\n## Data Download and Preparation\n\nThe CICIDS2017 dataset is used in this project. Follow these steps to download and prepare the data:\n\n 1. Clone this repository:\n\n```bash\ngit clone https://github.com/chenxingqiang/cotkg-network-intrusion-detection.git\ncd cotkg-network-intrusion-detection\n```\n\n 2. Install the required dependencies:\n\n```bash\npip install -r requirements.txt\n```\n\n 3. Run the data download script:\n\n```bash\npython src/download_data.py\n```\n\nThis script will:\n\n- Download the MachineLearningCSV.zip file from the CICIDS2017 dataset\n\n- Check the integrity of the downloaded file\n\n- Extract the contents to the `data/raw/` directory\n\nNote: The download might take some time as the file is about 224MB.\n\n4. After running the script, the data will be available in the `data/raw/MachineLearningCVE/` directory.\n\n## Installation\n\nTo install the required dependencies, run:\n\n```bash\npip install -r requirements.txt\n```\n\n## Usage\n\nAfter preparing the data, you can run the main script to train and evaluate the model:\n\n```bash\npython src/main.py\n\n```\n\nThis script will:\n\n- Load and preprocess the data\n- Perform feature engineering\n- Construct the knowledge graph\n- Train the GraphSAGE model\n- Evaluate the model\n- Generate explanations for the predictions\n\n## Note\n\nThe raw data files are large and are not included in the git repository. They will be downloaded when you run the `download_data.py` script. If you need to share the project, others can use the same script to download the data.\n\n## Author\n\nChen Xingqiang\nHanghzou Turing AI Co.,Ltd.\nEmail: <chen.xingqiang@iechor.com>\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## Acknowledgments\n\n- CICIDS2017 dataset: <https://www.unb.ca/cic/datasets/ids-2017.html>\n",
"bugtrack_url": null,
"license": null,
"summary": "A network intrusion detection system using Chain of Thought, knowledge graphs and GraphSAGE",
"version": "0.1.1",
"project_urls": {
"Homepage": "https://github.com/chenxingqiang/cotkg-network-intrusion-detection"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "beca92fc7a2a90910bdb42e8c490812ac88f755734dc85814bd3032584c8c35a",
"md5": "61f99b659c3de7621507dae6981ec7f0",
"sha256": "8dc6cc8f678d343ecdea0a7210a7fcfd280702ed1a2439a31adfe78de11da20f"
},
"downloads": -1,
"filename": "cotkg_network_intrusion_detection-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "61f99b659c3de7621507dae6981ec7f0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 26100,
"upload_time": "2024-11-11T23:36:40",
"upload_time_iso_8601": "2024-11-11T23:36:40.919267Z",
"url": "https://files.pythonhosted.org/packages/be/ca/92fc7a2a90910bdb42e8c490812ac88f755734dc85814bd3032584c8c35a/cotkg_network_intrusion_detection-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "2711e79d3bef3fe83e1e7089685f18e57656f62df016b6dae7d55155787d93d8",
"md5": "5731589f558026b124a406c989395259",
"sha256": "b8822ee247466d1f5d54adf6ac8b9bceda71ff7788c39be80fba59823f059b55"
},
"downloads": -1,
"filename": "cotkg_network_intrusion_detection-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "5731589f558026b124a406c989395259",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 56909,
"upload_time": "2024-11-11T23:36:42",
"upload_time_iso_8601": "2024-11-11T23:36:42.941562Z",
"url": "https://files.pythonhosted.org/packages/27/11/e79d3bef3fe83e1e7089685f18e57656f62df016b6dae7d55155787d93d8/cotkg_network_intrusion_detection-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-11 23:36:42",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "chenxingqiang",
"github_project": "cotkg-network-intrusion-detection",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "numpy",
"specs": [
[
"==",
"1.21.0"
]
]
},
{
"name": "pandas",
"specs": [
[
"==",
"1.3.0"
]
]
},
{
"name": "scikit-learn",
"specs": [
[
"==",
"0.24.2"
]
]
},
{
"name": "torch",
"specs": [
[
"==",
"1.9.0"
]
]
},
{
"name": "torch-geometric",
"specs": [
[
"==",
"2.0.1"
]
]
},
{
"name": "networkx",
"specs": [
[
"==",
"2.6.2"
]
]
},
{
"name": "py2neo",
"specs": [
[
"==",
"2021.1.1"
]
]
},
{
"name": "matplotlib",
"specs": [
[
"==",
"3.4.2"
]
]
},
{
"name": "seaborn",
"specs": [
[
"==",
"0.11.1"
]
]
},
{
"name": "tsfresh",
"specs": [
[
"==",
"0.18.0"
]
]
},
{
"name": "shap",
"specs": [
[
"==",
"0.39.0"
]
]
},
{
"name": "transformers",
"specs": [
[
"==",
"4.9.2"
]
]
},
{
"name": "openai",
"specs": [
[
"==",
"0.27.0"
]
]
},
{
"name": "tqdm",
"specs": [
[
"==",
"4.61.2"
]
]
},
{
"name": "jupyter",
"specs": [
[
"==",
"1.0.0"
]
]
},
{
"name": "pytest",
"specs": [
[
"==",
"6.2.4"
]
]
},
{
"name": "requests",
"specs": [
[
"==",
"2.26.0"
]
]
}
],
"lcname": "cotkg-network-intrusion-detection"
}
JSON
