# captures and processes network packets asynchronously using the Scapy library, and converts the results to a pandas DataFrame
## pip install pdsniff
#### Tested against Windows 10 ( Necessary: https://npcap.com/#download ) / Python 3.10 / Anaconda
### Function start_sniffing
The start_sniffing function is designed to capture and process network packets asynchronously using the Scapy library.
It takes several parameters, including count, result_list, killkey, and dumpfolder, to control the packet capturing and processing behavior.
#### Here's what the function does:
It initializes the result_list as a deque with a maximum length of 1 if not provided by the caller.
If a dumpfolder is provided, it creates the folder if it doesn't exist and determines the starting
index for saving pickled files based on existing files in the folder.
It defines an internal function \_get_frame that handles the actual packet capturing and processing.
The packet capturing is done using a separate thread (KThread) to avoid blocking the main program execution.
Packets are captured and processed in batches of size count.
The processed data is converted into a pandas DataFrame with proper data types based on the provided dtypes.
If a dumpfolder is provided, the processed data frames are periodically saved as pickled files.
```python
Args:
count (int): The number of packets to capture and process in each batch.
result_list (None, list, deque, optional): List or deque to store the processed data frames.
If None, a deque with a maximum length of 1 will be used. Defaults to None.
killkey (str, optional): The hotkey combination to stop packet capturing and processing.
Defaults to "ctrl+alt+x".
dumpfolder (str, optional): The folder path to save processed data frames as pickled files.
If provided, captured packets will be periodically saved as pickle files in this folder.
Defaults to None.
**kwargs: Additional keyword arguments to be passed to the Scapy sniff function.
Returns:
KThread: A thread object that captures and processes the network packets.
Note:
This function starts a new thread (KThread) for packet capturing and processing. The function
will keep running and capturing packets until the specified `killkey` hotkey is pressed.
Captured packets will be processed in batches of `count` and stored in the `result_list`.
If `dumpfolder` is provided, the processed data frames will also be saved as pickle files.
The `result_list` parameter can be used to access the captured and processed data frames
from the calling code.
```
#### Advantages:
- Asynchronous packet capturing allows the main program to continue executing without waiting for packets to arrive, leading to better program responsiveness.
- Batch processing reduces processing overhead and allows handling large volumes of packets efficiently.
- Saving processed data frames as pickled files facilitates data persistence and easy data retrieval for analysis or further processing.
### Function load_dump_files
The load_dump_files function is responsible for loading and concatenating processed data frames from pickled files in the specified folder.
#### Here's what the function does
It reads all pickled files (.pkl) in the provided folder that have numeric filenames.
It concatenates the data frames from these files into a single pandas DataFrame.
The resulting DataFrame contains the concatenated data from the pickled files.
```python
Load and concatenate processed data frames from pickled files in the specified folder.
Args:
folder (str): The folder path where the pickled data frames are stored.
Returns:
pd.DataFrame: A DataFrame containing the concatenated data from the pickled files.
Note:
This function reads all pickled files (*.pkl) in the specified folder that have numeric filenames.
It concatenates the data frames from these files and returns a single DataFrame.
The `folder` parameter should be the path to the folder containing the pickled files.
```
```python
from pdsniff import start_sniffing, load_dump_files
count = 1000
result_list = [] # results will be appended
killkey = "ctrl+alt+x"
folder_path="c:\\internetdump"
t2 = start_sniffing(
count=count, result_list=result_list, killkey=killkey, dumpfolder=folder_path
)
df = load_dump_files(folder_path)
```
Raw data
{
"_id": null,
"home_page": "https://github.com/hansalemaos/pdsniff",
"name": "pdsniff",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "network,sniffing",
"author": "Johannes Fischer",
"author_email": "aulasparticularesdealemaosp@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/f7/11/7f3a8a68b66be35db33f1dd450711712d5e8ea3d66283c00a3389c0402fe/pdsniff-0.10.tar.gz",
"platform": null,
"description": "\r\n# captures and processes network packets asynchronously using the Scapy library, and converts the results to a pandas DataFrame\r\n\r\n## pip install pdsniff \r\n\r\n#### Tested against Windows 10 ( Necessary: https://npcap.com/#download ) / Python 3.10 / Anaconda \r\n\r\n### Function start_sniffing\r\n\r\nThe start_sniffing function is designed to capture and process network packets asynchronously using the Scapy library. \r\nIt takes several parameters, including count, result_list, killkey, and dumpfolder, to control the packet capturing and processing behavior.\r\n\r\n#### Here's what the function does:\r\n\r\nIt initializes the result_list as a deque with a maximum length of 1 if not provided by the caller.\r\nIf a dumpfolder is provided, it creates the folder if it doesn't exist and determines the starting \r\nindex for saving pickled files based on existing files in the folder.\r\nIt defines an internal function \\_get_frame that handles the actual packet capturing and processing.\r\nThe packet capturing is done using a separate thread (KThread) to avoid blocking the main program execution.\r\nPackets are captured and processed in batches of size count.\r\nThe processed data is converted into a pandas DataFrame with proper data types based on the provided dtypes.\r\nIf a dumpfolder is provided, the processed data frames are periodically saved as pickled files.\r\n\r\n```python\r\n Args:\r\n count (int): The number of packets to capture and process in each batch.\r\n result_list (None, list, deque, optional): List or deque to store the processed data frames.\r\n If None, a deque with a maximum length of 1 will be used. Defaults to None.\r\n killkey (str, optional): The hotkey combination to stop packet capturing and processing.\r\n Defaults to \"ctrl+alt+x\".\r\n dumpfolder (str, optional): The folder path to save processed data frames as pickled files.\r\n If provided, captured packets will be periodically saved as pickle files in this folder.\r\n Defaults to None.\r\n **kwargs: Additional keyword arguments to be passed to the Scapy sniff function.\r\n\r\n Returns:\r\n KThread: A thread object that captures and processes the network packets.\r\n\r\n Note:\r\n This function starts a new thread (KThread) for packet capturing and processing. The function\r\n will keep running and capturing packets until the specified `killkey` hotkey is pressed.\r\n Captured packets will be processed in batches of `count` and stored in the `result_list`.\r\n If `dumpfolder` is provided, the processed data frames will also be saved as pickle files.\r\n\r\n The `result_list` parameter can be used to access the captured and processed data frames\r\n from the calling code.\r\n```\r\n\r\n\r\n\r\n#### Advantages:\r\n\r\n- Asynchronous packet capturing allows the main program to continue executing without waiting for packets to arrive, leading to better program responsiveness.\r\n- Batch processing reduces processing overhead and allows handling large volumes of packets efficiently.\r\n- Saving processed data frames as pickled files facilitates data persistence and easy data retrieval for analysis or further processing.\r\n\r\n### Function load_dump_files\r\n\r\nThe load_dump_files function is responsible for loading and concatenating processed data frames from pickled files in the specified folder.\r\n\r\n#### Here's what the function does\r\n\r\nIt reads all pickled files (.pkl) in the provided folder that have numeric filenames.\r\nIt concatenates the data frames from these files into a single pandas DataFrame.\r\nThe resulting DataFrame contains the concatenated data from the pickled files.\r\n\r\n```python\r\n\r\n Load and concatenate processed data frames from pickled files in the specified folder.\r\n\r\n Args:\r\n folder (str): The folder path where the pickled data frames are stored.\r\n\r\n Returns:\r\n pd.DataFrame: A DataFrame containing the concatenated data from the pickled files.\r\n\r\n Note:\r\n This function reads all pickled files (*.pkl) in the specified folder that have numeric filenames.\r\n It concatenates the data frames from these files and returns a single DataFrame.\r\n\r\n The `folder` parameter should be the path to the folder containing the pickled files.\r\n```\r\n\r\n\r\n\r\n\r\n```python\r\n\r\nfrom pdsniff import start_sniffing, load_dump_files\r\ncount = 1000\r\nresult_list = [] # results will be appended\r\nkillkey = \"ctrl+alt+x\"\r\nfolder_path=\"c:\\\\internetdump\"\r\nt2 = start_sniffing(\r\n\tcount=count, result_list=result_list, killkey=killkey, dumpfolder=folder_path\r\n)\r\n\r\n\r\ndf = load_dump_files(folder_path)\r\n\r\n\t\t\r\n```\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "captures and processes network packets asynchronously using the Scapy library, and converts the results to a pandas DataFrame",
"version": "0.10",
"project_urls": {
"Homepage": "https://github.com/hansalemaos/pdsniff"
},
"split_keywords": [
"network",
"sniffing"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "fcb85ad284fc067dbdc5b448e3eb7fc0d82fde033176f4e6414061b798d8d06e",
"md5": "a8da9fd825235b09e2c289bec598417c",
"sha256": "eae2504c9c43b023d7864cf3db5f735c1a29a9b5781910da91becfe0cc11b811"
},
"downloads": -1,
"filename": "pdsniff-0.10-py3-none-any.whl",
"has_sig": false,
"md5_digest": "a8da9fd825235b09e2c289bec598417c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 17050,
"upload_time": "2023-07-23T09:11:20",
"upload_time_iso_8601": "2023-07-23T09:11:20.628338Z",
"url": "https://files.pythonhosted.org/packages/fc/b8/5ad284fc067dbdc5b448e3eb7fc0d82fde033176f4e6414061b798d8d06e/pdsniff-0.10-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f7117f3a8a68b66be35db33f1dd450711712d5e8ea3d66283c00a3389c0402fe",
"md5": "d01512d7971f403588dd9246211d355f",
"sha256": "f106963f3a1662196079b16a5055ea0eefdbb9999536567c884f82d29e151e4b"
},
"downloads": -1,
"filename": "pdsniff-0.10.tar.gz",
"has_sig": false,
"md5_digest": "d01512d7971f403588dd9246211d355f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 16318,
"upload_time": "2023-07-23T09:11:22",
"upload_time_iso_8601": "2023-07-23T09:11:22.459838Z",
"url": "https://files.pythonhosted.org/packages/f7/11/7f3a8a68b66be35db33f1dd450711712d5e8ea3d66283c00a3389c0402fe/pdsniff-0.10.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-23 09:11:22",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "hansalemaos",
"github_project": "pdsniff",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "pdsniff"
}
JSON
