# provides a convenient and efficient solution for capturing and analyzing system activity logs using Procmon and converting them to the pandas compatible Parquet file format (2% of the original pml file size)
## pip install procmondf
### Tested against Windows 10 / Python 3.10 / Anaconda
The ProcMonDf class and its associated functions are used for logging and analyzing system processes and
events using Microsoft's Procmon https://learn.microsoft.com/en-us/sysinternals/downloads/procmon utility.
Procmon is a Windows-based tool that monitors and captures real-time system activity,
including file system and registry operations, process and thread activity, network activity, and more.
The ProcMonDf class provides a convenient way to start and stop logging with Procmon,
save the log as a Parquet file, and optionally return the log as a pandas DataFrame.
It leverages the functionality of the Procmon utility and combines
it with the data processing capabilities of pandas and
Parquet file format for efficient storage (between 1% to 5% of the original pml file size) and analysis.
600.000 rows require about 5 mb of disk space.
This functionality might be interesting for developers, system administrators, and analysts who need to track
and analyze system behavior, diagnose issues, understand resource usage patterns,
or monitor software performance.
By using the ProcMonDf class, they can automate the process of capturing system activity logs,
convert them to a structured DataFrame, and perform further analysis, filtering,
or visualization using the powerful tools available in the pandas ecosystem.
The advantages of using ProcMonDf and the associated functions include:
### Simplified interface:
The class encapsulates the complex logic required to start and stop Procmon logging,
convert logs to Parquet format, and manage temporary files.
### Data processing capabilities:
The captured logs are converted into a structured DataFrame, allowing users to leverage the powerful data manipulation and analysis functionalities provided by pandas.
### Storage efficiency:
The logs are saved in Parquet format, which offers a columnar storage layout and compression, resulting in reduced storage space requirements and improved query performance.
### Flexibility and extensibility:
Users can customize the logging parameters and additional arguments
passed to Procmon to suit their specific requirements.
### Integration with existing workflows:
The ability to return the log as a DataFrame allows users to seamlessly integrate it
into their existing data analysis pipelines or leverage other libraries and tools compatible with pandas.
```python
class ProcMonDf:
Initializes a ProcMonDf object.
Args:
output (str): The path to the output Parquet file. Doesn't have to exist yet.
Returns:
None
Example:
from procmondf import ProcMonDf
pm = ProcMonDf("c:\\proggi.pqt")
pm.start_logging()
# when you are done, use the following command to stop logging
df=pm.stop_logging(deltmp=True, returndf=True)
print(df[:5].to_string())
aa_time_of_day aa_process_name aa_pid aa_operation aa_path aa_result aa_detail
0 2023-06-25 09:55:29.951767 Explorer.EXE 3904 RegOpenKey HKCU\Software\Classes\CLSID\{xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx}\InprocServer32 NAME NOT FOUND Desired Access: Read
1 2023-06-25 09:55:29.951783 Explorer.EXE 3904 RegQueryKey HKCR\CLSID\{xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx} SUCCESS Query: HandleTags, HandleTags: 0x0
2 2023-06-25 09:55:29.951792 Explorer.EXE 3904 RegOpenKey HKCR\CLSID\{xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx}\InprocServer32 SUCCESS Desired Access: Read
3 2023-06-25 09:55:29.951836 Explorer.EXE 3904 RegQueryKey HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\SessionInfo\1\ApplicationViewManagement SUCCESS Query: HandleTags, HandleTags: 0x0
4 2023-06-25 09:55:29.951837 Procmon64.exe 11160 RegQueryValue HKLM\System\CurrentControlSet\Control\WMI\Securityxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx NAME NOT FOUND Length: 528
```
Raw data
{
"_id": null,
"home_page": "https://github.com/hansalemaos/procmondf",
"name": "procmondf",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "microsoft,procmon,pandas,DataFrame",
"author": "Johannes Fischer",
"author_email": "aulasparticularesdealemaosp@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/4b/b9/2d49d57b4622746dc7176d826fb1e2c7a380f9b0b89472cc4d21ced64ba8/procmondf-0.10.tar.gz",
"platform": null,
"description": "\r\n# provides a convenient and efficient solution for capturing and analyzing system activity logs using Procmon and converting them to the pandas compatible Parquet file format (2% of the original pml file size)\r\n\r\n\r\n## pip install procmondf\r\n\r\n### Tested against Windows 10 / Python 3.10 / Anaconda \r\n\r\n\r\nThe ProcMonDf class and its associated functions are used for logging and analyzing system processes and \r\nevents using Microsoft's Procmon https://learn.microsoft.com/en-us/sysinternals/downloads/procmon utility. \r\nProcmon is a Windows-based tool that monitors and captures real-time system activity, \r\nincluding file system and registry operations, process and thread activity, network activity, and more.\r\n\r\nThe ProcMonDf class provides a convenient way to start and stop logging with Procmon, \r\nsave the log as a Parquet file, and optionally return the log as a pandas DataFrame. \r\nIt leverages the functionality of the Procmon utility and combines \r\nit with the data processing capabilities of pandas and \r\nParquet file format for efficient storage (between 1% to 5% of the original pml file size) and analysis.\r\n600.000 rows require about 5 mb of disk space.\r\n\r\nThis functionality might be interesting for developers, system administrators, and analysts who need to track \r\nand analyze system behavior, diagnose issues, understand resource usage patterns, \r\nor monitor software performance. \r\nBy using the ProcMonDf class, they can automate the process of capturing system activity logs, \r\nconvert them to a structured DataFrame, and perform further analysis, filtering, \r\nor visualization using the powerful tools available in the pandas ecosystem.\r\n\r\nThe advantages of using ProcMonDf and the associated functions include:\r\n\r\n### Simplified interface: \r\n\r\nThe class encapsulates the complex logic required to start and stop Procmon logging, \r\nconvert logs to Parquet format, and manage temporary files.\r\n\r\n### Data processing capabilities: \r\n\r\nThe captured logs are converted into a structured DataFrame, allowing users to leverage the powerful data manipulation and analysis functionalities provided by pandas.\r\n\r\n### Storage efficiency: \r\n\r\nThe logs are saved in Parquet format, which offers a columnar storage layout and compression, resulting in reduced storage space requirements and improved query performance.\r\n\r\n### Flexibility and extensibility: \r\n\r\nUsers can customize the logging parameters and additional arguments \r\npassed to Procmon to suit their specific requirements.\r\n\r\n### Integration with existing workflows: \r\n\r\nThe ability to return the log as a DataFrame allows users to seamlessly integrate it \r\ninto their existing data analysis pipelines or leverage other libraries and tools compatible with pandas.\r\n\r\n```python\r\nclass ProcMonDf:\r\n Initializes a ProcMonDf object.\r\n\r\n Args:\r\n output (str): The path to the output Parquet file. Doesn't have to exist yet.\r\n\r\n Returns:\r\n None\r\n\r\n Example:\r\n from procmondf import ProcMonDf\r\n pm = ProcMonDf(\"c:\\\\proggi.pqt\")\r\n pm.start_logging()\r\n # when you are done, use the following command to stop logging\r\n df=pm.stop_logging(deltmp=True, returndf=True)\r\n\r\n print(df[:5].to_string())\r\n aa_time_of_day aa_process_name aa_pid aa_operation aa_path aa_result aa_detail\r\n 0 2023-06-25 09:55:29.951767 Explorer.EXE 3904 RegOpenKey HKCU\\Software\\Classes\\CLSID\\{xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx}\\InprocServer32 NAME NOT FOUND Desired Access: Read\r\n 1 2023-06-25 09:55:29.951783 Explorer.EXE 3904 RegQueryKey HKCR\\CLSID\\{xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx} SUCCESS Query: HandleTags, HandleTags: 0x0\r\n 2 2023-06-25 09:55:29.951792 Explorer.EXE 3904 RegOpenKey HKCR\\CLSID\\{xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx}\\InprocServer32 SUCCESS Desired Access: Read\r\n 3 2023-06-25 09:55:29.951836 Explorer.EXE 3904 RegQueryKey HKCU\\Software\\Microsoft\\Windows\\CurrentVersion\\Explorer\\SessionInfo\\1\\ApplicationViewManagement SUCCESS Query: HandleTags, HandleTags: 0x0\r\n 4 2023-06-25 09:55:29.951837 Procmon64.exe 11160 RegQueryValue HKLM\\System\\CurrentControlSet\\Control\\WMI\\Securityxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx NAME NOT FOUND Length: 528\r\n \r\n```\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "provides a convenient and efficient solution for capturing and analyzing system activity logs using Procmon and converting them to the pandas compatible Parquet file format (2% of the original pml file size)",
"version": "0.10",
"project_urls": {
"Homepage": "https://github.com/hansalemaos/procmondf"
},
"split_keywords": [
"microsoft",
"procmon",
"pandas",
"dataframe"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a2be18a7e625fea6660d498e76bb13929220da25c43bdfd12c82352c247b4bd1",
"md5": "473375a9643b095d48ba8f2310bcf13c",
"sha256": "43468611a9bc74e5be73fa1e48e30ed845ceda01f5172f074bde9f6620a40230"
},
"downloads": -1,
"filename": "procmondf-0.10-py3-none-any.whl",
"has_sig": false,
"md5_digest": "473375a9643b095d48ba8f2310bcf13c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 3524134,
"upload_time": "2023-06-25T13:10:10",
"upload_time_iso_8601": "2023-06-25T13:10:10.070947Z",
"url": "https://files.pythonhosted.org/packages/a2/be/18a7e625fea6660d498e76bb13929220da25c43bdfd12c82352c247b4bd1/procmondf-0.10-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4bb92d49d57b4622746dc7176d826fb1e2c7a380f9b0b89472cc4d21ced64ba8",
"md5": "194d7de3695fd7df5024ba9556c4c409",
"sha256": "7cfd79af72bc1172325ad7840705721c4cb9468f0aa67be74e1471ec1933576d"
},
"downloads": -1,
"filename": "procmondf-0.10.tar.gz",
"has_sig": false,
"md5_digest": "194d7de3695fd7df5024ba9556c4c409",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 3496558,
"upload_time": "2023-06-25T13:10:18",
"upload_time_iso_8601": "2023-06-25T13:10:18.079399Z",
"url": "https://files.pythonhosted.org/packages/4b/b9/2d49d57b4622746dc7176d826fb1e2c7a380f9b0b89472cc4d21ced64ba8/procmondf-0.10.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-25 13:10:18",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "hansalemaos",
"github_project": "procmondf",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "procmondf"
}
JSON
