# Analyzer SQream Logs
The log analyzer has the ability to help us find the correct logs faster and in a more intuitive way.
* **Version:** 0.0.3
Requirements
=====================
* Python 3.9+
How to use the tool
=====================
run either:
./sq_log_analyzer.py --help
python3 sq_log_analyzer.py --help
This will show the full syntax with all options and acronyms, as well as, help details if you are not sure of the use.
Please note, using --help (or -h) will override any other arguments, which can help when in doubt
Flags
======
1.Load logs
------------
The tool needs to know which files to use, we can specify a directory or a specific file:
--file-path (-p), requires a specific file path
specify the full path (e.g. /media/weka_delivery/ti_folder/sqream_20230830_results.csv)
specify just file name, the tool will search in the current directory
Usage: --file-path=”sqream_20230830_results.csv” or --file-path=”/media/weka_delivery/ti_folder/sqream_20230830_results.csv”
--dir-path (-P), require a specific directory path
specify the full path (e.g. /media/weka_delivery/ti_folder/10082023)
specify just file name, the tool will search in the current directory
Usage: --file-path=”10082023” or --file-path=”/media/weka_delivery/ti_folder/10082023”
the tool will search in all the subdirectories for sqream_logs files
Use the file filters (--file-date-filter/-fdf or --file-month-filter/-fmf) to help the tool search only the relevant files
2.Filters
----------
Use as many filters as relevant to your search:
--from-date/--to-date: date and time filters
--message-filter: any message in the logs, including, queries, known phrases (e.g. query before parsing)
--info-level: INFO, ERROR or DEBUG (most logs are info)
--ip-address: the server ip address
--worker-port: the worker port
--statement-id: the statement id if you are looking for a specific statement
--connection-id: the connection id
--service-name: service name
--message-type: the number of the message, like 200 for execution plan
3.Additional Parameters
-------------------------
The tool support querying the results using mysql syntax:
use --query=”specific query” or --query-file=”file_path.sql” for query in a file
if you use column names with space in the name (like “statement id”), you must use the query file as the name must come in double quotes.
The tool support saving results to file:
use --output-dir=”directory_name” to save the file as out.csv in the directory
use --output-path=”file_path.csv” to save to a specific file
please note, using the same directory dir will override old files
Important
===========
* The tool currently supports only the SQreamDB logs (the one inside the cluster)
* The tool will only read files with the pattern: sqream_yyyymmdd_*.log
* When specifying a directory or a file, it will search in the current directory unless a full path is specified.
* The name of the table if you add a query is df, always
* When running a query a warning may appear (“UserWarning: the 'timedelta' type is not supported”), ignore it
* Queries may take long to run, so make sure it runs on the smallest dataset possible
Raw data
{
"_id": null,
"home_page": "https://github.com/SQream/pysqream_log_analyzer",
"name": "pysqream-log-analyzer",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "",
"keywords": "",
"author": "SQream",
"author_email": "info@sqream.com",
"download_url": "https://files.pythonhosted.org/packages/e4/96/ec5372484cbda37ae47c5b71e6392e5176bbcb7ad25fa53fcddda9d25078/pysqream_log_analyzer-0.0.3.tar.gz",
"platform": null,
"description": "# Analyzer SQream Logs\n\nThe log analyzer has the ability to help us find the correct logs faster and in a more intuitive way.\n\n* **Version:** 0.0.3\n\nRequirements\n=====================\n\n* Python 3.9+\n\nHow to use the tool\n=====================\nrun either:\n\n ./sq_log_analyzer.py --help \n python3 sq_log_analyzer.py --help\n\nThis will show the full syntax with all options and acronyms, as well as, help details if you are not sure of the use.\n\nPlease note, using --help (or -h) will override any other arguments, which can help when in doubt\n\n\nFlags\n======\n\n1.Load logs\n------------\n\nThe tool needs to know which files to use, we can specify a directory or a specific file:\n\n --file-path (-p), requires a specific file path\n specify the full path (e.g. /media/weka_delivery/ti_folder/sqream_20230830_results.csv)\n specify just file name, the tool will search in the current directory\n Usage: --file-path=\u201dsqream_20230830_results.csv\u201d or --file-path=\u201d/media/weka_delivery/ti_folder/sqream_20230830_results.csv\u201d\n\n --dir-path (-P), require a specific directory path\n specify the full path (e.g. /media/weka_delivery/ti_folder/10082023)\n specify just file name, the tool will search in the current directory\n Usage: --file-path=\u201d10082023\u201d or --file-path=\u201d/media/weka_delivery/ti_folder/10082023\u201d\n\nthe tool will search in all the subdirectories for sqream_logs files\n\nUse the file filters (--file-date-filter/-fdf or --file-month-filter/-fmf) to help the tool search only the relevant files\n\n2.Filters\n----------\nUse as many filters as relevant to your search:\n\n --from-date/--to-date: date and time filters\n --message-filter: any message in the logs, including, queries, known phrases (e.g. query before parsing)\n --info-level: INFO, ERROR or DEBUG (most logs are info)\n --ip-address: the server ip address\n --worker-port: the worker port\n --statement-id: the statement id if you are looking for a specific statement\n --connection-id: the connection id\n --service-name: service name\n --message-type: the number of the message, like 200 for execution plan\n\n3.Additional Parameters\n-------------------------\nThe tool support querying the results using mysql syntax:\n\n use --query=\u201dspecific query\u201d or --query-file=\u201dfile_path.sql\u201d for query in a file\n\nif you use column names with space in the name (like \u201cstatement id\u201d), you must use the query file as the name must come in double quotes.\n\nThe tool support saving results to file:\n\n use --output-dir=\u201ddirectory_name\u201d to save the file as out.csv in the directory\n use --output-path=\u201dfile_path.csv\u201d to save to a specific file\n\nplease note, using the same directory dir will override old files\n\nImportant\n===========\n* The tool currently supports only the SQreamDB logs (the one inside the cluster)\n\n* The tool will only read files with the pattern: sqream_yyyymmdd_*.log\n\n* When specifying a directory or a file, it will search in the current directory unless a full path is specified.\n\n* The name of the table if you add a query is df, always\n\n* When running a query a warning may appear (\u201cUserWarning: the 'timedelta' type is not supported\u201d), ignore it\n\n* Queries may take long to run, so make sure it runs on the smallest dataset possible\n\n\n",
"bugtrack_url": null,
"license": "",
"summary": "The log analyzer has the ability to help us find the correct logs faster and in a more intuitive way.",
"version": "0.0.3",
"project_urls": {
"Homepage": "https://github.com/SQream/pysqream_log_analyzer"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f100f27e9b175d8f7a8201c819d4b5c7dc7edb28130a179763f065c3ded5f72f",
"md5": "c4ceb9a5db9e5b63e55099db4842c2b5",
"sha256": "642c919c008c34a11ac1663f3d0fc339f37a1733570c2b84050fe280136709b9"
},
"downloads": -1,
"filename": "pysqream_log_analyzer-0.0.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c4ceb9a5db9e5b63e55099db4842c2b5",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 3707,
"upload_time": "2023-11-26T08:19:42",
"upload_time_iso_8601": "2023-11-26T08:19:42.378061Z",
"url": "https://files.pythonhosted.org/packages/f1/00/f27e9b175d8f7a8201c819d4b5c7dc7edb28130a179763f065c3ded5f72f/pysqream_log_analyzer-0.0.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e496ec5372484cbda37ae47c5b71e6392e5176bbcb7ad25fa53fcddda9d25078",
"md5": "d07bd42fc308fdd7d175da4c2651d941",
"sha256": "540ebf10d1de4411320077a6805ee33edfdc69a80b3e001c5a3ae85e491be4e2"
},
"downloads": -1,
"filename": "pysqream_log_analyzer-0.0.3.tar.gz",
"has_sig": false,
"md5_digest": "d07bd42fc308fdd7d175da4c2651d941",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 3624,
"upload_time": "2023-11-26T08:19:44",
"upload_time_iso_8601": "2023-11-26T08:19:44.141620Z",
"url": "https://files.pythonhosted.org/packages/e4/96/ec5372484cbda37ae47c5b71e6392e5176bbcb7ad25fa53fcddda9d25078/pysqream_log_analyzer-0.0.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-26 08:19:44",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "SQream",
"github_project": "pysqream_log_analyzer",
"github_not_found": true,
"lcname": "pysqream-log-analyzer"
}
JSON
