# Web Statistics and Analytics
This package produces web statistics and analytical output based on nginx access log files.
[](https://pypi.org/project/a9x-webstatistics/) [](https://github.com/ava007/a9x-webstatistics/releases) [](https://github.com/ava007/a9x-webstatistics/actions) [](https://github.com/ava007/a9x-webstatistics/blob/main/LICENSE)
[](https://pypistats.org/packages/a9x-webstatistics)
[](https://pypi.org/project/a9x-webstatistics)
## Live Example
Visit the live example on [logikfabrik.com](https://www.logikfabrik.com/webstatsLF24.html).
<img src="https://github.com/ava007/a9x-webstatistics/blob/main/a9x-webstatistics_20241123.png">
## Installation
```bash
pip install a9x-webstatistics
```
## Requirements on Linux / Unix
- access log file from web server
- standard access log file from nginx, freenginx or apache
- access log file in json format
- python3.9 onwards
- optional: maxmind db which need to be download by end user after registration
## Deployment on Linux / Unix
```bash
#!/bin/sh
export LOG=/usr/local/www/webstats.log
export PATH=/usr/local/www/django5/envpy311/bin:$PATH
python3.11 -m a9x_webstatistics.main \
--infile /var/log/nginx-access.log \
--geoip /usr/local/share/GeoIP/GeoLite2-Country.mmdb \
--domain https://www.logikfabrik.com \
--statfile /usr/local/www/lf_static/webstatsLF24.json &> $LOG
python3.11 -m a9x_webstatistics.gencockpit \
--infile /usr/local/www/lf_static/webstatsLF24.json \
--outfile /usr/local/www/lf_static/webstatsLF24.html &>> $LOG
```
Cron-Job Entry for Linux / FreeBSD:
```bash
#minute hour mday month wday who command
13,58 * * * * dj1 /usr/local/www/cronjobs/cron_webstatistics.sh
```
## Installation on Windows
```bash
# install on windows:
## install a virtual env for python
python -m venv c:\temp\webstatsenv
## activate env
c:\temp\webstatsenv\Scripts\activate.bat
## install a9x-webstatistics
pip install a9x-webstatistics
python -m pip install --upgrade pip
python -m pip list
```
## Deployment on Windows
```bash
# run
c:
cd \temp
wget -URI https://www.logikfabrik.com/wlog/access.log -OutFile access.log
## active env
c:\temp\webstatsenv\Scripts\activate.bat
## create calculate statistics
## there are no country statistics created as not geoip defined
python -m a9x_webstatistics.main --infile access.log --statfile webstatsLF.json
## create html file for webstatistics
python -m a9x_webstatistics.gencockpit --infile webstatsLF.json --outfile webstatsLF.html
```
## Extended Example
```bash
#!/bin/sh
export LOG=/usr/local/www/webstats.log
export PATH=/usr/local/www/django5/envpy311/bin:$PATH
_YEARNUM="$(date +'%Y')";
_MONTHNUM="$(date +'%m')";
# make a archive copy for every month:
cp /usr/local/www/lf_static/webstatsLF24.json /usr/local/www/lf_static/webstatsLF24$_YEARNUM$_MONTHNUM.json
cp /usr/local/www/lf_static/webstatsLF24.html /usr/local/www/lf_static/webstatsLF24$_YEARNUM$_MONTHNUM.html
# updates all packages:
pip install -U `pip list --outdated | awk 'NR > 2 {print $1}'` >> $LOG
python3.11 -m a9x_webstatistics.main \
--infile /var/log/nginx-access.log \
--geoip /usr/local/share/GeoIP/GeoLite2-Country.mmdb \
--domain https://www.logikfabrik.com \
--statfile /usr/local/www/lf_static/webstatsLF24.json &> $LOG
python3.11 -m a9x_webstatistics.gencockpit \
--infile /usr/local/www/lf_static/webstatsLF24.json \
--outfile /usr/local/www/lf_static/webstatsLF24.html &>> $LOG
```
# Supported Access Log Formats
## Nginx Combined Format
predefined combined format: access_log /.../logs/nginx-access.log combined;
## Nginx Json Format
```bash
log_format a9x_logs escape=json '{'
'"tl":"$time_local",'
'"ip":"$remote_addr",' # must; client IP
'"rq":"$request_uri",' # must; full original request URI (with arguments)
'"qs":"$query_string",' # optional
'"st":"$status",' # must; response status code
'"ru":"$remote_user",' # optional; client HTTP username
'"bs":"$bytes_sent", ' # must; the number of bytes sent to a client
'"rf":"$http_referer",' # optional;
'"rt":"$upstream_response_time", ' # optional; time spend receiving upstream body
'"sc":"$scheme", ' # optional; http or https
'"ua":"$http_user_agent",' # optional
'"cs":"$upstream_cache_status",' # optional
'"al":"$http_accept_language",' # optional
'}';
access_log /.../log/nginx-access-log.json a9x_logs buffer=1k;
```
## Apache Combined Format
CustomLog /.../log/apache2/apache_access.log combined
Raw data
{
"_id": null,
"home_page": "https://github.com/ava007/a9x-webstatistics",
"name": "a9x-webstatistics",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "webstats, statistics, analytics",
"author": "Andr\u00e9 von Arx",
"author_email": "avonarx@outlook.de",
"download_url": "https://files.pythonhosted.org/packages/74/49/3f59b041f4d78c8d78db8a424a0d9ee7cb57a72bc500f28a36907704f5d6/a9x_webstatistics-1.4.17.tar.gz",
"platform": null,
"description": "# Web Statistics and Analytics\n\nThis package produces web statistics and analytical output based on nginx access log files.\n\n[](https://pypi.org/project/a9x-webstatistics/) [](https://github.com/ava007/a9x-webstatistics/releases) [](https://github.com/ava007/a9x-webstatistics/actions) [](https://github.com/ava007/a9x-webstatistics/blob/main/LICENSE)\n\n[](https://pypistats.org/packages/a9x-webstatistics)\n[](https://pypi.org/project/a9x-webstatistics)\n\n## Live Example\n\nVisit the live example on [logikfabrik.com](https://www.logikfabrik.com/webstatsLF24.html).\n\n<img src=\"https://github.com/ava007/a9x-webstatistics/blob/main/a9x-webstatistics_20241123.png\">\n\n## Installation\n\n```bash\npip install a9x-webstatistics\n```\n\n\n## Requirements on Linux / Unix\n- access log file from web server\n - standard access log file from nginx, freenginx or apache\n - access log file in json format\n- python3.9 onwards\n- optional: maxmind db which need to be download by end user after registration\n\n\n## Deployment on Linux / Unix\n\n```bash\n#!/bin/sh\n\nexport LOG=/usr/local/www/webstats.log\nexport PATH=/usr/local/www/django5/envpy311/bin:$PATH\n\npython3.11 -m a9x_webstatistics.main \\\n --infile /var/log/nginx-access.log \\\n --geoip /usr/local/share/GeoIP/GeoLite2-Country.mmdb \\\n --domain https://www.logikfabrik.com \\\n --statfile /usr/local/www/lf_static/webstatsLF24.json &> $LOG\n\npython3.11 -m a9x_webstatistics.gencockpit \\\n --infile /usr/local/www/lf_static/webstatsLF24.json \\\n --outfile /usr/local/www/lf_static/webstatsLF24.html &>> $LOG\n```\n\nCron-Job Entry for Linux / FreeBSD:\n```bash\n#minute hour mday month wday who command\n13,58 * * * * dj1 /usr/local/www/cronjobs/cron_webstatistics.sh\n\n```\n\n\n## Installation on Windows\n\n```bash\n# install on windows:\n\n## install a virtual env for python\npython -m venv c:\\temp\\webstatsenv\n\n## activate env\nc:\\temp\\webstatsenv\\Scripts\\activate.bat\n\n## install a9x-webstatistics\npip install a9x-webstatistics\npython -m pip install --upgrade pip\npython -m pip list\n```\n\n## Deployment on Windows\n\n```bash\n# run\nc:\ncd \\temp\nwget -URI https://www.logikfabrik.com/wlog/access.log -OutFile access.log\n\n## active env\nc:\\temp\\webstatsenv\\Scripts\\activate.bat\n\n## create calculate statistics\n## there are no country statistics created as not geoip defined\npython -m a9x_webstatistics.main --infile access.log --statfile webstatsLF.json\n\n## create html file for webstatistics\npython -m a9x_webstatistics.gencockpit --infile webstatsLF.json --outfile webstatsLF.html\n```\n\n## Extended Example\n\n```bash\n#!/bin/sh\n\nexport LOG=/usr/local/www/webstats.log\nexport PATH=/usr/local/www/django5/envpy311/bin:$PATH\n\n_YEARNUM=\"$(date +'%Y')\";\n_MONTHNUM=\"$(date +'%m')\";\n\n# make a archive copy for every month:\ncp /usr/local/www/lf_static/webstatsLF24.json /usr/local/www/lf_static/webstatsLF24$_YEARNUM$_MONTHNUM.json\ncp /usr/local/www/lf_static/webstatsLF24.html /usr/local/www/lf_static/webstatsLF24$_YEARNUM$_MONTHNUM.html\n\n# updates all packages:\npip install -U `pip list --outdated | awk 'NR > 2 {print $1}'` >> $LOG\n\n\npython3.11 -m a9x_webstatistics.main \\\n --infile /var/log/nginx-access.log \\\n --geoip /usr/local/share/GeoIP/GeoLite2-Country.mmdb \\\n --domain https://www.logikfabrik.com \\\n --statfile /usr/local/www/lf_static/webstatsLF24.json &> $LOG\n\npython3.11 -m a9x_webstatistics.gencockpit \\\n --infile /usr/local/www/lf_static/webstatsLF24.json \\\n --outfile /usr/local/www/lf_static/webstatsLF24.html &>> $LOG\n```\n# Supported Access Log Formats\n\n\n\n## Nginx Combined Format\n\npredefined combined format: access_log /.../logs/nginx-access.log combined;\n\n\n## Nginx Json Format\n\n```bash\nlog_format a9x_logs escape=json '{'\n '\"tl\":\"$time_local\",'\n '\"ip\":\"$remote_addr\",' # must; client IP\n '\"rq\":\"$request_uri\",' # must; full original request URI (with arguments)\n '\"qs\":\"$query_string\",' # optional\n '\"st\":\"$status\",' # must; response status code\n '\"ru\":\"$remote_user\",' # optional; client HTTP username\n '\"bs\":\"$bytes_sent\", ' # must; the number of bytes sent to a client\n '\"rf\":\"$http_referer\",' # optional;\n '\"rt\":\"$upstream_response_time\", ' # optional; time spend receiving upstream body\n '\"sc\":\"$scheme\", ' # optional; http or https\n '\"ua\":\"$http_user_agent\",' # optional\n '\"cs\":\"$upstream_cache_status\",' # optional\n '\"al\":\"$http_accept_language\",' # optional\n'}';\n\naccess_log /.../log/nginx-access-log.json a9x_logs buffer=1k;\n```\n\n## Apache Combined Format\n\n\nCustomLog /.../log/apache2/apache_access.log combined\n\n\n",
"bugtrack_url": null,
"license": null,
"summary": "Web Statistics and Analytics Package",
"version": "1.4.17",
"project_urls": {
"Bug Reports": "https://github.com/ava007/a9x-webstatistics/issues",
"Documentation": "https://github.com/ava007/a9x-webstatistics",
"Homepage": "https://github.com/ava007/a9x-webstatistics",
"Source Code": "https://github.com/ava007/a9x-webstatistics"
},
"split_keywords": [
"webstats",
" statistics",
" analytics"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8f3a85a214814e196d00c431a968a25d99c495b9e99f9d750d20fabf02cdf2d2",
"md5": "e0d6725f2f007064b5e8472cb31510b9",
"sha256": "085be7a4341fd87cfd1587d57b414760ebaa27c21f5ee7ce2047c7fc8318eebf"
},
"downloads": -1,
"filename": "a9x_webstatistics-1.4.17-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e0d6725f2f007064b5e8472cb31510b9",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 36992,
"upload_time": "2025-01-21T19:42:36",
"upload_time_iso_8601": "2025-01-21T19:42:36.409107Z",
"url": "https://files.pythonhosted.org/packages/8f/3a/85a214814e196d00c431a968a25d99c495b9e99f9d750d20fabf02cdf2d2/a9x_webstatistics-1.4.17-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "74493f59b041f4d78c8d78db8a424a0d9ee7cb57a72bc500f28a36907704f5d6",
"md5": "2357a8c64b34c3e8643fcca03e5d1358",
"sha256": "cc129b577e4d26a92637295354d5dd31eb14b5e9f0893bef7452b7883230c718"
},
"downloads": -1,
"filename": "a9x_webstatistics-1.4.17.tar.gz",
"has_sig": false,
"md5_digest": "2357a8c64b34c3e8643fcca03e5d1358",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 202106,
"upload_time": "2025-01-21T19:42:38",
"upload_time_iso_8601": "2025-01-21T19:42:38.431739Z",
"url": "https://files.pythonhosted.org/packages/74/49/3f59b041f4d78c8d78db8a424a0d9ee7cb57a72bc500f28a36907704f5d6/a9x_webstatistics-1.4.17.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-21 19:42:38",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ava007",
"github_project": "a9x-webstatistics",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "a9x-webstatistics"
}
JSON
