# pyreports
<img src="https://pyreports.readthedocs.io/en/latest/_static/pyreports.svg" alt="pyreports" title="pyreports" width="300" height="300" />
[](https://www.codacy.com/gh/MatteoGuadrini/pyreports/dashboard?utm_source=github.com&utm_medium=referral&utm_content=MatteoGuadrini/pyreports&utm_campaign=Badge_Grade)
[](https://circleci.com/gh/MatteoGuadrini/pyreports)
_pyreports_ is a python library that allows you to create complex reports from various sources such as databases,
text files, ldap, etc. and perform processing, filters, counters, etc.
and then export or write them in various formats or in databases.
## Test package
To test the package, follow these instructions:
```console
$ git clone https://github.com/MatteoGuadrini/pyreports.git
$ cd pyreports
$ python -m unittest discover tests
```
## Install package
To install package, follow these instructions:
```console
$ pip install pyreports #from pypi
$ git clone https://github.com/MatteoGuadrini/pyreports.git #from official repo
$ cd pyreports
$ python setup.py install
```
## Why choose this library?
_pyreports_ wants to be a library that simplifies the collection of data from multiple sources such as databases,
files and directory servers (through LDAP), the processing of them through built-in and customized functions,
and the saving in various formats (or, by inserting the data in a database).
## How does it work
_pyreports_ uses the [**tablib**](https://tablib.readthedocs.io/en/stable/) library to organize the data into _Dataset_ object.
### Simple report
I take the data from a database table, filter the data I need and save it in a csv file
```python
import pyreports
# Select source: this is a DatabaseManager object
mydb = pyreports.manager('mysql', host='mysql1.local', database='login_users', user='dba', password='dba0000')
# Get data
mydb.execute('SELECT * FROM site_login')
site_login = mydb.fetchall()
# Filter data
error_login = pyreports.Executor(site_login)
error_login.filter([400, 401, 403, 404, 500])
# Save report: this is a FileManager object
output = pyreports.manager('csv', '/home/report/error_login.csv')
output.write(error_login.get_data())
```
### Combine source
I take the data from a database table, and a log file, and save the report in json format
```python
import pyreports
# Select source: this is a DatabaseManager object
mydb = pyreports.manager('mysql', host='mysql1.local', database='login_users', user='dba', password='dba0000')
# Select another source: this is a FileManager object
mylog = pyreports.manager('file', '/var/log/httpd/error.log')
# Get data
mydb.execute('SELECT * FROM site_login')
site_login = mydb.fetchall()
error_log = mylog.read()
# Filter database
error_login = pyreports.Executor(site_login)
error_login.filter([400, 401, 403, 404, 500])
users_in_error = set(error_login.select_column('users'))
# Prepare log
myreport = dict()
log_user_error = pyreports.Executor(error_log)
log_user_error.filter(list(users_in_error))
for line in log_user_error:
for user in users_in_error:
myreport.setdefault(user, [])
myreport[user].append(line)
# Save report: this is a FileManager object
output = pyreports.manager('json', '/home/report/error_login.json')
output.write(myreport)
```
### Report object
```python
import pyreports
# Select source: this is a DatabaseManager object
mydb = pyreports.manager('mysql', host='mysql1.local', database='login_users', user='dba', password='dba0000')
output = pyreports.manager('xlsx', '/home/report/error_login.xlsx', mode='w')
# Get data
mydb.execute('SELECT * FROM site_login')
site_login = mydb.fetchall()
# Create report data
report = pyreports.Report(site_login, title='Site login failed', filters=[400, 401, 403, 404, 500], output=output)
# Filter data
report.exec()
# Save data on file
report.export()
```
### ReportBook collection object
```python
import pyreports
# Select source: this is a DatabaseManager object
mydb = pyreports.manager('mysql', host='mysql1.local', database='login_users', user='dba', password='dba0000')
# Get data
mydb.execute('SELECT * FROM site_login')
site_login = mydb.fetchall()
# Create report data
report_failed = pyreports.Report(site_login, title='Site login failed', filters=[400, 401, 403, 404, 500])
report_success = pyreports.Report(site_login, title='Site login success', filters=[200, 201, 202, 'OK'])
# Filter data
report_failed.exec()
report_success.exec()
# Create my ReportBook object
my_report = pyreports.ReportBook([report_failed, report_success])
# Save data on Excel file, with two worksheet ('Site login failed' and 'Site login success')
my_report.export(output='/home/report/site_login.xlsx')
```
## Tools for dataset
This library includes many tools for handling data received from databases and files.
Here are some practical examples of data manipulation.
```python
import pyreports
# Select source: this is a DatabaseManager object
mydb = pyreports.manager('mysql', host='mysql1.local', database='login_users', user='dba', password='dba0000')
# Get data
mydb.execute('SELECT * FROM site_login')
site_login = mydb.fetchall()
# Most common error
most_common_error_code = pyreports.most_common(site_login, 'code') # args: Dataset, column name
print(most_common_error_code) # 200
# Percentage of error 404
percentage_error_404 = pyreports.percentage(site_login, 404) # args: Dataset, filter
print(percentage_error_404) # 16.088264794 (percent)
# Count every error code
count_error_code = pyreports.counter(site_login, 'code') # args: Dataset, column name
print(count_error_code) # Counter({200: 4032, 201: 42, 202: 1, 400: 40, 401: 38, 403: 27, 404: 802, 500: 3})
```
### Command line
```console
$ cat car.yml
reports:
- report:
title: 'Red ford machine'
input:
manager: 'mysql'
source:
# Connection parameters of my mysql database
host: 'mysql1.local'
database: 'cars'
user: 'admin'
password: 'dba0000'
params:
query: 'SELECT * FROM cars WHERE brand = %s AND color = %s'
params: ['ford', 'red']
# Filter km
filters: [40000, 45000]
output:
manager: 'csv'
filename: '/tmp/car_csv.csv'
$ report car.yaml
```
## Official docs
In the following links there is the [official documentation](https://pyreports.readthedocs.io/en/latest/), for the use and development of the library.
* Managers: [doc](https://pyreports.readthedocs.io/en/latest/managers.html)
* Executor: [doc](https://pyreports.readthedocs.io/en/latest/executors.html)
* Report: [doc](https://pyreports.readthedocs.io/en/latest/report.html)
* data tools: [doc](https://pyreports.readthedocs.io/en/latest/datatools.html)
* examples: [doc](https://pyreports.readthedocs.io/en/latest/example.html)
* API: [io](https://pyreports.readthedocs.io/en/latest/dev/io.html), [core](https://pyreports.readthedocs.io/en/latest/dev/core.html)
* CLI: [cli](https://pyreports.readthedocs.io/en/latest/dev/cli.html)
## Open source
_pyreports_ is an open source project. Any contribute, It's welcome.
**A great thanks**.
For donations, press this
For me
[](https://www.paypal.me/guos)
For [Telethon](http://www.telethon.it/)
The Telethon Foundation is a non-profit organization recognized by the Ministry of University and Scientific and Technological Research.
They were born in 1990 to respond to the appeal of patients suffering from rare diseases.
Come today, we are organized to dare to listen to them and answers, every day of the year.
<a href="https://www.telethon.it/sostienici/dona-ora"> <img src="https://www.telethon.it/dev/_nuxt/img/c6d474e.svg" alt="Telethon" title="Telethon" width="200" height="104" /> </a>
[Adopt the future](https://www.ioadottoilfuturo.it/)
## Acknowledgments
Thanks to Mark Lutz for writing the _Learning Python_ and _Programming Python_ books that make up my python foundation.
Thanks to Kenneth Reitz and Tanya Schlusser for writing the _The Hitchhiker’s Guide to Python_ books.
Thanks to Dane Hillard for writing the _Practices of the Python Pro_ books.
Special thanks go to my wife, who understood the hours of absence for this development.
Thanks to my children, for the daily inspiration they give me and to make me realize, that life must be simple.
Thanks Python!
Raw data
{
"_id": null,
"home_page": "https://github.com/MatteoGuadrini/pyreports",
"name": "pyreports",
"maintainer": "Matteo Guadrini",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "Matteo Guadrini <matteo.guadrini@hotmail.it>",
"keywords": "pyreports, reports, report, csv, yaml, export, excel, database, ldap, dataset, file, executor, book",
"author": "Matteo Guadrini",
"author_email": "Matteo Guadrini <matteo.guadrini@hotmail.it>",
"download_url": "https://files.pythonhosted.org/packages/ba/5b/a51ecf33d41e65a45d1718fab572335f1b9fe0b204e3ce1d487e44a86c46/pyreports-1.7.0.tar.gz",
"platform": null,
"description": "# pyreports\n\n<img src=\"https://pyreports.readthedocs.io/en/latest/_static/pyreports.svg\" alt=\"pyreports\" title=\"pyreports\" width=\"300\" height=\"300\" />\n\n[](https://www.codacy.com/gh/MatteoGuadrini/pyreports/dashboard?utm_source=github.com&utm_medium=referral&utm_content=MatteoGuadrini/pyreports&utm_campaign=Badge_Grade)\n[](https://circleci.com/gh/MatteoGuadrini/pyreports)\n\n_pyreports_ is a python library that allows you to create complex reports from various sources such as databases, \ntext files, ldap, etc. and perform processing, filters, counters, etc. \nand then export or write them in various formats or in databases.\n\n## Test package\n\nTo test the package, follow these instructions:\n\n```console\n$ git clone https://github.com/MatteoGuadrini/pyreports.git\n$ cd pyreports\n$ python -m unittest discover tests\n```\n\n## Install package\n\nTo install package, follow these instructions:\n\n```console\n$ pip install pyreports #from pypi\n\n$ git clone https://github.com/MatteoGuadrini/pyreports.git #from official repo\n$ cd pyreports\n$ python setup.py install\n```\n\n## Why choose this library?\n\n_pyreports_ wants to be a library that simplifies the collection of data from multiple sources such as databases, \nfiles and directory servers (through LDAP), the processing of them through built-in and customized functions, \nand the saving in various formats (or, by inserting the data in a database).\n\n## How does it work\n\n_pyreports_ uses the [**tablib**](https://tablib.readthedocs.io/en/stable/) library to organize the data into _Dataset_ object.\n\n### Simple report\n\nI take the data from a database table, filter the data I need and save it in a csv file\n\n```python\nimport pyreports\n\n# Select source: this is a DatabaseManager object\nmydb = pyreports.manager('mysql', host='mysql1.local', database='login_users', user='dba', password='dba0000')\n\n# Get data\nmydb.execute('SELECT * FROM site_login')\nsite_login = mydb.fetchall()\n\n# Filter data\nerror_login = pyreports.Executor(site_login)\nerror_login.filter([400, 401, 403, 404, 500])\n\n# Save report: this is a FileManager object\noutput = pyreports.manager('csv', '/home/report/error_login.csv')\noutput.write(error_login.get_data())\n\n```\n\n### Combine source\n\nI take the data from a database table, and a log file, and save the report in json format\n\n```python\nimport pyreports\n\n# Select source: this is a DatabaseManager object\nmydb = pyreports.manager('mysql', host='mysql1.local', database='login_users', user='dba', password='dba0000')\n# Select another source: this is a FileManager object\nmylog = pyreports.manager('file', '/var/log/httpd/error.log')\n\n# Get data\nmydb.execute('SELECT * FROM site_login')\nsite_login = mydb.fetchall()\nerror_log = mylog.read()\n\n# Filter database\nerror_login = pyreports.Executor(site_login)\nerror_login.filter([400, 401, 403, 404, 500])\nusers_in_error = set(error_login.select_column('users'))\n\n# Prepare log\nmyreport = dict()\nlog_user_error = pyreports.Executor(error_log)\nlog_user_error.filter(list(users_in_error))\nfor line in log_user_error:\n for user in users_in_error:\n myreport.setdefault(user, [])\n myreport[user].append(line)\n\n# Save report: this is a FileManager object\noutput = pyreports.manager('json', '/home/report/error_login.json')\noutput.write(myreport)\n\n```\n\n### Report object\n\n```python\nimport pyreports\n\n# Select source: this is a DatabaseManager object\nmydb = pyreports.manager('mysql', host='mysql1.local', database='login_users', user='dba', password='dba0000')\noutput = pyreports.manager('xlsx', '/home/report/error_login.xlsx', mode='w')\n\n# Get data\nmydb.execute('SELECT * FROM site_login')\nsite_login = mydb.fetchall()\n\n# Create report data\nreport = pyreports.Report(site_login, title='Site login failed', filters=[400, 401, 403, 404, 500], output=output)\n# Filter data\nreport.exec()\n# Save data on file\nreport.export()\n\n```\n\n### ReportBook collection object\n\n```python\nimport pyreports\n\n# Select source: this is a DatabaseManager object\nmydb = pyreports.manager('mysql', host='mysql1.local', database='login_users', user='dba', password='dba0000')\n\n# Get data\nmydb.execute('SELECT * FROM site_login')\nsite_login = mydb.fetchall()\n\n# Create report data\nreport_failed = pyreports.Report(site_login, title='Site login failed', filters=[400, 401, 403, 404, 500])\nreport_success = pyreports.Report(site_login, title='Site login success', filters=[200, 201, 202, 'OK'])\n# Filter data\nreport_failed.exec()\nreport_success.exec()\n# Create my ReportBook object\nmy_report = pyreports.ReportBook([report_failed, report_success])\n# Save data on Excel file, with two worksheet ('Site login failed' and 'Site login success')\nmy_report.export(output='/home/report/site_login.xlsx')\n\n```\n\n## Tools for dataset\n\nThis library includes many tools for handling data received from databases and files. \nHere are some practical examples of data manipulation.\n\n```python\nimport pyreports\n\n# Select source: this is a DatabaseManager object\nmydb = pyreports.manager('mysql', host='mysql1.local', database='login_users', user='dba', password='dba0000')\n\n# Get data\nmydb.execute('SELECT * FROM site_login')\nsite_login = mydb.fetchall()\n\n# Most common error\nmost_common_error_code = pyreports.most_common(site_login, 'code') # args: Dataset, column name\nprint(most_common_error_code) # 200\n\n# Percentage of error 404\npercentage_error_404 = pyreports.percentage(site_login, 404) # args: Dataset, filter\nprint(percentage_error_404) # 16.088264794 (percent)\n\n# Count every error code\ncount_error_code = pyreports.counter(site_login, 'code') # args: Dataset, column name\nprint(count_error_code) # Counter({200: 4032, 201: 42, 202: 1, 400: 40, 401: 38, 403: 27, 404: 802, 500: 3})\n```\n\n### Command line\n\n```console\n$ cat car.yml\nreports:\n- report:\n title: 'Red ford machine'\n input:\n manager: 'mysql'\n source:\n # Connection parameters of my mysql database\n host: 'mysql1.local'\n database: 'cars'\n user: 'admin'\n password: 'dba0000'\n params:\n query: 'SELECT * FROM cars WHERE brand = %s AND color = %s'\n params: ['ford', 'red']\n # Filter km\n filters: [40000, 45000]\n output:\n manager: 'csv'\n filename: '/tmp/car_csv.csv'\n\n$ report car.yaml\n```\n\n## Official docs\n\nIn the following links there is the [official documentation](https://pyreports.readthedocs.io/en/latest/), for the use and development of the library.\n\n* Managers: [doc](https://pyreports.readthedocs.io/en/latest/managers.html)\n* Executor: [doc](https://pyreports.readthedocs.io/en/latest/executors.html)\n* Report: [doc](https://pyreports.readthedocs.io/en/latest/report.html)\n* data tools: [doc](https://pyreports.readthedocs.io/en/latest/datatools.html)\n* examples: [doc](https://pyreports.readthedocs.io/en/latest/example.html)\n* API: [io](https://pyreports.readthedocs.io/en/latest/dev/io.html), [core](https://pyreports.readthedocs.io/en/latest/dev/core.html)\n* CLI: [cli](https://pyreports.readthedocs.io/en/latest/dev/cli.html)\n\n## Open source\n_pyreports_ is an open source project. Any contribute, It's welcome.\n\n**A great thanks**.\n\nFor donations, press this\n\nFor me\n\n[](https://www.paypal.me/guos)\n\nFor [Telethon](http://www.telethon.it/)\n\nThe Telethon Foundation is a non-profit organization recognized by the Ministry of University and Scientific and Technological Research.\nThey were born in 1990 to respond to the appeal of patients suffering from rare diseases.\nCome today, we are organized to dare to listen to them and answers, every day of the year.\n\n<a href=\"https://www.telethon.it/sostienici/dona-ora\"> <img src=\"https://www.telethon.it/dev/_nuxt/img/c6d474e.svg\" alt=\"Telethon\" title=\"Telethon\" width=\"200\" height=\"104\" /> </a>\n\n[Adopt the future](https://www.ioadottoilfuturo.it/)\n\n\n## Acknowledgments\n\nThanks to Mark Lutz for writing the _Learning Python_ and _Programming Python_ books that make up my python foundation.\n\nThanks to Kenneth Reitz and Tanya Schlusser for writing the _The Hitchhiker\u2019s Guide to Python_ books.\n\nThanks to Dane Hillard for writing the _Practices of the Python Pro_ books.\n\nSpecial thanks go to my wife, who understood the hours of absence for this development. \nThanks to my children, for the daily inspiration they give me and to make me realize, that life must be simple.\n\nThanks Python!\n",
"bugtrack_url": null,
"license": "GNU General Public License v3.0",
"summary": "pyreports is a python library that allows you to create complex report from various sources.",
"version": "1.7.0",
"project_urls": {
"Homepage": "https://github.com/MatteoGuadrini/pyreports",
"changelog": "https://github.com/MatteoGuadrini/pyreports/blob/master/CHANGES.md",
"documentation": "https://pyreports.readthedocs.io/en/latest/",
"homepage": "https://github.com/MatteoGuadrini/pyreports'",
"repository": "https://github.com/MatteoGuadrini/pyreports.git"
},
"split_keywords": [
"pyreports",
" reports",
" report",
" csv",
" yaml",
" export",
" excel",
" database",
" ldap",
" dataset",
" file",
" executor",
" book"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "35dd3744b66060e6dc2728a97b3a9cefd99a9e18a799e59f6c2e58b202b93d9a",
"md5": "59bfdb9d98ed30f5041cf9e0d5282664",
"sha256": "31e6aa268ec05351c8b8a7e2da885ab04fb4cc486e08cf7143501e855e426a7f"
},
"downloads": -1,
"filename": "pyreports-1.7.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "59bfdb9d98ed30f5041cf9e0d5282664",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 33313,
"upload_time": "2024-04-19T08:15:15",
"upload_time_iso_8601": "2024-04-19T08:15:15.829888Z",
"url": "https://files.pythonhosted.org/packages/35/dd/3744b66060e6dc2728a97b3a9cefd99a9e18a799e59f6c2e58b202b93d9a/pyreports-1.7.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ba5ba51ecf33d41e65a45d1718fab572335f1b9fe0b204e3ce1d487e44a86c46",
"md5": "f146ee02d4d7324021a70aaee1ad97d5",
"sha256": "7d159b76d969c71d87310c8f85db8e611e69fa62f7da0e54b165b87b3044e412"
},
"downloads": -1,
"filename": "pyreports-1.7.0.tar.gz",
"has_sig": false,
"md5_digest": "f146ee02d4d7324021a70aaee1ad97d5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 83671,
"upload_time": "2024-04-19T08:15:18",
"upload_time_iso_8601": "2024-04-19T08:15:18.932754Z",
"url": "https://files.pythonhosted.org/packages/ba/5b/a51ecf33d41e65a45d1718fab572335f1b9fe0b204e3ce1d487e44a86c46/pyreports-1.7.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-19 08:15:18",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "MatteoGuadrini",
"github_project": "pyreports",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"circle": true,
"lcname": "pyreports"
}
JSON
