| Name | sweetviz JSON |
| Version |
2.3.1
JSON |
| download |
| home_page | |
| Summary | A pandas-based library to visualize and compare datasets. |
| upload_time | 2023-11-29 13:27:56 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.7 |
| license | MIT License Copyright (c) 2020 fbdesignpro Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
pandas
data-science
data-analysis
python
eda
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
 
### UPDATE (November 2023) - Version 2.3.0: Verbosity parameter added, long-standing issues fixed
---
_In-depth EDA **(target analysis, comparison, feature analysis, correlation)** in two lines of code!_
Sweetviz is an open-source Python library that generates beautiful, high-density visualizations to kickstart EDA (Exploratory Data Analysis) with just two lines of code. Output is a fully self-contained HTML application.
The system is built around quickly **visualizing target values** and **comparing datasets**. Its goal is to help quick analysis of target characteristics, training vs testing data, and other such data characterization tasks.
Usage and parameters are described below, [you can also find an article describing its features in depth and see examples in action HERE](https://towardsdatascience.com/powerful-eda-exploratory-data-analysis-in-just-two-lines-of-code-using-sweetviz-6c943d32f34).
**Sweetviz development is still ongoing!** Please let me know if you run into any data, compatibility or install issues! Thank you for [reporting any BUGS in the issue tracking system here](https://github.com/fbdesignpro/sweetviz/issues), and I welcome your feedback and questions on usage/features [in the brand-new GitHub "Discussions" tab right here!](https://github.com/fbdesignpro/sweetviz/discussions).
## Examples & mentions
[**Example HTML report** using the Titanic dataset](http://cooltiming.com/SWEETVIZ_REPORT.html)
[**Example Notebook w/docs** on Colab (Jupyter/other notebooks should also work)](https://colab.research.google.com/drive/1-md6YEwcVGWVnQWTBirQSYQYgdNoeSWg?usp=sharing)
[**Medium Article** describing its features in depth](https://towardsdatascience.com/powerful-eda-exploratory-data-analysis-in-just-two-lines-of-code-using-sweetviz-6c943d32f34)
KD Nugget articles:
[](https://www.kdnuggets.com/2021/02/powerful-exploratory-data-analysis-sweetviz.html) [](https://www.kdnuggets.com/2021/03/know-your-data-much-faster-sweetviz-python-library.html)
# Features
- **Target analysis**
- Shows how a target value (e.g. "Survived" in the Titanic dataset) relates to other features
- **Visualize and compare**
- Distinct datasets (e.g. training vs test data)
- Intra-set characteristics (e.g. male versus female)
- **Mixed-type associations**
- Sweetviz integrates associations for numerical (Pearson's correlation), categorical (uncertainty coefficient) and categorical-numerical (correlation ratio) datatypes seamlessly, to provide maximum information for all data types.
- **Type inference**
- Automatically detects numerical, categorical and text features, with optional manual overrides
- **Summary information**
- Type, unique values, missing values, duplicate rows, most frequent values
- Numerical analysis:
- min/max/range, quartiles, mean, mode, standard deviation, sum, median absolute deviation, coefficient of variation, kurtosis, skewness
## New & notable
- Version 2.2: Big compatibility update for python 3.7+ and numpy versions
- Version 2.1: **Comet.ml** support
- Version 2.0: **Jupyter, Colab & other notebook** support, report **scaling & vertical layout**
_(see below for docs on these features)_
# Upgrading
Some people have experienced mixed results behavior upgrading through `pip`. To update to the latest from an existing install, it is recommended to `pip uninstall sweetviz` first, then simply install.
# Installation
Sweetviz currently supports Python 3.6+ and Pandas 0.25.3+. Reports are output using the base "os" module, so custom environments such as Google Colab which require custom file operations are not yet supported, although I am looking into a solution.
## Using pip
The best way to install sweetviz (other than from source) is to use pip:
```
pip install sweetviz
```
#### Installation issues & fixes
In some rare cases, users have reported errors such as `ModuleNotFoundError: No module named 'sweetviz'` and `AttributeError: module 'sweetviz' has no attribute 'analyze'`.
In those cases, we suggest the following:
- Make sure none of your scripts are named `sweetviz.py`, as that interferes with the library itself. Delete or rename that script (and any associated `.pyc` files), and try again.
- Try uninstalling the library using `pip uninstall sweetviz`, then reinstalling
- The issue may stem from using multiple versions of Python, or from OS permissions. The following Stack Overflow articles have resolved many of these issues reported: [Article 1](https://stackoverflow.com/questions/32680081/importerror-after-successful-pip-installation/32680082), [Article 2](https://stackoverflow.com/questions/14295680/unable-to-import-a-module-that-is-definitely-installed), [Article 3](https://stackoverflow.com/questions/44528638/after-pip-successful-installed-modulenotfounderror)
- If all else fails, post a bug issue [here on github](https://github.com/fbdesignpro/sweetviz/issues). Thank you for taking the time, it may help resolve the issue for you and everyone else!
# Basic Usage
Creating a report is a quick 2-line process:
1. Create a `DataframeReport` object using one of: `analyze()`, `compare()` or `compare_intra()`
2. Use a `show_xxx()` function to render the report. You can now use either **html** or **notebook** report options, as well as scaling: (more info on these options below)
## Step 1: Create the report
There are 3 main functions for creating reports:
- analyze(...)
- compare(...)
- compare_intra(...)
#### Analyzing a single dataframe (and its optional target feature)
To analyze a single dataframe, simply use the `analyze(...)` function, then the `show_html(...)` function:
```
import sweetviz as sv
my_report = sv.analyze(my_dataframe)
my_report.show_html() # Default arguments will generate to "SWEETVIZ_REPORT.html"
```
When run, this will output a 1080p widescreen html app in your default browser:
##### Optional arguments
The `analyze()` function can take multiple other arguments:
```
analyze(source: Union[pd.DataFrame, Tuple[pd.DataFrame, str]],
target_feat: str = None,
feat_cfg: FeatureConfig = None,
pairwise_analysis: str = 'auto',
verbosity: str = 'default'):
```
- **source:** Either the data frame (as in the example) or a tuple containing the data frame and a name to show in the report.
e.g. `my_df` or `[my_df, "Training"]`
- **target_feat:** A string representing the name of the feature to be marked as "target". *Only BOOLEAN and NUMERICAL features can be targets for now.*
- **feat_cfg:** A FeatureConfig object representing features to be skipped, or to be forced a certain type in the analysis. The arguments can either be a single string or list of strings. Parameters are `skip`, `force_cat`, `force_num` and `force_text`. The "force_" arguments override the built-in type detection. They can be constructed as follows:
```
feature_config = sv.FeatureConfig(skip="PassengerId", force_text=["Age"])
```
- **verbosity:** **[NEW]** Can be set to `full`, `progress_only` (to only display the progress bar but not report generation messages) and `off` (fully quiet, except for errors or warnings). Default verbosity can also be set in the INI override, under the "General" heading (see "The Config file" section below for details).
- **pairwise_analysis:** Correlations and other associations can take quadratic time (n^2) to complete. The default setting ("auto") will run without warning until a data set contains "association_auto_threshold" features. Past that threshold, you need to explicitly pass the parameter `pairwise_analysis="on"` (or `="off"`) since processing that many features would take a long time. This parameter also covers the generation of the association graphs (based on [Drazen Zaric's concept](https://towardsdatascience.com/better-heatmaps-and-correlation-matrix-plots-in-python-41445d0f2bec)):
#### Comparing two dataframes (e.g. Test vs Training sets)
To compare two data sets, simply use the `compare()` function. Its parameters are the same as `analyze()`, except with an inserted second parameter to cover the comparison dataframe. It is recommended to use the [dataframe, "name"] format of parameters to better differentiate between the base and compared dataframes. (e.g. `[my_df, "Train"]` vs `my_df`)
```
my_report = sv.compare([my_dataframe, "Training Data"], [test_df, "Test Data"], "Survived", feature_config)
```
#### Comparing two subsets of the same dataframe (e.g. Male vs Female)
Another way to get great insights is to use the comparison functionality to split your dataset into 2 sub-populations.
Support for this is built in through the `compare_intra()` function. This function takes a boolean series as one of the arguments, as well as an explicit "name" tuple for naming the (true, false) resulting datasets. Note that internally, this creates 2 separate dataframes to represent each resulting group. As such, it is more of a shorthand function of doing such processing manually.
```
my_report = sv.compare_intra(my_dataframe, my_dataframe["Sex"] == "male", ["Male", "Female"], "Survived", feature_config)
```
## Step 2: Show the report
Once you have created your report object (e.g. `my_report` in the examples above), simply pass it into one of the two `show' functions:
### show_html()
```
show_html( filepath='SWEETVIZ_REPORT.html',
open_browser=True,
layout='widescreen',
scale=None)
```
**show_html(...)** will create and save an HTML report at the given file path. There are options for:
- **layout**: Either `'widescreen'` or `'vertical'`. The widescreen layout displays details on the right side of the screen, as the mouse goes over each feature. The new (as of 2.0) vertical layout is more compact horizontally and enables expanding each detail area upon clicking.
- **scale**: Use a floating-point number (e.g. `scale = 0.8` or `None`) to scale the entire report. This is very useful to fit reports to any output.
- **open_browser**: Enables the automatic opening of a web browser to show the report. Since under some circumstances this is not desired (or causes issues with some IDE's), you can disable it here.
### show_notebook()
```
show_notebook( w=None,
h=None,
scale=None,
layout='widescreen',
filepath=None,
file_layout=None,
file_scale=None)
```
**show_notebook(...)** is new as of 2.0 and will embed an IFRAME element showing the report right inside a notebook (e.g. Jupyter, Google Colab, etc.).
Note that since notebooks are generally a more constrained visual environment, it is probably a good idea to use custom width/height/scale values (`w`, `h`, `scale`) and even **set custom default values in an INI override** (see below). The options are:
- **w** (width): Sets the width of the output _window_ for the report (the full report may not fit; use `layout` and/or `scale` for the report itself). Can be as a percentage string (`w="100%"`) or number of pixels (`w=900`).
- **h** (height): Sets the height of the output _window_ for the report. Can be as a number of pixels (`h=700`) or "Full" to stretch the window to be as tall as all the features (`h="Full"`).
- **scale**: Same as for `show_html()`, above.
- **layout**: Same as for `show_html()`, above.
- **filepath**: An OPTIONAL output HTML report.
- **file_layout**: Layout for the OPTIONAL file output ONLY (same as `layout` for `show_html()`, above)
- **file_scale**: Scale for the OPTIONAL file output ONLY (same as `scale` for `show_html()`, above)
# Customizing defaults: the Config file
The package contains an INI file for configuration. You can override any setting by providing your own then calling this before creating a report:
```
sv.config_parser.read("Override.ini")
```
**IMPORTANT #1:** it is best to load overrides **before any other command**, as many of the INI options are used in the report generation.
**IMPORTANT #2:** always **put the header line** (e.g. `[General]`) before a set of values in your override INI file, **otherwise your settings will be ignored**. See examples below. If setting multiple values, only include the `[General]` line once.
### Most useful config overrides
You can look into the file `sweetviz_defaults.ini` for what can be overriden (warning: much of it is a work in progress and not well documented), but the most useful overrides are as follows.
#### Default report layout, size
Override any of these (by putting them in your own INI, again do not forget the header), to avoid having to set them every time you do a "show" command:
**Important**: note the double '%' if specifying a percentage
```
[Output_Defaults]
html_layout = widescreen
html_scale = 1.0
notebook_layout = vertical
notebook_scale = 0.9
notebook_width = 100%%
notebook_height = 700
```
##### Chinese, Japanse, Korean (CJK) character support
```
[General]
use_cjk_font = 1
```
*\*If setting multiple values for `[general]` only include the `[General]` line once*.
Will switch the font in the graphs to use a CJK-compatible font. Although this font is not as compact, it will get rid of any warnings and "unknown character" symbols for these languages.
##### Remove Sweetviz logo
```
[Layout]
show_logo = 0
```
Will remove the Sweetviz logo from the top of the page.
##### Set default verbosity level
```
[General]
default_verbosity = off
```
*\*If setting multiple values for `[general]` only include the `[General]` line once*.
Can be set to `full`, `progress_only` (to only display the progress bar but not report generation messages) and `off` (fully quiet, except for errors or warnings).
# Correlation/Association analysis
A major source of insight and unique feature of Sweetviz' associations graph and analysis is that **it unifies in a single graph** (and detail views):
- Numerical correlation (between numerical features)
- Uncertainty coefficient (for categorical-categorical)
- Correlation ratio (for categorical-numerical)
Squares represent categorical-featured-related variables and circles represent numerical-numerical correlations. Note that the trivial diagonal is left empty, for clarity.
IMPORTANT: categorical-categorical associations (provided by the SQUARES showing the uncertainty coefficient) are ASSYMMETRICAL, meaning that each row represents **how much the row title (on the left) gives information on each column**. _For example, "Sex", "Pclass" and "Fare" are the elements that give the most information on "Survived"._
For the Titanic dataset, this information is rather symmetrical but it is not always the case!
Correlations are also displayed in the detail section of each feature, with the target value highlighted when applicable. e.g.:
Finally, it is worth noting these correlation/association methods shouldn’t be taken as gospel as they make some assumptions on the underlying distribution of data and relationships. However they can be a _very_ useful starting point.
# Comet.ml integration
As of 2.1, Sweetviz now fully integrates [Comet.ml](https://www.comet.ml). This means Sweetviz will **automatically log any reports generated** using `show_html()` and `show_notebook()` to your workspace, as long as your API key is set up correctly in your environment.
Additionally, you can also use the new function `report.log_comet(experiment_object)` to explicitly upload a report for a given experiment to your workspace.
You can see an example of a [Colab notebook](https://colab.research.google.com/drive/1SK1I-gU6nLchesbMtFD9ZuzJHyzleFAr?usp=sharing) to generate the report, and its corresponding report in a [Comet.ml workspace](https://www.comet.ml/fbdesignpro/sweetviz-comet/d005158117c24924b07476887cd5ddfa?experiment-tab=html).
## Comet report parameters
You can customize how the Sweetviz report looks in your Comet workspace by overriding the `[comet_ml_defaults]` section of configuration file. See above for more information on using the INI override.
You can choose to use either the `widescreen` (horizontal) or `vertical` layouts, as well as set your preferred scale, by putting the following in your override INI file:
```
[comet_ml_defaults]
html_layout = vertical
html_scale = 0.85
```
# Troubleshooting / FAQ
- **Installation issues**
Please see the "Installation issues & fixes" section at the top of this document
- **Asian characters, "RuntimeWarning: Glyph ### missing from current font"**
See section above regarding CJK characters support. If you find the need for additional character types, definitely [post a request in the issue tracking system.](https://github.com/fbdesignpro/sweetviz/issues)
- **...any other issues**
Development is ongoing so absolutely feel free to report any issues and/or suggestions [in the issue tracking system here](https://github.com/fbdesignpro/sweetviz/issues) or [in our forum (you should be able to log in with your Github account!)](https://sweetviz.fbdesignpro.com)
# Contribute
This is my first open-source project! I built it to be the most useful tool possible and help as many people as possible with their data science work. If it is useful to you, your contribution is more than welcome and can take many forms:
### 1. Spread the word!
A STAR here on GitHub, and a Twitter or Instagram post are the easiest contribution and can potentially help grow this project tremendously! If you find this project useful, these quick actions from you would mean a lot and could go a long way.
Kaggle notebooks/posts, Medium articles, YouTube video tutorials and other content take more time but will help all the more!
### 2. Report bugs & issues
I expect there to be many quirks once the project is used by more and more people with a variety of new (& "unclean") data. If you found a bug, please [open a new issue here](https://github.com/fbdesignpro/sweetviz/issues).
### 3. Suggest and discuss usage/features
To make Sweetviz as useful as possible we need to hear what you would like it to do, or what it could do better! [Head on to our Discourse server and post your suggestions there; no login required!](https://sweetviz.fbdesignpro.com).
### 4. Contribute to the development
I definitely welcome the help I can get on this project, simply get in touch on the issue tracker and/or our Discourse forum.
Please note that after a hectic development period, the code itself right now needs a bit of cleanup. :)
# Special thanks & related materials
### Contributors
**A very special thanks to everyone who have contributed on Github, through reports, feedback and commits!** I want to give a special shout out to **Frank Male** who has been of tremendous help for fixing issues and setting up the new build pipeline for 2.2.0.
[](https://github.com/fbdesignpro/sweetviz/graphs/contributors)
Made with [contrib.rocks](https://contrib.rocks).
### Related materials
I want Sweetviz to be a hub of the best of what's out there, a way to get the most valuable information and visualization, without reinventing the wheel.
As such, I want to point some of those great resources that were inspiring and integrated into Sweetviz:
- [Pandas-Profiling](https://github.com/pandas-profiling/pandas-profiling) was the original inspiration for this project. Some of its type-detection code was included in Sweetviz.
- [Shaked Zychlinski: The Search for Categorical Correlation](https://towardsdatascience.com/the-search-for-categorical-correlation-a1cf7f1888c9) is a great article about different types of variable interactions that was the basis of that analysis in Sweetviz.
- [Drazen Zaric: Better Heatmaps and Correlation Matrix Plots in Python](https://towardsdatascience.com/better-heatmaps-and-correlation-matrix-plots-in-python-41445d0f2bec) was the basis for our association graphs.
Raw data
{
"_id": null,
"home_page": "",
"name": "sweetviz",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "pandas,data-science,data-analysis,python,eda",
"author": "",
"author_email": "Francois Bertrand <fb@fbdesignpro.com>",
"download_url": "https://files.pythonhosted.org/packages/ea/cc/a30efc5d6c8de367e01933262154c4cd60585c8db60db6f31bd4859a4a0f/sweetviz-2.3.1.tar.gz",
"platform": null,
"description": " \r\n\r\n### UPDATE (November 2023) - Version 2.3.0: Verbosity parameter added, long-standing issues fixed\r\n\r\n---\r\n \r\n\r\n_In-depth EDA **(target analysis, comparison, feature analysis, correlation)** in two lines of code!_\r\n\r\n\r\n\r\nSweetviz is an open-source Python library that generates beautiful, high-density visualizations to kickstart EDA (Exploratory Data Analysis) with just two lines of code. Output is a fully self-contained HTML application.\r\n\r\nThe system is built around quickly **visualizing target values** and **comparing datasets**. Its goal is to help quick analysis of target characteristics, training vs testing data, and other such data characterization tasks. \r\n\r\nUsage and parameters are described below, [you can also find an article describing its features in depth and see examples in action HERE](https://towardsdatascience.com/powerful-eda-exploratory-data-analysis-in-just-two-lines-of-code-using-sweetviz-6c943d32f34).\r\n\r\n**Sweetviz development is still ongoing!** Please let me know if you run into any data, compatibility or install issues! Thank you for [reporting any BUGS in the issue tracking system here](https://github.com/fbdesignpro/sweetviz/issues), and I welcome your feedback and questions on usage/features [in the brand-new GitHub \"Discussions\" tab right here!](https://github.com/fbdesignpro/sweetviz/discussions).\r\n\r\n## Examples & mentions\r\n[**Example HTML report** using the Titanic dataset](http://cooltiming.com/SWEETVIZ_REPORT.html)\r\n\r\n[**Example Notebook w/docs** on Colab (Jupyter/other notebooks should also work)](https://colab.research.google.com/drive/1-md6YEwcVGWVnQWTBirQSYQYgdNoeSWg?usp=sharing)\r\n\r\n[**Medium Article** describing its features in depth](https://towardsdatascience.com/powerful-eda-exploratory-data-analysis-in-just-two-lines-of-code-using-sweetviz-6c943d32f34)\r\n\r\nKD Nugget articles:\r\n[](https://www.kdnuggets.com/2021/02/powerful-exploratory-data-analysis-sweetviz.html) [](https://www.kdnuggets.com/2021/03/know-your-data-much-faster-sweetviz-python-library.html)\r\n\r\n# Features\r\n- **Target analysis** \r\n - Shows how a target value (e.g. \"Survived\" in the Titanic dataset) relates to other features\r\n- **Visualize and compare**\r\n - Distinct datasets (e.g. training vs test data)\r\n - Intra-set characteristics (e.g. male versus female)\r\n- **Mixed-type associations**\r\n - Sweetviz integrates associations for numerical (Pearson's correlation), categorical (uncertainty coefficient) and categorical-numerical (correlation ratio) datatypes seamlessly, to provide maximum information for all data types.\r\n- **Type inference**\r\n - Automatically detects numerical, categorical and text features, with optional manual overrides \r\n- **Summary information** \r\n - Type, unique values, missing values, duplicate rows, most frequent values\r\n - Numerical analysis: \r\n - min/max/range, quartiles, mean, mode, standard deviation, sum, median absolute deviation, coefficient of variation, kurtosis, skewness\r\n\r\n## New & notable\r\n- Version 2.2: Big compatibility update for python 3.7+ and numpy versions\r\n- Version 2.1: **Comet.ml** support\r\n- Version 2.0: **Jupyter, Colab & other notebook** support, report **scaling & vertical layout** \r\n\r\n_(see below for docs on these features)_\r\n\r\n# Upgrading\r\nSome people have experienced mixed results behavior upgrading through `pip`. To update to the latest from an existing install, it is recommended to `pip uninstall sweetviz` first, then simply install.\r\n\r\n# Installation\r\nSweetviz currently supports Python 3.6+ and Pandas 0.25.3+. Reports are output using the base \"os\" module, so custom environments such as Google Colab which require custom file operations are not yet supported, although I am looking into a solution. \r\n## Using pip\r\nThe best way to install sweetviz (other than from source) is to use pip:\r\n```\r\npip install sweetviz\r\n```\r\n#### Installation issues & fixes\r\nIn some rare cases, users have reported errors such as `ModuleNotFoundError: No module named 'sweetviz'` and `AttributeError: module 'sweetviz' has no attribute 'analyze'`.\r\nIn those cases, we suggest the following:\r\n- Make sure none of your scripts are named `sweetviz.py`, as that interferes with the library itself. Delete or rename that script (and any associated `.pyc` files), and try again.\r\n- Try uninstalling the library using `pip uninstall sweetviz`, then reinstalling\r\n- The issue may stem from using multiple versions of Python, or from OS permissions. The following Stack Overflow articles have resolved many of these issues reported: [Article 1](https://stackoverflow.com/questions/32680081/importerror-after-successful-pip-installation/32680082), [Article 2](https://stackoverflow.com/questions/14295680/unable-to-import-a-module-that-is-definitely-installed), [Article 3](https://stackoverflow.com/questions/44528638/after-pip-successful-installed-modulenotfounderror) \r\n- If all else fails, post a bug issue [here on github](https://github.com/fbdesignpro/sweetviz/issues). Thank you for taking the time, it may help resolve the issue for you and everyone else!\r\n# Basic Usage\r\nCreating a report is a quick 2-line process:\r\n1. Create a `DataframeReport` object using one of: `analyze()`, `compare()` or `compare_intra()`\r\n2. Use a `show_xxx()` function to render the report. You can now use either **html** or **notebook** report options, as well as scaling: (more info on these options below)\r\n\r\n \r\n\r\n## Step 1: Create the report\r\nThere are 3 main functions for creating reports:\r\n- analyze(...)\r\n- compare(...)\r\n- compare_intra(...)\r\n\r\n#### Analyzing a single dataframe (and its optional target feature)\r\nTo analyze a single dataframe, simply use the `analyze(...)` function, then the `show_html(...)` function:\r\n```\r\nimport sweetviz as sv\r\n\r\nmy_report = sv.analyze(my_dataframe)\r\nmy_report.show_html() # Default arguments will generate to \"SWEETVIZ_REPORT.html\"\r\n```\r\nWhen run, this will output a 1080p widescreen html app in your default browser:\r\n\r\n##### Optional arguments\r\nThe `analyze()` function can take multiple other arguments:\r\n```\r\nanalyze(source: Union[pd.DataFrame, Tuple[pd.DataFrame, str]],\r\n target_feat: str = None,\r\n feat_cfg: FeatureConfig = None,\r\n pairwise_analysis: str = 'auto',\r\n verbosity: str = 'default'):\r\n```\r\n- **source:** Either the data frame (as in the example) or a tuple containing the data frame and a name to show in the report. \r\ne.g. `my_df` or `[my_df, \"Training\"]`\r\n- **target_feat:** A string representing the name of the feature to be marked as \"target\". *Only BOOLEAN and NUMERICAL features can be targets for now.*\r\n- **feat_cfg:** A FeatureConfig object representing features to be skipped, or to be forced a certain type in the analysis. The arguments can either be a single string or list of strings. Parameters are `skip`, `force_cat`, `force_num` and `force_text`. The \"force_\" arguments override the built-in type detection. They can be constructed as follows:\r\n```\r\nfeature_config = sv.FeatureConfig(skip=\"PassengerId\", force_text=[\"Age\"])\r\n```\r\n- **verbosity:** **[NEW]** Can be set to `full`, `progress_only` (to only display the progress bar but not report generation messages) and `off` (fully quiet, except for errors or warnings). Default verbosity can also be set in the INI override, under the \"General\" heading (see \"The Config file\" section below for details).\r\n- **pairwise_analysis:** Correlations and other associations can take quadratic time (n^2) to complete. The default setting (\"auto\") will run without warning until a data set contains \"association_auto_threshold\" features. Past that threshold, you need to explicitly pass the parameter `pairwise_analysis=\"on\"` (or `=\"off\"`) since processing that many features would take a long time. This parameter also covers the generation of the association graphs (based on [Drazen Zaric's concept](https://towardsdatascience.com/better-heatmaps-and-correlation-matrix-plots-in-python-41445d0f2bec)):\r\n\r\n\r\n\r\n#### Comparing two dataframes (e.g. Test vs Training sets)\r\nTo compare two data sets, simply use the `compare()` function. Its parameters are the same as `analyze()`, except with an inserted second parameter to cover the comparison dataframe. It is recommended to use the [dataframe, \"name\"] format of parameters to better differentiate between the base and compared dataframes. (e.g. `[my_df, \"Train\"]` vs `my_df`)\r\n```\r\nmy_report = sv.compare([my_dataframe, \"Training Data\"], [test_df, \"Test Data\"], \"Survived\", feature_config)\r\n```\r\n#### Comparing two subsets of the same dataframe (e.g. Male vs Female)\r\nAnother way to get great insights is to use the comparison functionality to split your dataset into 2 sub-populations.\r\n\r\nSupport for this is built in through the `compare_intra()` function. This function takes a boolean series as one of the arguments, as well as an explicit \"name\" tuple for naming the (true, false) resulting datasets. Note that internally, this creates 2 separate dataframes to represent each resulting group. As such, it is more of a shorthand function of doing such processing manually.\r\n```\r\nmy_report = sv.compare_intra(my_dataframe, my_dataframe[\"Sex\"] == \"male\", [\"Male\", \"Female\"], \"Survived\", feature_config)\r\n```\r\n## Step 2: Show the report\r\nOnce you have created your report object (e.g. `my_report` in the examples above), simply pass it into one of the two `show' functions:\r\n\r\n### show_html()\r\n```\r\nshow_html( filepath='SWEETVIZ_REPORT.html', \r\n open_browser=True, \r\n layout='widescreen', \r\n scale=None)\r\n``` \r\n**show_html(...)** will create and save an HTML report at the given file path. There are options for:\r\n- **layout**: Either `'widescreen'` or `'vertical'`. The widescreen layout displays details on the right side of the screen, as the mouse goes over each feature. The new (as of 2.0) vertical layout is more compact horizontally and enables expanding each detail area upon clicking.\r\n- **scale**: Use a floating-point number (e.g. `scale = 0.8` or `None`) to scale the entire report. This is very useful to fit reports to any output.\r\n- **open_browser**: Enables the automatic opening of a web browser to show the report. Since under some circumstances this is not desired (or causes issues with some IDE's), you can disable it here.\r\n\r\n### show_notebook()\r\n```\r\nshow_notebook( w=None, \r\n h=None, \r\n scale=None,\r\n layout='widescreen',\r\n filepath=None,\r\n file_layout=None,\r\n file_scale=None)\r\n``` \r\n**show_notebook(...)** is new as of 2.0 and will embed an IFRAME element showing the report right inside a notebook (e.g. Jupyter, Google Colab, etc.). \r\n\r\nNote that since notebooks are generally a more constrained visual environment, it is probably a good idea to use custom width/height/scale values (`w`, `h`, `scale`) and even **set custom default values in an INI override** (see below). The options are:\r\n- **w** (width): Sets the width of the output _window_ for the report (the full report may not fit; use `layout` and/or `scale` for the report itself). Can be as a percentage string (`w=\"100%\"`) or number of pixels (`w=900`).\r\n- **h** (height): Sets the height of the output _window_ for the report. Can be as a number of pixels (`h=700`) or \"Full\" to stretch the window to be as tall as all the features (`h=\"Full\"`).\r\n- **scale**: Same as for `show_html()`, above.\r\n- **layout**: Same as for `show_html()`, above.\r\n- **filepath**: An OPTIONAL output HTML report.\r\n- **file_layout**: Layout for the OPTIONAL file output ONLY (same as `layout` for `show_html()`, above)\r\n- **file_scale**: Scale for the OPTIONAL file output ONLY (same as `scale` for `show_html()`, above)\r\n# Customizing defaults: the Config file\r\nThe package contains an INI file for configuration. You can override any setting by providing your own then calling this before creating a report:\r\n```\r\nsv.config_parser.read(\"Override.ini\")\r\n```\r\n**IMPORTANT #1:** it is best to load overrides **before any other command**, as many of the INI options are used in the report generation. \r\n\r\n**IMPORTANT #2:** always **put the header line** (e.g. `[General]`) before a set of values in your override INI file, **otherwise your settings will be ignored**. See examples below. If setting multiple values, only include the `[General]` line once.\r\n\r\n\r\n### Most useful config overrides\r\nYou can look into the file `sweetviz_defaults.ini` for what can be overriden (warning: much of it is a work in progress and not well documented), but the most useful overrides are as follows.\r\n\r\n#### Default report layout, size\r\nOverride any of these (by putting them in your own INI, again do not forget the header), to avoid having to set them every time you do a \"show\" command:\r\n\r\n**Important**: note the double '%' if specifying a percentage\r\n```\r\n[Output_Defaults]\r\nhtml_layout = widescreen\r\nhtml_scale = 1.0\r\nnotebook_layout = vertical\r\nnotebook_scale = 0.9\r\nnotebook_width = 100%%\r\nnotebook_height = 700\r\n```\r\n\r\n##### Chinese, Japanse, Korean (CJK) character support\r\n```\r\n[General]\r\nuse_cjk_font = 1 \r\n```\r\n*\\*If setting multiple values for `[general]` only include the `[General]` line once*.\r\n\r\nWill switch the font in the graphs to use a CJK-compatible font. Although this font is not as compact, it will get rid of any warnings and \"unknown character\" symbols for these languages.\r\n##### Remove Sweetviz logo\r\n```\r\n[Layout]\r\nshow_logo = 0\r\n```\r\nWill remove the Sweetviz logo from the top of the page. \r\n\r\n##### Set default verbosity level\r\n```\r\n[General]\r\ndefault_verbosity = off \r\n```\r\n*\\*If setting multiple values for `[general]` only include the `[General]` line once*.\r\n\r\nCan be set to `full`, `progress_only` (to only display the progress bar but not report generation messages) and `off` (fully quiet, except for errors or warnings).\r\n\r\n# Correlation/Association analysis\r\nA major source of insight and unique feature of Sweetviz' associations graph and analysis is that **it unifies in a single graph** (and detail views):\r\n - Numerical correlation (between numerical features)\r\n - Uncertainty coefficient (for categorical-categorical)\r\n - Correlation ratio (for categorical-numerical)\r\n\r\n\r\n Squares represent categorical-featured-related variables and circles represent numerical-numerical correlations. Note that the trivial diagonal is left empty, for clarity.\r\n \r\nIMPORTANT: categorical-categorical associations (provided by the SQUARES showing the uncertainty coefficient) are ASSYMMETRICAL, meaning that each row represents **how much the row title (on the left) gives information on each column**. _For example, \"Sex\", \"Pclass\" and \"Fare\" are the elements that give the most information on \"Survived\"._ \r\n\r\nFor the Titanic dataset, this information is rather symmetrical but it is not always the case!\r\n\r\nCorrelations are also displayed in the detail section of each feature, with the target value highlighted when applicable. e.g.:\r\n\r\n\r\n\r\nFinally, it is worth noting these correlation/association methods shouldn\u2019t be taken as gospel as they make some assumptions on the underlying distribution of data and relationships. However they can be a _very_ useful starting point.\r\n\r\n# Comet.ml integration\r\nAs of 2.1, Sweetviz now fully integrates [Comet.ml](https://www.comet.ml). This means Sweetviz will **automatically log any reports generated** using `show_html()` and `show_notebook()` to your workspace, as long as your API key is set up correctly in your environment.\r\n\r\nAdditionally, you can also use the new function `report.log_comet(experiment_object)` to explicitly upload a report for a given experiment to your workspace.\r\n\r\nYou can see an example of a [Colab notebook](https://colab.research.google.com/drive/1SK1I-gU6nLchesbMtFD9ZuzJHyzleFAr?usp=sharing) to generate the report, and its corresponding report in a [Comet.ml workspace](https://www.comet.ml/fbdesignpro/sweetviz-comet/d005158117c24924b07476887cd5ddfa?experiment-tab=html).\r\n\r\n## Comet report parameters\r\nYou can customize how the Sweetviz report looks in your Comet workspace by overriding the `[comet_ml_defaults]` section of configuration file. See above for more information on using the INI override.\r\n\r\nYou can choose to use either the `widescreen` (horizontal) or `vertical` layouts, as well as set your preferred scale, by putting the following in your override INI file:\r\n```\r\n[comet_ml_defaults]\r\nhtml_layout = vertical\r\nhtml_scale = 0.85\r\n```\r\n\r\n# Troubleshooting / FAQ\r\n- **Installation issues**\r\n\r\nPlease see the \"Installation issues & fixes\" section at the top of this document\r\n- **Asian characters, \"RuntimeWarning: Glyph ### missing from current font\"**\r\n\r\nSee section above regarding CJK characters support. If you find the need for additional character types, definitely [post a request in the issue tracking system.](https://github.com/fbdesignpro/sweetviz/issues)\r\n\r\n- **...any other issues**\r\n\r\nDevelopment is ongoing so absolutely feel free to report any issues and/or suggestions [in the issue tracking system here](https://github.com/fbdesignpro/sweetviz/issues) or [in our forum (you should be able to log in with your Github account!)](https://sweetviz.fbdesignpro.com)\r\n\r\n# Contribute\r\nThis is my first open-source project! I built it to be the most useful tool possible and help as many people as possible with their data science work. If it is useful to you, your contribution is more than welcome and can take many forms:\r\n### 1. Spread the word!\r\nA STAR here on GitHub, and a Twitter or Instagram post are the easiest contribution and can potentially help grow this project tremendously! If you find this project useful, these quick actions from you would mean a lot and could go a long way. \r\n\r\nKaggle notebooks/posts, Medium articles, YouTube video tutorials and other content take more time but will help all the more!\r\n\r\n### 2. Report bugs & issues\r\nI expect there to be many quirks once the project is used by more and more people with a variety of new (& \"unclean\") data. If you found a bug, please [open a new issue here](https://github.com/fbdesignpro/sweetviz/issues).\r\n\r\n### 3. Suggest and discuss usage/features\r\nTo make Sweetviz as useful as possible we need to hear what you would like it to do, or what it could do better! [Head on to our Discourse server and post your suggestions there; no login required!](https://sweetviz.fbdesignpro.com).\r\n\r\n### 4. Contribute to the development\r\nI definitely welcome the help I can get on this project, simply get in touch on the issue tracker and/or our Discourse forum. \r\n\r\nPlease note that after a hectic development period, the code itself right now needs a bit of cleanup. :)\r\n\r\n# Special thanks & related materials\r\n### Contributors\r\n**A very special thanks to everyone who have contributed on Github, through reports, feedback and commits!** I want to give a special shout out to **Frank Male** who has been of tremendous help for fixing issues and setting up the new build pipeline for 2.2.0.\r\n\r\n[](https://github.com/fbdesignpro/sweetviz/graphs/contributors)\r\n\r\nMade with [contrib.rocks](https://contrib.rocks).\r\n### Related materials\r\nI want Sweetviz to be a hub of the best of what's out there, a way to get the most valuable information and visualization, without reinventing the wheel.\r\n\r\nAs such, I want to point some of those great resources that were inspiring and integrated into Sweetviz:\r\n- [Pandas-Profiling](https://github.com/pandas-profiling/pandas-profiling) was the original inspiration for this project. Some of its type-detection code was included in Sweetviz.\r\n- [Shaked Zychlinski: The Search for Categorical Correlation](https://towardsdatascience.com/the-search-for-categorical-correlation-a1cf7f1888c9) is a great article about different types of variable interactions that was the basis of that analysis in Sweetviz.\r\n- [Drazen Zaric: Better Heatmaps and Correlation Matrix Plots in Python](https://towardsdatascience.com/better-heatmaps-and-correlation-matrix-plots-in-python-41445d0f2bec) was the basis for our association graphs.\r\n\r\n",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2020 fbdesignpro Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ",
"summary": "A pandas-based library to visualize and compare datasets.",
"version": "2.3.1",
"project_urls": {
"Bug Tracker": "https://github.com/fbdesignpro/sweetviz/issues",
"Changelog": "https://github.com/fbdesignpro/sweetviz/releases",
"Discussions": "https://github.com/fbdesignpro/sweetviz/discussions",
"Homepage": "https://github.com/fbdesignpro/sweetviz"
},
"split_keywords": [
"pandas",
"data-science",
"data-analysis",
"python",
"eda"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "89508d3f7ca820267a38b42c64778db0f186d85cec6e3f0c5210eab9c16e783f",
"md5": "cf8dd60fa1de73cfb979826f7a3fa21b",
"sha256": "9ac785dd78f8555f6d57a2e4fee3853e6c5ace9706de19f30697c409ff9de429"
},
"downloads": -1,
"filename": "sweetviz-2.3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "cf8dd60fa1de73cfb979826f7a3fa21b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 15112426,
"upload_time": "2023-11-29T13:27:52",
"upload_time_iso_8601": "2023-11-29T13:27:52.923619Z",
"url": "https://files.pythonhosted.org/packages/89/50/8d3f7ca820267a38b42c64778db0f186d85cec6e3f0c5210eab9c16e783f/sweetviz-2.3.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "eacca30efc5d6c8de367e01933262154c4cd60585c8db60db6f31bd4859a4a0f",
"md5": "fb701130c394eb33c5b165b5135a1209",
"sha256": "076bcab57835db167789e68949340f074dee2da78a24eaaaf211d2735f38b691"
},
"downloads": -1,
"filename": "sweetviz-2.3.1.tar.gz",
"has_sig": false,
"md5_digest": "fb701130c394eb33c5b165b5135a1209",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 15099229,
"upload_time": "2023-11-29T13:27:56",
"upload_time_iso_8601": "2023-11-29T13:27:56.990319Z",
"url": "https://files.pythonhosted.org/packages/ea/cc/a30efc5d6c8de367e01933262154c4cd60585c8db60db6f31bd4859a4a0f/sweetviz-2.3.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-29 13:27:56",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "fbdesignpro",
"github_project": "sweetviz",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "sweetviz"
}
