# Watson Assistant Improve Notebooks
[](https://travis-ci.org/github/watson-developer-cloud/assistant-improve-recommendations-notebook)
[](https://wdc-slack-inviter.mybluemix.net)
[](https://pypi.org/project/assistant-improve-toolkit/)
[](https://cla-assistant.io/watson-developer-cloud/assistant-improve-recommendations-notebook)
This repository houses Watson Assistant Improve notebooks and the underlying assistant improve toolkit library.
## Introduction
To help improving your Watson Assistant after you have deployed it to production, we prepared the following Jupyter notebooks. These notebooks include practical steps for measuring, analyzing, and actively improving your assistant in a continuous manner. Check out [IBM Watson Assistant Continuous Improvement Best Practices](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/raw/master/notebook/IBM%20Watson%20Assistant%20Continuous%20Improvement%20Best%20Practices.pdf) for more details.
- __Measure notebook__ contains a set of automated metrics that help you monitor and understand the behavior of your system. The goal is to understand where your assistant is doing well vs where it isn’t, and to focus your improvement effort to one of the problem areas identified. This notebook generates an assessment spreadsheet for you to use to label problematic conversations, and then feed to the Effectiveness notebook.
- __Effectiveness notebook__ helps you understand the relative performance of each intent and entity as well as the confusion between your intents. This information helps you prioritize your improvement effort. The input to this notebook is an assessment spreadsheet generated from the Measure notebook. Update the marked columns in the spreadsheet with your labels and load it into the Effectiveness notebook for analysis.
- __Logs notebook__ helps you fetch logs using Watson Assistant API. You can fetch logs with various filters, and save them as a JSON file, or export the utterances in the logs into a CSV file. The JSON file can be loaded into the Measure notebook. The CSV file can be used for [intent recommendation service](https://cloud.ibm.com/docs/assistant?topic=assistant-intent-recommendations#intent-recommendations-get-intent-recommendations-task). Alternatively, you can run python scripts [`fetch_logs`](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/blob/master/src/main/python/fetch_logs.py) and [`export_csv_for_intent_recommendation`](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/blob/master/src/main/python/export_csv_for_intent_recommendation.py) to fetch logs and export them to [intent recommendation CSV](https://cloud.ibm.com/docs/assistant?topic=assistant-intent-recommendations#intent-recommendations-data-resources), respectively. Run `python get_logs -h` and `python export_csv_for_intent_recommendation.py -h` for usage.
- __Dialog Flow Analysis notebook__ help you assess and analyze user journeys and issues related to the dialog flow of ineffective (low quality) conversations based on production logs. Check out [Dialog Flow Analysis](https://github.com/watson-developer-cloud/assistant-dialog-flow-analysis) for more details.
- __Dialog Skill Analysis notebook__ help you analyze characteristics of your data such as the number of training examples for each intent or the terms which seem to be characteristic of a specific intent. Check out [Dialog Skill Analysis](https://github.com/watson-developer-cloud/assistant-dialog-skill-analysis) for more details.
## Getting Started
You can either run the notebooks locally or in [IBM Watson Studio](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/notebooks-parent.html).
- **Run locally**
1. Install Jupyter Notebook, see [Jupyter/IPython Notebook Quick Start Guide](https://jupyter-notebook-beginner-guide.readthedocs.io/en/latest/install.html) for more details.
2. Download the Jupyter notebooks available in this repository's [notebook](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/tree/master/notebook) directory. __Note: These notebook files are not designed for Watson Studio environment__
3. Start jupyter server `jupyter notebook`
4. Follow the instructions in each of the notebooks. Be sure to add your Watson Assistant credentials if necessary.
- **Run in Watson Studio**
1. Create a Watson Studio account.
Sign up in [Watson Studio](https://www.ibm.com/cloud/watson-studio), or use an existing account. Lite plan is free to use.
2. Create a new project and add a Cloud Object Storage (COS) account.
For more information regarding COS plans, see [Pricing](https://www.ibm.com/cloud-computing/bluemix/pricing-object-storage).
3. Copy [Measure](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/133dfc4cd1480bbe4eaa78d3f635e568) or [Effectiveness](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/133dfc4cd1480bbe4eaa78d3f636921c) notebook from Watson Studio community into your project.
4. Follow the instructions in each notebook to add project tokens and Watson Assistant credentials if necessary.
## Guides
* Learn more about our measure and effectiveness notebook on Medium: [Continuously Improve Your Watson Assistant with Jupyter Notebooks](https://medium.com/ibm-watson/continuously-improve-your-watson-assistant-with-jupiter-notebooks-60231df4f01f)
## Contributing
See [CONTRIBUTING.md](CONTRIBUTING.md) for more details on how to contribute
## License
This library is licensed under the [Apache 2.0 license](http://www.apache.org/licenses/LICENSE-2.0).
Raw data
{
"_id": null,
"home_page": "https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook",
"name": "assistant-improve-toolkit",
"maintainer": "Zhe Zhang",
"docs_url": null,
"requires_python": "",
"maintainer_email": "zhangzhe@us.ibm.com",
"keywords": "",
"author": "IBM Watson",
"author_email": "watdevex@us.ibm.com",
"download_url": "https://files.pythonhosted.org/packages/d5/20/5d1a26195ec2e891f150c9095fcef54e18472f56130e2f898b687643efcc/assistant_improve_toolkit-1.4.1.tar.gz",
"platform": "any",
"description": "# Watson Assistant Improve Notebooks\n\n[](https://travis-ci.org/github/watson-developer-cloud/assistant-improve-recommendations-notebook)\n[](https://wdc-slack-inviter.mybluemix.net)\n[](https://pypi.org/project/assistant-improve-toolkit/)\n[](https://cla-assistant.io/watson-developer-cloud/assistant-improve-recommendations-notebook)\n\nThis repository houses Watson Assistant Improve notebooks and the underlying assistant improve toolkit library.\n\n## Introduction\nTo help improving your Watson Assistant after you have deployed it to production, we prepared the following Jupyter notebooks. These notebooks include practical steps for measuring, analyzing, and actively improving your assistant in a continuous manner. Check out [IBM Watson Assistant Continuous Improvement Best Practices](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/raw/master/notebook/IBM%20Watson%20Assistant%20Continuous%20Improvement%20Best%20Practices.pdf) for more details.\n\n- __Measure notebook__ contains a set of automated metrics that help you monitor and understand the behavior of your system. The goal is to understand where your assistant is doing well vs where it isn\u2019t, and to focus your improvement effort to one of the problem areas identified. This notebook generates an assessment spreadsheet for you to use to label problematic conversations, and then feed to the Effectiveness notebook. \n\n- __Effectiveness notebook__ helps you understand the relative performance of each intent and entity as well as the confusion between your intents. This information helps you prioritize your improvement effort. The input to this notebook is an assessment spreadsheet generated from the Measure notebook. Update the marked columns in the spreadsheet with your labels and load it into the Effectiveness notebook for analysis.\n\n- __Logs notebook__ helps you fetch logs using Watson Assistant API. You can fetch logs with various filters, and save them as a JSON file, or export the utterances in the logs into a CSV file. The JSON file can be loaded into the Measure notebook. The CSV file can be used for [intent recommendation service](https://cloud.ibm.com/docs/assistant?topic=assistant-intent-recommendations#intent-recommendations-get-intent-recommendations-task). Alternatively, you can run python scripts [`fetch_logs`](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/blob/master/src/main/python/fetch_logs.py) and [`export_csv_for_intent_recommendation`](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/blob/master/src/main/python/export_csv_for_intent_recommendation.py) to fetch logs and export them to [intent recommendation CSV](https://cloud.ibm.com/docs/assistant?topic=assistant-intent-recommendations#intent-recommendations-data-resources), respectively. Run `python get_logs -h` and `python export_csv_for_intent_recommendation.py -h` for usage. \n\n- __Dialog Flow Analysis notebook__ help you assess and analyze user journeys and issues related to the dialog flow of ineffective (low quality) conversations based on production logs. Check out [Dialog Flow Analysis](https://github.com/watson-developer-cloud/assistant-dialog-flow-analysis) for more details.\n\n- __Dialog Skill Analysis notebook__ help you analyze characteristics of your data such as the number of training examples for each intent or the terms which seem to be characteristic of a specific intent. Check out [Dialog Skill Analysis](https://github.com/watson-developer-cloud/assistant-dialog-skill-analysis) for more details.\n\n## Getting Started\n\nYou can either run the notebooks locally or in [IBM Watson Studio](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/notebooks-parent.html).\n\n- **Run locally**\n\n 1. Install Jupyter Notebook, see [Jupyter/IPython Notebook Quick Start Guide](https://jupyter-notebook-beginner-guide.readthedocs.io/en/latest/install.html) for more details.\n 2. Download the Jupyter notebooks available in this repository's [notebook](https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook/tree/master/notebook) directory. __Note: These notebook files are not designed for Watson Studio environment__\n 3. Start jupyter server `jupyter notebook`\n 4. Follow the instructions in each of the notebooks. Be sure to add your Watson Assistant credentials if necessary.\n\n- **Run in Watson Studio**\n \n 1. Create a Watson Studio account. \n Sign up in [Watson Studio](https://www.ibm.com/cloud/watson-studio), or use an existing account. Lite plan is free to use.\n\n 2. Create a new project and add a Cloud Object Storage (COS) account. \n For more information regarding COS plans, see [Pricing](https://www.ibm.com/cloud-computing/bluemix/pricing-object-storage).\n\n 3. Copy [Measure](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/133dfc4cd1480bbe4eaa78d3f635e568) or [Effectiveness](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/133dfc4cd1480bbe4eaa78d3f636921c) notebook from Watson Studio community into your project.\n \n 4. Follow the instructions in each notebook to add project tokens and Watson Assistant credentials if necessary.\n\n## Guides\n* Learn more about our measure and effectiveness notebook on Medium: [Continuously Improve Your Watson Assistant with Jupyter Notebooks](https://medium.com/ibm-watson/continuously-improve-your-watson-assistant-with-jupiter-notebooks-60231df4f01f)\n\n## Contributing \nSee [CONTRIBUTING.md](CONTRIBUTING.md) for more details on how to contribute\n\n## License\nThis library is licensed under the [Apache 2.0 license](http://www.apache.org/licenses/LICENSE-2.0).\n\n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "Assistant Improve Toolkit",
"version": "1.4.1",
"project_urls": {
"Homepage": "https://github.com/watson-developer-cloud/assistant-improve-recommendations-notebook"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "775a808b5a0447466dff31488b0a78065e6a7547cdc4d870a5227c2a40e7ec75",
"md5": "54147657258350d9fc3e77ff5f518afc",
"sha256": "987ac685ace243d6e14f3ae1e589ffdb131158443f907b830a725b48feaa941e"
},
"downloads": -1,
"filename": "assistant_improve_toolkit-1.4.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "54147657258350d9fc3e77ff5f518afc",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 74368,
"upload_time": "2023-07-08T22:45:08",
"upload_time_iso_8601": "2023-07-08T22:45:08.206915Z",
"url": "https://files.pythonhosted.org/packages/77/5a/808b5a0447466dff31488b0a78065e6a7547cdc4d870a5227c2a40e7ec75/assistant_improve_toolkit-1.4.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d5205d1a26195ec2e891f150c9095fcef54e18472f56130e2f898b687643efcc",
"md5": "ec81063e2a5fe1db17928508d3b810e7",
"sha256": "219c481e4a0d43cc5ed5a05b0f508c6788d6fe3093bd4a61fe1461e5bad01404"
},
"downloads": -1,
"filename": "assistant_improve_toolkit-1.4.1.tar.gz",
"has_sig": false,
"md5_digest": "ec81063e2a5fe1db17928508d3b810e7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 68435,
"upload_time": "2023-07-08T22:45:09",
"upload_time_iso_8601": "2023-07-08T22:45:09.552930Z",
"url": "https://files.pythonhosted.org/packages/d5/20/5d1a26195ec2e891f150c9095fcef54e18472f56130e2f898b687643efcc/assistant_improve_toolkit-1.4.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-08 22:45:09",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "watson-developer-cloud",
"github_project": "assistant-improve-recommendations-notebook",
"travis_ci": true,
"coveralls": false,
"github_actions": true,
"requirements": [],
"tox": true,
"lcname": "assistant-improve-toolkit"
}
JSON
