| Name | translation-canvas JSON |
| Version |
1.0.0
JSON |
| download |
| home_page | None |
| Summary | Translation Canvas - A tool for evaluating and visualizing machine translation models |
| upload_time | 2024-08-20 00:19:18 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.12.2 |
| license | None |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Translation Canvas
[](https://badge.fury.io/py/translation-canvas)
[](https://opensource.org/licenses/MIT)
## Overview
**Translation Canvas** is a Python package designed for in-depth analysis and visualization of machine translation (MT) model outputs. It facilitates both system-level and instance-level evaluations, helping researchers identify, analyze, and comprehend the strengths and weaknesses of translation models.
The tool integrates multiple evaluation metrics, including BLEU, COMET, and InstructScore, to provide a comprehensive view of translation quality. Moreover, it offers detailed natural language explanations for identified errors, powered by InstructScore, and presents the results in an intuitive and interactive dashboard.
### Why Translation Canvas?
With the rapid development of machine translation systems, traditional evaluation tools like COMET and SacreBLEU often fall short in providing fine-grained insights. **Translation Canvas** bridges this gap by offering:
**Instance-level Error Analysis:** Highlight specific errors in translation instances and explain their nature using natural language descriptions.
**System-level Insights:** Aggregate error analysis to identify common pitfalls and strengths across entire datasets.
**Visual Comparisons:** Interactive dashboard for comparing the performance of different models on a granular level.
## Installation
You can easily install **Translation Canvas** via `pip`:
```bash
pip install translation-canvas
```
After installation, run a one-time setup script to configure necessary dependencies:
```bash
translation-canvas-setup
```
For the latest development version, you can install directly from the GitHub repository:
```bash
pip install git+https://github.com/ChinDandekar/translation_canvas
```
## Features
<div style="text-align: center;">
<img src="https://raw.githubusercontent.com/ChinDandekar/translation_canvas/main/images/instance-3-compare.png" alt="Compare Instances" width="1000"/>
</div>
### Error Analysis
**Translation Canvas** highlights errors at the instance level using color-coded spans. Hovering over these spans reveals natural language explanations, making it easier to understand the type of errors encountered.
### Comparison and Search
**Translation Canvas** allows users to compare multiple models simulatneously. It also provides a powerful search feature to users, allowing them to filter instancs by text, error type, error scale and error explanation
### System-level Dashboard
<div style="text-align: center;">
<img src="https://raw.githubusercontent.com/ChinDandekar/translation_canvas/main/images/system-3-compare.png" alt="Compare Systems" width="1000"/>
</div>
**Translation Canvas** provides a system-level dashboard to understand model performance at a system level.
### Instance Submission
<div style="text-align: center;">
<img src="https://raw.githubusercontent.com/ChinDandekar/translation_canvas/main/images/submit-workflow.png" alt="Submit Workflow" width="1000"/>
</div>
Submit source-prediction-reference triplets through the user interface or by directly uploading files. The tool will process these instances, evaluate them using the integrated metrics, and provide detailed feedback.
## Usage
**Translation Canvas** operates as a web application, running in your browser. To start the application:
```bash
translation-canvas-start
```
By default, the app will be available at
[http://127.0.0.1:5000](http://127.0.0.1:5000).
To run the app on a different port:
```bash
translation-canvas-start --port your-port
```
### Port Forwarding
If you are running **Translation Canvas** on a remote server via SSH, use port forwarding to access the app:
```bash
ssh -L your-port:127.0.0.1:5000 username@yourserver.com
```
## Evaluation and Feedback
**Translation Canvas** has been tested with machine translation experts, who found it to be both effective and user-friendly. The tool has shown to be particularly useful in pinpointing subtle errors that might be overlooked by traditional evaluation methods.
Raw data
{
"_id": null,
"home_page": null,
"name": "translation-canvas",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.12.2",
"maintainer_email": null,
"keywords": null,
"author": null,
"author_email": "Chinmay Dandekar <cdandekar@ucsb.edu>",
"download_url": "https://files.pythonhosted.org/packages/fc/d3/5939b8bc5e99d23a897cf8ca06e9f80152868b1a0039e0f2a5aeaec2559b/translation_canvas-1.0.0.tar.gz",
"platform": null,
"description": "# Translation Canvas\n\n[](https://badge.fury.io/py/translation-canvas)\n[](https://opensource.org/licenses/MIT)\n\n## Overview\n\n**Translation Canvas** is a Python package designed for in-depth analysis and visualization of machine translation (MT) model outputs. It facilitates both system-level and instance-level evaluations, helping researchers identify, analyze, and comprehend the strengths and weaknesses of translation models.\n\nThe tool integrates multiple evaluation metrics, including BLEU, COMET, and InstructScore, to provide a comprehensive view of translation quality. Moreover, it offers detailed natural language explanations for identified errors, powered by InstructScore, and presents the results in an intuitive and interactive dashboard.\n\n### Why Translation Canvas?\n\nWith the rapid development of machine translation systems, traditional evaluation tools like COMET and SacreBLEU often fall short in providing fine-grained insights. **Translation Canvas** bridges this gap by offering:\n\n**Instance-level Error Analysis:** Highlight specific errors in translation instances and explain their nature using natural language descriptions.\n\n**System-level Insights:** Aggregate error analysis to identify common pitfalls and strengths across entire datasets.\n\n**Visual Comparisons:** Interactive dashboard for comparing the performance of different models on a granular level.\n\n## Installation\n\nYou can easily install **Translation Canvas** via `pip`:\n\n```bash\npip install translation-canvas\n```\n\nAfter installation, run a one-time setup script to configure necessary dependencies:\n\n```bash\ntranslation-canvas-setup\n```\n\nFor the latest development version, you can install directly from the GitHub repository:\n\n```bash\npip install git+https://github.com/ChinDandekar/translation_canvas\n```\n\n## Features \n<div style=\"text-align: center;\">\n <img src=\"https://raw.githubusercontent.com/ChinDandekar/translation_canvas/main/images/instance-3-compare.png\" alt=\"Compare Instances\" width=\"1000\"/>\n</div>\n\n### Error Analysis\n\n**Translation Canvas** highlights errors at the instance level using color-coded spans. Hovering over these spans reveals natural language explanations, making it easier to understand the type of errors encountered.\n\n### Comparison and Search\n\n**Translation Canvas** allows users to compare multiple models simulatneously. It also provides a powerful search feature to users, allowing them to filter instancs by text, error type, error scale and error explanation\n\n\n### System-level Dashboard\n\n<div style=\"text-align: center;\">\n <img src=\"https://raw.githubusercontent.com/ChinDandekar/translation_canvas/main/images/system-3-compare.png\" alt=\"Compare Systems\" width=\"1000\"/>\n</div>\n\n**Translation Canvas** provides a system-level dashboard to understand model performance at a system level.\n\n### Instance Submission\n\n<div style=\"text-align: center;\">\n <img src=\"https://raw.githubusercontent.com/ChinDandekar/translation_canvas/main/images/submit-workflow.png\" alt=\"Submit Workflow\" width=\"1000\"/>\n</div>\n\nSubmit source-prediction-reference triplets through the user interface or by directly uploading files. The tool will process these instances, evaluate them using the integrated metrics, and provide detailed feedback.\n\n## Usage\n\n**Translation Canvas** operates as a web application, running in your browser. To start the application:\n\n```bash\ntranslation-canvas-start\n```\n\nBy default, the app will be available at \n[http://127.0.0.1:5000](http://127.0.0.1:5000).\n\nTo run the app on a different port:\n\n```bash\ntranslation-canvas-start --port your-port\n```\n\n\n### Port Forwarding\n\nIf you are running **Translation Canvas** on a remote server via SSH, use port forwarding to access the app:\n\n```bash\nssh -L your-port:127.0.0.1:5000 username@yourserver.com\n```\n\n\n## Evaluation and Feedback\n\n**Translation Canvas** has been tested with machine translation experts, who found it to be both effective and user-friendly. The tool has shown to be particularly useful in pinpointing subtle errors that might be overlooked by traditional evaluation methods.\n\n\n\n",
"bugtrack_url": null,
"license": null,
"summary": "Translation Canvas - A tool for evaluating and visualizing machine translation models",
"version": "1.0.0",
"project_urls": {
"Homepage": "https://github.com/ChinDandekar/instructscore_visualizer",
"Issues": "https://github.com/ChinDandekar/instructscore_visualizer/issues"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "32eedd0a33f4e31634e8895103b636154010953ab53d38497a2c6e9f6dc5a80f",
"md5": "cfce7f3b0f2379a988bee46e154fe08e",
"sha256": "509638388ddddc73f242be7e96f96e39ad248e46066b202b41c35fe50e21276a"
},
"downloads": -1,
"filename": "translation_canvas-1.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "cfce7f3b0f2379a988bee46e154fe08e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.12.2",
"size": 1830410,
"upload_time": "2024-08-20T00:19:15",
"upload_time_iso_8601": "2024-08-20T00:19:15.581154Z",
"url": "https://files.pythonhosted.org/packages/32/ee/dd0a33f4e31634e8895103b636154010953ab53d38497a2c6e9f6dc5a80f/translation_canvas-1.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "fcd35939b8bc5e99d23a897cf8ca06e9f80152868b1a0039e0f2a5aeaec2559b",
"md5": "63a5fc898dba35b92a1177b1a8983ef0",
"sha256": "8670785ed3a8ec41bf328443902483be175099d80a61c108fe243374c6e4ad42"
},
"downloads": -1,
"filename": "translation_canvas-1.0.0.tar.gz",
"has_sig": false,
"md5_digest": "63a5fc898dba35b92a1177b1a8983ef0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.12.2",
"size": 1809493,
"upload_time": "2024-08-20T00:19:18",
"upload_time_iso_8601": "2024-08-20T00:19:18.364050Z",
"url": "https://files.pythonhosted.org/packages/fc/d3/5939b8bc5e99d23a897cf8ca06e9f80152868b1a0039e0f2a5aeaec2559b/translation_canvas-1.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-20 00:19:18",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ChinDandekar",
"github_project": "instructscore_visualizer",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "translation-canvas"
}
