# robotframework-doctestlibrary
----
[Robot Framework](https://robotframework.org) DocTest library.
Simple Automated Visual Document Testing.
See **keyword documentation** for
- [Visual Document Tests](https://manykarim.github.io/robotframework-doctestlibrary/VisualTest.html)
- [Print Job Tests](https://manykarim.github.io/robotframework-doctestlibrary/PrintJobTest.html)
- [Pdf Tests (very basic)](https://manykarim.github.io/robotframework-doctestlibrary/PdfTest.html)
- [Ai Keywords (optional)](https://manykarim.github.io/robotframework-doctestlibrary/Ai.html)
```RobotFramework
*** Settings ***
Library DocTest.VisualTest
*** Test Cases ***
Compare two Images and highlight differences
Compare Images Reference.jpg Candidate.jpg
```
[Optional LLM for image comparison](https://manykarim.github.io/robotframework-doctestlibrary/Ai.html)
```RobotFramework
*** Settings ***
Library DocTest.Ai
Library DocTest.VisualTest
Library DocTest.PdfTest
*** Test Cases ***
Review Visual Differences With LLM
Compare Images With LLM Reference.pdf Candidate.pdf llm_override=${True}
Extract Text From Document With LLM
${text}= Get Text With LLM Candidate.pdf prompt=Return text and table contents
Log ${text}
Image Should Contain Object With LLM
Image Should Contain Candidate.png Missing product logo
Count Items With LLM
${count}= Get Item Count From Image Candidate.png item_description=number of pallets
Should Be True ${count} >= 0
```
[](https://youtu.be/qmpwlQoJ-nE "DocTest Library presentation at robocon.io 2021")
# Installation instructions
`pip install --upgrade robotframework-doctestlibrary`
## Optional LLM-Assisted Comparisons
You can optionally rely on a large language model to review detected differences and
decide whether a comparison should pass. This path is fully opt-in; nothing changes
for users who skip these dependencies.
1. Install the optional extra only when needed:
```bash
pip install "robotframework-doctestlibrary[ai]"
```
2. Create a `.env` file at the repository root (values here override existing environment variables):
```ini
# OpenAI-compatible endpoints
OPENAI_API_KEY=sk-...
DOCTEST_LLM_MODEL=gpt-5,gpt-4o
DOCTEST_LLM_VISION_MODEL=gpt-5-mini,gpt-4o-mini
```
Azure OpenAI deployments can be configured with:
```ini
AZURE_OPENAI_ENDPOINT=https://your-endpoint.openai.azure.com/
AZURE_OPENAI_API_KEY=...
AZURE_OPENAI_DEPLOYMENT=gpt-4o
AZURE_OPENAI_API_VERSION=2024-06-01
DOCTEST_LLM_PROVIDER=azure
```
See `.env.example` for a combined template covering both providers.
3. Use the dedicated keywords (or pass `llm_enabled=${True}` to existing ones):
```RobotFramework
*** Test Cases ***
Review Visual Differences With LLM
Compare Images With LLM Reference.pdf Candidate.pdf llm_override=${True}
Review Pdf Structure With LLM
Compare Pdf Documents With LLM reference.pdf candidate.pdf compare=structure
```
Set `llm_override=${True}` when an LLM approval should override SSIM/DeepDiff failures.
Without the override flag the AI feedback is logged for investigation while the original
assertion result is preserved.
Pass `llm_prompt=` (or the specialty variants `llm_visual_prompt=` / `llm_pdf_prompt=`) to
customise the prompt sent to the model for a particular comparison.
Only Python 3.X or newer is supported. Tested with Python 3.8/3.11/3.12
## Install robotframework-doctestlibrary
### Installation via `pip` from PyPI (recommended)
* `pip install --upgrade robotframework-doctestlibrary`
### Installation via `pip` from GitHub
* `pip install git+https://github.com/manykarim/robotframework-doctestlibrary.git`
or
* `git clone https://github.com/manykarim/robotframework-doctestlibrary.git`
* `cd robotframework-doctestlibrary`
* `pip install -e .`
## Install dependencies
Install Tesseract, Ghostscript, GhostPCL, ImageMagick binaries and barcode libraries (libdmtx, zbar) on your system.
<br>Hint: Since `0.2.0` Ghostscript, GhostPCL and ImageMagick are only needed for rendering `.ps` and `.pcl`files.
<br> Rendering and content parsing of `.pdf` is done via `MuPDF`
<br>In the future there might be a separate pypi package for `.pcl` and `.ps` files to get rid of those dependencies.
Linux
```bash
apt-get install imagemagick tesseract-ocr ghostscript libdmtx0b libzbar0
```
Windows
* https://github.com/UB-Mannheim/tesseract/wiki
* https://ghostscript.com/releases/gsdnld.html
* https://ghostscript.com/releases/gpcldnld.html
* https://imagemagick.org/script/download.php
## Some special instructions for Windows
### Rename executable for GhostPCL to pcl6.exe (only needed for `.pcl` support)
The executable for GhostPCL `gpcl6win64.exe` needs to be renamed to `pcl6.exe`
Otherwise it will not be possible to render .pcl files successfully for visual comparison.
### Add tesseract, ghostscript and imagemagick to system path in windows (only needed for OCR, `.pcl` and `.ps` support)
* C:\Program Files\ImageMagick-7.0.10-Q16-HDRI
* C:\Program Files\Tesseract-OCR
* C:\Program Files\gs\gs9.53.1\bin
* C:\Program Files\gs\ghostpcl-9.53.1-win64
(The folder names and versions on your system might be different)
That means: When you open the CMD shell you can run the commands
* `magick.exe`
* `tesseract.exe`
* `gswin64.exe`
* `pcl6.exe`
successfully from any folder/location
### Windows error message regarding pylibdmtx
[How to solve ImportError for pylibdmtx](https://github.com/NaturalHistoryMuseum/pylibdmtx/#windows-error-message)
If you see an ugly `ImportError` when importing `pylibdmtx` on
Windows you will most likely need the [Visual C++ Redistributable Packages for
Visual Studio 2013](https://www.microsoft.com/en-US/download/details.aspx?id=40784). Install `vcredist_x64.exe` if using 64-bit Python, `vcredist_x86.exe` if using 32-bit Python.
## ImageMagick
The library might return the error `File could not be converted by ImageMagick to OpenCV Image: <path to the file>` when comparing PDF files.
This is due to ImageMagick permissions. Verify this as follows with the `sample.pdf` in the `testdata` directory:
```bash
convert sample.pdf sample.jpg
convert-im6.q16: attempt to perform an operation not allowed by the security policy
```
Solution is to copy the `policy.xml` from the repository to the ImageMagick installation directory.
## Docker
You can also use the [docker images](https://github.com/manykarim/robotframework-doctestlibrary/packages) or create your own Docker Image
`docker build -t robotframework-doctest .`
Afterwards you can, e.g., start the container and run the povided examples like this:
* Windows
* `docker run -t -v "%cd%":/opt/test -w /opt/test robotframework-doctest robot atest/Compare.robot`
* Linux
* `docker run -t -v $PWD:/opt/test -w /opt/test robotframework-doctest robot atest/Compare.robot`
## Gitpod.io
[](https://gitpod.io/#https://github.com/manykarim/robotframework-doctestlibrary)
Try out the library using [Gitpod](https://gitpod.io/#https://github.com/manykarim/robotframework-doctestlibrary)
# Examples
Have a look at
* [Visual Comparison Tests](./atest/Compare.robot)
* [PDF Content Tests](./atest/PdfContent.robot)
* [Print Job Tests](./atest/PrintJobs.robot)
* [AI-Assisted Tests](./atest/LLM.robot)
for more examples.
### Testing with [Robot Framework](https://robotframework.org)
```RobotFramework
*** Settings ***
Library DocTest.VisualTest
*** Test Cases ***
Compare two Images and highlight differences
Compare Images Reference.jpg Candidate.jpg
```
### Use masks/placeholders to exclude parts from visual comparison
```RobotFramework
*** Settings ***
Library DocTest.VisualTest
*** Test Cases ***
Compare two Images and ignore parts by using masks
Compare Images Reference.jpg Candidate.jpg placeholder_file=masks.json
Compare two PDF Docments and ignore parts by using masks
Compare Images Reference.jpg Candidate.jpg placeholder_file=masks.json
Compare two Farm images with date pattern
Compare Images Reference.jpg Candidate.jpg placeholder_file=testdata/pattern_mask.json
Compare two Farm images with area mask as list
${top_mask} Create Dictionary page=1 type=area location=top percent=10
${bottom_mask} Create Dictionary page=all type=area location=bottom percent=10
${masks} Create List ${top_mask} ${bottom_mask}
Compare Images Reference.jpg Candidate.jpg mask=${masks}
Compare two Farm images with area mask as string
Compare Images Reference.jpg Candidate.jpg mask=top:10;bottom:10
```
#### Different Mask Types to Ignore Parts When Comparing
##### Areas, Coordinates, Text Patterns
```python
[
{
"page": "all",
"name": "Date Pattern",
"type": "pattern",
"pattern": ".*[0-9]{2}-[a-zA-Z]{3}-[0-9]{4}.*"
},
{
"page": "1",
"name": "Top Border",
"type": "area",
"location": "top",
"percent": 5
},
{
"page": "1",
"name": "Left Border",
"type": "area",
"location": "left",
"percent": 5
},
{
"page": 1,
"name": "Top Rectangle",
"type": "coordinates",
"x": 0,
"y": 0,
"height": 10,
"width": 210,
"unit": "mm"
}
]
```
### Accept visual different by checking move distance or text content
```RobotFramework
*** Settings ***
Library DocTest.VisualTest
*** Test Cases ***
Accept if parts are moved up to 20 pixels by pure visual check
Compare Images Reference.jpg Candidate.jpg move_tolerance=20
Accept if parts are moved up to 20 pixels by reading PDF Data
Compare Images Reference.pdf Candidate.pdf move_tolerance=20 get_pdf_content=${true}
Accept differences if text content is the same via OCR
Compare Images Reference.jpg Candidate.jpg check_text_content=${true}
Accept differences if text content is the same from PDF Data
Compare Images Reference.pdf Candidate.pdf check_text_content=${true} get_pdf_content=${true}
```
#### Different options to detect moved parts/objects
```RobotFramework
*** Settings ***
Library DocTest.VisualTest movement_detection=orb
*** Test Cases ***
Accept if parts are moved up to 20 pixels by pure visual check
Compare Images Reference.jpg Candidate.jpg move_tolerance=20
```
```RobotFramework
*** Settings ***
Library DocTest.VisualTest movement_detection=template
*** Test Cases ***
Accept if parts are moved up to 20 pixels by pure visual check
Compare Images Reference.jpg Candidate.jpg move_tolerance=20
```
```RobotFramework
*** Settings ***
Library DocTest.VisualTest movement_detection=classic
*** Test Cases ***
Accept if parts are moved up to 20 pixels by pure visual check
Compare Images Reference.jpg Candidate.jpg move_tolerance=20
```
### Options for taking additional screenshots, screenshot format and render resolution
Take additional screenshots or reference and candidate file.
```RobotFramework
*** Settings ***
Library DocTest.VisualTest take_screenshots=${true} screenshot_format=png
```
Take diff screenshots to highlight differences
```RobotFramework
*** Settings ***
Library DocTest.VisualTest show_diff=${true} DPI=300
```
### Experimental usage of Open CV East Text Detection to improve OCR
```RobotFramework
*** Settings ***
Library DocTest.VisualTest
*** Test Cases ***
Compare two Farm images with date pattern and east detection
Compare Images Reference.jpg Candidate.jpg placeholder_file=masks.json ocr_engine=east
```
### Check content of PDF files
```RobotFramework
*** Settings ***
Library DocTest.PdfTest
*** Test Cases ***
Check if list of strings exists in PDF File
@{strings}= Create List First String Second String
PDF Should Contain Strings ${strings} Candidate.pdf
Compare two PDF Files and only check text content
Compare Pdf Documents Reference.pdf Candidate.pdf compare=text
Compare two PDF Files and only check text content and metadata
Compare Pdf Documents Reference.pdf Candidate.pdf compare=text,metadata
Compare two PDF Files and check all possible content
Compare Pdf Documents Reference.pdf Candidate.pdf
```
### Ignore Watermarks for Visual Comparisons
Store the watermark in a separate B/W image or PDF.
<br>
Watermark area needs to be filled with black color.
<br>
Watermark content will be subtracted from Visual Comparison result.
```RobotFramework
*** Settings ***
Library DocTest.VisualTest
*** Test Cases ***
Compare two Images and ignore jpg watermark
Compare Images Reference.jpg Candidate.jpg watermark_file=Watermark.jpg
Compare two Images and ignore pdf watermark
Compare Images Reference.pdf Candidate.pdf watermark_file=Watermark.pdf
Compare two Images and ignore watermark folder
Compare Images Reference.pdf Candidate.pdf watermark_file=${CURDIR}${/}watermarks
```
Watermarks can also be passed on Library import. This setting will apply to all Test Cases in Test Suite
```RobotFramework
*** Settings ***
Library DocTest.VisualTest watermark_file=${CURDIR}${/}watermarks
*** Test Cases ***
Compare two Images and ignore watermarks
Compare Images Reference.jpg Candidate.jpg
```
### Get Text From Documents or Images
```RobotFramework
*** Settings ***
Library DocTest.VisualTest
*** Test Cases ***
Get Text Content And Compare
${text} Get Text From Document Reference.pdf
List Should Contain Value ${text} Test String
```
### Get Barcodes From Documents or Images
```RobotFramework
*** Settings ***
Library DocTest.VisualTest
*** Test Cases ***
Get Text Content And Compare
${text} Get Barcodes From Document reference.jpg
List Should Contain Value ${text} 123456789
```
### Using pabot to run tests in parallel
Document Testing can be run in parallel using [pabot](https://pabot.org/).
However, you need to pass the additional arguments `--artifacts` and `--artifactsinsubfolders` to the `pabot` command, to move the screenshots to the correct subfolder.
Otherwise the screenshots will not be visible in the `log.html`
```
pabot --testlevelsplit --processes 8 --artifacts png,jpg,pdf,xml --artifactsinsubfolders /path/to/your/tests/
```
### Visual Testing of Web Applications
I experimented a bit and tried to use this library for Visual Testing of Web Applications.
Please have a look at this pilot example [here](https://github.com/manykarim/robotframework-doctestlibrary/blob/main/atest/Browser.robot)
# Development
Feel free to create issues or pull requests.
I'm always happy for any feedback.
## Core team
In order of appearance.
* Many Kasiriha
* April Wang
## Contributors
This project is community driven and becomes a reality only through the work of all the people who contribute.
Raw data
{
"_id": null,
"home_page": "https://github.com/manykarim/robotframework-doctestlibrary",
"name": "robotframework-doctestlibrary",
"maintainer": "Many Kasiriha",
"docs_url": null,
"requires_python": "<4.0.0,>=3.9.0",
"maintainer_email": "many.kasiriha@dbschenker.com",
"keywords": null,
"author": "Many Kasiriha",
"author_email": "many.kasiriha@dbschenker.com",
"download_url": "https://files.pythonhosted.org/packages/3a/84/10472a529b3028c367e6fe2e1224aba58709db7899f1a0455d19775b4e35/robotframework_doctestlibrary-0.28.2.tar.gz",
"platform": null,
"description": "# robotframework-doctestlibrary\n----\n[Robot Framework](https://robotframework.org) DocTest library. \nSimple Automated Visual Document Testing.\n\nSee **keyword documentation** for\n\n- [Visual Document Tests](https://manykarim.github.io/robotframework-doctestlibrary/VisualTest.html)\n- [Print Job Tests](https://manykarim.github.io/robotframework-doctestlibrary/PrintJobTest.html)\n- [Pdf Tests (very basic)](https://manykarim.github.io/robotframework-doctestlibrary/PdfTest.html)\n- [Ai Keywords (optional)](https://manykarim.github.io/robotframework-doctestlibrary/Ai.html)\n\n```RobotFramework\n*** Settings ***\nLibrary DocTest.VisualTest\n\n*** Test Cases ***\nCompare two Images and highlight differences\n Compare Images Reference.jpg Candidate.jpg\n```\n\n[Optional LLM for image comparison](https://manykarim.github.io/robotframework-doctestlibrary/Ai.html)\n\n```RobotFramework\n*** Settings ***\nLibrary DocTest.Ai\nLibrary DocTest.VisualTest\nLibrary DocTest.PdfTest\n\n*** Test Cases ***\nReview Visual Differences With LLM\n Compare Images With LLM Reference.pdf Candidate.pdf llm_override=${True}\n\nExtract Text From Document With LLM\n ${text}= Get Text With LLM Candidate.pdf prompt=Return text and table contents\n Log ${text}\n\nImage Should Contain Object With LLM\n Image Should Contain Candidate.png Missing product logo\n\nCount Items With LLM\n ${count}= Get Item Count From Image Candidate.png item_description=number of pallets\n Should Be True ${count} >= 0\n```\n\n[](https://youtu.be/qmpwlQoJ-nE \"DocTest Library presentation at robocon.io 2021\")\n\n# Installation instructions\n\n`pip install --upgrade robotframework-doctestlibrary`\n\n## Optional LLM-Assisted Comparisons\n\nYou can optionally rely on a large language model to review detected differences and\ndecide whether a comparison should pass. This path is fully opt-in; nothing changes\nfor users who skip these dependencies.\n\n1. Install the optional extra only when needed:\n ```bash\n pip install \"robotframework-doctestlibrary[ai]\"\n ```\n2. Create a `.env` file at the repository root (values here override existing environment variables):\n ```ini\n # OpenAI-compatible endpoints\n OPENAI_API_KEY=sk-...\n DOCTEST_LLM_MODEL=gpt-5,gpt-4o\n DOCTEST_LLM_VISION_MODEL=gpt-5-mini,gpt-4o-mini\n ```\n Azure OpenAI deployments can be configured with:\n ```ini\n AZURE_OPENAI_ENDPOINT=https://your-endpoint.openai.azure.com/\n AZURE_OPENAI_API_KEY=...\n AZURE_OPENAI_DEPLOYMENT=gpt-4o\n AZURE_OPENAI_API_VERSION=2024-06-01\n DOCTEST_LLM_PROVIDER=azure\n ```\n See `.env.example` for a combined template covering both providers.\n3. Use the dedicated keywords (or pass `llm_enabled=${True}` to existing ones):\n\n ```RobotFramework\n *** Test Cases ***\n Review Visual Differences With LLM\n Compare Images With LLM Reference.pdf Candidate.pdf llm_override=${True}\n\n Review Pdf Structure With LLM\n Compare Pdf Documents With LLM reference.pdf candidate.pdf compare=structure\n ```\n\nSet `llm_override=${True}` when an LLM approval should override SSIM/DeepDiff failures.\nWithout the override flag the AI feedback is logged for investigation while the original\nassertion result is preserved.\n\nPass `llm_prompt=` (or the specialty variants `llm_visual_prompt=` / `llm_pdf_prompt=`) to\ncustomise the prompt sent to the model for a particular comparison.\n\nOnly Python 3.X or newer is supported. Tested with Python 3.8/3.11/3.12\n\n## Install robotframework-doctestlibrary\n\n### Installation via `pip` from PyPI (recommended)\n\n* `pip install --upgrade robotframework-doctestlibrary`\n\n### Installation via `pip` from GitHub\n\n* `pip install git+https://github.com/manykarim/robotframework-doctestlibrary.git` \n\nor\n\n* `git clone https://github.com/manykarim/robotframework-doctestlibrary.git`\n* `cd robotframework-doctestlibrary`\n* `pip install -e .`\n\n## Install dependencies\n\nInstall Tesseract, Ghostscript, GhostPCL, ImageMagick binaries and barcode libraries (libdmtx, zbar) on your system.\n<br>Hint: Since `0.2.0` Ghostscript, GhostPCL and ImageMagick are only needed for rendering `.ps` and `.pcl`files.\n<br> Rendering and content parsing of `.pdf` is done via `MuPDF`\n<br>In the future there might be a separate pypi package for `.pcl` and `.ps` files to get rid of those dependencies.\n\nLinux\n```bash\napt-get install imagemagick tesseract-ocr ghostscript libdmtx0b libzbar0\n```\n\nWindows\n * https://github.com/UB-Mannheim/tesseract/wiki\n * https://ghostscript.com/releases/gsdnld.html\n * https://ghostscript.com/releases/gpcldnld.html\n * https://imagemagick.org/script/download.php\n\n\n## Some special instructions for Windows \n\n### Rename executable for GhostPCL to pcl6.exe (only needed for `.pcl` support)\nThe executable for GhostPCL `gpcl6win64.exe` needs to be renamed to `pcl6.exe`\n\nOtherwise it will not be possible to render .pcl files successfully for visual comparison.\n\n### Add tesseract, ghostscript and imagemagick to system path in windows (only needed for OCR, `.pcl` and `.ps` support)\n* C:\\Program Files\\ImageMagick-7.0.10-Q16-HDRI\n* C:\\Program Files\\Tesseract-OCR\n* C:\\Program Files\\gs\\gs9.53.1\\bin\n* C:\\Program Files\\gs\\ghostpcl-9.53.1-win64\n\n(The folder names and versions on your system might be different)\n\nThat means: When you open the CMD shell you can run the commands\n* `magick.exe`\n* `tesseract.exe`\n* `gswin64.exe`\n* `pcl6.exe`\n\nsuccessfully from any folder/location\n\n### Windows error message regarding pylibdmtx\n\n[How to solve ImportError for pylibdmtx](https://github.com/NaturalHistoryMuseum/pylibdmtx/#windows-error-message)\n\nIf you see an ugly `ImportError` when importing `pylibdmtx` on\nWindows you will most likely need the [Visual C++ Redistributable Packages for\nVisual Studio 2013](https://www.microsoft.com/en-US/download/details.aspx?id=40784). Install `vcredist_x64.exe` if using 64-bit Python, `vcredist_x86.exe` if using 32-bit Python.\n\n## ImageMagick\n\nThe library might return the error `File could not be converted by ImageMagick to OpenCV Image: <path to the file>` when comparing PDF files.\nThis is due to ImageMagick permissions. Verify this as follows with the `sample.pdf` in the `testdata` directory:\n```bash\nconvert sample.pdf sample.jpg \nconvert-im6.q16: attempt to perform an operation not allowed by the security policy\n```\n\nSolution is to copy the `policy.xml` from the repository to the ImageMagick installation directory.\n\n## Docker\n\nYou can also use the [docker images](https://github.com/manykarim/robotframework-doctestlibrary/packages) or create your own Docker Image\n`docker build -t robotframework-doctest .`\nAfterwards you can, e.g., start the container and run the povided examples like this:\n* Windows\n * `docker run -t -v \"%cd%\":/opt/test -w /opt/test robotframework-doctest robot atest/Compare.robot`\n* Linux\n * `docker run -t -v $PWD:/opt/test -w /opt/test robotframework-doctest robot atest/Compare.robot`\n\n## Gitpod.io\n[](https://gitpod.io/#https://github.com/manykarim/robotframework-doctestlibrary) \nTry out the library using [Gitpod](https://gitpod.io/#https://github.com/manykarim/robotframework-doctestlibrary)\n\n# Examples\n\nHave a look at \n* [Visual Comparison Tests](./atest/Compare.robot)\n* [PDF Content Tests](./atest/PdfContent.robot)\n* [Print Job Tests](./atest/PrintJobs.robot)\n* [AI-Assisted Tests](./atest/LLM.robot)\n\nfor more examples.\n\n### Testing with [Robot Framework](https://robotframework.org)\n\n```RobotFramework\n*** Settings ***\nLibrary DocTest.VisualTest\n\n*** Test Cases ***\nCompare two Images and highlight differences\n Compare Images Reference.jpg Candidate.jpg\n```\n\n### Use masks/placeholders to exclude parts from visual comparison\n\n```RobotFramework\n*** Settings ***\nLibrary DocTest.VisualTest\n\n*** Test Cases ***\nCompare two Images and ignore parts by using masks\n Compare Images Reference.jpg Candidate.jpg placeholder_file=masks.json\n\nCompare two PDF Docments and ignore parts by using masks\n Compare Images Reference.jpg Candidate.jpg placeholder_file=masks.json\n\nCompare two Farm images with date pattern\n Compare Images Reference.jpg Candidate.jpg placeholder_file=testdata/pattern_mask.json\n\nCompare two Farm images with area mask as list\n ${top_mask} Create Dictionary page=1 type=area location=top percent=10\n ${bottom_mask} Create Dictionary page=all type=area location=bottom percent=10\n ${masks} Create List ${top_mask} ${bottom_mask}\n Compare Images Reference.jpg Candidate.jpg mask=${masks}\n\nCompare two Farm images with area mask as string\n Compare Images Reference.jpg Candidate.jpg mask=top:10;bottom:10\n\n```\n#### Different Mask Types to Ignore Parts When Comparing\n##### Areas, Coordinates, Text Patterns\n```python\n[\n {\n \"page\": \"all\",\n \"name\": \"Date Pattern\",\n \"type\": \"pattern\",\n \"pattern\": \".*[0-9]{2}-[a-zA-Z]{3}-[0-9]{4}.*\"\n },\n {\n \"page\": \"1\",\n \"name\": \"Top Border\",\n \"type\": \"area\",\n \"location\": \"top\",\n \"percent\": 5\n },\n {\n \"page\": \"1\",\n \"name\": \"Left Border\",\n \"type\": \"area\",\n \"location\": \"left\",\n \"percent\": 5\n },\n {\n \"page\": 1,\n \"name\": \"Top Rectangle\",\n \"type\": \"coordinates\",\n \"x\": 0,\n \"y\": 0,\n \"height\": 10,\n \"width\": 210,\n \"unit\": \"mm\"\n }\n]\n```\n### Accept visual different by checking move distance or text content\n\n```RobotFramework\n*** Settings ***\nLibrary DocTest.VisualTest\n\n*** Test Cases ***\nAccept if parts are moved up to 20 pixels by pure visual check\n Compare Images Reference.jpg Candidate.jpg move_tolerance=20\n\nAccept if parts are moved up to 20 pixels by reading PDF Data\n Compare Images Reference.pdf Candidate.pdf move_tolerance=20 get_pdf_content=${true}\n\nAccept differences if text content is the same via OCR\n Compare Images Reference.jpg Candidate.jpg check_text_content=${true}\n\nAccept differences if text content is the same from PDF Data\n Compare Images Reference.pdf Candidate.pdf check_text_content=${true} get_pdf_content=${true}\n```\n\n#### Different options to detect moved parts/objects\n\n```RobotFramework\n*** Settings ***\nLibrary DocTest.VisualTest movement_detection=orb\n\n*** Test Cases ***\nAccept if parts are moved up to 20 pixels by pure visual check\n Compare Images Reference.jpg Candidate.jpg move_tolerance=20\n```\t\n\n```RobotFramework\n*** Settings ***\nLibrary DocTest.VisualTest movement_detection=template\n\n*** Test Cases ***\nAccept if parts are moved up to 20 pixels by pure visual check\n Compare Images Reference.jpg Candidate.jpg move_tolerance=20\n```\t\n\n```RobotFramework\n*** Settings ***\nLibrary DocTest.VisualTest movement_detection=classic\n\n*** Test Cases ***\nAccept if parts are moved up to 20 pixels by pure visual check\n Compare Images Reference.jpg Candidate.jpg move_tolerance=20\n```\t\n\n### Options for taking additional screenshots, screenshot format and render resolution\n\nTake additional screenshots or reference and candidate file.\n\n```RobotFramework\n*** Settings ***\nLibrary DocTest.VisualTest take_screenshots=${true} screenshot_format=png\n```\nTake diff screenshots to highlight differences\n\n```RobotFramework\n*** Settings ***\nLibrary DocTest.VisualTest show_diff=${true} DPI=300\n```\n\n### Experimental usage of Open CV East Text Detection to improve OCR\n\n```RobotFramework\n*** Settings ***\nLibrary DocTest.VisualTest\n\n*** Test Cases ***\nCompare two Farm images with date pattern and east detection\n Compare Images Reference.jpg Candidate.jpg placeholder_file=masks.json ocr_engine=east\n```\n\n### Check content of PDF files\n\n```RobotFramework\n*** Settings ***\nLibrary DocTest.PdfTest\n\n*** Test Cases ***\nCheck if list of strings exists in PDF File\n @{strings}= Create List First String Second String\n PDF Should Contain Strings ${strings} Candidate.pdf\n \nCompare two PDF Files and only check text content\n Compare Pdf Documents Reference.pdf Candidate.pdf compare=text\n\nCompare two PDF Files and only check text content and metadata\n Compare Pdf Documents Reference.pdf Candidate.pdf compare=text,metadata\n \nCompare two PDF Files and check all possible content\n Compare Pdf Documents Reference.pdf Candidate.pdf\n```\n\n### Ignore Watermarks for Visual Comparisons\nStore the watermark in a separate B/W image or PDF.\n<br>\nWatermark area needs to be filled with black color.\n<br>\nWatermark content will be subtracted from Visual Comparison result.\n\n```RobotFramework\n*** Settings ***\nLibrary DocTest.VisualTest\n\n*** Test Cases ***\nCompare two Images and ignore jpg watermark\n Compare Images Reference.jpg Candidate.jpg watermark_file=Watermark.jpg\n\nCompare two Images and ignore pdf watermark\n Compare Images Reference.pdf Candidate.pdf watermark_file=Watermark.pdf\n\nCompare two Images and ignore watermark folder\n Compare Images Reference.pdf Candidate.pdf watermark_file=${CURDIR}${/}watermarks\n```\n\nWatermarks can also be passed on Library import. This setting will apply to all Test Cases in Test Suite\n\n```RobotFramework\n*** Settings ***\nLibrary DocTest.VisualTest watermark_file=${CURDIR}${/}watermarks\n\n*** Test Cases ***\nCompare two Images and ignore watermarks\n Compare Images Reference.jpg Candidate.jpg\n```\n\n### Get Text From Documents or Images\n\n```RobotFramework\n*** Settings ***\nLibrary DocTest.VisualTest\n\n*** Test Cases ***\nGet Text Content And Compare\n ${text} Get Text From Document Reference.pdf\n List Should Contain Value ${text} Test String\n```\n\n### Get Barcodes From Documents or Images\n\n```RobotFramework\n*** Settings ***\nLibrary DocTest.VisualTest\n\n*** Test Cases ***\nGet Text Content And Compare\n ${text} Get Barcodes From Document reference.jpg\n List Should Contain Value ${text} 123456789\n```\n\n### Using pabot to run tests in parallel\n\nDocument Testing can be run in parallel using [pabot](https://pabot.org/). \nHowever, you need to pass the additional arguments `--artifacts` and `--artifactsinsubfolders` to the `pabot` command, to move the screenshots to the correct subfolder. \nOtherwise the screenshots will not be visible in the `log.html`\n\n```\npabot --testlevelsplit --processes 8 --artifacts png,jpg,pdf,xml --artifactsinsubfolders /path/to/your/tests/\n```\n\n### Visual Testing of Web Applications\n\nI experimented a bit and tried to use this library for Visual Testing of Web Applications. \nPlease have a look at this pilot example [here](https://github.com/manykarim/robotframework-doctestlibrary/blob/main/atest/Browser.robot)\n\n# Development\n\nFeel free to create issues or pull requests. \nI'm always happy for any feedback.\n\n## Core team\n\nIn order of appearance.\n\n * Many Kasiriha\n * April Wang\n\n## Contributors\n\nThis project is community driven and becomes a reality only through the work of all the people who contribute.\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "A library for Visual Document Testing",
"version": "0.28.2",
"project_urls": {
"Homepage": "https://github.com/manykarim/robotframework-doctestlibrary"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3198a83924acf77ad6bd5252cdb8331acbc943380ff18c1bceb19cb1e492ea70",
"md5": "e0d954c76f9025079db1830969e0483a",
"sha256": "a5fc7b6525b9a824452e8977fb978303fad8e40436708da9b0761c89d88325af"
},
"downloads": -1,
"filename": "robotframework_doctestlibrary-0.28.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e0d954c76f9025079db1830969e0483a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0.0,>=3.9.0",
"size": 95607,
"upload_time": "2025-10-29T14:37:26",
"upload_time_iso_8601": "2025-10-29T14:37:26.698992Z",
"url": "https://files.pythonhosted.org/packages/31/98/a83924acf77ad6bd5252cdb8331acbc943380ff18c1bceb19cb1e492ea70/robotframework_doctestlibrary-0.28.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3a8410472a529b3028c367e6fe2e1224aba58709db7899f1a0455d19775b4e35",
"md5": "045229460e6897189a67232fdd19af48",
"sha256": "da3c23e5083fbe504354607bffaaa722b29c260278fd30801a4a6fdd6ba8d969"
},
"downloads": -1,
"filename": "robotframework_doctestlibrary-0.28.2.tar.gz",
"has_sig": false,
"md5_digest": "045229460e6897189a67232fdd19af48",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0.0,>=3.9.0",
"size": 86954,
"upload_time": "2025-10-29T14:37:28",
"upload_time_iso_8601": "2025-10-29T14:37:28.431448Z",
"url": "https://files.pythonhosted.org/packages/3a/84/10472a529b3028c367e6fe2e1224aba58709db7899f1a0455d19775b4e35/robotframework_doctestlibrary-0.28.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-29 14:37:28",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "manykarim",
"github_project": "robotframework-doctestlibrary",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "robotframework-doctestlibrary"
}
JSON
