# Test runner for LaVague
LaVague Test Runner is a tool designed to run tests on real websites and generate comprehensive reports. It can be easily launched using the lavague-test command.
## Usage
To run the LaVague Test Runner, use the lavague-test command. Below are the available options and their descriptions:
### Command Options
- --directory / -d
- Default: current working directory + "/lavague-tests/sites"
- Type: str
- Description: Specifies the directory where the site test files are located.
- --site / -s
- Default: test on all sites described in -d directory
- Type: str
- Description: Specifies the site names to run tests on. This option can be used multiple times to specify multiple sites.
- --display
- Type: flag
- Description: If set, the browser will be displayed during the tests.
### Site Test Folder Structure
Each site to be tested must be a folder containing a `config.yml` file with the test configuration. Below is an example structure:
```arduino
lavague-tests/sites/
└── example-site/
└── config.yml
```
The config.yml file define tasks with the following structure:
```yaml
type: web
max_steps: 5 # max number of action engine retry
n_attempts: 1 # max number of agent retry
user_data: # optional user data to feed the Agent
key: value
tasks:
- name: Name # Optional display name
max_steps: 5 # to override global value
n_attempts: 1 # to override global value
url: https://example.com # the intial task URL
prompt: Prompt for the agent # the agent prompt
expect: # the list of tests to perform on task completion, see below for details
- <property> <operator> <value>
user_data: # optional task-scoped user data to feed the Agent
key: value
```
### Available operators:
| Operator | Python operation |
|-------------------|-----------------------|
| is | operator.eq |
| is not | operator.ne |
| is lower than | operator.lt |
| is greater than | operator.gt |
| contains | operator.contains |
| does not contain | not operator.contains |
### Available properties:
| Property | Type |
|----------|-------------------|
| URL | string |
| Status | success / failure |
| Output | string |
| Steps | number |
| HTML | string |
| Tabs | string |
Example of `config.yml`
```yaml
tasks:
- name: HuggingFace navigation
url: https://huggingface.co/docs
prompt: Go on the quicktour of PEFT
expect:
- URL is https://huggingface.co/docs/peft/quicktour
- Status is success
- HTML contains PEFT offers parameter-efficient methods for finetuning large pretrained models
- name: HuggingFace search
url: https://huggingface.co
prompt: Find the-wave-250 dataset
expect:
- URL is https://huggingface.co/datasets/BigAction/the-wave-250
- Status is success
```
## Output
`lavague-test` will output a report with successes and failures.
If all tests passed exit code will be 0, and -1 if at least one test failed.
Example:
```text
[o] HuggingFace navigation
URL is https://huggingface.co/docs/peft/quicktour
Status is success
HTML contains PEFT offers parameter-efficient methods for finetuning large pretrained models
[o] HuggingFace search
URL is https://huggingface.co/datasets/BigAction/the-wave-250
Status is success
[x] Go to LaVague.ai from https://google.com
(x) URL is https://lavague.ai - was: https://www.google.com/
(x) Status is success - was: failure
[o] Navigate using link
URL is http://localhost:8000/menu.html
Status is success
HTML contains <h1>Menu</h1>
Result: 80 % (8 / 10)
```
## Use with static local website
In `config.yml` set `type` to `static` to serve a static website locally.
The following options are available:
- directory : the directory to serve files from. Defaults to `www` ;
- port : the port to serve the files on. Defaults to 8000.
## Use with dynamic local website
Initialization command for local websites will be available soon.
Raw data
{
"_id": null,
"home_page": "https://lavague.ai",
"name": "lavague-tests",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0.0,>=3.10.0",
"maintainer_email": null,
"keywords": "LAM, action, automation, LLM, NLP, RAG, selenium, selenium",
"author": "lavague-ai",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/53/63/f2bda9ebe0fa524198ea217e473558b9168ece106d5d2961de599bc54fef/lavague_tests-0.0.3.tar.gz",
"platform": null,
"description": "# Test runner for LaVague\n\nLaVague Test Runner is a tool designed to run tests on real websites and generate comprehensive reports. It can be easily launched using the lavague-test command.\n\n## Usage\n\nTo run the LaVague Test Runner, use the lavague-test command. Below are the available options and their descriptions:\n\n### Command Options\n\n- --directory / -d\n - Default: current working directory + \"/lavague-tests/sites\"\n - Type: str\n - Description: Specifies the directory where the site test files are located.\n\n- --site / -s\n - Default: test on all sites described in -d directory\n - Type: str\n - Description: Specifies the site names to run tests on. This option can be used multiple times to specify multiple sites.\n\n- --display\n - Type: flag\n - Description: If set, the browser will be displayed during the tests.\n\n\n### Site Test Folder Structure\n\nEach site to be tested must be a folder containing a `config.yml` file with the test configuration. Below is an example structure:\n\n```arduino\nlavague-tests/sites/\n\u2514\u2500\u2500 example-site/\n \u2514\u2500\u2500 config.yml\n```\n\nThe config.yml file define tasks with the following structure:\n\n```yaml\ntype: web\nmax_steps: 5 # max number of action engine retry\nn_attempts: 1 # max number of agent retry\nuser_data: # optional user data to feed the Agent\n key: value\ntasks:\n - name: Name # Optional display name\n max_steps: 5 # to override global value\n n_attempts: 1 # to override global value\n url: https://example.com # the intial task URL\n prompt: Prompt for the agent # the agent prompt\n expect: # the list of tests to perform on task completion, see below for details\n - <property> <operator> <value>\n user_data: # optional task-scoped user data to feed the Agent\n key: value\n```\n\n\n### Available operators:\n\n| Operator | Python operation |\n|-------------------|-----------------------|\n| is | operator.eq |\n| is not | operator.ne |\n| is lower than | operator.lt |\n| is greater than | operator.gt |\n| contains | operator.contains |\n| does not contain | not operator.contains |\n\n\n### Available properties:\n\n| Property | Type |\n|----------|-------------------|\n| URL | string |\n| Status | success / failure |\n| Output | string |\n| Steps | number |\n| HTML | string |\n| Tabs | string |\n\nExample of `config.yml`\n\n```yaml\ntasks:\n - name: HuggingFace navigation\n url: https://huggingface.co/docs\n prompt: Go on the quicktour of PEFT\n expect:\n - URL is https://huggingface.co/docs/peft/quicktour\n - Status is success\n - HTML contains PEFT offers parameter-efficient methods for finetuning large pretrained models\n - name: HuggingFace search\n url: https://huggingface.co\n prompt: Find the-wave-250 dataset\n expect:\n - URL is https://huggingface.co/datasets/BigAction/the-wave-250\n - Status is success\n```\n\n\n## Output\n\n`lavague-test` will output a report with successes and failures.\nIf all tests passed exit code will be 0, and -1 if at least one test failed.\n\nExample:\n```text\n[o] HuggingFace navigation\n URL is https://huggingface.co/docs/peft/quicktour\n Status is success\n HTML contains PEFT offers parameter-efficient methods for finetuning large pretrained models\n\n[o] HuggingFace search\n URL is https://huggingface.co/datasets/BigAction/the-wave-250\n Status is success\n\n[x] Go to LaVague.ai from https://google.com\n (x) URL is https://lavague.ai - was: https://www.google.com/\n (x) Status is success - was: failure\n\n[o] Navigate using link\n URL is http://localhost:8000/menu.html\n Status is success\n HTML contains <h1>Menu</h1>\n\nResult: 80 % (8 / 10)\n```\n\n## Use with static local website\n\nIn `config.yml` set `type` to `static` to serve a static website locally.\n\nThe following options are available:\n- directory : the directory to serve files from. Defaults to `www` ;\n- port : the port to serve the files on. Defaults to 8000.\n\n## Use with dynamic local website\n\nInitialization command for local websites will be available soon.",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Test runner for Lavague",
"version": "0.0.3",
"project_urls": {
"Documentation": "https://docs.lavague.ai/en/latest/",
"Homepage": "https://lavague.ai",
"Repository": "https://github.com/lavague-ai/LaVague/"
},
"split_keywords": [
"lam",
" action",
" automation",
" llm",
" nlp",
" rag",
" selenium",
" selenium"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c443fff0fda9cf6d6c0e00d1674894d3f08f573c807fd5daafac0b3b7974230c",
"md5": "4fef869b8cec1271301e2404971d540b",
"sha256": "2db11ebc4570365ce513ef1911450eb5325d799acbe5a58a4c82504266f2e6b7"
},
"downloads": -1,
"filename": "lavague_tests-0.0.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4fef869b8cec1271301e2404971d540b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0.0,>=3.10.0",
"size": 8539,
"upload_time": "2024-07-29T13:00:08",
"upload_time_iso_8601": "2024-07-29T13:00:08.778280Z",
"url": "https://files.pythonhosted.org/packages/c4/43/fff0fda9cf6d6c0e00d1674894d3f08f573c807fd5daafac0b3b7974230c/lavague_tests-0.0.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5363f2bda9ebe0fa524198ea217e473558b9168ece106d5d2961de599bc54fef",
"md5": "31659610b215cbe54c09d62b510e2774",
"sha256": "f931662b777a8bf6b1751d4fd150df32a13bdab4933ad32d89760a95a7663ba8"
},
"downloads": -1,
"filename": "lavague_tests-0.0.3.tar.gz",
"has_sig": false,
"md5_digest": "31659610b215cbe54c09d62b510e2774",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0.0,>=3.10.0",
"size": 8124,
"upload_time": "2024-07-29T13:00:10",
"upload_time_iso_8601": "2024-07-29T13:00:10.456596Z",
"url": "https://files.pythonhosted.org/packages/53/63/f2bda9ebe0fa524198ea217e473558b9168ece106d5d2961de599bc54fef/lavague_tests-0.0.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-29 13:00:10",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "lavague-ai",
"github_project": "LaVague",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "lavague-tests"
}
JSON
