# Neon Minerva
Neon Minerva (Modular INtelligent Evaluation for a Reliable Voice Assistant)
provides tools for testing skills.
Install the Minerva Python package with: `pip install neon-minerva`
The `minerva` entrypoint is available to interact with a bus via CLI.
Help is available via `minerva --help`.
## Installation
If testing Padatious intents, the following system packages must be
installed before installing this package:
```shell
sudo apt install swig libfann-dev
```
To install this package from PyPI, simply run:
```shell
pip install neon-minerva
```
If testing with Padatious, install with the `padatious` extras:
```shell
pip install neon-minerva[padatious]
```
## Usage
This package provides a CLI for local testing of skills. Skills installed with
`pip` can be specified by entrypoint, or skills cloned locally can be specified
by root directory.
### Resource Tests
To test that skill resources are defined for all supported languages,
`minerva test-resources <skill-entrypoint> <test-file>`
> - <skill-entrypoint\> is the string entrypoint for the skill to test as specified in `setup.py` OR the path to
the skill's root directory
> - <test-file\> is a relative or absolute path to the resource test file, usually `test_resources.yaml`
example `test_resources.yaml`:
```yaml
# Specify resources to test here.
# Specify languages to be tested
languages:
- "en-us"
- "uk-ua"
# vocab is lowercase .voc file basenames
vocab:
- ip
- public
- query
# dialog is .dialog file basenames (case-sensitive)
dialog:
- dot
- my address is
- my address on X is Y
- no network connection
- word_public
- word_local
# regex entities, not necessarily filenames
regex: []
intents:
# Padatious intents are the `.intent` file names
padatious: []
# Adapt intents are the name passed to the constructor
adapt:
- IPIntent
```
### Intent Tests
To test that skill intents match as expected for all supported languages,
`minerva test-intents <skill-entrypoint> <test-file>`
> - <skill-entrypoint\> is the string entrypoint for the skill to test as specified in `setup.py` OR the path to
the skill's root directory
> - <test-file\> is a relative or absolute path to the resource test file, usually `test_intents.yaml`
> - The `--padacioso` flag can be added to test with Padacioso instead of Padatious for relevant intents
example `test_intents.yaml`:
```yaml
en-us:
IPIntent:
- what is your ip address
- what is my ip address:
- IP
- what is my i.p. address
- What is your I.P. address?
- what is my public IP address?:
- public: public
uk-ua:
IPIntent:
- шо в мене за ай пі:
- IP
- покажи яка в мене за мережа:
- IP
- покажи яка в мене публічний ай пі адреса:
- public: публічний
```
#### Test Configuration
The following top-level sections can be added to intent test configuration:
- `unmatched intents`: dict of `lang` to list of `utterances` that should match
no intents. Note that this does not test for CommonQuery or CommonPlay matches.
- `common query`: dict of `lang` to list of `utterances` OR dict of `utterances`
to expected: `callback_data` (list keys or dict data), `min_confidence`, and
`max_confidence`
- `common play`: TBD
## Advanced Usage
In addition to convenient CLI methods, this package also provides test cases that
may be extended.
### Skill Unit Tests
`neon_minerva.tests.skill_unit_test_base` provides `SkillTestCase`, a class
that supplies boilerplate setup/teardown/mocking for testing a skill. An example
skill test implementation could look like:
```python
from os import environ
from neon_minerva.tests.skill_unit_test_base import SkillTestCase
environ['TEST_SKILL_ENTRYPOINT'] = "my_skill.test"
class MySkillTest(SkillTestCase):
def test_skill_init(self):
self.assertEqual(self.skill.skill_id, "my_skill.test")
...
```
Be sure to review the base class for mocked methods and test paths as these may
change in the future.
### Chatbot Unit Tests
`neon_minerva.chatbots` contains mocked data for testing as well as some utility
methods. `neon_minerva.tests.chatbot_v1_test_base` provides `TestSubmind` which
may be extended to test a submind bot in a mocked v1 environment. For example:
```python
from os import environ
from datetime import datetime
from chatbot_core.utils.enum import ConversationState
from neon_minerva.tests.chatbot_v1_test_base import TestSubmind
from neon_minerva.chatbots.test_constants import PROMPT, RESPONSES
environ["TEST_BOT_ENTRYPOINT"] = "tester"
class TestTester(TestSubmind):
def test_submind_chatbot(self):
self.submind.state = ConversationState.RESP
response = self.submind.ask_chatbot("testrunner", PROMPT,
datetime.now().strftime(
"%I:%M:%S %p"))
self.assertIsInstance(response, str)
self.assertIsNotNone(response)
```
> Make sure to install the `chatbots` extra to use this test case
Raw data
{
"_id": null,
"home_page": "https://github.com/neongeckocom/neon-minerva",
"name": "neon-minerva",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "",
"author": "Neongecko",
"author_email": "developers@neon.ai",
"download_url": "https://files.pythonhosted.org/packages/11/be/fe28fc09cffa5dcdc4ee77c6720152c21a114181e0917e1c74288bc0f856/neon-minerva-0.2.0.tar.gz",
"platform": null,
"description": "# Neon Minerva\nNeon Minerva (Modular INtelligent Evaluation for a Reliable Voice Assistant) \nprovides tools for testing skills.\n\nInstall the Minerva Python package with: `pip install neon-minerva`\nThe `minerva` entrypoint is available to interact with a bus via CLI. \nHelp is available via `minerva --help`.\n\n## Installation\nIf testing Padatious intents, the following system packages must be \ninstalled before installing this package:\n```shell\nsudo apt install swig libfann-dev\n```\nTo install this package from PyPI, simply run:\n```shell\npip install neon-minerva\n```\n\nIf testing with Padatious, install with the `padatious` extras:\n```shell\npip install neon-minerva[padatious]\n```\n\n## Usage\nThis package provides a CLI for local testing of skills. Skills installed with \n`pip` can be specified by entrypoint, or skills cloned locally can be specified\nby root directory.\n\n### Resource Tests\nTo test that skill resources are defined for all supported languages,\n`minerva test-resources <skill-entrypoint> <test-file>`\n> - <skill-entrypoint\\> is the string entrypoint for the skill to test as specified in `setup.py` OR the path to \n the skill's root directory \n> - <test-file\\> is a relative or absolute path to the resource test file, usually `test_resources.yaml`\n\nexample `test_resources.yaml`:\n```yaml\n# Specify resources to test here.\n\n# Specify languages to be tested\nlanguages:\n - \"en-us\"\n - \"uk-ua\"\n\n# vocab is lowercase .voc file basenames\nvocab:\n - ip\n - public\n - query\n\n# dialog is .dialog file basenames (case-sensitive)\ndialog:\n - dot\n - my address is\n - my address on X is Y\n - no network connection\n - word_public\n - word_local\n# regex entities, not necessarily filenames\nregex: []\nintents:\n # Padatious intents are the `.intent` file names\n padatious: []\n # Adapt intents are the name passed to the constructor\n adapt:\n - IPIntent\n```\n\n### Intent Tests\nTo test that skill intents match as expected for all supported languages,\n`minerva test-intents <skill-entrypoint> <test-file>`\n> - <skill-entrypoint\\> is the string entrypoint for the skill to test as specified in `setup.py` OR the path to \n the skill's root directory \n> - <test-file\\> is a relative or absolute path to the resource test file, usually `test_intents.yaml`\n> - The `--padacioso` flag can be added to test with Padacioso instead of Padatious for relevant intents\n\nexample `test_intents.yaml`:\n```yaml\nen-us:\n IPIntent:\n - what is your ip address\n - what is my ip address:\n - IP\n - what is my i.p. address\n - What is your I.P. address?\n - what is my public IP address?:\n - public: public\n\nuk-ua:\n IPIntent:\n - \u0448\u043e \u0432 \u043c\u0435\u043d\u0435 \u0437\u0430 \u0430\u0439 \u043f\u0456:\n - IP \n - \u043f\u043e\u043a\u0430\u0436\u0438 \u044f\u043a\u0430 \u0432 \u043c\u0435\u043d\u0435 \u0437\u0430 \u043c\u0435\u0440\u0435\u0436\u0430:\n - IP\n - \u043f\u043e\u043a\u0430\u0436\u0438 \u044f\u043a\u0430 \u0432 \u043c\u0435\u043d\u0435 \u043f\u0443\u0431\u043b\u0456\u0447\u043d\u0438\u0439 \u0430\u0439 \u043f\u0456 \u0430\u0434\u0440\u0435\u0441\u0430:\n - public: \u043f\u0443\u0431\u043b\u0456\u0447\u043d\u0438\u0439\n```\n\n#### Test Configuration\nThe following top-level sections can be added to intent test configuration:\n\n- `unmatched intents`: dict of `lang` to list of `utterances` that should match \n no intents. Note that this does not test for CommonQuery or CommonPlay matches.\n- `common query`: dict of `lang` to list of `utterances` OR dict of `utterances`\n to expected: `callback_data` (list keys or dict data), `min_confidence`, and\n `max_confidence`\n- `common play`: TBD\n\n## Advanced Usage\nIn addition to convenient CLI methods, this package also provides test cases that\nmay be extended.\n\n### Skill Unit Tests\n`neon_minerva.tests.skill_unit_test_base` provides `SkillTestCase`, a class\nthat supplies boilerplate setup/teardown/mocking for testing a skill. An example\nskill test implementation could look like:\n\n```python\nfrom os import environ\nfrom neon_minerva.tests.skill_unit_test_base import SkillTestCase\n\nenviron['TEST_SKILL_ENTRYPOINT'] = \"my_skill.test\"\n\nclass MySkillTest(SkillTestCase):\n def test_skill_init(self):\n self.assertEqual(self.skill.skill_id, \"my_skill.test\")\n ...\n```\n\nBe sure to review the base class for mocked methods and test paths as these may\nchange in the future.\n\n### Chatbot Unit Tests\n`neon_minerva.chatbots` contains mocked data for testing as well as some utility\nmethods. `neon_minerva.tests.chatbot_v1_test_base` provides `TestSubmind` which\nmay be extended to test a submind bot in a mocked v1 environment. For example:\n\n```python\nfrom os import environ\nfrom datetime import datetime\nfrom chatbot_core.utils.enum import ConversationState\n\nfrom neon_minerva.tests.chatbot_v1_test_base import TestSubmind\nfrom neon_minerva.chatbots.test_constants import PROMPT, RESPONSES\n\nenviron[\"TEST_BOT_ENTRYPOINT\"] = \"tester\"\n\n\nclass TestTester(TestSubmind):\n def test_submind_chatbot(self):\n self.submind.state = ConversationState.RESP\n response = self.submind.ask_chatbot(\"testrunner\", PROMPT,\n datetime.now().strftime(\n \"%I:%M:%S %p\"))\n self.assertIsInstance(response, str)\n self.assertIsNotNone(response)\n```\n> Make sure to install the `chatbots` extra to use this test case\n",
"bugtrack_url": null,
"license": "BSD-3-Clause",
"summary": "Modular INtelligent Evaluation for a Reliable Voice Assistant",
"version": "0.2.0",
"project_urls": {
"Homepage": "https://github.com/neongeckocom/neon-minerva"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1c22854d9703d401fc0299cf2c726f455a22d7fb8f664a03f7fde75f52e3b4f1",
"md5": "d66cb60a962e8af38164c45e607f6c8c",
"sha256": "e8f56af281327142b6dd5db96006e09054676d74e5a2aa50f7468a22e541a3fa"
},
"downloads": -1,
"filename": "neon_minerva-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d66cb60a962e8af38164c45e607f6c8c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 40115,
"upload_time": "2024-03-01T23:21:04",
"upload_time_iso_8601": "2024-03-01T23:21:04.568296Z",
"url": "https://files.pythonhosted.org/packages/1c/22/854d9703d401fc0299cf2c726f455a22d7fb8f664a03f7fde75f52e3b4f1/neon_minerva-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "11befe28fc09cffa5dcdc4ee77c6720152c21a114181e0917e1c74288bc0f856",
"md5": "13922e4ce0e06b6c7cc621e560ac29c5",
"sha256": "4bdcf208d3c01ed3a88029745fd04201adff91212cdc95585afdf02aa6ec38f5"
},
"downloads": -1,
"filename": "neon-minerva-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "13922e4ce0e06b6c7cc621e560ac29c5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 20741,
"upload_time": "2024-03-01T23:21:06",
"upload_time_iso_8601": "2024-03-01T23:21:06.348077Z",
"url": "https://files.pythonhosted.org/packages/11/be/fe28fc09cffa5dcdc4ee77c6720152c21a114181e0917e1c74288bc0f856/neon-minerva-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-01 23:21:06",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "neongeckocom",
"github_project": "neon-minerva",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "neon-minerva"
}
JSON
