# Smartloop NLP framework
Natural language processing framework
# Train a bot
Use the `sample.json` file in the `\data` folder, you will pass the name of bot as an argument in the next step.
Below is as training JSON sample containing the pattern and name of the intent that wil be resolved for a user input.
```json
{
"examples": {
"intents": [
{
"text": "about",
"intent": "about"
},
{
"text": "company",
"intent": "about"
},
{
"text": "what is smartloop",
"intent": "about"
},
{
"text": "start",
"intent": "start"
},
{
"text": "menu",
"intent": "start"
},
{
"text": "hi",
"intent": "start"
}
]
},
"lang": "en"
}
```
From the command line type the following to train the bot:
```
python main.py train -i sample
```
Testing the bot
To test the type the following command:
```
python main.py parse -i sample -t "I need a chabot"
```
This should return the intent name followed by the confidence level
```
{
"topIntent": {
"intent": "i-need-chatbot",
"confidence": 0.9999436140060425
},
"intents": [
{
"intent": "i-need-chatbot",
"confidence": 0.9999436140060425
},
{
"intent": "chatter-good-afternoon",
"confidence": 4.835660001845099e-05
},
{
"intent": "bizbot-no-way",
"confidence": 3.6056665067008e-06
},
{
"intent": "about-chatbot",
"confidence": 1.9573460576793877e-06
},
{
"intent": "contact",
"confidence": 1.095663265004987e-06
}
]
}
```
## Tunning your model (Advanced)
It is possible to override the default training parameters to create a model that fits your need, override `config.yaml` to tune your model:
```yaml
# number of epochs
epochs: 100
# Use tensorboard callback
logs: True
# classifier parameters
embedded_intent_classifier:
# base neurons, this will be increased based on the intent size
neurons: 16
# length of input len("hello how are you") = 4
input_length: 100
learning_rate: 1e-2
flatten: False
hidden_layers: 2
# drop rate to avoid overfitting
drop_rate: 0.2
# early stop training in case of not improving
early_stopping: True
```
This can vary based on model size, can be tuned using the grid search capabablites to find the optimal settings.
Here is a list of basic parameters and their meaning:
* epochs - This is the number of iterations where 1 epoch = 1 complete neural net cycle
* learning_rate - How fast or slow, the model is learning through iterations
* drop_rate - Adjust to prevent overfitting of the data to fine tune your model
## Configuration
Install stop words dictionary using following command
```
python -m nltk.downloader stopwords
```
## Debugging
Set `logs:True` in config.yaml to enable debugging using `tensorboard`. Once you have trained the bot. Type the following command to start tensorboard:
```commandline
tensorboard serve --logdir logs/nlp_data/<bot_id>/<model_id>
```
## Supported Language
* English (en)
* Spanish (es)
* German (de)
## Requirements
* Tensorflow (>=2.12.0)
## License
[](https://opensource.org/licenses/mit)
See [LICENSE](LICENSE) for full details.
```text
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
```
Raw data
{
"_id": null,
"home_page": "https://github.com/SmartloopAI/sl-core",
"name": "smartloop-core",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "NLP,framework,tensorflow,smartloop",
"author": "Smartloop Inc.",
"author_email": "mehfuz@smartloop.ai",
"download_url": "https://files.pythonhosted.org/packages/3a/1e/d1b56f95f01270024909de98e0ebec60d5f5cc00626ba69730fa4eab3a12/smartloop-core-1.0.2.tar.gz",
"platform": null,
"description": "# Smartloop NLP framework\nNatural language processing framework\n\n# Train a bot\n\nUse the `sample.json` file in the `\\data` folder, you will pass the name of bot as an argument in the next step. \n\nBelow is as training JSON sample containing the pattern and name of the intent that wil be resolved for a user input.\n\n```json\n{\n \"examples\": {\n \"intents\": [\n {\n \"text\": \"about\",\n \"intent\": \"about\"\n },\n {\n \"text\": \"company\",\n \"intent\": \"about\"\n },\n {\n \"text\": \"what is smartloop\",\n \"intent\": \"about\"\n },\n {\n \"text\": \"start\",\n \"intent\": \"start\"\n },\n {\n \"text\": \"menu\",\n \"intent\": \"start\"\n },\n {\n \"text\": \"hi\",\n \"intent\": \"start\"\n }\n ]\n },\n \"lang\": \"en\"\n}\n```\n\nFrom the command line type the following to train the bot:\n\n```\npython main.py train -i sample\n\n```\n\nTesting the bot\n\nTo test the type the following command:\n\n```\npython main.py parse -i sample -t \"I need a chabot\"\n```\n\nThis should return the intent name followed by the confidence level\n\n```\n{\n \"topIntent\": {\n \"intent\": \"i-need-chatbot\",\n \"confidence\": 0.9999436140060425\n },\n \"intents\": [\n {\n \"intent\": \"i-need-chatbot\",\n \"confidence\": 0.9999436140060425\n },\n {\n \"intent\": \"chatter-good-afternoon\",\n \"confidence\": 4.835660001845099e-05\n },\n {\n \"intent\": \"bizbot-no-way\",\n \"confidence\": 3.6056665067008e-06\n },\n {\n \"intent\": \"about-chatbot\",\n \"confidence\": 1.9573460576793877e-06\n },\n {\n \"intent\": \"contact\",\n \"confidence\": 1.095663265004987e-06\n }\n ]\n}\n```\n\n## Tunning your model (Advanced)\n\nIt is possible to override the default training parameters to create a model that fits your need, override `config.yaml` to tune your model:\n\n```yaml\n# number of epochs\nepochs: 100\n\n# Use tensorboard callback\nlogs: True\n\n# classifier parameters\nembedded_intent_classifier:\n # base neurons, this will be increased based on the intent size\n neurons: 16\n # length of input len(\"hello how are you\") = 4\n input_length: 100\n learning_rate: 1e-2\n flatten: False\n hidden_layers: 2\n # drop rate to avoid overfitting\n drop_rate: 0.2\n # early stop training in case of not improving\n early_stopping: True\n```\n\nThis can vary based on model size, can be tuned using the grid search capabablites to find the optimal settings. \n\nHere is a list of basic parameters and their meaning:\n\n* epochs - This is the number of iterations where 1 epoch = 1 complete neural net cycle\n* learning_rate - How fast or slow, the model is learning through iterations\n* drop_rate - Adjust to prevent overfitting of the data to fine tune your model\n\n\n## Configuration\n\nInstall stop words dictionary using following command\n\n```\npython -m nltk.downloader stopwords \n```\n\n## Debugging\n\nSet `logs:True` in config.yaml to enable debugging using `tensorboard`. Once you have trained the bot. Type the following command to start tensorboard:\n\n```commandline\ntensorboard serve --logdir logs/nlp_data/<bot_id>/<model_id>\n```\n\n## Supported Language\n\n* English (en)\n* Spanish (es)\n* German (de)\n\n## Requirements\n\n* Tensorflow (>=2.12.0)\n\n## License\n\n[](https://opensource.org/licenses/mit)\n\nSee [LICENSE](LICENSE) for full details.\n\n```text\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n```\n",
"bugtrack_url": null,
"license": "LICENSE.txt",
"summary": "Natural language processing framework for text processing",
"version": "1.0.2",
"project_urls": {
"Download": "https://github.com/SmartloopAI/sl-core/archive/refs/tags/1.0.1.tar.gz",
"Homepage": "https://github.com/SmartloopAI/sl-core"
},
"split_keywords": [
"nlp",
"framework",
"tensorflow",
"smartloop"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3a1ed1b56f95f01270024909de98e0ebec60d5f5cc00626ba69730fa4eab3a12",
"md5": "0bd6ced3d11805a9c13c45d832c1d795",
"sha256": "232b0388cc947bf3fe5b8736d97f1902642e547ad140421abe2277aa37757996"
},
"downloads": -1,
"filename": "smartloop-core-1.0.2.tar.gz",
"has_sig": false,
"md5_digest": "0bd6ced3d11805a9c13c45d832c1d795",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 13945,
"upload_time": "2023-07-28T21:36:30",
"upload_time_iso_8601": "2023-07-28T21:36:30.033976Z",
"url": "https://files.pythonhosted.org/packages/3a/1e/d1b56f95f01270024909de98e0ebec60d5f5cc00626ba69730fa4eab3a12/smartloop-core-1.0.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-28 21:36:30",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "SmartloopAI",
"github_project": "sl-core",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "smartloop-core"
}
JSON
