# autoFillMaskWithCandy
Under construction! Not ready for use yet! Currently experimenting and planning!
Developed by Elatot (c) 2024
# PyTorch and Transformers
This Python package provides advanced text processing functionalities utilizing PyTorch and Hugging Face's Transformers library. It includes capabilities for setting up a tokenizer and model from Hugging Face, automatically masking differing words in sentences, calculating the probability of candidate words, and more.
# Features
- Initialize tokenizer and model from Hugging Face's Transformer library.
- Automatically mask differing words in a set of input sentences.
- Calculate the probability of a candidate word.
- Show masked inputs and provide scores for candidate words.
- Replace masked words with the most probable candidates.
# Limitations
- May not be compatible with model using <mask> inference key like roBerta LLM. This algorithm uses [CLS] and [SEP] which are compatible with Bert and electra.
# Installation
- To use this package, you need to have Python installed on your system, along with PyTorch and the Transformers library. If you haven't installed these dependencies yet, you can do so using pip:
```bash
pip install torch transformers
pip install autoFillMaskWithCandy==0.0.4
```
# Examples of Usage
```python
from autoFillMaskWithCandy import setTokenModel, show_mask_fill, mask_fill_replaced
# Set the model for tokenizer
model_name = "Your-Model-Name-Here"
setTokenModel(model_name)
# Example sentences
input_sentences = [
"Pasensya heto lng ako, bobo sa pagaral",
"Pasensya hito lng ako, bobo sa pagaral",
"Pasensya heto lng ako, bubo sa pagaral",
"Pasensya hito lng ako, bubo sa pagaral"
]
# Show masked input and scores for candidate words
show_mask_fill(input_sentences)
# Print the sentence with masked words replaced
print(mask_fill_replaced(input_sentences))
```
- Replace "Your-Model-Name-Here" with the model name you want to use from Hugging Face.
Raw data
{
"_id": null,
"home_page": "https://github.com/Elashico/autoFillMaskWithCandy.git",
"name": "autoFillMaskWithCandy",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "python,hugging-face,fill-mask,natural language processing,pretrained langugae model",
"author": "Elatot",
"author_email": "<elashiishii@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/2f/c5/efb787ef29216a97a7663a93cf6a7f4430c847cddf5170f945f0bc6e4231/autoFillMaskWithCandy-0.0.4.tar.gz",
"platform": null,
"description": "# autoFillMaskWithCandy\r\n\r\nUnder construction! Not ready for use yet! Currently experimenting and planning!\r\n\r\nDeveloped by Elatot (c) 2024\r\n\r\n# PyTorch and Transformers\r\n\r\nThis Python package provides advanced text processing functionalities utilizing PyTorch and Hugging Face's Transformers library. It includes capabilities for setting up a tokenizer and model from Hugging Face, automatically masking differing words in sentences, calculating the probability of candidate words, and more.\r\n\r\n# Features\r\n\r\n- Initialize tokenizer and model from Hugging Face's Transformer library.\r\n- Automatically mask differing words in a set of input sentences.\r\n- Calculate the probability of a candidate word.\r\n- Show masked inputs and provide scores for candidate words.\r\n- Replace masked words with the most probable candidates.\r\n\r\n# Limitations\r\n\r\n- May not be compatible with model using <mask> inference key like roBerta LLM. This algorithm uses [CLS] and [SEP] which are compatible with Bert and electra.\r\n\r\n# Installation\r\n- To use this package, you need to have Python installed on your system, along with PyTorch and the Transformers library. If you haven't installed these dependencies yet, you can do so using pip:\r\n\r\n```bash\r\npip install torch transformers\r\npip install autoFillMaskWithCandy==0.0.4\r\n```\r\n\r\n# Examples of Usage\r\n\r\n```python\r\nfrom autoFillMaskWithCandy import setTokenModel, show_mask_fill, mask_fill_replaced\r\n\r\n# Set the model for tokenizer\r\nmodel_name = \"Your-Model-Name-Here\"\r\nsetTokenModel(model_name)\r\n\r\n# Example sentences\r\ninput_sentences = [\r\n \"Pasensya heto lng ako, bobo sa pagaral\",\r\n \"Pasensya hito lng ako, bobo sa pagaral\",\r\n \"Pasensya heto lng ako, bubo sa pagaral\",\r\n \"Pasensya hito lng ako, bubo sa pagaral\"\r\n]\r\n\r\n# Show masked input and scores for candidate words\r\nshow_mask_fill(input_sentences)\r\n\r\n# Print the sentence with masked words replaced\r\nprint(mask_fill_replaced(input_sentences))\r\n\r\n```\r\n- Replace \"Your-Model-Name-Here\" with the model name you want to use from Hugging Face.\r\n\r\n\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Automatically mask sentences from a given input where certain words vary, and fil-mask from given candidates",
"version": "0.0.4",
"project_urls": {
"Homepage": "https://github.com/Elashico/autoFillMaskWithCandy.git"
},
"split_keywords": [
"python",
"hugging-face",
"fill-mask",
"natural language processing",
"pretrained langugae model"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "62bc6c37a22c3e8dc09d584ef3e1dbd53bb41b9801d0dd8ff6b2cb37801c575c",
"md5": "21eec9073e9ba80cc35699154721836b",
"sha256": "f603766bef9dc4d414d0bf343295c23c7c95743d7e45141f9853edce21a9e110"
},
"downloads": -1,
"filename": "autoFillMaskWithCandy-0.0.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "21eec9073e9ba80cc35699154721836b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 5222,
"upload_time": "2024-02-11T08:37:37",
"upload_time_iso_8601": "2024-02-11T08:37:37.107208Z",
"url": "https://files.pythonhosted.org/packages/62/bc/6c37a22c3e8dc09d584ef3e1dbd53bb41b9801d0dd8ff6b2cb37801c575c/autoFillMaskWithCandy-0.0.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "2fc5efb787ef29216a97a7663a93cf6a7f4430c847cddf5170f945f0bc6e4231",
"md5": "c96128515d114d0b22bf3ba7dcc01157",
"sha256": "e0e05e48278815e3ec4e551dcabb271c39bd60cc86e4484c3c46da7bbb948137"
},
"downloads": -1,
"filename": "autoFillMaskWithCandy-0.0.4.tar.gz",
"has_sig": false,
"md5_digest": "c96128515d114d0b22bf3ba7dcc01157",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 4538,
"upload_time": "2024-02-11T08:37:40",
"upload_time_iso_8601": "2024-02-11T08:37:40.089005Z",
"url": "https://files.pythonhosted.org/packages/2f/c5/efb787ef29216a97a7663a93cf6a7f4430c847cddf5170f945f0bc6e4231/autoFillMaskWithCandy-0.0.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-11 08:37:40",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Elashico",
"github_project": "autoFillMaskWithCandy",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "autofillmaskwithcandy"
}
JSON
