<p align="center">
<img src="https://raw.githubusercontent.com/RAravindDS/CharLLMs/main/images/charllms.webp" height=300>
<br/>
<br/>
</p>
# **Character-Level Language Models Repo 🕺🏽**
This repository contains multiple character-level language models (charLLM). Each language model is designed to generate text at the character level, providing a granular level of control and flexibility.
## 🌟 Available Language Models
- **Character-Level MLP LLM (<i>First MLP LLM</i>)**
- **GPT-2 (under process)**
## Character-Level MLP
The Character-Level MLP language model is implemented based on the approach described in the paper "[A Neural Probabilistic Language Model](https://www.jmlr.org/papers/volume3/bengio03a/bengio03a.pdf)" by Bential et al. (2002).
It utilizes a multilayer perceptron architecture to generate text at the character level.
## Installation
### With PIP
This repository is tested on Python 3.8+, and PyTorch 2.0.0+.
First, create a **virtual environment** with the version of Python you're going to use and activate it.
Then, you will need to install **PyTorch**.
When backends has been installed, CharLLMs can be installed using pip as follows:
```python
pip install charLLM
```
### With GIT
CharLLMs can be installed using conda as follows:
```zsh
git clone https://github.com/RAravindDS/Neural-Probabilistic-Language-Model.git
```
### Quick Tour
To use the Character-Level MLP language model, follow these steps:
1. Install the package dependencies.
2. Import the `CharMLP` class from the `charLLM` module.
3. Create an instance of the `CharMLP` class.
4. Train the model on a suitable dataset.
5. Generate text using the trained model.
**Demo for NPLM** (A Neural Probabilistic Language Model)
```python
# Import the class
>>> from charLLM import NPLM # Neural Probabilistic Language Model
>>> text_path = "path-to-text-file.txt"
>>> model_parameters = {
"block_size" :3,
"train_size" :0.8,
'epochs' :10000,
'batch_size' :32,
'hidden_layer' :100,
'embedding_dimension' :50,
'learning_rate' :0.1
}
>>> obj = NPLM(text_path, model_parameters) # Initialize the class
>>> obj.train_model()
## It outputs the val_loss and image
>>> obj.sampling(words_needed=10) #It samples 10 tokens.
```
**Model Output Graph**
<img src="https://raw.githubusercontent.com/RAravindDS/CharLLMs/main/images/nplm_plt.png" height=350>
Feel free to explore the repository and experiment with the different language models provided.
## Contributions
Contributions to this repository are welcome. If you have implemented a novel character-level language model or would like to enhance the existing models, please consider contributing to the project.
## License
This repository is licensed under the [MIT License](https://raw.githubusercontent.com/RAravindDS/CharLLMs/main/LICENCE).
Raw data
{
"_id": null,
"home_page": "https://github.com/RAravindDS/CharLLMs",
"name": "charLLM",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "",
"author": "Aravind",
"author_email": "aravindan22052001@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/65/3f/634043208a51154d56ac5dbbea0ca76d809083104808f548ae974c0076fa/charLLM-0.0.7.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n <img src=\"https://raw.githubusercontent.com/RAravindDS/CharLLMs/main/images/charllms.webp\" height=300>\n <br/>\n <br/>\n</p>\n\n\n\n\n# **Character-Level Language Models Repo \ud83d\udd7a\ud83c\udffd**\n\nThis repository contains multiple character-level language models (charLLM). Each language model is designed to generate text at the character level, providing a granular level of control and flexibility.\n\n\n## \ud83c\udf1f Available Language Models \n\n- **Character-Level MLP LLM (<i>First MLP LLM</i>)**\n- **GPT-2 (under process)**\n\n## Character-Level MLP\n\nThe Character-Level MLP language model is implemented based on the approach described in the paper \"[A Neural Probabilistic Language Model](https://www.jmlr.org/papers/volume3/bengio03a/bengio03a.pdf)\" by Bential et al. (2002). \nIt utilizes a multilayer perceptron architecture to generate text at the character level.\n\n## Installation\n\n### With PIP\n\nThis repository is tested on Python 3.8+, and PyTorch 2.0.0+.\n\nFirst, create a **virtual environment** with the version of Python you're going to use and activate it.\n\nThen, you will need to install **PyTorch**.\n\nWhen backends has been installed, CharLLMs can be installed using pip as follows:\n\n```python\npip install charLLM\n```\n### With GIT \n\nCharLLMs can be installed using conda as follows:\n\n```zsh\ngit clone https://github.com/RAravindDS/Neural-Probabilistic-Language-Model.git\n```\n\n\n### Quick Tour\n\n\nTo use the Character-Level MLP language model, follow these steps:\n\n1. Install the package dependencies.\n2. Import the `CharMLP` class from the `charLLM` module.\n3. Create an instance of the `CharMLP` class.\n4. Train the model on a suitable dataset.\n5. Generate text using the trained model.\n\n**Demo for NPLM** (A Neural Probabilistic Language Model)\n```python \n# Import the class \n>>> from charLLM import NPLM # Neural Probabilistic Language Model\n>>> text_path = \"path-to-text-file.txt\" \n>>> model_parameters = {\n \"block_size\" :3, \n \"train_size\" :0.8, \n 'epochs' :10000, \n 'batch_size' :32, \n 'hidden_layer' :100, \n 'embedding_dimension' :50,\n 'learning_rate' :0.1 \n }\n>>> obj = NPLM(text_path, model_parameters) # Initialize the class \n>>> obj.train_model() \n## It outputs the val_loss and image \n>>> obj.sampling(words_needed=10) #It samples 10 tokens. \n```\n\n\n**Model Output Graph**\n\n\n<img src=\"https://raw.githubusercontent.com/RAravindDS/CharLLMs/main/images/nplm_plt.png\" height=350>\n\n\nFeel free to explore the repository and experiment with the different language models provided.\n\n## Contributions\n\nContributions to this repository are welcome. If you have implemented a novel character-level language model or would like to enhance the existing models, please consider contributing to the project.\n\n## License\n\nThis repository is licensed under the [MIT License](https://raw.githubusercontent.com/RAravindDS/CharLLMs/main/LICENCE).\n",
"bugtrack_url": null,
"license": "",
"summary": "Character Level Language Models \ud83d\udd7a\ud83c\udffd",
"version": "0.0.7",
"project_urls": {
"Bug Tracker": "https://github.com/RAravindDS/CharLLMs/issues",
"Homepage": "https://github.com/RAravindDS/CharLLMs"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "920512e91fd03360e22bbb11fe76332b3b8057a15ab593059554a2dba346e0c1",
"md5": "621f0eb3ea0d38a60e99a10dd9a48a24",
"sha256": "d40bc4f0d2a4714fcf4091bcb1d6a32fa38077f7e32d0892f8982a4350f17241"
},
"downloads": -1,
"filename": "charLLM-0.0.7-py3-none-any.whl",
"has_sig": false,
"md5_digest": "621f0eb3ea0d38a60e99a10dd9a48a24",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 8807,
"upload_time": "2023-06-09T14:45:36",
"upload_time_iso_8601": "2023-06-09T14:45:36.187852Z",
"url": "https://files.pythonhosted.org/packages/92/05/12e91fd03360e22bbb11fe76332b3b8057a15ab593059554a2dba346e0c1/charLLM-0.0.7-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "653f634043208a51154d56ac5dbbea0ca76d809083104808f548ae974c0076fa",
"md5": "7b10c116f65427f5d15afc4e0dd19531",
"sha256": "54766a6f4b5907e224abc3692e6f3d5cee865818f0e6d49bff24aa2b32a3efe8"
},
"downloads": -1,
"filename": "charLLM-0.0.7.tar.gz",
"has_sig": false,
"md5_digest": "7b10c116f65427f5d15afc4e0dd19531",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 8408,
"upload_time": "2023-06-09T14:45:37",
"upload_time_iso_8601": "2023-06-09T14:45:37.862540Z",
"url": "https://files.pythonhosted.org/packages/65/3f/634043208a51154d56ac5dbbea0ca76d809083104808f548ae974c0076fa/charLLM-0.0.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-09 14:45:37",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "RAravindDS",
"github_project": "CharLLMs",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "torch",
"specs": []
},
{
"name": "numpy",
"specs": []
},
{
"name": "matplotlib",
"specs": []
},
{
"name": "tqdm",
"specs": []
}
],
"lcname": "charllm"
}
JSON
