ngram-ml

Name	ngram-ml JSON
Version	0.1.0 JSON
	download
home_page	https://github.com/anil-gurbuz/ngram_ml
Summary	Basic python package for creating n-gram language models from text files
upload_time	2023-03-16 22:29:18
maintainer
docs_url	None
author	Anil Gurbuz
requires_python
license	MIT
keywords	nlp ngram mle simple language model neural network
VCS
bugtrack_url
requirements	No requirements were recorded.
Travis-CI	No Travis.
coveralls test coverage	No coveralls.

            # Maximum Likelihood fit for N-grams 
A small library for quickly deriving the Maximum Likelihood estimates and Neural Network training for N-grams.

## Installation
```bash
pip install ngram-ml
```

## Usage
```python
from ngram_ml import *
```

## Example
- Maximum Likelihood Estimator Example
```python
mle = NGramMLEstimator(sentences=tokens, n_grams=2, label_smoothing=1)
mle.calculate_cross_entropy(tokens)
mle.calculate_cross_entropy([['<S>', 'the', 'cat', 'sat', 'on', 'the', 'mat', '</S>']])

mle.generate_sentence(30, initial_pre_seq= tuple([mle.word_to_idx['pencil']]))
mle.generate_most_probable_sentence(30, initial_pre_seq= tuple([mle.word_to_idx['book']]))
```
- Neural Network Example
```python
# Neural Network Example
dataset = NGramDataset(sentences=tokens, n_grams=2)
NN = NGramNeuralNet(n_grams=2, in_size=dataset.n_unique_words, embed_size=200)
NN.train(dataset.x, dataset.y, n_epochs=100, lr=0.01)
```



            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/anil-gurbuz/ngram_ml",
    "name": "ngram-ml",
    "maintainer": "",
    "docs_url": null,
    "requires_python": "",
    "maintainer_email": "",
    "keywords": "NLP,ngram,MLE,Simple Language Model,Neural Network",
    "author": "Anil Gurbuz",
    "author_email": "anlgrbz91@gmail.com",
    "download_url": "https://files.pythonhosted.org/packages/a9/3d/f0af05129c8b41ff6d3692a621327765e36479304bf17d15334c18bb0e3f/ngram_ml-0.1.0.tar.gz",
    "platform": null,
    "description": "# Maximum Likelihood fit for N-grams \nA small library for quickly deriving the Maximum Likelihood estimates and Neural Network training for N-grams.\n\n## Installation\n```bash\npip install ngram-ml\n```\n\n## Usage\n```python\nfrom ngram_ml import *\n```\n\n## Example\n- Maximum Likelihood Estimator Example\n```python\nmle = NGramMLEstimator(sentences=tokens, n_grams=2, label_smoothing=1)\nmle.calculate_cross_entropy(tokens)\nmle.calculate_cross_entropy([['<S>', 'the', 'cat', 'sat', 'on', 'the', 'mat', '</S>']])\n\nmle.generate_sentence(30, initial_pre_seq= tuple([mle.word_to_idx['pencil']]))\nmle.generate_most_probable_sentence(30, initial_pre_seq= tuple([mle.word_to_idx['book']]))\n```\n- Neural Network Example\n```python\n# Neural Network Example\ndataset = NGramDataset(sentences=tokens, n_grams=2)\nNN = NGramNeuralNet(n_grams=2, in_size=dataset.n_unique_words, embed_size=200)\nNN.train(dataset.x, dataset.y, n_epochs=100, lr=0.01)\n```\n\n\n",
    "bugtrack_url": null,
    "license": "MIT",
    "summary": "Basic python package for creating n-gram language models from text files",
    "version": "0.1.0",
    "split_keywords": [
        "nlp",
        "ngram",
        "mle",
        "simple language model",
        "neural network"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "abe4b0f207997dbe1c81508c87da625d807de6373bf4bec28c1f17fb5ee401cb",
                "md5": "76981d96c158cb6cd42d8968c1b531ba",
                "sha256": "4ece709a1220e898542347cefc56a89f1f232954d995cf6ee20ce1e0510f60ba"
            },
            "downloads": -1,
            "filename": "ngram_ml-0.1.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "76981d96c158cb6cd42d8968c1b531ba",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 4054,
            "upload_time": "2023-03-16T22:29:14",
            "upload_time_iso_8601": "2023-03-16T22:29:14.806937Z",
            "url": "https://files.pythonhosted.org/packages/ab/e4/b0f207997dbe1c81508c87da625d807de6373bf4bec28c1f17fb5ee401cb/ngram_ml-0.1.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "a93df0af05129c8b41ff6d3692a621327765e36479304bf17d15334c18bb0e3f",
                "md5": "8a719fde046623dce59542def441b10f",
                "sha256": "3a090be042570016fea27ccb9b3a6a24db1757ada26762592076df2438ee8621"
            },
            "downloads": -1,
            "filename": "ngram_ml-0.1.0.tar.gz",
            "has_sig": false,
            "md5_digest": "8a719fde046623dce59542def441b10f",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 9513,
            "upload_time": "2023-03-16T22:29:18",
            "upload_time_iso_8601": "2023-03-16T22:29:18.421205Z",
            "url": "https://files.pythonhosted.org/packages/a9/3d/f0af05129c8b41ff6d3692a621327765e36479304bf17d15334c18bb0e3f/ngram_ml-0.1.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-03-16 22:29:18",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "github_user": "anil-gurbuz",
    "github_project": "ngram_ml",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "requirements": [],
    "lcname": "ngram-ml"
}