tfIdfInheritVectorizer


NametfIdfInheritVectorizer JSON
Version 0.1 PyPI version JSON
download
home_page
Summary
upload_time2023-02-02 11:35:19
maintainer
docs_urlNone
authorBerke Dilekoglu
requires_python
licenseMIT
keywords machine-learning tf-idf
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # TFIDFVectorizer

TFIDFVectorizer is a custom implementation of the TF-IDF transformation algorithm, using scikit-learn's TfidfVectorizer as a base. The implementation is written in Python, making use of numpy, scikit-learn and other commonly used packages.

The main aim of this implementation is to provide a simple and efficient way of transforming a collection of text documents into a matrix representation, which can then be used as input to various machine learning algorithms.

## Installation

The package can be installed using pip:

```bash
pip install tfIdfInheritVectorizer
```

## Usage

To use the TFIDFVectorizer, simply create an instance of the class and call its fit_transform method. The method takes a list of text documents as input, and returns a sparse matrix representation of the TF-IDF scores for each document.

```python
from tfIdfInheritVectorizer.feature_extraction.vectorizer import TFIDFVectorizer


text_data = [    "This is the first document.",    "This is the second document.",    "And this is the third one.",    "Is this the first document?"]

vectorizer = TFIDFVectorizer()
tfidf = vectorizer.fit_transform(text_data)

```

In addition to the fit_transform method, the TFIDFVectorizer also has a transform method that can be used to transform new text data into a matrix representation, given the model has already been fit to the training data.

```python
new_text_data = [
    "This is a new document.",
    "Is this a new one?"
]

new_tfidf = vectorizer.transform(new_text_data)
```

## Configuration

The TFIDFVectorizer has several parameters that can be configured to customize its behavior. Some of the most important parameters are:

- stop_words: a list of stop words that will be ignored during the tokenization process

```python
vectorizer = TFIDFVectorizer(stop_words=["is", "the", "this"])

```

- max_features: the maximum number of features to keep, based on term frequency across the entire corpus.

```python
vectorizer = TFIDFVectorizer(max_features=50)

```

- use_idf: a flag indicating whether to use the inverse document frequency (IDF) weighting.

```python
vectorizer = TFIDFVectorizer(use_idf=False)

```

For a full list of parameters, see the [scikit-learn documentation](https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html)

## Conclusion

TFIDFVectorizer is a simple and efficient implementation of the TF-IDF transformation algorithm, suitable for use in various machine learning applications. By using scikit-learn as a base, it provides a wide range of customization options and can be easily integrated into existing machine learning workflows.

## License

[MIT](https://choosealicense.com/licenses/mit/)

            

Raw data

            {
    "_id": null,
    "home_page": "",
    "name": "tfIdfInheritVectorizer",
    "maintainer": "",
    "docs_url": null,
    "requires_python": "",
    "maintainer_email": "",
    "keywords": "machine-learning tf-idf",
    "author": "Berke Dilekoglu",
    "author_email": "",
    "download_url": "https://files.pythonhosted.org/packages/fc/80/14a1edd3773972d02da249a71b92262b543f56e064d7ba44b3bf433a6ec0/tfIdfInheritVectorizer-0.1.tar.gz",
    "platform": null,
    "description": "# TFIDFVectorizer\n\nTFIDFVectorizer is a custom implementation of the TF-IDF transformation algorithm, using scikit-learn's TfidfVectorizer as a base. The implementation is written in Python, making use of numpy, scikit-learn and other commonly used packages.\n\nThe main aim of this implementation is to provide a simple and efficient way of transforming a collection of text documents into a matrix representation, which can then be used as input to various machine learning algorithms.\n\n## Installation\n\nThe package can be installed using pip:\n\n```bash\npip install tfIdfInheritVectorizer\n```\n\n## Usage\n\nTo use the TFIDFVectorizer, simply create an instance of the class and call its fit_transform method. The method takes a list of text documents as input, and returns a sparse matrix representation of the TF-IDF scores for each document.\n\n```python\nfrom tfIdfInheritVectorizer.feature_extraction.vectorizer import TFIDFVectorizer\n\n\ntext_data = [    \"This is the first document.\",    \"This is the second document.\",    \"And this is the third one.\",    \"Is this the first document?\"]\n\nvectorizer = TFIDFVectorizer()\ntfidf = vectorizer.fit_transform(text_data)\n\n```\n\nIn addition to the fit_transform method, the TFIDFVectorizer also has a transform method that can be used to transform new text data into a matrix representation, given the model has already been fit to the training data.\n\n```python\nnew_text_data = [\n    \"This is a new document.\",\n    \"Is this a new one?\"\n]\n\nnew_tfidf = vectorizer.transform(new_text_data)\n```\n\n## Configuration\n\nThe TFIDFVectorizer has several parameters that can be configured to customize its behavior. Some of the most important parameters are:\n\n- stop_words: a list of stop words that will be ignored during the tokenization process\n\n```python\nvectorizer = TFIDFVectorizer(stop_words=[\"is\", \"the\", \"this\"])\n\n```\n\n- max_features: the maximum number of features to keep, based on term frequency across the entire corpus.\n\n```python\nvectorizer = TFIDFVectorizer(max_features=50)\n\n```\n\n- use_idf: a flag indicating whether to use the inverse document frequency (IDF) weighting.\n\n```python\nvectorizer = TFIDFVectorizer(use_idf=False)\n\n```\n\nFor a full list of parameters, see the [scikit-learn documentation](https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html)\n\n## Conclusion\n\nTFIDFVectorizer is a simple and efficient implementation of the TF-IDF transformation algorithm, suitable for use in various machine learning applications. By using scikit-learn as a base, it provides a wide range of customization options and can be easily integrated into existing machine learning workflows.\n\n## License\n\n[MIT](https://choosealicense.com/licenses/mit/)\n",
    "bugtrack_url": null,
    "license": "MIT",
    "summary": "",
    "version": "0.1",
    "split_keywords": [
        "machine-learning",
        "tf-idf"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "fc8014a1edd3773972d02da249a71b92262b543f56e064d7ba44b3bf433a6ec0",
                "md5": "f0f15a5e868dba3c0f7412899439020d",
                "sha256": "5466c939bc6b2471100fcb18aedaf5712653c800d4844ee8eb05dd81630f46fd"
            },
            "downloads": -1,
            "filename": "tfIdfInheritVectorizer-0.1.tar.gz",
            "has_sig": false,
            "md5_digest": "f0f15a5e868dba3c0f7412899439020d",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 4475,
            "upload_time": "2023-02-02T11:35:19",
            "upload_time_iso_8601": "2023-02-02T11:35:19.632389Z",
            "url": "https://files.pythonhosted.org/packages/fc/80/14a1edd3773972d02da249a71b92262b543f56e064d7ba44b3bf433a6ec0/tfIdfInheritVectorizer-0.1.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-02-02 11:35:19",
    "github": false,
    "gitlab": false,
    "bitbucket": false,
    "lcname": "tfidfinheritvectorizer"
}
        
Elapsed time: 0.04110s