MistralAI Questionnaire
=================================
This project provides a toolkit for generating questionnaire from documents: [``txt``, ``docx``, ``pdf``] to ``.csv`` dataset format.
Requirements
------------
Before starting, you need to install the following libraries:
.. code-block:: python
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
- ``langchain``
- ``langchain_community``
- ``langchain-huggingface``
- ``playwright``
- ``html2text``
- ``sentence_transformers``
- ``faiss-cpu``
- ``pandas``
- ``peft==0.4.0``
- ``trl==0.4.7``
- ``pypdf``
- ``bitsandbytes``
- ``accelerate``
Description
-----------
ModelManager
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This class is responsible for loading mistralai model and generating QA.
Constructor
^^^^^^^^^^^
.. code-block:: python
__init__(self, model_name)
- **model_name**: The path or name of the pre-trained model.
Methods
^^^^^^^
- **setup_tokenizer()**: Loads and configures the tokenizer for the model.
- **setup_bitsandbytes_parameters()**: Configures parameters for bit quantization (BitsAndBytes).
- **from_pretrained()**: Loads the model with pre-trained weights and quantization configuration.
- **print_model_parameters(examples)**: Prints the number of trainable and total parameters of the model.
- **__call__(self, *args, **kwargs)**: The main method for running the generate tasks.
Usage
-----
To start generating QA, you should create an instance of the ``ModelManager`` class and call its ``__call__`` method, passing the necessary arguments.
.. code-block:: python
from questionnaire_mistral.models import ModelManager
model = ModelManager(model_name="path_to_model")
model(document=document, task=task, document_content=document_content, task_count=task_count)
License
-------
The project is distributed under the MIT License.
Raw data
{
"_id": null,
"home_page": "https://github.com/skillfi/questionnairemistral",
"name": "questionnaire-mistral",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": null,
"author": "Alex",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/29/6a/365fc60ea532448fa17dbe0ad3d370b3ba22ecda708e9047bcd6412126cc/questionnaire_mistral-3.1.tar.gz",
"platform": null,
"description": "MistralAI Questionnaire\n=================================\n\nThis project provides a toolkit for generating questionnaire from documents: [``txt``, ``docx``, ``pdf``] to ``.csv`` dataset format.\n\nRequirements\n------------\n\nBefore starting, you need to install the following libraries:\n .. code-block:: python\n\n pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124\n\n- ``langchain``\n- ``langchain_community``\n- ``langchain-huggingface``\n- ``playwright``\n- ``html2text``\n- ``sentence_transformers``\n- ``faiss-cpu``\n- ``pandas``\n- ``peft==0.4.0``\n- ``trl==0.4.7``\n- ``pypdf``\n- ``bitsandbytes``\n- ``accelerate``\n\nDescription\n-----------\n\nModelManager\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nThis class is responsible for loading mistralai model and generating QA.\n\nConstructor\n^^^^^^^^^^^\n\n.. code-block:: python\n\n __init__(self, model_name)\n\n- **model_name**: The path or name of the pre-trained model.\n\n\nMethods\n^^^^^^^\n\n- **setup_tokenizer()**: Loads and configures the tokenizer for the model.\n- **setup_bitsandbytes_parameters()**: Configures parameters for bit quantization (BitsAndBytes).\n- **from_pretrained()**: Loads the model with pre-trained weights and quantization configuration.\n- **print_model_parameters(examples)**: Prints the number of trainable and total parameters of the model.\n- **__call__(self, *args, **kwargs)**: The main method for running the generate tasks.\n\nUsage\n-----\n\nTo start generating QA, you should create an instance of the ``ModelManager`` class and call its ``__call__`` method, passing the necessary arguments.\n\n.. code-block:: python\n from questionnaire_mistral.models import ModelManager\n model = ModelManager(model_name=\"path_to_model\")\n model(document=document, task=task, document_content=document_content, task_count=task_count)\n\nLicense\n-------\n\nThe project is distributed under the MIT License.\n",
"bugtrack_url": null,
"license": "Apache 2.0 License",
"summary": null,
"version": "3.1",
"project_urls": {
"Homepage": "https://github.com/skillfi/questionnairemistral"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2137188cfc2c3b3523be5e6ecd2819cf8b30d53a5ba9a6d701edebccab819970",
"md5": "4a208c1df22e4ea32152063bad21d714",
"sha256": "946e42b63053bdc0e6f070631addad81b0e42e78f011dc0072e24d24c22d1c04"
},
"downloads": -1,
"filename": "questionnaire_mistral-3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4a208c1df22e4ea32152063bad21d714",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 9524,
"upload_time": "2024-10-07T13:43:09",
"upload_time_iso_8601": "2024-10-07T13:43:09.874768Z",
"url": "https://files.pythonhosted.org/packages/21/37/188cfc2c3b3523be5e6ecd2819cf8b30d53a5ba9a6d701edebccab819970/questionnaire_mistral-3.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "296a365fc60ea532448fa17dbe0ad3d370b3ba22ecda708e9047bcd6412126cc",
"md5": "c1ca963266847f6cf9d8a567803788ba",
"sha256": "f7a681a17bb83dbe8e4faae8232c3179614226ff76030861ce2c5ddf458d5599"
},
"downloads": -1,
"filename": "questionnaire_mistral-3.1.tar.gz",
"has_sig": false,
"md5_digest": "c1ca963266847f6cf9d8a567803788ba",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 9441,
"upload_time": "2024-10-07T13:43:11",
"upload_time_iso_8601": "2024-10-07T13:43:11.852779Z",
"url": "https://files.pythonhosted.org/packages/29/6a/365fc60ea532448fa17dbe0ad3d370b3ba22ecda708e9047bcd6412126cc/questionnaire_mistral-3.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-07 13:43:11",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "skillfi",
"github_project": "questionnairemistral",
"github_not_found": true,
"lcname": "questionnaire-mistral"
}
JSON
