bcqa


Namebcqa JSON
Version 1.1.4 PyPI version JSON
download
home_pagehttps://github.com/VenkteshV/BCQA
SummaryA Benchmark for Complex Heterogeneous Question answering
upload_time2024-05-30 23:17:18
maintainerNone
docs_urlNone
authorVenktesh V, Deepali Prabhu
requires_python>=3.8
licenseApache License 2.0
keywords information retrieval transformer networks bert pytorch complex question answering ir nlp deep learning
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # BCQA (Benchmarking Complex QA)

BCQA is a benchmark for a wide range of complex Qa tasks. It also aims to provide a easy to use framework for evaluating retrieval and reasoning approaches for answering complex multi-hop questions.


# Setup
1) Clone the repo <br />
2) Create a conda environment conda create -n bcqa  <br />
3) pip install -e .<br />

# Running Evaluation
The evaluation scripts for retreival and LLMs are in the evaluation folder 

For instance to run dpr retreival for Wikimultihopqa run <br/>
python3 evaluation/wikimultihop/run_dpr_inference.py <br />

Before running the above script make sure you have configured the correct paths for the data and corpus files in evaluation/config.ini <br />

Example: 
wikimultihopqa = /home/bcqa/BCQA/2wikimultihopQA <br />
wikimultihopqa-corpus = /home/bcqa/BCQA/wiki_musique_corpus.json <br />


## Coding Practices

### Auto-formatting code
1. Install `black`: ```pip install black``` or ```conda install black```
2. In your IDE: Enable formatting on save.
3. Install `isort`: ```pip install isort``` or ```conda install isort```
4. In your IDE: Enable sorting import on save.

In VS Code, you can do this using the following config:
```json
{
    "editor.formatOnSave": true,
    "editor.codeActionsOnSave": {
        "source.organizeImports": true
    }
}
```

### Type hints
Use [type hints](https://docs.python.org/3/library/typing.html) for __everything__! No exceptions.

### Docstrings
Write a docstring for __every__ function (except the main function). We use the [Google format](https://github.com/NilsJPWerner/autoDocstring/blob/HEAD/docs/google.md). In VS Code, you can use [autoDocstring](https://marketplace.visualstudio.com/items?itemName=njpwerner.autodocstring).

### Example
```python
def sum(a: float, b: float) -> float:
    """Compute the sum of a and b.

    Args:
        a (float): First number.
        b (float): Second number.
    
    Returns:
        float: The sum of a and b.
    """

    return a + b
```

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/VenkteshV/BCQA",
    "name": "bcqa",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.8",
    "maintainer_email": null,
    "keywords": "Information Retrieval Transformer Networks BERT PyTorch Complex Question Answering IR NLP deep learning",
    "author": "Venktesh V, Deepali Prabhu",
    "author_email": "venkyviswa12@gmail.com",
    "download_url": "https://files.pythonhosted.org/packages/f6/36/0e745920ef0437a5e4a2b3ce79968fea448ffa6cd43dc5f327464c08f512/bcqa-1.1.4.tar.gz",
    "platform": null,
    "description": "# BCQA (Benchmarking Complex QA)\n\nBCQA is a benchmark for a wide range of complex Qa tasks. It also aims to provide a easy to use framework for evaluating retrieval and reasoning approaches for answering complex multi-hop questions.\n\n\n# Setup\n1) Clone the repo <br />\n2) Create a conda environment conda create -n bcqa  <br />\n3) pip install -e .<br />\n\n# Running Evaluation\nThe evaluation scripts for retreival and LLMs are in the evaluation folder \n\nFor instance to run dpr retreival for Wikimultihopqa run <br/>\npython3 evaluation/wikimultihop/run_dpr_inference.py <br />\n\nBefore running the above script make sure you have configured the correct paths for the data and corpus files in evaluation/config.ini <br />\n\nExample: \nwikimultihopqa = /home/bcqa/BCQA/2wikimultihopQA <br />\nwikimultihopqa-corpus = /home/bcqa/BCQA/wiki_musique_corpus.json <br />\n\n\n## Coding Practices\n\n### Auto-formatting code\n1. Install `black`: ```pip install black``` or ```conda install black```\n2. In your IDE: Enable formatting on save.\n3. Install `isort`: ```pip install isort``` or ```conda install isort```\n4. In your IDE: Enable sorting import on save.\n\nIn VS Code, you can do this using the following config:\n```json\n{\n    \"editor.formatOnSave\": true,\n    \"editor.codeActionsOnSave\": {\n        \"source.organizeImports\": true\n    }\n}\n```\n\n### Type hints\nUse [type hints](https://docs.python.org/3/library/typing.html) for __everything__! No exceptions.\n\n### Docstrings\nWrite a docstring for __every__ function (except the main function). We use the [Google format](https://github.com/NilsJPWerner/autoDocstring/blob/HEAD/docs/google.md). In VS Code, you can use [autoDocstring](https://marketplace.visualstudio.com/items?itemName=njpwerner.autodocstring).\n\n### Example\n```python\ndef sum(a: float, b: float) -> float:\n    \"\"\"Compute the sum of a and b.\n\n    Args:\n        a (float): First number.\n        b (float): Second number.\n    \n    Returns:\n        float: The sum of a and b.\n    \"\"\"\n\n    return a + b\n```\n",
    "bugtrack_url": null,
    "license": "Apache License 2.0",
    "summary": "A Benchmark for Complex Heterogeneous Question answering",
    "version": "1.1.4",
    "project_urls": {
        "Homepage": "https://github.com/VenkteshV/BCQA"
    },
    "split_keywords": [
        "information",
        "retrieval",
        "transformer",
        "networks",
        "bert",
        "pytorch",
        "complex",
        "question",
        "answering",
        "ir",
        "nlp",
        "deep",
        "learning"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "78d03e83b61fc1ccda5c21998a28c3b27dd8a231ef39bec504ea26194b322c0e",
                "md5": "9880934a253fd791a5555815821a663c",
                "sha256": "25a03667978202e2a7eca49bd0dd07444f55274566fe10c05d3c902ca40498ce"
            },
            "downloads": -1,
            "filename": "bcqa-1.1.4-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "9880934a253fd791a5555815821a663c",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.8",
            "size": 144660,
            "upload_time": "2024-05-30T23:17:16",
            "upload_time_iso_8601": "2024-05-30T23:17:16.353321Z",
            "url": "https://files.pythonhosted.org/packages/78/d0/3e83b61fc1ccda5c21998a28c3b27dd8a231ef39bec504ea26194b322c0e/bcqa-1.1.4-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "f6360e745920ef0437a5e4a2b3ce79968fea448ffa6cd43dc5f327464c08f512",
                "md5": "af7043230247a9ba84ef16b4811b7ed1",
                "sha256": "d31652e2651c600e2c7f89041bb8ab7938828be3841845f030e157116a525a5b"
            },
            "downloads": -1,
            "filename": "bcqa-1.1.4.tar.gz",
            "has_sig": false,
            "md5_digest": "af7043230247a9ba84ef16b4811b7ed1",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.8",
            "size": 92612,
            "upload_time": "2024-05-30T23:17:18",
            "upload_time_iso_8601": "2024-05-30T23:17:18.048835Z",
            "url": "https://files.pythonhosted.org/packages/f6/36/0e745920ef0437a5e4a2b3ce79968fea448ffa6cd43dc5f327464c08f512/bcqa-1.1.4.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-05-30 23:17:18",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "VenkteshV",
    "github_project": "BCQA",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "lcname": "bcqa"
}
        
Elapsed time: 0.25722s