[](https://github.com/pre-commit/pre-commit)
[](https://experimaestro-ir.readthedocs.io/en/latest/?badge=latest)
[](https://badge.fury.io/py/experimaestro-ir)
# Information Retrieval for experimaestro
Information Retrieval module for [experimaestro](https://experimaestro-python.readthedocs.io/)
The full documentation can be read at [IR@experimaestro](https://experimaestro-ir.readthedocs.io/).
You can find experiments built on top of XPMIR on the [xpmir github workspace](https://github.com/orgs/xpmir/repositories).
Finally, you can find the [roadmap](https://github.com/experimaestro/experimaestro-ir/issues/9).
## Install
Base experimaestro-IR can be installed with `pip install xpmir`.
Functionalities can be added by installing optional dependencies:
- `pip install xpmir[neural]` to install neural-IR packages (torch, etc.)
- `pip install xpmir[anserini]` to install Anserini related packages
For the development version, you can:
- If you just want the development version: install with `pip install git+https://github.com/experimaestro/experimaestro-ir.git`
- If you want to edit the code: clone and then do a `pip install -e .` within the directory
## What's inside?
- Collection management (using datamaestro)
- Interface for the [IR datasets library](https://ir-datasets.com/)
- Splitting IR datasets
- Shuffling training triplets
- Representation
- Word Embeddings
- HuggingFace transformers
- Indices
- dense: [FAISS](https://github.com/facebookresearch/faiss) interface
- sparse: [xpmir-rust library](https://github.com/experimaestro/experimaestro-ir-rust)
- Standard Indexing and Retrieval
- Anserini
- Learning to Rank
- Pointwise
- Pairwise
- Distillation
- Neural IR
- Cross-Encoder
- Splade
- DRMM
- ColBERT
- Paper reproduction:
- *MonoBERT* (Passage Re-ranking with BERT. Rodrigo Nogueira and Kyunghyun Cho. 2019)
- (alpha) *DuoBERT* (Multi-Stage Document Ranking with BERT. Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, Jimmy Lin. 2019)
- (beta) *Splade v2* (SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval, Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and Stéphane Clinchant. SIGIR 2021)
- (planned) ANCE
- Pre-trained models
- [HuggingFace](https://huggingface.co) [integration](https://experimaestro-ir.readthedocs.io/en/latest/pretrained.html) (direct, through the Sentence Transformers library)
## Thanks
Some parts of the code have been adapted from [OpenNIR](https://github.com/Georgetown-IR-Lab/OpenNIR)
Raw data
{
"_id": null,
"home_page": "",
"name": "experimaestro-ir",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "neural information retrieval,information retrieval,experiments",
"author": "",
"author_email": "Benjamin Piwowarski <benjamin@piwowarski.fr>",
"download_url": "https://files.pythonhosted.org/packages/c6/eb/24b6d4f0eb12cd66aad487b139047dfe737dbfbdce92317a90780545139d/experimaestro-ir-1.2.0.tar.gz",
"platform": null,
"description": "[](https://github.com/pre-commit/pre-commit)\n[](https://experimaestro-ir.readthedocs.io/en/latest/?badge=latest)\n[](https://badge.fury.io/py/experimaestro-ir)\n\n# Information Retrieval for experimaestro\n\nInformation Retrieval module for [experimaestro](https://experimaestro-python.readthedocs.io/)\n\nThe full documentation can be read at [IR@experimaestro](https://experimaestro-ir.readthedocs.io/).\n\nYou can find experiments built on top of XPMIR on the [xpmir github workspace](https://github.com/orgs/xpmir/repositories).\n\nFinally, you can find the [roadmap](https://github.com/experimaestro/experimaestro-ir/issues/9).\n\n## Install\n\nBase experimaestro-IR can be installed with `pip install xpmir`.\nFunctionalities can be added by installing optional dependencies:\n\n- `pip install xpmir[neural]` to install neural-IR packages (torch, etc.)\n- `pip install xpmir[anserini]` to install Anserini related packages\n\nFor the development version, you can:\n\n- If you just want the development version: install with `pip install git+https://github.com/experimaestro/experimaestro-ir.git`\n- If you want to edit the code: clone and then do a `pip install -e .` within the directory\n\n## What's inside?\n\n- Collection management (using datamaestro)\n - Interface for the [IR datasets library](https://ir-datasets.com/)\n - Splitting IR datasets\n - Shuffling training triplets\n- Representation\n - Word Embeddings\n - HuggingFace transformers\n- Indices\n - dense: [FAISS](https://github.com/facebookresearch/faiss) interface\n - sparse: [xpmir-rust library](https://github.com/experimaestro/experimaestro-ir-rust)\n- Standard Indexing and Retrieval\n - Anserini\n- Learning to Rank\n - Pointwise\n - Pairwise\n - Distillation\n- Neural IR\n - Cross-Encoder\n - Splade\n - DRMM\n - ColBERT\n- Paper reproduction:\n - *MonoBERT* (Passage Re-ranking with BERT. Rodrigo Nogueira and Kyunghyun Cho. 2019)\n - (alpha) *DuoBERT* (Multi-Stage Document Ranking with BERT. Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, Jimmy Lin. 2019)\n - (beta) *Splade v2* (SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval, Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and St\u00e9phane Clinchant. SIGIR 2021)\n - (planned) ANCE\n- Pre-trained models\n - [HuggingFace](https://huggingface.co) [integration](https://experimaestro-ir.readthedocs.io/en/latest/pretrained.html) (direct, through the Sentence Transformers library)\n\n## Thanks\n\nSome parts of the code have been adapted from [OpenNIR](https://github.com/Georgetown-IR-Lab/OpenNIR)\n",
"bugtrack_url": null,
"license": "GPL-3",
"summary": "Experimaestro common module for IR experiments",
"version": "1.2.0",
"project_urls": {
"documentation": "https://experimaestro-ir.readthedocs.io/en/latest/",
"homepage": "https://github.com/bpiwowar/experimaestro-ir",
"repository": "https://github.com/bpiwowar/experimaestro-ir"
},
"split_keywords": [
"neural information retrieval",
"information retrieval",
"experiments"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "db2336784a4b410b8c7882ae40186e107f4ec4940e23feed30feba1bfd667bfe",
"md5": "1e270a0921353ea2f772476bc352ff8c",
"sha256": "4027542ce0fb2bb9bf581847bb7c356e1b9322ac20e3a97a63845a615c16bb62"
},
"downloads": -1,
"filename": "experimaestro_ir-1.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1e270a0921353ea2f772476bc352ff8c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 159124,
"upload_time": "2023-12-08T16:45:00",
"upload_time_iso_8601": "2023-12-08T16:45:00.080898Z",
"url": "https://files.pythonhosted.org/packages/db/23/36784a4b410b8c7882ae40186e107f4ec4940e23feed30feba1bfd667bfe/experimaestro_ir-1.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c6eb24b6d4f0eb12cd66aad487b139047dfe737dbfbdce92317a90780545139d",
"md5": "1e356fd50da29afc61e841091061541b",
"sha256": "6ac3999e57288733bf6df024a6d1be790b2ce4c0c908b37491c00308a1a02430"
},
"downloads": -1,
"filename": "experimaestro-ir-1.2.0.tar.gz",
"has_sig": false,
"md5_digest": "1e356fd50da29afc61e841091061541b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 139235,
"upload_time": "2023-12-08T16:45:03",
"upload_time_iso_8601": "2023-12-08T16:45:03.444057Z",
"url": "https://files.pythonhosted.org/packages/c6/eb/24b6d4f0eb12cd66aad487b139047dfe737dbfbdce92317a90780545139d/experimaestro-ir-1.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-12-08 16:45:03",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "bpiwowar",
"github_project": "experimaestro-ir",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"tox": true,
"lcname": "experimaestro-ir"
}
JSON
