<div align="center">
<img height="170x" src="https://ibm.github.io/zshot/img/graph.png" />
<h1>Zshot</h1>
<p>
<strong>Zero and Few shot named entity & relationships recognition</strong>
</p>
<p>
<a href="https://ibm.github.io/zshot/"><img alt="Tutorials" src="https://img.shields.io/badge/docs-tutorials-green" /></a>
<a href="https://pypi.org/project/zshot/"><img src="https://img.shields.io/pypi/v/zshot" /></a>
<a href="https://pypi.org/project/zshot/"><img src="https://img.shields.io/pypi/dm/zshot" /></a>
<a href="https://github.com/IBM/zshot/actions/workflows/python-tests.yml"> <img alt="Build" src="https://github.com/IBM/zshot/actions/workflows/python-tests.yml/badge.svg" /></a>
<a href="https://app.codecov.io/github/ibm/zshot"> <img alt="Build" src="https://codecov.io/github/ibm/zshot/branch/main/graph/badge.svg" /></a>
</p>
</div>
**Documentation**: <a href="https://ibm.github.io/zshot/" target="_blank">https://ibm.github.io/zshot</a>
**Source Code**: <a href="https://github.com/IBM/zshot" target="_blank">https://github.com/IBM/zshot</a>
**Paper**: <a href="https://aclanthology.org/2023.acl-demo.34/" target="_blank">https://aclanthology.org/2023.acl-demo.34/</a>
Zshot is a highly customisable framework for performing Zero and Few shot named entity recognition.
Can be used to perform:
- **Mentions extraction**: Identify globally relevant mentions or mentions relevant for a given domain
- **Wikification**: The task of linking textual mentions to entities in Wikipedia
- **Zero and Few Shot named entity recognition**: using language description perform NER to generalize to unseen domains
- **Zero and Few Shot named relationship recognition**
- **Visualization: Zero-shot NER and RE extraction**
## Requirements
* `Python 3.6+`
* <a href="https://spacy.io/" target="_blank"><code>spacy</code></a> - Zshot rely on <a href="https://spacy.io/" class="external-link" target="_blank">Spacy</a> for pipelining and visualization
* <a href="https://pytorch.org/get-started" target="_blank"><code>torch</code></a> - PyTorch is required to run pytorch models.
* <a href="https://huggingface.co/docs/transformers/index" target="_blank"><code>transformers</code></a> - Required for pre-trained language models.
* <a href="https://huggingface.co/docs/evaluate/index" target="_blank"><code>evaluate</code></a> - Required for evaluation.
* <a href="https://huggingface.co/docs/datasets/index" target="_blank"><code>datasets</code></a> - Required to evaluate over datasets (e.g.: OntoNotes).
### Optional Dependencies
* <a href="https://github.com/flairNLP/flair" target="_blank"><code>flair</code></a> - Required if you want to use Flair mentions extractor and for TARS linker.
* <a href="https://github.com/facebookresearch/BLINK" target="_blank"><code>blink</code></a> - Required if you want to use Blink for linking to Wikipedia pages.
## Installation
<div class="termy">
```console
$ pip install zshot
---> 100%
```
</div>
## Examples
| Example | Notebook |
|:------------------------------:|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Installation and Visualization | [](https://colab.research.google.com/github/IBM/zshot/blob/examples/Zshot%20Example.ipynb) |
| Knowledge Extractor | [](https://colab.research.google.com/github/IBM/zshot/blob/examples/Zshot%20Example%20-%20Knowledge%20Extractor.ipynb) |
| Wikification | [](https://colab.research.google.com/github/IBM/zshot/blob/examples/Zshot%20Example%20-%20Wikification.ipynb) |
| Custom Components | [](https://colab.research.google.com/github/IBM/zshot/blob/examples/Zshot%20Example%20-%20Custom%20Components.ipynb) |
| Evaluation | [](https://colab.research.google.com/github/IBM/zshot/blob/examples/Zshot%20Example%20-%20Evaluation.ipynb) |
## Zshot Approach
ZShot contains two different components, the **mentions extractor** and the **linker**.
### Mentions Extractor
The **mentions extractor** will detect the possible entities (a.k.a. mentions), that will be then linked to a data source (e.g.: Wikidata) by the **linker**.
Currently, there are 6 different **mentions extractors** supported, SMXM, TARS, 2 based on *SpaCy*, and 2 that are based on *Flair*. The two different versions for *SpaCy* and *Flair* are similar, one is based on Named Entity Recognition and Classification (NERC) and the other one is based on the linguistics (i.e.: using Part Of the Speech tagging (PoS) and Dependency Parsing(DP)).
The NERC approach will use NERC models to detect all the entities that have to be linked. This approach depends on the model that is being used, and the entities the model has been trained on, so depending on the use case and the target entities it may be not the best approach, as the entities may be not recognized by the NERC model and thus won't be linked.
The linguistic approach relies on the idea that mentions will usually be a syntagma or a noun. Therefore, this approach detects nouns that are included in a syntagma and that act like objects, subjects, etc. This approach do not depend on the model (although the performance does), but a noun in a text should be always a noun, it doesn't depend on the dataset the model has been trained on.
### Linker
The **linker** will link the detected entities to a existing set of labels. Some of the **linkers**, however, are *end-to-end*, i.e. they don't need the **mentions extractor**, as they detect and link the entities at the same time.
Again, there are 4 **linkers** available currently, 2 of them are *end-to-end* and 2 are not. Let's start with those thar are not *end-to-end*:
| Linker Name | end-to-end | Source Code | Paper |
|:-----------:|:----------:|----------------------------------------------------------|--------------------------------------------------------------------|
| Blink | X | [Source Code](https://github.com/facebookresearch/BLINK) | [Paper](https://arxiv.org/pdf/1911.03814.pdf) |
| GENRE | X | [Source Code](https://github.com/facebookresearch/GENRE) | [Paper](https://arxiv.org/pdf/2010.00904.pdf) |
| SMXM | ✓ | [Source Code](https://github.com/Raldir/Zero-shot-NERC) | [Paper](https://aclanthology.org/2021.acl-long.120/) |
| TARS | ✓ | [Source Code](https://github.com/flairNLP/flair) | [Paper](https://kishaloyhalder.github.io/pdfs/tars_coling2020.pdf) |
### Relations Extractor
The **relations extractor** will extract relations among different entities *previously* extracted by a **linker**..
Currently, the is only one Relation Extractor available:
- ZS-Bert
- [Paper](https://arxiv.org/abs/2104.04697)
- [Source Code](https://github.com/dinobby/ZS-BERT)
### Knowledge Extractor
The **knowledge extractor** will perform at the same time the extraction and classification of named entities and the extraction of relations among them. The pipeline with this component doesn't need any **mentions extractor**, **linker** or **relation extractor** to work.
Currently, the is only one Knowledge Extractor available:
- KnowGL
- [Rossiello et al. (AAAI 2023)](https://arxiv.org/pdf/2210.13952.pdf)
- [Mihindukulasooriya et al. (ISWC 2022)](https://arxiv.org/pdf/2207.05188.pdf)
### How to use it
* Install requirements: `pip install -r requirements.txt`
* Install a spacy pipeline to use it for mentions extraction: `python -m spacy download en_core_web_sm`
* Create a file `main.py` with the pipeline configuration and entities definition (*Wikipedia abstract are usually a good starting point for descriptions*):
```Python
import spacy
from zshot import PipelineConfig, displacy
from zshot.linker import LinkerRegen
from zshot.mentions_extractor import MentionsExtractorSpacy
from zshot.utils.data_models import Entity
nlp = spacy.load("en_core_web_sm")
nlp_config = PipelineConfig(
mentions_extractor=MentionsExtractorSpacy(),
linker=LinkerRegen(),
entities=[
Entity(name="Paris",
description="Paris is located in northern central France, in a north-bending arc of the river Seine"),
Entity(name="IBM",
description="International Business Machines Corporation (IBM) is an American multinational technology corporation headquartered in Armonk, New York"),
Entity(name="New York", description="New York is a city in U.S. state"),
Entity(name="Florida", description="southeasternmost U.S. state"),
Entity(name="American",
description="American, something of, from, or related to the United States of America, commonly known as the United States or America"),
Entity(name="Chemical formula",
description="In chemistry, a chemical formula is a way of presenting information about the chemical proportions of atoms that constitute a particular chemical compound or molecule"),
Entity(name="Acetamide",
description="Acetamide (systematic name: ethanamide) is an organic compound with the formula CH3CONH2. It is the simplest amide derived from acetic acid. It finds some use as a plasticizer and as an industrial solvent."),
Entity(name="Armonk",
description="Armonk is a hamlet and census-designated place (CDP) in the town of North Castle, located in Westchester County, New York, United States."),
Entity(name="Acetic Acid",
description="Acetic acid, systematically named ethanoic acid, is an acidic, colourless liquid and organic compound with the chemical formula CH3COOH"),
Entity(name="Industrial solvent",
description="Acetamide (systematic name: ethanamide) is an organic compound with the formula CH3CONH2. It is the simplest amide derived from acetic acid. It finds some use as a plasticizer and as an industrial solvent."),
]
)
nlp.add_pipe("zshot", config=nlp_config, last=True)
text = "International Business Machines Corporation (IBM) is an American multinational technology corporation" \
" headquartered in Armonk, New York, with operations in over 171 countries."
doc = nlp(text)
displacy.serve(doc, style="ent")
```
### Run it
Run with
```console
$ python main.py
Using the 'ent' visualizer
Serving on http://0.0.0.0:5000 ...
```
The script will annotate the text using Zshot and use Displacy for visualising the annotations
### Check it
Open your browser at <a href="http://127.0.0.1:5000" class="external-link" target="_blank">http://127.0.0.1:5000</a> .
You will see the annotated sentence:
<img src="https://ibm.github.io/zshot/img/annotations.png" />
### How to create a custom component
If you want to implement your own mentions_extractor or linker and use it with ZShot you can do it. To make it easier for the user to implement a new component, some base classes are provided that you have to extend with your code.
It is as simple as create a new class extending the base class (`MentionsExtractor` or `Linker`). You will have to implement the predict method, which will receive the SpaCy Documents and will return a list of `zshot.utils.data_models.Span` for each document.
This is a simple mentions_extractor that will extract as mentions all words that contain the letter s:
```python
from typing import Iterable
import spacy
from spacy.tokens import Doc
from zshot import PipelineConfig
from zshot.utils.data_models import Span
from zshot.mentions_extractor import MentionsExtractor
class SimpleMentionExtractor(MentionsExtractor):
def predict(self, docs: Iterable[Doc], batch_size=None):
spans = [[Span(tok.idx, tok.idx + len(tok)) for tok in doc if "s" in tok.text] for doc in docs]
return spans
new_nlp = spacy.load("en_core_web_sm")
config = PipelineConfig(
mentions_extractor=SimpleMentionExtractor()
)
new_nlp.add_pipe("zshot", config=config, last=True)
text_acetamide = "CH2O2 is a chemical compound similar to Acetamide used in International Business " \
"Machines Corporation (IBM)."
doc = new_nlp(text_acetamide)
print(doc._.mentions)
>>> [is, similar, used, Business, Machines, materials]
```
### How to evaluate ZShot
Evaluation is an important process to keep improving the performance of the models, that's why ZShot allows to evaluate the component with two predefined datasets: OntoNotes and MedMentions, in a Zero-Shot version in which the entities of the test and validation splits don't appear in the train set.
The package `evaluation` contains all the functionalities to evaluate the ZShot components. The main function is `zshot.evaluation.zshot_evaluate.evaluate`, that will take as input the SpaCy `nlp` model and the dataset to evaluate. It will return a `str` containing a table with the results of the evaluation. For instance the evaluation of the TARS linker in ZShot for the *Ontonotes validation* set would be:
```python
import spacy
from zshot import PipelineConfig
from zshot.linker import LinkerTARS
from zshot.evaluation.dataset import load_ontonotes_zs
from zshot.evaluation.zshot_evaluate import evaluate, prettify_evaluate_report
from zshot.evaluation.metrics.seqeval.seqeval import Seqeval
ontonotes_zs = load_ontonotes_zs('validation')
nlp = spacy.blank("en")
nlp_config = PipelineConfig(
linker=LinkerTARS(),
entities=ontonotes_zs.entities
)
nlp.add_pipe("zshot", config=nlp_config, last=True)
evaluation = evaluate(nlp, ontonotes_zs, metric=Seqeval())
prettify_evaluate_report(evaluation)
```
## Citation
```
@inproceedings{picco-etal-2023-zshot,
title = "Zshot: An Open-source Framework for Zero-Shot Named Entity Recognition and Relation Extraction",
author = "Picco, Gabriele and
Martinez Galindo, Marcos and
Purpura, Alberto and
Fuchs, Leopold and
Lopez, Vanessa and
Hoang, Thanh Lam",
booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)",
month = jul,
year = "2023",
address = "Toronto, Canada",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2023.acl-demo.34",
doi = "10.18653/v1/2023.acl-demo.34",
pages = "357--368",
abstract = "The Zero-Shot Learning (ZSL) task pertains to the identification of entities or relations in texts that were not seen during training. ZSL has emerged as a critical research area due to the scarcity of labeled data in specific domains, and its applications have grown significantly in recent years. With the advent of large pretrained language models, several novel methods have been proposed, resulting in substantial improvements in ZSL performance. There is a growing demand, both in the research community and industry, for a comprehensive ZSL framework that facilitates the development and accessibility of the latest methods and pretrained models.In this study, we propose a novel ZSL framework called Zshot that aims to address the aforementioned challenges. Our primary objective is to provide a platform that allows researchers to compare different state-of-the-art ZSL methods with standard benchmark datasets. Additionally, we have designed our framework to support the industry with readily available APIs for production under the standard SpaCy NLP pipeline. Our API is extendible and evaluable, moreover, we include numerous enhancements such as boosting the accuracy with pipeline ensembling and visualization utilities available as a SpaCy extension.",
}
```
Raw data
{
"_id": null,
"home_page": "https://ibm.github.io/zshot",
"name": "zshot",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "NER Zero-Shot Few-Shot",
"author": "IBM Research",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/79/b9/7354d498128cab4c531185b7004a030889746cf343e3fc04c455cffb1f2f/zshot-0.0.9.tar.gz",
"platform": null,
"description": "<div align=\"center\">\n \n <img height=\"170x\" src=\"https://ibm.github.io/zshot/img/graph.png\" />\n \n <h1>Zshot</h1>\n\n <p>\n <strong>Zero and Few shot named entity & relationships recognition</strong>\n </p>\n\n <p> \n \n<a href=\"https://ibm.github.io/zshot/\"><img alt=\"Tutorials\" src=\"https://img.shields.io/badge/docs-tutorials-green\" /></a>\n<a href=\"https://pypi.org/project/zshot/\"><img src=\"https://img.shields.io/pypi/v/zshot\" /></a>\n<a href=\"https://pypi.org/project/zshot/\"><img src=\"https://img.shields.io/pypi/dm/zshot\" /></a>\n<a href=\"https://github.com/IBM/zshot/actions/workflows/python-tests.yml\"> <img alt=\"Build\" src=\"https://github.com/IBM/zshot/actions/workflows/python-tests.yml/badge.svg\" /></a>\n<a href=\"https://app.codecov.io/github/ibm/zshot\"> <img alt=\"Build\" src=\"https://codecov.io/github/ibm/zshot/branch/main/graph/badge.svg\" /></a>\n\n </p>\n</div>\n\n**Documentation**: <a href=\"https://ibm.github.io/zshot/\" target=\"_blank\">https://ibm.github.io/zshot</a>\n\n**Source Code**: <a href=\"https://github.com/IBM/zshot\" target=\"_blank\">https://github.com/IBM/zshot</a>\n\n**Paper**: <a href=\"https://aclanthology.org/2023.acl-demo.34/\" target=\"_blank\">https://aclanthology.org/2023.acl-demo.34/</a>\n\nZshot is a highly customisable framework for performing Zero and Few shot named entity recognition.\n\nCan be used to perform:\n\n- **Mentions extraction**: Identify globally relevant mentions or mentions relevant for a given domain \n- **Wikification**: The task of linking textual mentions to entities in Wikipedia\n- **Zero and Few Shot named entity recognition**: using language description perform NER to generalize to unseen domains\n- **Zero and Few Shot named relationship recognition**\n- **Visualization: Zero-shot NER and RE extraction**\n\n## Requirements\n\n* `Python 3.6+`\n\n* <a href=\"https://spacy.io/\" target=\"_blank\"><code>spacy</code></a> - Zshot rely on <a href=\"https://spacy.io/\" class=\"external-link\" target=\"_blank\">Spacy</a> for pipelining and visualization\n* <a href=\"https://pytorch.org/get-started\" target=\"_blank\"><code>torch</code></a> - PyTorch is required to run pytorch models.\n* <a href=\"https://huggingface.co/docs/transformers/index\" target=\"_blank\"><code>transformers</code></a> - Required for pre-trained language models.\n* <a href=\"https://huggingface.co/docs/evaluate/index\" target=\"_blank\"><code>evaluate</code></a> - Required for evaluation.\n* <a href=\"https://huggingface.co/docs/datasets/index\" target=\"_blank\"><code>datasets</code></a> - Required to evaluate over datasets (e.g.: OntoNotes).\n\n### Optional Dependencies\n\n* <a href=\"https://github.com/flairNLP/flair\" target=\"_blank\"><code>flair</code></a> - Required if you want to use Flair mentions extractor and for TARS linker.\n* <a href=\"https://github.com/facebookresearch/BLINK\" target=\"_blank\"><code>blink</code></a> - Required if you want to use Blink for linking to Wikipedia pages.\n\n\n## Installation\n\n<div class=\"termy\">\n\n```console\n$ pip install zshot\n\n---> 100%\n```\n\n</div>\n\n## Examples\n\n| Example | Notebook |\n|:------------------------------:|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| Installation and Visualization | [](https://colab.research.google.com/github/IBM/zshot/blob/examples/Zshot%20Example.ipynb) |\n| Knowledge Extractor | [](https://colab.research.google.com/github/IBM/zshot/blob/examples/Zshot%20Example%20-%20Knowledge%20Extractor.ipynb) |\n| Wikification | [](https://colab.research.google.com/github/IBM/zshot/blob/examples/Zshot%20Example%20-%20Wikification.ipynb) |\n| Custom Components | [](https://colab.research.google.com/github/IBM/zshot/blob/examples/Zshot%20Example%20-%20Custom%20Components.ipynb) |\n| Evaluation | [](https://colab.research.google.com/github/IBM/zshot/blob/examples/Zshot%20Example%20-%20Evaluation.ipynb) |\n\n\n## Zshot Approach\n\nZShot contains two different components, the **mentions extractor** and the **linker**.\n\n### Mentions Extractor\nThe **mentions extractor** will detect the possible entities (a.k.a. mentions), that will be then linked to a data source (e.g.: Wikidata) by the **linker**. \n\nCurrently, there are 6 different **mentions extractors** supported, SMXM, TARS, 2 based on *SpaCy*, and 2 that are based on *Flair*. The two different versions for *SpaCy* and *Flair* are similar, one is based on Named Entity Recognition and Classification (NERC) and the other one is based on the linguistics (i.e.: using Part Of the Speech tagging (PoS) and Dependency Parsing(DP)).\n\nThe NERC approach will use NERC models to detect all the entities that have to be linked. This approach depends on the model that is being used, and the entities the model has been trained on, so depending on the use case and the target entities it may be not the best approach, as the entities may be not recognized by the NERC model and thus won't be linked.\n\nThe linguistic approach relies on the idea that mentions will usually be a syntagma or a noun. Therefore, this approach detects nouns that are included in a syntagma and that act like objects, subjects, etc. This approach do not depend on the model (although the performance does), but a noun in a text should be always a noun, it doesn't depend on the dataset the model has been trained on.\n\n### Linker\nThe **linker** will link the detected entities to a existing set of labels. Some of the **linkers**, however, are *end-to-end*, i.e. they don't need the **mentions extractor**, as they detect and link the entities at the same time. \n\nAgain, there are 4 **linkers** available currently, 2 of them are *end-to-end* and 2 are not. Let's start with those thar are not *end-to-end*:\n\n| Linker Name | end-to-end | Source Code | Paper |\n|:-----------:|:----------:|----------------------------------------------------------|--------------------------------------------------------------------|\n| Blink | X | [Source Code](https://github.com/facebookresearch/BLINK) | [Paper](https://arxiv.org/pdf/1911.03814.pdf) |\n| GENRE | X | [Source Code](https://github.com/facebookresearch/GENRE) | [Paper](https://arxiv.org/pdf/2010.00904.pdf) |\n| SMXM | ✓ | [Source Code](https://github.com/Raldir/Zero-shot-NERC) | [Paper](https://aclanthology.org/2021.acl-long.120/) |\n| TARS | ✓ | [Source Code](https://github.com/flairNLP/flair) | [Paper](https://kishaloyhalder.github.io/pdfs/tars_coling2020.pdf) |\n\n### Relations Extractor\nThe **relations extractor** will extract relations among different entities *previously* extracted by a **linker**.. \n\nCurrently, the is only one Relation Extractor available: \n\n- ZS-Bert\n - [Paper](https://arxiv.org/abs/2104.04697)\n - [Source Code](https://github.com/dinobby/ZS-BERT)\n\n\n### Knowledge Extractor\nThe **knowledge extractor** will perform at the same time the extraction and classification of named entities and the extraction of relations among them. The pipeline with this component doesn't need any **mentions extractor**, **linker** or **relation extractor** to work.\n\nCurrently, the is only one Knowledge Extractor available: \n\n- KnowGL\n\n - [Rossiello et al. (AAAI 2023)](https://arxiv.org/pdf/2210.13952.pdf)\n - [Mihindukulasooriya et al. (ISWC 2022)](https://arxiv.org/pdf/2207.05188.pdf)\n\n\n### How to use it\n\n* Install requirements: `pip install -r requirements.txt`\n* Install a spacy pipeline to use it for mentions extraction: `python -m spacy download en_core_web_sm`\n* Create a file `main.py` with the pipeline configuration and entities definition (*Wikipedia abstract are usually a good starting point for descriptions*):\n\n```Python\nimport spacy\n\nfrom zshot import PipelineConfig, displacy\nfrom zshot.linker import LinkerRegen\nfrom zshot.mentions_extractor import MentionsExtractorSpacy\nfrom zshot.utils.data_models import Entity\n\nnlp = spacy.load(\"en_core_web_sm\")\nnlp_config = PipelineConfig(\n mentions_extractor=MentionsExtractorSpacy(),\n linker=LinkerRegen(),\n entities=[\n Entity(name=\"Paris\",\n description=\"Paris is located in northern central France, in a north-bending arc of the river Seine\"),\n Entity(name=\"IBM\",\n description=\"International Business Machines Corporation (IBM) is an American multinational technology corporation headquartered in Armonk, New York\"),\n Entity(name=\"New York\", description=\"New York is a city in U.S. state\"),\n Entity(name=\"Florida\", description=\"southeasternmost U.S. state\"),\n Entity(name=\"American\",\n description=\"American, something of, from, or related to the United States of America, commonly known as the United States or America\"),\n Entity(name=\"Chemical formula\",\n description=\"In chemistry, a chemical formula is a way of presenting information about the chemical proportions of atoms that constitute a particular chemical compound or molecule\"),\n Entity(name=\"Acetamide\",\n description=\"Acetamide (systematic name: ethanamide) is an organic compound with the formula CH3CONH2. It is the simplest amide derived from acetic acid. It finds some use as a plasticizer and as an industrial solvent.\"),\n Entity(name=\"Armonk\",\n description=\"Armonk is a hamlet and census-designated place (CDP) in the town of North Castle, located in Westchester County, New York, United States.\"),\n Entity(name=\"Acetic Acid\",\n description=\"Acetic acid, systematically named ethanoic acid, is an acidic, colourless liquid and organic compound with the chemical formula CH3COOH\"),\n Entity(name=\"Industrial solvent\",\n description=\"Acetamide (systematic name: ethanamide) is an organic compound with the formula CH3CONH2. It is the simplest amide derived from acetic acid. It finds some use as a plasticizer and as an industrial solvent.\"),\n ]\n)\nnlp.add_pipe(\"zshot\", config=nlp_config, last=True)\n\ntext = \"International Business Machines Corporation (IBM) is an American multinational technology corporation\" \\\n \" headquartered in Armonk, New York, with operations in over 171 countries.\"\n\ndoc = nlp(text)\ndisplacy.serve(doc, style=\"ent\")\n```\n\n\n### Run it\n\nRun with\n\n```console\n$ python main.py\n\nUsing the 'ent' visualizer\nServing on http://0.0.0.0:5000 ...\n```\n\n\nThe script will annotate the text using Zshot and use Displacy for visualising the annotations\n\n### Check it\n\nOpen your browser at <a href=\"http://127.0.0.1:5000\" class=\"external-link\" target=\"_blank\">http://127.0.0.1:5000</a> .\n\nYou will see the annotated sentence:\n\n<img src=\"https://ibm.github.io/zshot/img/annotations.png\" />\n\n### How to create a custom component\n\nIf you want to implement your own mentions_extractor or linker and use it with ZShot you can do it. To make it easier for the user to implement a new component, some base classes are provided that you have to extend with your code.\n\nIt is as simple as create a new class extending the base class (`MentionsExtractor` or `Linker`). You will have to implement the predict method, which will receive the SpaCy Documents and will return a list of `zshot.utils.data_models.Span` for each document.\n\nThis is a simple mentions_extractor that will extract as mentions all words that contain the letter s:\n\n```python\nfrom typing import Iterable\nimport spacy\nfrom spacy.tokens import Doc\nfrom zshot import PipelineConfig\nfrom zshot.utils.data_models import Span\nfrom zshot.mentions_extractor import MentionsExtractor\n\nclass SimpleMentionExtractor(MentionsExtractor):\n def predict(self, docs: Iterable[Doc], batch_size=None):\n spans = [[Span(tok.idx, tok.idx + len(tok)) for tok in doc if \"s\" in tok.text] for doc in docs]\n return spans\n\nnew_nlp = spacy.load(\"en_core_web_sm\")\n\nconfig = PipelineConfig(\n mentions_extractor=SimpleMentionExtractor()\n)\nnew_nlp.add_pipe(\"zshot\", config=config, last=True)\ntext_acetamide = \"CH2O2 is a chemical compound similar to Acetamide used in International Business \" \\\n \"Machines Corporation (IBM).\"\n\ndoc = new_nlp(text_acetamide)\nprint(doc._.mentions)\n\n>>> [is, similar, used, Business, Machines, materials]\n```\n\n### How to evaluate ZShot\n\nEvaluation is an important process to keep improving the performance of the models, that's why ZShot allows to evaluate the component with two predefined datasets: OntoNotes and MedMentions, in a Zero-Shot version in which the entities of the test and validation splits don't appear in the train set. \n\nThe package `evaluation` contains all the functionalities to evaluate the ZShot components. The main function is `zshot.evaluation.zshot_evaluate.evaluate`, that will take as input the SpaCy `nlp` model and the dataset to evaluate. It will return a `str` containing a table with the results of the evaluation. For instance the evaluation of the TARS linker in ZShot for the *Ontonotes validation* set would be:\n\n```python\nimport spacy\n\nfrom zshot import PipelineConfig\nfrom zshot.linker import LinkerTARS\nfrom zshot.evaluation.dataset import load_ontonotes_zs\nfrom zshot.evaluation.zshot_evaluate import evaluate, prettify_evaluate_report\nfrom zshot.evaluation.metrics.seqeval.seqeval import Seqeval\n\nontonotes_zs = load_ontonotes_zs('validation')\n\n\nnlp = spacy.blank(\"en\")\nnlp_config = PipelineConfig(\n linker=LinkerTARS(),\n entities=ontonotes_zs.entities\n)\n\nnlp.add_pipe(\"zshot\", config=nlp_config, last=True)\n\nevaluation = evaluate(nlp, ontonotes_zs, metric=Seqeval())\nprettify_evaluate_report(evaluation)\n```\n\n## Citation\n```\n@inproceedings{picco-etal-2023-zshot,\n title = \"Zshot: An Open-source Framework for Zero-Shot Named Entity Recognition and Relation Extraction\",\n author = \"Picco, Gabriele and\n Martinez Galindo, Marcos and\n Purpura, Alberto and\n Fuchs, Leopold and\n Lopez, Vanessa and\n Hoang, Thanh Lam\",\n booktitle = \"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)\",\n month = jul,\n year = \"2023\",\n address = \"Toronto, Canada\",\n publisher = \"Association for Computational Linguistics\",\n url = \"https://aclanthology.org/2023.acl-demo.34\",\n doi = \"10.18653/v1/2023.acl-demo.34\",\n pages = \"357--368\",\n abstract = \"The Zero-Shot Learning (ZSL) task pertains to the identification of entities or relations in texts that were not seen during training. ZSL has emerged as a critical research area due to the scarcity of labeled data in specific domains, and its applications have grown significantly in recent years. With the advent of large pretrained language models, several novel methods have been proposed, resulting in substantial improvements in ZSL performance. There is a growing demand, both in the research community and industry, for a comprehensive ZSL framework that facilitates the development and accessibility of the latest methods and pretrained models.In this study, we propose a novel ZSL framework called Zshot that aims to address the aforementioned challenges. Our primary objective is to provide a platform that allows researchers to compare different state-of-the-art ZSL methods with standard benchmark datasets. Additionally, we have designed our framework to support the industry with readily available APIs for production under the standard SpaCy NLP pipeline. Our API is extendible and evaluable, moreover, we include numerous enhancements such as boosting the accuracy with pipeline ensembling and visualization utilities available as a SpaCy extension.\",\n}\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Zero and Few shot named entity recognition",
"version": "0.0.9",
"project_urls": {
"Homepage": "https://ibm.github.io/zshot"
},
"split_keywords": [
"ner",
"zero-shot",
"few-shot"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7949987f8211866ebc1fd1ecfa7a6dac4643e51389ec55fbdd9f8b66936f5a0c",
"md5": "0a82b6b709ea08a6fcbb231ece78a853",
"sha256": "8a532a952538b21b3409d88bc16133dcb6a00b34e9a4fb1985e6df6073505876"
},
"downloads": -1,
"filename": "zshot-0.0.9-py3-none-any.whl",
"has_sig": false,
"md5_digest": "0a82b6b709ea08a6fcbb231ece78a853",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 114311,
"upload_time": "2024-07-23T15:51:19",
"upload_time_iso_8601": "2024-07-23T15:51:19.897029Z",
"url": "https://files.pythonhosted.org/packages/79/49/987f8211866ebc1fd1ecfa7a6dac4643e51389ec55fbdd9f8b66936f5a0c/zshot-0.0.9-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "79b97354d498128cab4c531185b7004a030889746cf343e3fc04c455cffb1f2f",
"md5": "bd7ddefb6c5a0aece12065feb5bf318f",
"sha256": "761713119593542101ae6ae271dadda9d62d94427eeb90e0ee7e5a522ad293f7"
},
"downloads": -1,
"filename": "zshot-0.0.9.tar.gz",
"has_sig": false,
"md5_digest": "bd7ddefb6c5a0aece12065feb5bf318f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 82143,
"upload_time": "2024-07-23T15:51:21",
"upload_time_iso_8601": "2024-07-23T15:51:21.320883Z",
"url": "https://files.pythonhosted.org/packages/79/b9/7354d498128cab4c531185b7004a030889746cf343e3fc04c455cffb1f2f/zshot-0.0.9.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-23 15:51:21",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "zshot"
}
JSON