# Named_Entity_Recognition_BERT_Multilingual_Library_LU
## Overview
**Named_Entity_Recognition_BERT_Multilingual_Library_LU** is a powerful and flexible library for Named Entity Recognition (NER) tasks using BERT models. This library supports multilingual NER and is suitable for key information extraction tasks across various domains such as biomedical, environmental, and technological.
The library simplifies NER tasks by providing an easy-to-use pipeline for loading data, training models, and making predictions. It is designed for developers, researchers, and data scientists looking for a robust NER solution.
## Features
- **Multilingual Support**: Leverage the power of BERT for NER tasks in multiple languages.
- **Flexible Input Format**: Works with CoNLL format data.
- **Easy Integration**: Provides a simple `NERPipeline` class for seamless integration into your projects.
- **Comprehensive Metrics**: Evaluate your models with precision, recall, F1-score, and accuracy.
- **Pretrained Models**: Supports any BERT-based pretrained models.
## Installation
Install the library using `pip`:
```bash
pip install named-entity-recognition-bert-multilingual-library-lu
```
## Usage
### Example Usage
Here is a complete example of how to use the library for training and predicting:
```bash
# Clone the dataset
!git clone https://github.com/spyysalo/bc2gm-corpus.git
```
```python
from Named_Entity_Recognition_BERT_Multilingual_Library_LU import NERPipeline
# Initialize pipeline
pipeline = NERPipeline(pretrained_model="bert-base-cased")
# Prepare data
train_dataset, val_dataset, test_dataset = pipeline.prepare_data(
"./bc2gm-corpus/conll/train.tsv",
"./bc2gm-corpus/conll/devel.tsv",
"./bc2gm-corpus/conll/test.tsv"
)
# Initialize model
pipeline.initialize_model(num_labels=len(pipeline.label_list))
# Train the model
pipeline.train(train_dataset, val_dataset)
# Test the model
test_metrics = pipeline.test(test_dataset)
print(test_metrics)
print("***************")
# Print metrics and predictions on a new sentence
test_sentence = "BRCA1 is a gene associated with breast cancer."
predictions = pipeline.predict(test_sentence)
```
## CoNLL Data Format
The input data should be in CoNLL format:
```mathematica
Token1 Label1
Token2 Label2
Token3 Label3
Token4 Label4
```
### Example:
```mathematica
BRCA1 B-GENE
is O
a O
gene O
associated O
with O
breast B-DISEASE
cancer I-DISEASE
. O
```
## Key Components
### 1. NERPipeline
The main class providing methods for:
- **Data Preparation**: Converts CoNLL format to a dataset suitable for BERT.
- **Model Initialization**: Loads a pretrained BERT model for NER.
- **Training**: Fine-tunes the model on your data.
- **Prediction**: Predicts labels for new sentences.
- **Evaluation**: Computes evaluation metrics (precision, recall, F1, etc.).
### 2. Helper Functions
The library also includes utility functions for advanced users:
- **`load_data`**: Load and parse CoNLL format data.
- **`convert_to_hf_format`**: Convert data to Hugging Face dataset format.
- **`compute_metrics`**: Evaluate predictions using `seqeval`.
## Evaluation Metrics
The library uses the `seqeval` library to compute the following metrics:
- Precision
- Recall
- F1-Score
- Accuracy
## Dependencies
- `torch`
- `transformers`
- `datasets`
- `evaluate`
- `numpy`
- `seqeval`
## Contributing
We welcome contributions! Please feel free to open issues or submit pull requests.
Raw data
{
"_id": null,
"home_page": "https://github.com/yourusername/Named_Entity_Recognition_BERT_Multilingual_Library_LU",
"name": "Named-Entity-Recognition-BERT-Multilingual-Library-LU",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "Named Entity Recognition, BERT, Multilingual NLP, Information Extraction, NER, Key Information Extraction",
"author": "Mehrdad ALMASI, Demival VASQUES FILHO, Tugce KARATAS",
"author_email": "mehrdad.al.2023@gmail.com, demival.vasques@uni.lu, tugce.karatas@uni.lu",
"download_url": "https://files.pythonhosted.org/packages/5d/c3/ec12010bf6ef27925cb386ea5aeabf7b3bcde3298e6da27ceaef3be934fb/named_entity_recognition_bert_multilingual_library_lu-0.1.31.tar.gz",
"platform": null,
"description": "# Named_Entity_Recognition_BERT_Multilingual_Library_LU\n\n## Overview\n\n**Named_Entity_Recognition_BERT_Multilingual_Library_LU** is a powerful and flexible library for Named Entity Recognition (NER) tasks using BERT models. This library supports multilingual NER and is suitable for key information extraction tasks across various domains such as biomedical, environmental, and technological.\n\nThe library simplifies NER tasks by providing an easy-to-use pipeline for loading data, training models, and making predictions. It is designed for developers, researchers, and data scientists looking for a robust NER solution.\n\n## Features\n\n- **Multilingual Support**: Leverage the power of BERT for NER tasks in multiple languages.\n- **Flexible Input Format**: Works with CoNLL format data.\n- **Easy Integration**: Provides a simple `NERPipeline` class for seamless integration into your projects.\n- **Comprehensive Metrics**: Evaluate your models with precision, recall, F1-score, and accuracy.\n- **Pretrained Models**: Supports any BERT-based pretrained models.\n\n## Installation\n\nInstall the library using `pip`:\n```bash\npip install named-entity-recognition-bert-multilingual-library-lu\n```\n\n## Usage\n\n### Example Usage\n\nHere is a complete example of how to use the library for training and predicting:\n\n```bash\n# Clone the dataset\n!git clone https://github.com/spyysalo/bc2gm-corpus.git\n```\n\n```python\nfrom Named_Entity_Recognition_BERT_Multilingual_Library_LU import NERPipeline\n\n# Initialize pipeline\npipeline = NERPipeline(pretrained_model=\"bert-base-cased\")\n\n# Prepare data\ntrain_dataset, val_dataset, test_dataset = pipeline.prepare_data(\n \"./bc2gm-corpus/conll/train.tsv\", \n \"./bc2gm-corpus/conll/devel.tsv\", \n \"./bc2gm-corpus/conll/test.tsv\"\n)\n\n# Initialize model\npipeline.initialize_model(num_labels=len(pipeline.label_list))\n\n# Train the model\npipeline.train(train_dataset, val_dataset)\n\n# Test the model\ntest_metrics = pipeline.test(test_dataset)\nprint(test_metrics)\nprint(\"***************\")\n\n# Print metrics and predictions on a new sentence\ntest_sentence = \"BRCA1 is a gene associated with breast cancer.\"\npredictions = pipeline.predict(test_sentence)\n\n```\n\n## CoNLL Data Format\n\nThe input data should be in CoNLL format:\n\n```mathematica\n\nToken1 Label1\nToken2 Label2\nToken3 Label3\nToken4 Label4\n```\n\n### Example:\n\n```mathematica\nBRCA1 B-GENE\nis O\na O\ngene O\nassociated O\nwith O\nbreast B-DISEASE\ncancer I-DISEASE\n. O\n```\n\n## Key Components\n\n### 1. NERPipeline\n\nThe main class providing methods for:\n\n- **Data Preparation**: Converts CoNLL format to a dataset suitable for BERT.\n- **Model Initialization**: Loads a pretrained BERT model for NER.\n- **Training**: Fine-tunes the model on your data.\n- **Prediction**: Predicts labels for new sentences.\n- **Evaluation**: Computes evaluation metrics (precision, recall, F1, etc.).\n\n### 2. Helper Functions\n\nThe library also includes utility functions for advanced users:\n\n- **`load_data`**: Load and parse CoNLL format data.\n- **`convert_to_hf_format`**: Convert data to Hugging Face dataset format.\n- **`compute_metrics`**: Evaluate predictions using `seqeval`.\n\n## Evaluation Metrics\n\nThe library uses the `seqeval` library to compute the following metrics:\n\n- Precision\n- Recall\n- F1-Score\n- Accuracy\n\n## Dependencies\n\n- `torch`\n- `transformers`\n- `datasets`\n- `evaluate`\n- `numpy`\n- `seqeval`\n\n## Contributing\n\nWe welcome contributions! Please feel free to open issues or submit pull requests.\n\n",
"bugtrack_url": null,
"license": null,
"summary": "A comprehensive multilingual Named Entity Recognition (NER) library leveraging BERT. Supports key information extraction tasks across various domains such as biomedical, environmental, and technological.",
"version": "0.1.31",
"project_urls": {
"Documentation": "https://github.com/yourusername/Named_Entity_Recognition_BERT_Multilingual_Library_LU/wiki",
"Homepage": "https://github.com/yourusername/Named_Entity_Recognition_BERT_Multilingual_Library_LU",
"Source": "https://github.com/yourusername/Named_Entity_Recognition_BERT_Multilingual_Library_LU",
"Tracker": "https://github.com/yourusername/Named_Entity_Recognition_BERT_Multilingual_Library_LU/issues"
},
"split_keywords": [
"named entity recognition",
" bert",
" multilingual nlp",
" information extraction",
" ner",
" key information extraction"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c1c65a13e50b83f98de565ad05d7c9eab9133adb93383b0c7fa6fb5e63c1a98b",
"md5": "cac75efcaf81d14bd155c719000e8afb",
"sha256": "e1012ac01cc447d264ba30d12e5f6d5bbc0ea86cfae0a9290f31dbad46a6200b"
},
"downloads": -1,
"filename": "Named_Entity_Recognition_BERT_Multilingual_Library_LU-0.1.31-py3-none-any.whl",
"has_sig": false,
"md5_digest": "cac75efcaf81d14bd155c719000e8afb",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 7695,
"upload_time": "2024-11-08T13:42:19",
"upload_time_iso_8601": "2024-11-08T13:42:19.765277Z",
"url": "https://files.pythonhosted.org/packages/c1/c6/5a13e50b83f98de565ad05d7c9eab9133adb93383b0c7fa6fb5e63c1a98b/Named_Entity_Recognition_BERT_Multilingual_Library_LU-0.1.31-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5dc3ec12010bf6ef27925cb386ea5aeabf7b3bcde3298e6da27ceaef3be934fb",
"md5": "6e05537ccc8829018c66dfb6cafe284b",
"sha256": "984506ac8ded3dd13779ec6defe814ff5e44d08621e3b62d212046430d688c5f"
},
"downloads": -1,
"filename": "named_entity_recognition_bert_multilingual_library_lu-0.1.31.tar.gz",
"has_sig": false,
"md5_digest": "6e05537ccc8829018c66dfb6cafe284b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 6887,
"upload_time": "2024-11-08T13:42:20",
"upload_time_iso_8601": "2024-11-08T13:42:20.866613Z",
"url": "https://files.pythonhosted.org/packages/5d/c3/ec12010bf6ef27925cb386ea5aeabf7b3bcde3298e6da27ceaef3be934fb/named_entity_recognition_bert_multilingual_library_lu-0.1.31.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-08 13:42:20",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "yourusername",
"github_project": "Named_Entity_Recognition_BERT_Multilingual_Library_LU",
"github_not_found": true,
"lcname": "named-entity-recognition-bert-multilingual-library-lu"
}
JSON
