[**NEW: v.1.4 supports starting multiple QuickUMLS matchers concurrently!**](https://giphy.com/embed/BlVnrxJgTGsUw) I've finally added support for [unqlite](https://github.com/coleifer/unqlite-python) as an alternative to leveldb for storage of CUIs and Semantic Types (see [here](https://github.com/Georgetown-IR-Lab/QuickUMLS/wiki/Migration-QuickUMLS-1.3-to-1.4) for more details). unqlite-backed QuickUMLS installation support multiple matchers running at the same time. Other than better multi-processing support, unqlite should have better support for unicode.
# QuickUMLS
QuickUMLS (Soldaini and Goharian, 2016) is a tool for fast, unsupervised biomedical concept extraction from medical text.
It takes advantage of [Simstring](http://www.chokkan.org/software/simstring/) (Okazaki and Tsujii, 2010) for approximate string matching.
For more details on how QuickUMLS works, we remand to our paper.
This project should be compatible with Python 3 (Python 2 is [no longer supported](https://pythonclock.org/)) and run on any UNIX system (support for Windows is experimental, please report bugs!). **If you find any bugs, please file an issue on GitHub or email the author at luca@ir.cs.georgetown.edu**.
## Installation
1. **Obtain a UMLS installation** This tool requires you to have a valid UMLS installation on disk. To install UMLS, you must first obtain a [license](https://uts.nlm.nih.gov/license.html) from the National Library of Medicine; then you should download all UMLS files from [this page](https://www.nlm.nih.gov/research/umls/licensedcontent/umlsknowledgesources.html); finally, you can install UMLS using the [MetamorphoSys](https://www.nlm.nih.gov/pubs/factsheets/umlsmetamorph.html) tool as [explained in this guide](https://www.nlm.nih.gov/research/umls/implementation_resources/metamorphosys/help.html). The installation can be removed once the system has been initialized.
2. **Install QuickUMLS**: You can do so by either running `pip install quickumls` or `python setup.py install`. On macOS, using anaconda is **strongly recommended**<sup>†</sup>.
3. **Create a QuickUMLS installation** Initialize the system by running `python -m quickumls.install <umls_installation_path> <destination_path>`, where `<umls_installation_path>` is where the installation files are (in particular, we need `MRCONSO.RRF` and `MRSTY.RRF`) and `<destination_path>` is the directory where the QuickUmls data files should be installed. This process will take between 5 and 30 minutes depending how fast the CPU and the drive where UMLS and QuickUMLS files are stored are (on a system with a Intel i7 6700K CPU and a 7200 RPM hard drive, initialization takes 8.5 minutes). `python -m quickumls.install` supports the following optional arguments:
- `-L` / `--lowercase`: if used, all concept terms are folded to lowercase before being processed. This option typically increases recall, but it might reduce precision;
- `-U` / `--normalize-unicode`: if used, expressions with non-ASCII characters are converted to the closest combination of ASCII characters.
- `-E` / `--language`: Specify the language to consider for UMLS concepts; by default, English is used. For a complete list of languages, please see [this table provided by NLM](https://www.nlm.nih.gov/research/umls/knowledge_sources/metathesaurus/release/abbreviations.html#LAT).
- `-d` / `--database-backend`: Specify which database backend to use for QuickUMLS. The two options are `leveldb` and `unqlite`. The latter supports multi-process reading and has better unicode compatibility, and it used as default for all new 1.4 installations; the former is still used as default when instantiating a QuickUMLS client. More info about differences between the two databases and migration info are available [here](https://github.com/Georgetown-IR-Lab/QuickUMLS/wiki/Migration-QuickUMLS-1.3-to-1.4).
**†**: If the installation fails on macOS when using Anaconda, install `leveldb` first by running `conda install -c conda-forge python-leveldb`.
## APIs
A QuickUMLS object can be instantiated as follows:
```python
from quickumls import QuickUMLS
matcher = QuickUMLS(quickumls_fp, overlapping_criteria, threshold,
similarity_name, window, accepted_semtypes)
```
Where:
- `quickumls_fp` is the directory where the QuickUMLS data files are installed.
- `overlapping_criteria` (optional, default: "score") is the criteria used to deal with overlapping concepts; choose "score" if the matching score of the concepts should be consider first, "length" if the longest should be considered first instead.
- `threshold` (optional, default: 0.7) is the minimum similarity value between strings.
- `similarity_name` (optional, default: "jaccard") is the name of similarity to use. Choose between "dice", "jaccard", "cosine", or "overlap".
- `window` (optional, default: 5) is the maximum number of tokens to consider for matching.
- `accepted_semtypes` (optional, default: see `constants.py`) is the set of UMLS semantic types concepts should belong to. Semantic types are identified by the letter "T" followed by three numbers (e.g., "T131", which identifies the type *"Hazardous or Poisonous Substance"*). See [here](https://metamap.nlm.nih.gov/Docs/SemanticTypes_2013AA.txt) for the full list.
To use the matcher, simply call
```python
text = "The ulna has dislocated posteriorly from the trochlea of the humerus."
matcher.match(text, best_match=True, ignore_syntax=False)
```
Set `best_match` to `False` if you want to return overlapping candidates, `ignore_syntax` to `True` to disable all heuristics introduced in (Soldaini and Goharian, 2016).
If the matcher throws a warning during initialization, read [this page](https://github.com/Georgetown-IR-Lab/QuickUMLS/wiki/Migration-QuickUMLS-1.3-to-1.4) to learn why and how to stop it from doing so.
## spaCy pipeline component
QuickUMLS can be used for standalone processing but it can also be use as a component in a modular spaCy pipeline. This follows traditional spaCy handling of concepts to be entity objects added to the Document object. These entity objects contain the CUI, similarity score and Semantic Types in the spacy "underscore" object.
Adding QuickUMLS as a component in a pipeline can be done as follows:
```python
from quickumls.spacy_component import SpacyQuickUMLS
# common English pipeline
nlp = spacy.load('en_core_web_sm')
quickumls_component = SpacyQuickUMLS(nlp, 'PATH_TO_QUICKUMLS_DATA')
nlp.add_pipe(quickumls_component)
doc = nlp('Pt c/o shortness of breath, chest pain, nausea, vomiting, diarrrhea')
for ent in doc.ents:
print('Entity text : {}'.format(ent.text))
print('Label (UMLS CUI) : {}'.format(ent.label_))
print('Similarity : {}'.format(ent._.similarity))
print('Semtypes : {}'.format(ent._.semtypes))
```
## Server / Client Support
Starting with v.1.2, QuickUMLS includes a support for being used in a client-server configuration. That is, you can start one QuickUMLS server, and query it from multiple scripts using a client.
To start the server, run `python -m quickumls.server`:
```bash
python -m quickumls.server /path/to/quickumls/files {-P QuickUMLS port} {-H QuickUMLS host} {QuickUMLS options}
```
Host and port are optional; by default, QuickUMLS runs on `localhost:4645`. You can also pass any QuickUMLS option mentioned above to the server. To obtain a list of options for the server, run `python -m quickumls.server -h`.
To load the client, import `get_quickumls_client` from `quickumls`:
```bash
from quickumls import get_quickumls_client
matcher = get_quickumls_client()
text = "The ulna has dislocated posteriorly from the trochlea of the humerus."
matcher.match(text, best_match=True, ignore_syntax=False)
```
The API of the client is the same of a QuickUMLS object.
In case you wish to run the server in the background, you can do so as follows:
```bash
nohup python -m quickumls.server /path/to/QuickUMLS {server options} > /dev/null 2>&1 & echo $! > nohup.pid
```
When you are done, don't forget to stop the server by running.
```bash
kill -9 `cat nohup.pid`
rm nohup.pid
```
## References
- Okazaki, Naoaki, and Jun'ichi Tsujii. "*Simple and efficient algorithm for approximate dictionary matching.*" COLING 2010.
- Luca Soldaini and Nazli Goharian. "*QuickUMLS: a fast, unsupervised approach for medical concept extraction.*" MedIR Workshop, SIGIR 2016.
Raw data
{
"_id": null,
"home_page": "https://github.com/Georgetown-IR-Lab/QuickUMLS",
"name": "quickumls",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "Luca Soldaini",
"author_email": "luca@soldaini.net",
"download_url": "https://files.pythonhosted.org/packages/57/d8/e0346e41de9c1c0cd3420a39b9415e6176080e94577821208909d3849f3e/quickumls-1.4.2.tar.gz",
"platform": null,
"description": "[**NEW: v.1.4 supports starting multiple QuickUMLS matchers concurrently!**](https://giphy.com/embed/BlVnrxJgTGsUw) I've finally added support for [unqlite](https://github.com/coleifer/unqlite-python) as an alternative to leveldb for storage of CUIs and Semantic Types (see [here](https://github.com/Georgetown-IR-Lab/QuickUMLS/wiki/Migration-QuickUMLS-1.3-to-1.4) for more details). unqlite-backed QuickUMLS installation support multiple matchers running at the same time. Other than better multi-processing support, unqlite should have better support for unicode.\n\n# QuickUMLS\n\nQuickUMLS (Soldaini and Goharian, 2016) is a tool for fast, unsupervised biomedical concept extraction from medical text.\nIt takes advantage of [Simstring](http://www.chokkan.org/software/simstring/) (Okazaki and Tsujii, 2010) for approximate string matching.\nFor more details on how QuickUMLS works, we remand to our paper.\n\nThis project should be compatible with Python 3 (Python 2 is [no longer supported](https://pythonclock.org/)) and run on any UNIX system (support for Windows is experimental, please report bugs!). **If you find any bugs, please file an issue on GitHub or email the author at luca@ir.cs.georgetown.edu**.\n\n## Installation\n\n1. **Obtain a UMLS installation** This tool requires you to have a valid UMLS installation on disk. To install UMLS, you must first obtain a [license](https://uts.nlm.nih.gov/license.html) from the National Library of Medicine; then you should download all UMLS files from [this page](https://www.nlm.nih.gov/research/umls/licensedcontent/umlsknowledgesources.html); finally, you can install UMLS using the [MetamorphoSys](https://www.nlm.nih.gov/pubs/factsheets/umlsmetamorph.html) tool as [explained in this guide](https://www.nlm.nih.gov/research/umls/implementation_resources/metamorphosys/help.html). The installation can be removed once the system has been initialized.\n2. **Install QuickUMLS**: You can do so by either running `pip install quickumls` or `python setup.py install`. On macOS, using anaconda is **strongly recommended**<sup>\u2020</sup>.\n3. **Create a QuickUMLS installation** Initialize the system by running `python -m quickumls.install <umls_installation_path> <destination_path>`, where `<umls_installation_path>` is where the installation files are (in particular, we need `MRCONSO.RRF` and `MRSTY.RRF`) and `<destination_path>` is the directory where the QuickUmls data files should be installed. This process will take between 5 and 30 minutes depending how fast the CPU and the drive where UMLS and QuickUMLS files are stored are (on a system with a Intel i7 6700K CPU and a 7200 RPM hard drive, initialization takes 8.5 minutes). `python -m quickumls.install` supports the following optional arguments:\n - `-L` / `--lowercase`: if used, all concept terms are folded to lowercase before being processed. This option typically increases recall, but it might reduce precision;\n - `-U` / `--normalize-unicode`: if used, expressions with non-ASCII characters are converted to the closest combination of ASCII characters.\n - `-E` / `--language`: Specify the language to consider for UMLS concepts; by default, English is used. For a complete list of languages, please see [this table provided by NLM](https://www.nlm.nih.gov/research/umls/knowledge_sources/metathesaurus/release/abbreviations.html#LAT).\n - `-d` / `--database-backend`: Specify which database backend to use for QuickUMLS. The two options are `leveldb` and `unqlite`. The latter supports multi-process reading and has better unicode compatibility, and it used as default for all new 1.4 installations; the former is still used as default when instantiating a QuickUMLS client. More info about differences between the two databases and migration info are available [here](https://github.com/Georgetown-IR-Lab/QuickUMLS/wiki/Migration-QuickUMLS-1.3-to-1.4).\n\n\n**\u2020**: If the installation fails on macOS when using Anaconda, install `leveldb` first by running `conda install -c conda-forge python-leveldb`.\n\n## APIs\n\nA QuickUMLS object can be instantiated as follows:\n\n```python\nfrom quickumls import QuickUMLS\n\nmatcher = QuickUMLS(quickumls_fp, overlapping_criteria, threshold,\n similarity_name, window, accepted_semtypes)\n```\n\nWhere:\n\n- `quickumls_fp` is the directory where the QuickUMLS data files are installed.\n- `overlapping_criteria` (optional, default: \"score\") is the criteria used to deal with overlapping concepts; choose \"score\" if the matching score of the concepts should be consider first, \"length\" if the longest should be considered first instead.\n- `threshold` (optional, default: 0.7) is the minimum similarity value between strings.\n- `similarity_name` (optional, default: \"jaccard\") is the name of similarity to use. Choose between \"dice\", \"jaccard\", \"cosine\", or \"overlap\".\n- `window` (optional, default: 5) is the maximum number of tokens to consider for matching.\n- `accepted_semtypes` (optional, default: see `constants.py`) is the set of UMLS semantic types concepts should belong to. Semantic types are identified by the letter \"T\" followed by three numbers (e.g., \"T131\", which identifies the type *\"Hazardous or Poisonous Substance\"*). See [here](https://metamap.nlm.nih.gov/Docs/SemanticTypes_2013AA.txt) for the full list.\n\nTo use the matcher, simply call\n\n```python\ntext = \"The ulna has dislocated posteriorly from the trochlea of the humerus.\"\nmatcher.match(text, best_match=True, ignore_syntax=False)\n```\n\nSet `best_match` to `False` if you want to return overlapping candidates, `ignore_syntax` to `True` to disable all heuristics introduced in (Soldaini and Goharian, 2016).\n\nIf the matcher throws a warning during initialization, read [this page](https://github.com/Georgetown-IR-Lab/QuickUMLS/wiki/Migration-QuickUMLS-1.3-to-1.4) to learn why and how to stop it from doing so.\n\n## spaCy pipeline component\n\nQuickUMLS can be used for standalone processing but it can also be use as a component in a modular spaCy pipeline. This follows traditional spaCy handling of concepts to be entity objects added to the Document object. These entity objects contain the CUI, similarity score and Semantic Types in the spacy \"underscore\" object.\n\nAdding QuickUMLS as a component in a pipeline can be done as follows:\n\n```python\nfrom quickumls.spacy_component import SpacyQuickUMLS\n\n# common English pipeline\nnlp = spacy.load('en_core_web_sm')\n\nquickumls_component = SpacyQuickUMLS(nlp, 'PATH_TO_QUICKUMLS_DATA')\nnlp.add_pipe(quickumls_component)\n\ndoc = nlp('Pt c/o shortness of breath, chest pain, nausea, vomiting, diarrrhea')\n\nfor ent in doc.ents:\n print('Entity text : {}'.format(ent.text))\n print('Label (UMLS CUI) : {}'.format(ent.label_))\n print('Similarity : {}'.format(ent._.similarity))\n print('Semtypes : {}'.format(ent._.semtypes))\n```\n\n## Server / Client Support\n\nStarting with v.1.2, QuickUMLS includes a support for being used in a client-server configuration. That is, you can start one QuickUMLS server, and query it from multiple scripts using a client.\n\nTo start the server, run `python -m quickumls.server`:\n\n```bash\npython -m quickumls.server /path/to/quickumls/files {-P QuickUMLS port} {-H QuickUMLS host} {QuickUMLS options}\n```\n\nHost and port are optional; by default, QuickUMLS runs on `localhost:4645`. You can also pass any QuickUMLS option mentioned above to the server. To obtain a list of options for the server, run `python -m quickumls.server -h`.\n\nTo load the client, import `get_quickumls_client` from `quickumls`:\n\n```bash\nfrom quickumls import get_quickumls_client\nmatcher = get_quickumls_client()\ntext = \"The ulna has dislocated posteriorly from the trochlea of the humerus.\"\nmatcher.match(text, best_match=True, ignore_syntax=False)\n```\n\nThe API of the client is the same of a QuickUMLS object.\n\n\nIn case you wish to run the server in the background, you can do so as follows:\n\n```bash\nnohup python -m quickumls.server /path/to/QuickUMLS {server options} > /dev/null 2>&1 & echo $! > nohup.pid\n\n```\n\nWhen you are done, don't forget to stop the server by running.\n```bash\nkill -9 `cat nohup.pid`\nrm nohup.pid\n```\n\n## References\n\n- Okazaki, Naoaki, and Jun'ichi Tsujii. \"*Simple and efficient algorithm for approximate dictionary matching.*\" COLING 2010.\n- Luca Soldaini and Nazli Goharian. \"*QuickUMLS: a fast, unsupervised approach for medical concept extraction.*\" MedIR Workshop, SIGIR 2016.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "QuickUMLS is a tool for fast, unsupervised biomedical concept extraction from medical text",
"version": "1.4.2",
"project_urls": {
"Homepage": "https://github.com/Georgetown-IR-Lab/QuickUMLS"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "76c585766a23bf62b3b6fe6a85ba790c8e210fc91ec18dd041bf3a0c20a03366",
"md5": "666cd3204886b6f91b4ef354f3d56642",
"sha256": "43e2b09b9a634e1412e73d760845ec0cea09b64bcef39b1d97c159355aa97a5c"
},
"downloads": -1,
"filename": "quickumls-1.4.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "666cd3204886b6f91b4ef354f3d56642",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 27564,
"upload_time": "2023-10-16T03:33:30",
"upload_time_iso_8601": "2023-10-16T03:33:30.872604Z",
"url": "https://files.pythonhosted.org/packages/76/c5/85766a23bf62b3b6fe6a85ba790c8e210fc91ec18dd041bf3a0c20a03366/quickumls-1.4.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "57d8e0346e41de9c1c0cd3420a39b9415e6176080e94577821208909d3849f3e",
"md5": "5ba83fab1ec203d4d8dd9ac72b1c8494",
"sha256": "d63f7039480eb685bfb85e360eb3773ea45cdb2c5a2cf2bdf9ac524aae300000"
},
"downloads": -1,
"filename": "quickumls-1.4.2.tar.gz",
"has_sig": false,
"md5_digest": "5ba83fab1ec203d4d8dd9ac72b1c8494",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 28199,
"upload_time": "2023-10-16T03:33:32",
"upload_time_iso_8601": "2023-10-16T03:33:32.548441Z",
"url": "https://files.pythonhosted.org/packages/57/d8/e0346e41de9c1c0cd3420a39b9415e6176080e94577821208909d3849f3e/quickumls-1.4.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-10-16 03:33:32",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Georgetown-IR-Lab",
"github_project": "QuickUMLS",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "quickumls"
}
JSON
