<div align="center">
<br>
<img src="https://raw.githubusercontent.com/allenai/allennlp/main/docs/img/allennlp-logo-dark.png" width="400"/>
<p>
An Apache 2.0 NLP research library, built on PyTorch, for developing state-of-the-art deep learning models on a wide variety of linguistic tasks.
</p>
<hr/>
</div>
<p align="center">
<a href="https://github.com/allenai/allennlp/actions">
<img alt="CI" src="https://github.com/allenai/allennlp/workflows/CI/badge.svg?event=push&branch=main">
</a>
<a href="https://pypi.org/project/allennlp/">
<img alt="PyPI" src="https://img.shields.io/pypi/v/allennlp">
</a>
<a href="https://github.com/allenai/allennlp/blob/main/LICENSE">
<img alt="License" src="https://img.shields.io/github/license/allenai/allennlp.svg?color=blue&cachedrop">
</a>
<a href="https://codecov.io/gh/allenai/allennlp">
<img alt="Codecov" src="https://codecov.io/gh/allenai/allennlp/branch/main/graph/badge.svg">
</a>
<a href="https://optuna.org">
<img alt="Optuna" src="https://img.shields.io/badge/Optuna-integrated-blue">
</a>
<br/>
</p>
⚠️ **NOTICE:** The AllenNLP library is now in maintenance mode. That means we are no longer adding new features or upgrading dependencies. We will still respond to questions and address bugs as they arise up until December 16th, 2022. If you have any concerns or are interested in maintaining AllenNLP going forward, please open an issue on this repository.
AllenNLP has been a big success, but as the field is advancing quickly it's time to focus on new initiatives. We're working hard to make [AI2 Tango](https://github.com/allenai/tango) the best way to organize research codebases. If you are an active user of AllenNLP, here are some suggested alternatives:
* If you like the trainer, the configuration language, or are simply looking for a better way to manage your experiments, check out [AI2 Tango](https://github.com/allenai/tango).
* If you like AllenNLP's `modules` and `nn` packages, check out [delmaksym/allennlp-light](https://github.com/delmaksym/allennlp-light). It's even compatible with [AI2 Tango](https://github.com/allenai/tango)!
* If you like the framework aspect of AllenNLP, check out [flair](https://github.com/flairNLP/flair). It has multiple state-of-art NLP models and allows you to easily use pretrained embeddings such as those from transformers.
* If you like the AllenNLP metrics package, check out [torchmetrics](https://torchmetrics.readthedocs.io/en/stable/). It has the same API as AllenNLP, so it should be a quick learning curve to make the switch.
* If you want to vectorize text, try [the transformers library](https://github.com/huggingface/transformers).
* If you want to maintain the AllenNLP Fairness or Interpret components, please get in touch. There is no alternative to it, so we are looking for a dedicated maintainer.
* If you are concerned about other AllenNLP functionality, please create an issue. Maybe we can find another way to continue supporting your use case.
## Quick Links
- ↗️ [Website](https://allennlp.org/)
- 🔦 [Guide](https://guide.allennlp.org/)
- 🖼 [Gallery](https://gallery.allennlp.org)
- 💻 [Demo](https://demo.allennlp.org)
- 📓 [Documentation](https://docs.allennlp.org/) ( [latest](https://docs.allennlp.org/latest/) | [stable](https://docs.allennlp.org/stable/) | [commit](https://docs.allennlp.org/main/) )
- ⬆️ [Upgrade Guide from 1.x to 2.0](https://github.com/allenai/allennlp/discussions/4933)
- ❓ [Stack Overflow](https://stackoverflow.com/questions/tagged/allennlp)
- ✋ [Contributing Guidelines](CONTRIBUTING.md)
- 🤖 [Officially Supported Models](https://github.com/allenai/allennlp-models)
- [Pretrained Models](https://github.com/allenai/allennlp-models/blob/main/allennlp_models/pretrained.py)
- [Documentation](https://docs.allennlp.org/models/) ( [latest](https://docs.allennlp.org/models/latest/) | [stable](https://docs.allennlp.org/models/stable/) | [commit](https://docs.allennlp.org/models/main/) )
- ⚙️ [Continuous Build](https://github.com/allenai/allennlp/actions)
- 🌙 [Nightly Releases](https://pypi.org/project/allennlp/#history)
## In this README
- [Getting Started Using the Library](#getting-started-using-the-library)
- [Plugins](#plugins)
- [Package Overview](#package-overview)
- [Installation](#installation)
- [Installing via pip](#installing-via-pip)
- [Installing using Docker](#installing-using-docker)
- [Installing from source](#installing-from-source)
- [Running AllenNLP](#running-allennlp)
- [Issues](#issues)
- [Contributions](#contributions)
- [Citing](#citing)
- [Team](#team)
## Getting Started Using the Library
If you're interested in using AllenNLP for model development, we recommend you check out the
[AllenNLP Guide](https://guide.allennlp.org) for a thorough introduction to the library, followed by our more advanced guides
on [GitHub Discussions](https://github.com/allenai/allennlp/discussions/categories/guides).
When you're ready to start your project, we've created a couple of template repositories that you can use as a starting place:
* If you want to use `allennlp train` and config files to specify experiments, use [this
template](https://github.com/allenai/allennlp-template-config-files). We recommend this approach.
* If you'd prefer to use python code to configure your experiments and run your training loop, use
[this template](https://github.com/allenai/allennlp-template-python-script). There are a few
things that are currently a little harder in this setup (loading a saved model, and using
distributed training), but otherwise it's functionality equivalent to the config files
setup.
In addition, there are external tutorials:
* [Hyperparameter optimization for AllenNLP using Optuna](https://medium.com/optuna/hyperparameter-optimization-for-allennlp-using-optuna-54b4bfecd78b)
* [Training with multiple GPUs in AllenNLP](https://medium.com/ai2-blog/tutorial-how-to-train-with-multiple-gpus-in-allennlp-c4d7c17eb6d6)
* [Training on larger batches with less memory in AllenNLP](https://medium.com/ai2-blog/tutorial-training-on-larger-batches-with-less-memory-in-allennlp-1cd2047d92ad)
* [How to upload transformer weights and tokenizers from AllenNLP to HuggingFace](https://medium.com/ai2-blog/tutorial-how-to-upload-transformer-weights-and-tokenizers-from-allennlp-to-huggingface-ecf6c0249bf)
And others on the [AI2 AllenNLP blog](https://medium.com/ai2-blog/allennlp/home).
## Plugins
AllenNLP supports loading "plugins" dynamically. A plugin is just a Python package that
provides custom registered classes or additional `allennlp` subcommands.
There is ecosystem of open source plugins, some of which are maintained by the AllenNLP
team here at AI2, and some of which are maintained by the broader community.
<table>
<tr>
<td><b> Plugin </b></td>
<td><b> Maintainer </b></td>
<td><b> CLI </b></td>
<td><b> Description </b></td>
</tr>
<tr>
<td> <a href="https://github.com/allenai/allennlp-models"><b>allennlp-models</b></a> </td>
<td> AI2 </td>
<td> No </td>
<td> A collection of state-of-the-art models </td>
</tr>
<tr>
<td> <a href="https://github.com/allenai/allennlp-semparse"><b>allennlp-semparse</b></a> </td>
<td> AI2 </td>
<td> No </td>
<td> A framework for building semantic parsers </td>
</tr>
<tr>
<td> <a href="https://github.com/allenai/allennlp-server"><b>allennlp-server</b></a> </td>
<td> AI2 </td>
<td> Yes </td>
<td> A simple demo server for serving models </td>
</tr>
<tr>
<td> <a href="https://github.com/himkt/allennlp-optuna"><b>allennlp-optuna</b></a> </td>
<td> <a href="https://himkt.github.io/profile/">Makoto Hiramatsu</a> </td>
<td> Yes </td>
<td> <a href="https://optuna.org/">Optuna</a> integration for hyperparameter optimization </td>
</tr>
</table>
AllenNLP will automatically find any official AI2-maintained plugins that you have installed,
but for AllenNLP to find personal or third-party plugins you've installed,
you also have to create either a local plugins file named `.allennlp_plugins`
in the directory where you run the `allennlp` command, or a global plugins file at `~/.allennlp/plugins`.
The file should list the plugin modules that you want to be loaded, one per line.
To test that your plugins can be found and imported by AllenNLP, you can run the `allennlp test-install` command.
Each discovered plugin will be logged to the terminal.
For more information about plugins, see the [plugins API docs](https://docs.allennlp.org/main/api/common/plugins/). And for information on how to create a custom subcommand
to distribute as a plugin, see the [subcommand API docs](https://docs.allennlp.org/main/api/commands/subcommand/).
## Package Overview
<table>
<tr>
<td><b> allennlp </b></td>
<td> An open-source NLP research library, built on PyTorch </td>
</tr>
<tr>
<td><b> allennlp.commands </b></td>
<td> Functionality for the CLI </td>
</tr>
<tr>
<td><b> allennlp.common </b></td>
<td> Utility modules that are used across the library </td>
</tr>
<tr>
<td><b> allennlp.data </b></td>
<td> A data processing module for loading datasets and encoding strings as integers for representation in matrices </td>
</tr>
<tr>
<td><b> allennlp.fairness </b></td>
<td> A module for bias mitigation and fairness algorithms and metrics </td>
</tr>
<tr>
<td><b> allennlp.modules </b></td>
<td> A collection of PyTorch modules for use with text </td>
</tr>
<tr>
<td><b> allennlp.nn </b></td>
<td> Tensor utility functions, such as initializers and activation functions </td>
</tr>
<tr>
<td><b> allennlp.training </b></td>
<td> Functionality for training models </td>
</tr>
</table>
## Installation
AllenNLP requires Python 3.6.1 or later and [PyTorch](https://pytorch.org/).
We support AllenNLP on Mac and Linux environments. We presently do not support Windows but are open to contributions.
### Installing via conda-forge
The simplest way to install AllenNLP is using conda (you can choose a different python version):
```
conda install -c conda-forge python=3.8 allennlp
```
To install optional packages, such as `checklist`, use
```
conda install -c conda-forge allennlp-checklist
```
or simply install `allennlp-all` directly. The plugins mentioned above are similarly installable, e.g.
```
conda install -c conda-forge allennlp-models allennlp-semparse allennlp-server allennlp-optuna
```
### Installing via pip
It's recommended that you install the PyTorch ecosystem **before** installing AllenNLP by following the instructions on [pytorch.org](https://pytorch.org/).
After that, just run `pip install allennlp`.
> ⚠️ If you're using Python 3.7 or greater, you should ensure that you don't have the PyPI version of `dataclasses` installed after running the above command, as this could cause issues on certain platforms. You can quickly check this by running `pip freeze | grep dataclasses`. If you see something like `dataclasses=0.6` in the output, then just run `pip uninstall -y dataclasses`.
If you need pointers on setting up an appropriate Python environment or would like to install AllenNLP using a different method, see below.
#### Setting up a virtual environment
[Conda](https://conda.io/) can be used set up a virtual environment with the
version of Python required for AllenNLP. If you already have a Python 3
environment you want to use, you can skip to the 'installing via pip' section.
1. [Download and install Conda](https://conda.io/projects/conda/en/latest/user-guide/install/index.html).
2. Create a Conda environment with Python 3.8 (3.7 or 3.9 would work as well):
```
conda create -n allennlp_env python=3.8
```
3. Activate the Conda environment. You will need to activate the Conda environment in each terminal in which you want to use AllenNLP:
```
conda activate allennlp_env
```
#### Installing the library and dependencies
Installing the library and dependencies is simple using `pip`.
```bash
pip install allennlp
```
To install the optional dependencies, such as `checklist`, run
```bash
pip install allennlp[checklist]
```
Or you can just install all optional dependencies with `pip install allennlp[all]`.
*Looking for bleeding edge features? You can install nightly releases directly from [pypi](https://pypi.org/project/allennlp/#history)*
AllenNLP installs a script when you install the python package, so you can run allennlp commands just by typing `allennlp` into a terminal. For example, you can now test your installation with `allennlp test-install`.
You may also want to install `allennlp-models`, which contains the NLP constructs to train and run our officially
supported models, many of which are hosted at [https://demo.allennlp.org](https://demo.allennlp.org).
```bash
pip install allennlp-models
```
### Installing using Docker
Docker provides a virtual machine with everything set up to run AllenNLP--
whether you will leverage a GPU or just run on a CPU. Docker provides more
isolation and consistency, and also makes it easy to distribute your
environment to a compute cluster.
AllenNLP provides [official Docker images](https://hub.docker.com/r/allennlp/allennlp) with the library and all of its dependencies installed.
Once you have [installed Docker](https://docs.docker.com/engine/installation/),
you should also install the [NVIDIA Container Toolkit](https://github.com/NVIDIA/nvidia-docker)
if you have GPUs available.
Then run the following command to get an environment that will run on GPU:
```bash
mkdir -p $HOME/.allennlp/
docker run --rm --gpus all -v $HOME/.allennlp:/root/.allennlp allennlp/allennlp:latest
```
You can test the Docker environment with
```bash
docker run --rm --gpus all -v $HOME/.allennlp:/root/.allennlp allennlp/allennlp:latest test-install
```
If you don't have GPUs available, just omit the `--gpus all` flag.
#### Building your own Docker image
For various reasons you may need to create your own AllenNLP Docker image, such as if you need a different version
of PyTorch. To do so, just run `make docker-image` from the root of your local clone of AllenNLP.
By default this builds an image with the tag `allennlp/allennlp`, but you can change this to anything you want
by setting the `DOCKER_IMAGE_NAME` flag when you call `make`. For example,
`make docker-image DOCKER_IMAGE_NAME=my-allennlp`.
If you want to use a different version of Python or PyTorch, set the flags `DOCKER_PYTHON_VERSION` and `DOCKER_TORCH_VERSION` to something like
`3.9` and `1.9.0-cuda10.2`, respectively. These flags together determine the base image that is used. You can see the list of valid
combinations in this GitHub Container Registry: [github.com/allenai/docker-images/pkgs/container/pytorch](https://github.com/allenai/docker-images/pkgs/container/pytorch).
After building the image you should be able to see it listed by running `docker images allennlp`.
```
REPOSITORY TAG IMAGE ID CREATED SIZE
allennlp/allennlp latest b66aee6cb593 5 minutes ago 2.38GB
```
### Installing from source
You can also install AllenNLP by cloning our git repository:
```bash
git clone https://github.com/allenai/allennlp.git
```
Create a Python 3.7 or 3.8 virtual environment, and install AllenNLP in `editable` mode by running:
```bash
pip install -U pip setuptools wheel
pip install --editable .[dev,all]
```
This will make `allennlp` available on your system but it will use the sources from the local clone
you made of the source repository.
You can test your installation with `allennlp test-install`.
See [https://github.com/allenai/allennlp-models](https://github.com/allenai/allennlp-models)
for instructions on installing `allennlp-models` from source.
## Running AllenNLP
Once you've installed AllenNLP, you can run the command-line interface
with the `allennlp` command (whether you installed from `pip` or from source).
`allennlp` has various subcommands such as `train`, `evaluate`, and `predict`.
To see the full usage information, run `allennlp --help`.
You can test your installation by running `allennlp test-install`.
## Issues
Everyone is welcome to file issues with either feature requests, bug reports, or general questions. As a small team with our own internal goals, we may ask for contributions if a prompt fix doesn't fit into our roadmap. To keep things tidy we will often close issues we think are answered, but don't hesitate to follow up if further discussion is needed.
## Contributions
The AllenNLP team at AI2 ([@allenai](https://github.com/allenai)) welcomes contributions from the community.
If you're a first time contributor, we recommend you start by reading our [CONTRIBUTING.md](https://github.com/allenai/allennlp/blob/main/CONTRIBUTING.md) guide.
Then have a look at our issues with the tag [**`Good First Issue`**](https://github.com/allenai/allennlp/issues?q=is%3Aissue+is%3Aopen+label%3A%22Good+First+Issue%22).
If you would like to contribute a larger feature, we recommend first creating an issue with a proposed design for discussion. This will prevent you from spending significant time on an implementation which has a technical limitation someone could have pointed out early on. Small contributions can be made directly in a pull request.
Pull requests (PRs) must have one approving review and no requested changes before they are merged. As AllenNLP is primarily driven by AI2 we reserve the right to reject or revert contributions that we don't think are good additions.
## Citing
If you use AllenNLP in your research, please cite [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://www.semanticscholar.org/paper/AllenNLP%3A-A-Deep-Semantic-Natural-Language-Platform-Gardner-Grus/a5502187140cdd98d76ae711973dbcdaf1fef46d).
```bibtex
@inproceedings{Gardner2017AllenNLP,
title={AllenNLP: A Deep Semantic Natural Language Processing Platform},
author={Matt Gardner and Joel Grus and Mark Neumann and Oyvind Tafjord
and Pradeep Dasigi and Nelson F. Liu and Matthew Peters and
Michael Schmitz and Luke S. Zettlemoyer},
year={2017},
Eprint = {arXiv:1803.07640},
}
```
## Team
AllenNLP is an open-source project backed by [the Allen Institute for Artificial Intelligence (AI2)](https://allenai.org/).
AI2 is a non-profit institute with the mission to contribute to humanity through high-impact AI research and engineering.
To learn more about who specifically contributed to this codebase, see [our contributors](https://github.com/allenai/allennlp/graphs/contributors) page.
Raw data
{
"_id": null,
"home_page": "https://github.com/allenai/allennlp",
"name": "allennlp",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7.1",
"maintainer_email": "",
"keywords": "allennlp NLP deep learning machine reading",
"author": "Allen Institute for Artificial Intelligence",
"author_email": "allennlp@allenai.org",
"download_url": "https://files.pythonhosted.org/packages/71/b2/c604c5cc387c0731922e4327749a9976be4a8ea6396b9bed1903c0f470b2/allennlp-2.10.1.tar.gz",
"platform": null,
"description": "<div align=\"center\">\n <br>\n <img src=\"https://raw.githubusercontent.com/allenai/allennlp/main/docs/img/allennlp-logo-dark.png\" width=\"400\"/>\n <p>\n An Apache 2.0 NLP research library, built on PyTorch, for developing state-of-the-art deep learning models on a wide variety of linguistic tasks.\n </p>\n <hr/>\n</div>\n<p align=\"center\">\n <a href=\"https://github.com/allenai/allennlp/actions\">\n <img alt=\"CI\" src=\"https://github.com/allenai/allennlp/workflows/CI/badge.svg?event=push&branch=main\">\n </a>\n <a href=\"https://pypi.org/project/allennlp/\">\n <img alt=\"PyPI\" src=\"https://img.shields.io/pypi/v/allennlp\">\n </a>\n <a href=\"https://github.com/allenai/allennlp/blob/main/LICENSE\">\n <img alt=\"License\" src=\"https://img.shields.io/github/license/allenai/allennlp.svg?color=blue&cachedrop\">\n </a>\n <a href=\"https://codecov.io/gh/allenai/allennlp\">\n <img alt=\"Codecov\" src=\"https://codecov.io/gh/allenai/allennlp/branch/main/graph/badge.svg\">\n </a>\n <a href=\"https://optuna.org\">\n <img alt=\"Optuna\" src=\"https://img.shields.io/badge/Optuna-integrated-blue\">\n </a>\n <br/>\n</p>\n\n\u26a0\ufe0f **NOTICE:** The AllenNLP library is now in maintenance mode. That means we are no longer adding new features or upgrading dependencies. We will still respond to questions and address bugs as they arise up until December 16th, 2022. If you have any concerns or are interested in maintaining AllenNLP going forward, please open an issue on this repository.\n\nAllenNLP has been a big success, but as the field is advancing quickly it's time to focus on new initiatives. We're working hard to make [AI2 Tango](https://github.com/allenai/tango) the best way to organize research codebases. If you are an active user of AllenNLP, here are some suggested alternatives:\n* If you like the trainer, the configuration language, or are simply looking for a better way to manage your experiments, check out [AI2 Tango](https://github.com/allenai/tango).\n* If you like AllenNLP's `modules` and `nn` packages, check out [delmaksym/allennlp-light](https://github.com/delmaksym/allennlp-light). It's even compatible with [AI2 Tango](https://github.com/allenai/tango)!\n* If you like the framework aspect of AllenNLP, check out [flair](https://github.com/flairNLP/flair). It has multiple state-of-art NLP models and allows you to easily use pretrained embeddings such as those from transformers.\n* If you like the AllenNLP metrics package, check out [torchmetrics](https://torchmetrics.readthedocs.io/en/stable/). It has the same API as AllenNLP, so it should be a quick learning curve to make the switch.\n* If you want to vectorize text, try [the transformers library](https://github.com/huggingface/transformers).\n* If you want to maintain the AllenNLP Fairness or Interpret components, please get in touch. There is no alternative to it, so we are looking for a dedicated maintainer.\n* If you are concerned about other AllenNLP functionality, please create an issue. Maybe we can find another way to continue supporting your use case.\n\n## Quick Links\n\n- \u2197\ufe0f [Website](https://allennlp.org/)\n- \ud83d\udd26 [Guide](https://guide.allennlp.org/)\n- \ud83d\uddbc [Gallery](https://gallery.allennlp.org)\n- \ud83d\udcbb [Demo](https://demo.allennlp.org)\n- \ud83d\udcd3 [Documentation](https://docs.allennlp.org/) ( [latest](https://docs.allennlp.org/latest/) | [stable](https://docs.allennlp.org/stable/) | [commit](https://docs.allennlp.org/main/) )\n- \u2b06\ufe0f [Upgrade Guide from 1.x to 2.0](https://github.com/allenai/allennlp/discussions/4933)\n- \u2753 [Stack Overflow](https://stackoverflow.com/questions/tagged/allennlp)\n- \u270b [Contributing Guidelines](CONTRIBUTING.md)\n- \ud83e\udd16 [Officially Supported Models](https://github.com/allenai/allennlp-models)\n - [Pretrained Models](https://github.com/allenai/allennlp-models/blob/main/allennlp_models/pretrained.py)\n - [Documentation](https://docs.allennlp.org/models/) ( [latest](https://docs.allennlp.org/models/latest/) | [stable](https://docs.allennlp.org/models/stable/) | [commit](https://docs.allennlp.org/models/main/) )\n- \u2699\ufe0f [Continuous Build](https://github.com/allenai/allennlp/actions)\n- \ud83c\udf19 [Nightly Releases](https://pypi.org/project/allennlp/#history)\n\n## In this README\n\n- [Getting Started Using the Library](#getting-started-using-the-library)\n- [Plugins](#plugins)\n- [Package Overview](#package-overview)\n- [Installation](#installation)\n - [Installing via pip](#installing-via-pip)\n - [Installing using Docker](#installing-using-docker)\n - [Installing from source](#installing-from-source)\n- [Running AllenNLP](#running-allennlp)\n- [Issues](#issues)\n- [Contributions](#contributions)\n- [Citing](#citing)\n- [Team](#team)\n\n## Getting Started Using the Library\n\nIf you're interested in using AllenNLP for model development, we recommend you check out the\n[AllenNLP Guide](https://guide.allennlp.org) for a thorough introduction to the library, followed by our more advanced guides\non [GitHub Discussions](https://github.com/allenai/allennlp/discussions/categories/guides).\n\nWhen you're ready to start your project, we've created a couple of template repositories that you can use as a starting place:\n\n* If you want to use `allennlp train` and config files to specify experiments, use [this\n template](https://github.com/allenai/allennlp-template-config-files). We recommend this approach.\n* If you'd prefer to use python code to configure your experiments and run your training loop, use\n [this template](https://github.com/allenai/allennlp-template-python-script). There are a few\n things that are currently a little harder in this setup (loading a saved model, and using\n distributed training), but otherwise it's functionality equivalent to the config files\n setup.\n\nIn addition, there are external tutorials:\n\n* [Hyperparameter optimization for AllenNLP using Optuna](https://medium.com/optuna/hyperparameter-optimization-for-allennlp-using-optuna-54b4bfecd78b)\n* [Training with multiple GPUs in AllenNLP](https://medium.com/ai2-blog/tutorial-how-to-train-with-multiple-gpus-in-allennlp-c4d7c17eb6d6)\n* [Training on larger batches with less memory in AllenNLP](https://medium.com/ai2-blog/tutorial-training-on-larger-batches-with-less-memory-in-allennlp-1cd2047d92ad)\n* [How to upload transformer weights and tokenizers from AllenNLP to HuggingFace](https://medium.com/ai2-blog/tutorial-how-to-upload-transformer-weights-and-tokenizers-from-allennlp-to-huggingface-ecf6c0249bf)\n\nAnd others on the [AI2 AllenNLP blog](https://medium.com/ai2-blog/allennlp/home).\n\n## Plugins\n\nAllenNLP supports loading \"plugins\" dynamically. A plugin is just a Python package that\nprovides custom registered classes or additional `allennlp` subcommands.\n\nThere is ecosystem of open source plugins, some of which are maintained by the AllenNLP\nteam here at AI2, and some of which are maintained by the broader community.\n\n<table>\n<tr>\n <td><b> Plugin </b></td>\n <td><b> Maintainer </b></td>\n <td><b> CLI </b></td>\n <td><b> Description </b></td>\n</tr>\n<tr>\n <td> <a href=\"https://github.com/allenai/allennlp-models\"><b>allennlp-models</b></a> </td>\n <td> AI2 </td>\n <td> No </td>\n <td> A collection of state-of-the-art models </td>\n</tr>\n<tr>\n <td> <a href=\"https://github.com/allenai/allennlp-semparse\"><b>allennlp-semparse</b></a> </td>\n <td> AI2 </td>\n <td> No </td>\n <td> A framework for building semantic parsers </td>\n</tr>\n<tr>\n <td> <a href=\"https://github.com/allenai/allennlp-server\"><b>allennlp-server</b></a> </td>\n <td> AI2 </td>\n <td> Yes </td>\n <td> A simple demo server for serving models </td>\n</tr>\n<tr>\n <td> <a href=\"https://github.com/himkt/allennlp-optuna\"><b>allennlp-optuna</b></a> </td>\n <td> <a href=\"https://himkt.github.io/profile/\">Makoto Hiramatsu</a> </td>\n <td> Yes </td>\n <td> <a href=\"https://optuna.org/\">Optuna</a> integration for hyperparameter optimization </td>\n</tr>\n</table>\n\nAllenNLP will automatically find any official AI2-maintained plugins that you have installed,\nbut for AllenNLP to find personal or third-party plugins you've installed,\nyou also have to create either a local plugins file named `.allennlp_plugins`\nin the directory where you run the `allennlp` command, or a global plugins file at `~/.allennlp/plugins`.\nThe file should list the plugin modules that you want to be loaded, one per line.\n\nTo test that your plugins can be found and imported by AllenNLP, you can run the `allennlp test-install` command.\nEach discovered plugin will be logged to the terminal.\n\nFor more information about plugins, see the [plugins API docs](https://docs.allennlp.org/main/api/common/plugins/). And for information on how to create a custom subcommand\nto distribute as a plugin, see the [subcommand API docs](https://docs.allennlp.org/main/api/commands/subcommand/).\n\n## Package Overview\n\n<table>\n<tr>\n <td><b> allennlp </b></td>\n <td> An open-source NLP research library, built on PyTorch </td>\n</tr>\n<tr>\n <td><b> allennlp.commands </b></td>\n <td> Functionality for the CLI </td>\n</tr>\n<tr>\n <td><b> allennlp.common </b></td>\n <td> Utility modules that are used across the library </td>\n</tr>\n<tr>\n <td><b> allennlp.data </b></td>\n <td> A data processing module for loading datasets and encoding strings as integers for representation in matrices </td>\n</tr>\n<tr>\n <td><b> allennlp.fairness </b></td>\n <td> A module for bias mitigation and fairness algorithms and metrics </td>\n</tr>\n<tr>\n <td><b> allennlp.modules </b></td>\n <td> A collection of PyTorch modules for use with text </td>\n</tr>\n<tr>\n <td><b> allennlp.nn </b></td>\n <td> Tensor utility functions, such as initializers and activation functions </td>\n</tr>\n<tr>\n <td><b> allennlp.training </b></td>\n <td> Functionality for training models </td>\n</tr>\n</table>\n\n## Installation\n\nAllenNLP requires Python 3.6.1 or later and [PyTorch](https://pytorch.org/).\n\nWe support AllenNLP on Mac and Linux environments. We presently do not support Windows but are open to contributions.\n\n### Installing via conda-forge\n\nThe simplest way to install AllenNLP is using conda (you can choose a different python version):\n\n```\nconda install -c conda-forge python=3.8 allennlp\n```\n\nTo install optional packages, such as `checklist`, use\n\n```\nconda install -c conda-forge allennlp-checklist\n```\n\nor simply install `allennlp-all` directly. The plugins mentioned above are similarly installable, e.g.\n\n```\nconda install -c conda-forge allennlp-models allennlp-semparse allennlp-server allennlp-optuna\n```\n\n### Installing via pip\n\nIt's recommended that you install the PyTorch ecosystem **before** installing AllenNLP by following the instructions on [pytorch.org](https://pytorch.org/).\n\nAfter that, just run `pip install allennlp`.\n\n\n\n> \u26a0\ufe0f If you're using Python 3.7 or greater, you should ensure that you don't have the PyPI version of `dataclasses` installed after running the above command, as this could cause issues on certain platforms. You can quickly check this by running `pip freeze | grep dataclasses`. If you see something like `dataclasses=0.6` in the output, then just run `pip uninstall -y dataclasses`.\n\nIf you need pointers on setting up an appropriate Python environment or would like to install AllenNLP using a different method, see below.\n\n#### Setting up a virtual environment\n\n[Conda](https://conda.io/) can be used set up a virtual environment with the\nversion of Python required for AllenNLP. If you already have a Python 3\nenvironment you want to use, you can skip to the 'installing via pip' section.\n\n1. [Download and install Conda](https://conda.io/projects/conda/en/latest/user-guide/install/index.html).\n\n2. Create a Conda environment with Python 3.8 (3.7 or 3.9 would work as well):\n\n ```\n conda create -n allennlp_env python=3.8\n ```\n\n3. Activate the Conda environment. You will need to activate the Conda environment in each terminal in which you want to use AllenNLP:\n\n ```\n conda activate allennlp_env\n ```\n\n#### Installing the library and dependencies\n\nInstalling the library and dependencies is simple using `pip`.\n\n```bash\npip install allennlp\n```\n\nTo install the optional dependencies, such as `checklist`, run\n\n```bash\npip install allennlp[checklist]\n```\nOr you can just install all optional dependencies with `pip install allennlp[all]`.\n\n*Looking for bleeding edge features? You can install nightly releases directly from [pypi](https://pypi.org/project/allennlp/#history)*\n\nAllenNLP installs a script when you install the python package, so you can run allennlp commands just by typing `allennlp` into a terminal. For example, you can now test your installation with `allennlp test-install`.\n\nYou may also want to install `allennlp-models`, which contains the NLP constructs to train and run our officially\nsupported models, many of which are hosted at [https://demo.allennlp.org](https://demo.allennlp.org).\n\n```bash\npip install allennlp-models\n```\n\n### Installing using Docker\n\nDocker provides a virtual machine with everything set up to run AllenNLP--\nwhether you will leverage a GPU or just run on a CPU. Docker provides more\nisolation and consistency, and also makes it easy to distribute your\nenvironment to a compute cluster.\n\nAllenNLP provides [official Docker images](https://hub.docker.com/r/allennlp/allennlp) with the library and all of its dependencies installed.\n\nOnce you have [installed Docker](https://docs.docker.com/engine/installation/),\nyou should also install the [NVIDIA Container Toolkit](https://github.com/NVIDIA/nvidia-docker)\nif you have GPUs available.\n\nThen run the following command to get an environment that will run on GPU:\n\n```bash\nmkdir -p $HOME/.allennlp/\ndocker run --rm --gpus all -v $HOME/.allennlp:/root/.allennlp allennlp/allennlp:latest\n```\n\nYou can test the Docker environment with\n\n```bash\ndocker run --rm --gpus all -v $HOME/.allennlp:/root/.allennlp allennlp/allennlp:latest test-install \n```\n\nIf you don't have GPUs available, just omit the `--gpus all` flag.\n\n#### Building your own Docker image\n\nFor various reasons you may need to create your own AllenNLP Docker image, such as if you need a different version\nof PyTorch. To do so, just run `make docker-image` from the root of your local clone of AllenNLP.\n\nBy default this builds an image with the tag `allennlp/allennlp`, but you can change this to anything you want\nby setting the `DOCKER_IMAGE_NAME` flag when you call `make`. For example,\n`make docker-image DOCKER_IMAGE_NAME=my-allennlp`.\n\nIf you want to use a different version of Python or PyTorch, set the flags `DOCKER_PYTHON_VERSION` and `DOCKER_TORCH_VERSION` to something like\n`3.9` and `1.9.0-cuda10.2`, respectively. These flags together determine the base image that is used. You can see the list of valid\ncombinations in this GitHub Container Registry: [github.com/allenai/docker-images/pkgs/container/pytorch](https://github.com/allenai/docker-images/pkgs/container/pytorch).\n\nAfter building the image you should be able to see it listed by running `docker images allennlp`.\n\n```\nREPOSITORY TAG IMAGE ID CREATED SIZE\nallennlp/allennlp latest b66aee6cb593 5 minutes ago 2.38GB\n```\n\n### Installing from source\n\nYou can also install AllenNLP by cloning our git repository:\n\n```bash\ngit clone https://github.com/allenai/allennlp.git\n```\n\nCreate a Python 3.7 or 3.8 virtual environment, and install AllenNLP in `editable` mode by running:\n\n```bash\npip install -U pip setuptools wheel\npip install --editable .[dev,all]\n```\n\nThis will make `allennlp` available on your system but it will use the sources from the local clone\nyou made of the source repository.\n\nYou can test your installation with `allennlp test-install`.\nSee [https://github.com/allenai/allennlp-models](https://github.com/allenai/allennlp-models)\nfor instructions on installing `allennlp-models` from source.\n\n## Running AllenNLP\n\nOnce you've installed AllenNLP, you can run the command-line interface\nwith the `allennlp` command (whether you installed from `pip` or from source).\n`allennlp` has various subcommands such as `train`, `evaluate`, and `predict`.\nTo see the full usage information, run `allennlp --help`.\n\nYou can test your installation by running `allennlp test-install`.\n\n## Issues\n\nEveryone is welcome to file issues with either feature requests, bug reports, or general questions. As a small team with our own internal goals, we may ask for contributions if a prompt fix doesn't fit into our roadmap. To keep things tidy we will often close issues we think are answered, but don't hesitate to follow up if further discussion is needed.\n\n## Contributions\n\nThe AllenNLP team at AI2 ([@allenai](https://github.com/allenai)) welcomes contributions from the community. \nIf you're a first time contributor, we recommend you start by reading our [CONTRIBUTING.md](https://github.com/allenai/allennlp/blob/main/CONTRIBUTING.md) guide.\nThen have a look at our issues with the tag [**`Good First Issue`**](https://github.com/allenai/allennlp/issues?q=is%3Aissue+is%3Aopen+label%3A%22Good+First+Issue%22).\n\nIf you would like to contribute a larger feature, we recommend first creating an issue with a proposed design for discussion. This will prevent you from spending significant time on an implementation which has a technical limitation someone could have pointed out early on. Small contributions can be made directly in a pull request.\n\nPull requests (PRs) must have one approving review and no requested changes before they are merged. As AllenNLP is primarily driven by AI2 we reserve the right to reject or revert contributions that we don't think are good additions.\n\n## Citing\n\nIf you use AllenNLP in your research, please cite [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://www.semanticscholar.org/paper/AllenNLP%3A-A-Deep-Semantic-Natural-Language-Platform-Gardner-Grus/a5502187140cdd98d76ae711973dbcdaf1fef46d).\n\n```bibtex\n@inproceedings{Gardner2017AllenNLP,\n title={AllenNLP: A Deep Semantic Natural Language Processing Platform},\n author={Matt Gardner and Joel Grus and Mark Neumann and Oyvind Tafjord\n and Pradeep Dasigi and Nelson F. Liu and Matthew Peters and\n Michael Schmitz and Luke S. Zettlemoyer},\n year={2017},\n Eprint = {arXiv:1803.07640},\n}\n```\n\n## Team\n\nAllenNLP is an open-source project backed by [the Allen Institute for Artificial Intelligence (AI2)](https://allenai.org/).\nAI2 is a non-profit institute with the mission to contribute to humanity through high-impact AI research and engineering.\nTo learn more about who specifically contributed to this codebase, see [our contributors](https://github.com/allenai/allennlp/graphs/contributors) page.\n",
"bugtrack_url": null,
"license": "Apache",
"summary": "An open-source NLP research library, built on PyTorch.",
"version": "2.10.1",
"split_keywords": [
"allennlp",
"nlp",
"deep",
"learning",
"machine",
"reading"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d38025f92a8dcc47d86da251766107aa39dae1b903f5cfaddbf0e7cf33bc5957",
"md5": "886301f10c09727b633c46edd1226b42",
"sha256": "d840f2cb285ded77f48fbe5c8f52fe955dc48d4462cae2baf4668eca10dd42fb"
},
"downloads": -1,
"filename": "allennlp-2.10.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "886301f10c09727b633c46edd1226b42",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7.1",
"size": 730227,
"upload_time": "2022-10-18T23:52:13",
"upload_time_iso_8601": "2022-10-18T23:52:13.959855Z",
"url": "https://files.pythonhosted.org/packages/d3/80/25f92a8dcc47d86da251766107aa39dae1b903f5cfaddbf0e7cf33bc5957/allennlp-2.10.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "71b2c604c5cc387c0731922e4327749a9976be4a8ea6396b9bed1903c0f470b2",
"md5": "54fe0cc25611d04c0ca7b2dda263d316",
"sha256": "5297e768f43a8471c90ed956d675cbc17eca6b8ecfd13626a2a95072913182f8"
},
"downloads": -1,
"filename": "allennlp-2.10.1.tar.gz",
"has_sig": false,
"md5_digest": "54fe0cc25611d04c0ca7b2dda263d316",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7.1",
"size": 574511,
"upload_time": "2022-10-18T23:52:16",
"upload_time_iso_8601": "2022-10-18T23:52:16.722443Z",
"url": "https://files.pythonhosted.org/packages/71/b2/c604c5cc387c0731922e4327749a9976be4a8ea6396b9bed1903c0f470b2/allennlp-2.10.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-10-18 23:52:16",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "allenai",
"github_project": "allennlp",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"requirements": [],
"lcname": "allennlp"
}
JSON
