frame-semantic-transformer


Nameframe-semantic-transformer JSON
Version 0.4.1 PyPI version JSON
download
home_pagehttps://github.com/chanind/frame-semantic-transformer
SummaryFrame Semantic Parser based on T5 and FrameNet
upload_time2022-05-25 12:53:48
maintainer
docs_urlNone
authorDavid Chanin
requires_python>=3.7,<4.0
licenseMIT
keywords
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # Frame Semantic Transformer

[![ci](https://img.shields.io/github/workflow/status/chanind/frame-semantic-transformer/CI/main)](https://github.com/chanind/frame-semantic-transformer)
[![PyPI](https://img.shields.io/pypi/v/frame-semantic-transformer?color=blue)](https://pypi.org/project/frame-semantic-transformer/)

Frame-based semantic parsing library trained on [FrameNet](https://framenet2.icsi.berkeley.edu/) and built on HuggingFace's [T5 Transformer](https://huggingface.co/docs/transformers/model_doc/t5). This library is designed to be easy to use, yet powerful.

**Live Demo: [chanind.github.io/frame-semantic-transformer](https://chanind.github.io/frame-semantic-transformer)**

This library draws heavily on [Open-Sesame](https://github.com/swabhs/open-sesame) ([paper](https://arxiv.org/abs/1706.09528)) for inspiration on training and evaluation on FrameNet 1.7, and uses ideas from the paper [Open-Domain Frame Semantic Parsing Using Transformers](https://arxiv.org/abs/2010.10998) for using T5 as a frame-semantic parser. [SimpleT5](https://github.com/Shivanandroy/simpleT5) was also used as a base for the initial training setup.

## Performance

This library uses the same train/dev/test documents and evaluation methodology as Open-Sesame, so that the results should be comparable between the 2 libraries. There are 2 pretrained models available, `base` and `small`, corresponding to `t5-base` and `t5-small` in Huggingface, respectively.

| Task                   | Sesame F1 (dev/test) | Small Model F1 (dev/test) | Base Model F1 (dev/test) |
| ---------------------- | -------------------- | ------------------------- | ------------------------ |
| Trigger identification | 0.80 / 0.73          | 0.74 / 0.70               | 0.78 / 0.71              |
| Frame classification   | 0.90 / 0.87          | 0.83 / 0.81               | 0.89 / 0.87              |
| Argument extraction    | 0.61 / 0.61          | 0.68 / 0.70               | 0.74 / 0.72              |

The base model performs similarly to Open-Sesame on trigger identification and frame classification tasks, but outperforms it by a significant margin on argument extraction. The small pretrained model has lower F1 than base across the board, but is 1/4 the size and is still comparable to Open-Sesame at argument extraction.

## Installation

```
pip install frame-semantic-transformer
```

## Usage

### Inference

The main entry to interacting with the library is the `FrameSemanticTransformer` class, as shown below. For inference the `detect_frames()` method is likely all that is needed to perform frame parsing.

```python
from frame_semantic_transformer import FrameSemanticTransformer

frame_transformer = FrameSemanticTransformer()

result = frame_transformer.detect_frames("The hallway smelt of boiled cabbage and old rag mats.")

print(f"Results found in: {result.sentence}")
for frames in result.frames:
    print(f"FRAME: {frame.name}")
    for element in frame.frame_elements:
        print(f"{element.name}: {element.text}")
```

The result returned from `detect_frames()` is an object containing `sentence`, a parsed version of the original sentence text, `trigger_locations`, the indices within the sentence where frame triggers were detected, and `frames`, a list of all detected frames in the sentence. Within `frames`, each object containes `name` which corresponds to the FrameNet name of the frame, `trigger_location` corresponding to which trigger in the text this frame this frame uses, and `frame_elements` containing a list of all relevant frame elements found in the text.

### Loading Models

There are currently 2 available pre-trained models for inference, called `base` and `small`, fine-tuned from HuggingFace's [t5-base](https://huggingface.co/t5-base) and [t5-small](https://huggingface.co/t5-base) model respectively. If a local fine-tuned t5 model exists that can be loaded as well. If no model is specified, the `base` model will be used.

```
base_transformer = FrameSemanticTransformer("base") # this is also the default
small_transformer = FrameSemanticTransformer("small") # a smaller pretrained model which is faster to run
custom_transformer = FrameSemanticTransformer("/path/to/model") # load a custom t5 model
```

By default, models are lazily loaded when `detect_frames()` is first called. If you want to load the model sooner, you can call `setup()` on a `FrameSemanticTransformer` instance to load models immediately.

```
frame_transformer = FrameSemanticTransformer()
frame_transformer.setup() # load models immediately
```

### Training

If you want to train a new model yourself, you can run the training script like below:

```
python -m frame_semantic_transformer.train \
    --base-model t5-base \
    --use-gpu \
    --batch-size 8 \
    --epochs 10 \
    --learning-rate 5e-5 \
    --output-dir ./outputs
```

Training uses [Pytorch Lightning](https://www.pytorchlightning.ai/) behind the scenes, and will place tensorboard logs into `./lightning_logs` as it trains.

If you need more control, you can also directly import the `train()` method from `frame_semantic_transformer.train` and run training directly in code.

## Contributing

Any contributions to improve this project are welcome! Please open an issue or pull request in this repo with any bugfixes / changes / improvements you have!

This project uses [Black](https://github.com/psf/black) for code formatting, [Flake8](https://flake8.pycqa.org/en/latest/) for linting, and [Pytest](https://docs.pytest.org/) for tests. Make sure any changes you submit pass these code checks in your PR. If you have trouble getting these to run feel free to open a pull-request regardless and we can discuss further in the PR.

## License

The code contained in this repo is released under a MIT license, however the pretrained models are released under an Apache 2.0 license in accordance with FrameNet training data and HuggingFace's T5 base models.


            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/chanind/frame-semantic-transformer",
    "name": "frame-semantic-transformer",
    "maintainer": "",
    "docs_url": null,
    "requires_python": ">=3.7,<4.0",
    "maintainer_email": "",
    "keywords": "",
    "author": "David Chanin",
    "author_email": "chanindav@gmail.com",
    "download_url": "https://files.pythonhosted.org/packages/91/61/40943238a0ed6ed31d35eca4f4dba9cba9d3632b41e4f8f9954be81cb884/frame-semantic-transformer-0.4.1.tar.gz",
    "platform": null,
    "description": "# Frame Semantic Transformer\n\n[![ci](https://img.shields.io/github/workflow/status/chanind/frame-semantic-transformer/CI/main)](https://github.com/chanind/frame-semantic-transformer)\n[![PyPI](https://img.shields.io/pypi/v/frame-semantic-transformer?color=blue)](https://pypi.org/project/frame-semantic-transformer/)\n\nFrame-based semantic parsing library trained on [FrameNet](https://framenet2.icsi.berkeley.edu/) and built on HuggingFace's [T5 Transformer](https://huggingface.co/docs/transformers/model_doc/t5). This library is designed to be easy to use, yet powerful.\n\n**Live Demo: [chanind.github.io/frame-semantic-transformer](https://chanind.github.io/frame-semantic-transformer)**\n\nThis library draws heavily on [Open-Sesame](https://github.com/swabhs/open-sesame) ([paper](https://arxiv.org/abs/1706.09528)) for inspiration on training and evaluation on FrameNet 1.7, and uses ideas from the paper [Open-Domain Frame Semantic Parsing Using Transformers](https://arxiv.org/abs/2010.10998) for using T5 as a frame-semantic parser. [SimpleT5](https://github.com/Shivanandroy/simpleT5) was also used as a base for the initial training setup.\n\n## Performance\n\nThis library uses the same train/dev/test documents and evaluation methodology as Open-Sesame, so that the results should be comparable between the 2 libraries. There are 2 pretrained models available, `base` and `small`, corresponding to `t5-base` and `t5-small` in Huggingface, respectively.\n\n| Task                   | Sesame F1 (dev/test) | Small Model F1 (dev/test) | Base Model F1 (dev/test) |\n| ---------------------- | -------------------- | ------------------------- | ------------------------ |\n| Trigger identification | 0.80 / 0.73          | 0.74 / 0.70               | 0.78 / 0.71              |\n| Frame classification   | 0.90 / 0.87          | 0.83 / 0.81               | 0.89 / 0.87              |\n| Argument extraction    | 0.61 / 0.61          | 0.68 / 0.70               | 0.74 / 0.72              |\n\nThe base model performs similarly to Open-Sesame on trigger identification and frame classification tasks, but outperforms it by a significant margin on argument extraction. The small pretrained model has lower F1 than base across the board, but is 1/4 the size and is still comparable to Open-Sesame at argument extraction.\n\n## Installation\n\n```\npip install frame-semantic-transformer\n```\n\n## Usage\n\n### Inference\n\nThe main entry to interacting with the library is the `FrameSemanticTransformer` class, as shown below. For inference the `detect_frames()` method is likely all that is needed to perform frame parsing.\n\n```python\nfrom frame_semantic_transformer import FrameSemanticTransformer\n\nframe_transformer = FrameSemanticTransformer()\n\nresult = frame_transformer.detect_frames(\"The hallway smelt of boiled cabbage and old rag mats.\")\n\nprint(f\"Results found in: {result.sentence}\")\nfor frames in result.frames:\n    print(f\"FRAME: {frame.name}\")\n    for element in frame.frame_elements:\n        print(f\"{element.name}: {element.text}\")\n```\n\nThe result returned from `detect_frames()` is an object containing `sentence`, a parsed version of the original sentence text, `trigger_locations`, the indices within the sentence where frame triggers were detected, and `frames`, a list of all detected frames in the sentence. Within `frames`, each object containes `name` which corresponds to the FrameNet name of the frame, `trigger_location` corresponding to which trigger in the text this frame this frame uses, and `frame_elements` containing a list of all relevant frame elements found in the text.\n\n### Loading Models\n\nThere are currently 2 available pre-trained models for inference, called `base` and `small`, fine-tuned from HuggingFace's [t5-base](https://huggingface.co/t5-base) and [t5-small](https://huggingface.co/t5-base) model respectively. If a local fine-tuned t5 model exists that can be loaded as well. If no model is specified, the `base` model will be used.\n\n```\nbase_transformer = FrameSemanticTransformer(\"base\") # this is also the default\nsmall_transformer = FrameSemanticTransformer(\"small\") # a smaller pretrained model which is faster to run\ncustom_transformer = FrameSemanticTransformer(\"/path/to/model\") # load a custom t5 model\n```\n\nBy default, models are lazily loaded when `detect_frames()` is first called. If you want to load the model sooner, you can call `setup()` on a `FrameSemanticTransformer` instance to load models immediately.\n\n```\nframe_transformer = FrameSemanticTransformer()\nframe_transformer.setup() # load models immediately\n```\n\n### Training\n\nIf you want to train a new model yourself, you can run the training script like below:\n\n```\npython -m frame_semantic_transformer.train \\\n    --base-model t5-base \\\n    --use-gpu \\\n    --batch-size 8 \\\n    --epochs 10 \\\n    --learning-rate 5e-5 \\\n    --output-dir ./outputs\n```\n\nTraining uses [Pytorch Lightning](https://www.pytorchlightning.ai/) behind the scenes, and will place tensorboard logs into `./lightning_logs` as it trains.\n\nIf you need more control, you can also directly import the `train()` method from `frame_semantic_transformer.train` and run training directly in code.\n\n## Contributing\n\nAny contributions to improve this project are welcome! Please open an issue or pull request in this repo with any bugfixes / changes / improvements you have!\n\nThis project uses [Black](https://github.com/psf/black) for code formatting, [Flake8](https://flake8.pycqa.org/en/latest/) for linting, and [Pytest](https://docs.pytest.org/) for tests. Make sure any changes you submit pass these code checks in your PR. If you have trouble getting these to run feel free to open a pull-request regardless and we can discuss further in the PR.\n\n## License\n\nThe code contained in this repo is released under a MIT license, however the pretrained models are released under an Apache 2.0 license in accordance with FrameNet training data and HuggingFace's T5 base models.\n\n",
    "bugtrack_url": null,
    "license": "MIT",
    "summary": "Frame Semantic Parser based on T5 and FrameNet",
    "version": "0.4.1",
    "split_keywords": [],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "49e8852a91a31fcd31fedb5c3c19d018",
                "sha256": "422313a1cf0b91c83846fe0f88f0d18237008556478b355790d27578e9654317"
            },
            "downloads": -1,
            "filename": "frame_semantic_transformer-0.4.1-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "49e8852a91a31fcd31fedb5c3c19d018",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.7,<4.0",
            "size": 28988,
            "upload_time": "2022-05-25T12:53:46",
            "upload_time_iso_8601": "2022-05-25T12:53:46.940771Z",
            "url": "https://files.pythonhosted.org/packages/3a/e2/35c1364134ef56ea79d4e6f5850dc846d7f045778579e73f39ed2a36d458/frame_semantic_transformer-0.4.1-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "cc221d6e8e3b51ead90a66e1025e3127",
                "sha256": "e6af151bfdd25e3dde3301b9b7907c54fb49b7946ea66b190ca598c19f992840"
            },
            "downloads": -1,
            "filename": "frame-semantic-transformer-0.4.1.tar.gz",
            "has_sig": false,
            "md5_digest": "cc221d6e8e3b51ead90a66e1025e3127",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.7,<4.0",
            "size": 21330,
            "upload_time": "2022-05-25T12:53:48",
            "upload_time_iso_8601": "2022-05-25T12:53:48.492807Z",
            "url": "https://files.pythonhosted.org/packages/91/61/40943238a0ed6ed31d35eca4f4dba9cba9d3632b41e4f8f9954be81cb884/frame-semantic-transformer-0.4.1.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2022-05-25 12:53:48",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "github_user": "chanind",
    "github_project": "frame-semantic-transformer",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "frame-semantic-transformer"
}
        
Elapsed time: 0.46039s