lmdx-flow

Name	lmdx-flow JSON
Version	0.1.2 JSON
	download
home_page	https://github.com/JazBern/LMDX-flow
Summary	Python toolkit for document information extraction using LMDX
upload_time	2023-10-15 05:58:37
maintainer
docs_url	None
author	JazBern
requires_python	>=3.9,<3.13
license
keywords
VCS
bugtrack_url
requirements	No requirements were recorded.
Travis-CI	No Travis.
coveralls test coverage	No coveralls.

            # LMDX-flow

LMDX-flow is a Python toolkit designed for document information extraction using LMDX. 
It simplifies the process of creating prompts which contain document layout information and decoding LLM responses to extract valuable information from documents.

## What is LMDX: LANGUAGE MODEL-BASED DOCUMENT INFORMATION EXTRACTION AND LOCALIZATION?

LMDX is a methodology for leveraging off-the-shelf LLMs for information extraction on
semi-structured documents.

Paper : https://arxiv.org/pdf/2309.10952.pdf

- Proposes a prompt that enables LLMs to perform the document IE task on leaf and
hierarchical entities with precise localization, including without any training data.
- Proposes a layout encoding scheme that communicate spatial information to the
LLM without any change to its architecture.
- Introduces a decoding algorithm transforming the LLM responses into extracted entities
and their bounding boxes on the document, while discarding all hallucination.


## Key Features

- **Prompt Generation:** Easily create effective prompts based on the LMDX methodology.
- **Response Decoding:** Extract entity values and bounding boxes by decoding and grounding the LLM responses.

## Getting Started

* [Install tesseract-OCR](https://tesseract-ocr.github.io/tessdoc/Installation.html)
* [Install poppler-utils](https://github.com/Belval/pdf2image#platform-independant-using-conda)

```shell
pip install lmdx-flow
```
```python
## Load the tokenizer
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")

from lmdx_flow import Pipeline
P = Pipeline(file_path,tokenizer)
prompts = P.generate_prompt(schema)
answers = P.postprocess_all_chunks(llm_responses)
```

## To-do

- Add support for hierarchical entities
- Add option to use OCR-words as segment (currently uses OCR-lines as segment)

## References
* [LMDX paper](https://arxiv.org/pdf/2309.10952.pdf)
* [Hugging Face Transformers Library](https://github.com/huggingface/transformers)

Explore the potential of LMDX-flow to enhance document information extraction using LLMs with ease.

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/JazBern/LMDX-flow",
    "name": "lmdx-flow",
    "maintainer": "",
    "docs_url": null,
    "requires_python": ">=3.9,<3.13",
    "maintainer_email": "",
    "keywords": "",
    "author": "JazBern",
    "author_email": "jasmine0696@gmail.com",
    "download_url": "https://files.pythonhosted.org/packages/ca/39/c87f0f34b0f68b0a58ca1ccc6ac36e0068e007b560506e630a32c0bfff2f/lmdx_flow-0.1.2.tar.gz",
    "platform": null,
    "description": "# LMDX-flow\n\nLMDX-flow is a Python toolkit designed for document information extraction using LMDX. \nIt simplifies the process of creating prompts which contain document layout information and decoding LLM responses to extract valuable information from documents.\n\n## What is LMDX: LANGUAGE MODEL-BASED DOCUMENT INFORMATION EXTRACTION AND LOCALIZATION?\n\nLMDX is a methodology for leveraging off-the-shelf LLMs for information extraction on\nsemi-structured documents.\n\nPaper : https://arxiv.org/pdf/2309.10952.pdf\n\n- Proposes a prompt that enables LLMs to perform the document IE task on leaf and\nhierarchical entities with precise localization, including without any training data.\n- Proposes a layout encoding scheme that communicate spatial information to the\nLLM without any change to its architecture.\n- Introduces a decoding algorithm transforming the LLM responses into extracted entities\nand their bounding boxes on the document, while discarding all hallucination.\n\n\n## Key Features\n\n- **Prompt Generation:** Easily create effective prompts based on the LMDX methodology.\n- **Response Decoding:** Extract entity values and bounding boxes by decoding and grounding the LLM responses.\n\n## Getting Started\n\n* [Install tesseract-OCR](https://tesseract-ocr.github.io/tessdoc/Installation.html)\n* [Install poppler-utils](https://github.com/Belval/pdf2image#platform-independant-using-conda)\n\n```shell\npip install lmdx-flow\n```\n```python\n## Load the tokenizer\nfrom transformers import AutoTokenizer\ntokenizer = AutoTokenizer.from_pretrained(\"bert-base-uncased\")\n\nfrom lmdx_flow import Pipeline\nP = Pipeline(file_path,tokenizer)\nprompts = P.generate_prompt(schema)\nanswers = P.postprocess_all_chunks(llm_responses)\n```\n\n## To-do\n\n- Add support for hierarchical entities\n- Add option to use OCR-words as segment (currently uses OCR-lines as segment)\n\n## References\n* [LMDX paper](https://arxiv.org/pdf/2309.10952.pdf)\n* [Hugging Face Transformers Library](https://github.com/huggingface/transformers)\n\nExplore the potential of LMDX-flow to enhance document information extraction using LLMs with ease.",
    "bugtrack_url": null,
    "license": "",
    "summary": "Python toolkit for document information extraction using LMDX",
    "version": "0.1.2",
    "project_urls": {
        "Homepage": "https://github.com/JazBern/LMDX-flow",
        "Repository": "https://github.com/JazBern/LMDX-flow"
    },
    "split_keywords": [],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "d9218b7d08aee58c497ac9a414b2a797abbbad099754137224ed2b5e7346e290",
                "md5": "8ff3176eaac4fe99ebc27d5bbcabfd2f",
                "sha256": "83b033688c4d0c9494e40089e6327fbfb75ad3e27b7bce603f886dd6841b2d50"
            },
            "downloads": -1,
            "filename": "lmdx_flow-0.1.2-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "8ff3176eaac4fe99ebc27d5bbcabfd2f",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.9,<3.13",
            "size": 5729,
            "upload_time": "2023-10-15T05:58:35",
            "upload_time_iso_8601": "2023-10-15T05:58:35.395907Z",
            "url": "https://files.pythonhosted.org/packages/d9/21/8b7d08aee58c497ac9a414b2a797abbbad099754137224ed2b5e7346e290/lmdx_flow-0.1.2-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "ca39c87f0f34b0f68b0a58ca1ccc6ac36e0068e007b560506e630a32c0bfff2f",
                "md5": "5c64fb1ebc9ad65ffe9eb475d6a0d0c6",
                "sha256": "2f49c1adb4b90b0e25b6ee4eef37cf7719894b666b6e85876fd40bbbaf141bc8"
            },
            "downloads": -1,
            "filename": "lmdx_flow-0.1.2.tar.gz",
            "has_sig": false,
            "md5_digest": "5c64fb1ebc9ad65ffe9eb475d6a0d0c6",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.9,<3.13",
            "size": 5071,
            "upload_time": "2023-10-15T05:58:37",
            "upload_time_iso_8601": "2023-10-15T05:58:37.034439Z",
            "url": "https://files.pythonhosted.org/packages/ca/39/c87f0f34b0f68b0a58ca1ccc6ac36e0068e007b560506e630a32c0bfff2f/lmdx_flow-0.1.2.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-10-15 05:58:37",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "JazBern",
    "github_project": "LMDX-flow",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "requirements": [],
    "lcname": "lmdx-flow"
}

JazBern