<div align="center">
<h1>UniMERNet: A Universal Network for Real-World Mathematical Expression Recognition</h1>
[](https://arxiv.org/abs/2404.15254)
[](https://huggingface.co/wanderkid/unimernet)
</div>
Welcome to the official repository of UniMERNet, a solution that converts images of mathematical expressions into LaTeX, suitable for a wide range of real-world scenarios.
## News
**2024.4.24** 🎉🎉 Paper now available on [ArXiv](https://arxiv.org/abs/2404.15254).
**2024.4.24** 🎉🎉 Inference code and checkpoints have been released.
## Quick Start
### Clone the repo and download the model
```bash
git clone https://github.com/opendatalab/UniMERNet.git
```
```bash
cd UniMERNet/models
# Download the model and tokenizer individually or use git-lfs
git lfs install
git clone https://huggingface.co/wanderkid/unimernet
```
### Installation
``` bash
conda create -n unimernet python=3.10
conda activate unimernet
pip install unimernet
```
### Run the demo
> Input an image and predict the LaTeX code
```bash
python demo.py
```
> Input an image and obtain the LaTeX code and corresponding rendered image
```bash
jupyter-lab ./demo.ipynb
```
## Performance Comparison (BLEU) with SOTA Methods.
> UniMERNet significantly outperforms mainstream models in recognizing real-world mathematical expressions, demonstrating superior performance across Simple Printed Expressions (SPE), Complex Printed Expressions (CPE), Screen-Captured Expressions (SCE), and Handwritten Expressions (HWE), as evidenced by the comparative BLEU Score evaluation.
## Visualization Result with Different Methods.
> UniMERNet excels in visual recognition of challenging samples, outperforming other methods.
## TODO
- [x] Release inference code and checkpoints of UniMERNet.
- [ ] Release UniMER-1M and UniMER-Test.
- [ ] Release the training code for UniMERNet.
## Citation
If you find our models / code / papers useful in your research, please consider giving us a star ⭐ and citing our work 📝, thank you :)
```bibtex
@misc{wang2024unimernet,
title={UniMERNet: A Universal Network for Real-World Mathematical Expression Recognition},
author={Bin Wang and Zhuangcheng Gu and Chao Xu and Bo Zhang and Botian Shi and Conghui He},
year={2024},
eprint={2404.15254},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```
## Acknowledgements
- [VIGC](https://github.com/opendatalab/VIGC). The model framework is dependent on VIGC.
- [Texify](https://github.com/VikParuchuri/texify). A mainstream MER algorithm, UniMERNet data processing refers to Texify.
- [Latex-OCR](https://github.com/lukas-blecher/LaTeX-OCR). Another mainstream MER algorithm.
- [Donut](https://huggingface.co/naver-clova-ix/donut-base). The UniMERNet's Transformer Encoder-Decoder are referenced from Donut.
- [Nougat](https://github.com/facebookresearch/nougat). The tokenizer uses Nougat.
## Contact Us
If you have any questions, comments, or suggestions, please do not hesitate to contact us at wangbin@pjlab.org.cn.
## License
[Apache License 2.0](LICENSE)
Raw data
{
"_id": null,
"home_page": "https://github.com/opendatalab/UniMERNet",
"name": "unimernet",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "MER, latex, markdown, pdf",
"author": "Bin Wang",
"author_email": "ictwangbin@gmail.com",
"download_url": null,
"platform": null,
"description": "<div align=\"center\">\n<h1>UniMERNet: A Universal Network for Real-World Mathematical Expression Recognition</h1>\n\n[](https://arxiv.org/abs/2404.15254)\n[](https://huggingface.co/wanderkid/unimernet)\n\n</div>\n\nWelcome to the official repository of UniMERNet, a solution that converts images of mathematical expressions into LaTeX, suitable for a wide range of real-world scenarios.\n\n## News\n**2024.4.24** \ud83c\udf89\ud83c\udf89 Paper now available on [ArXiv](https://arxiv.org/abs/2404.15254). \n**2024.4.24** \ud83c\udf89\ud83c\udf89 Inference code and checkpoints have been released. \n\n\n## Quick Start\n\n### Clone the repo and download the model\n```bash\ngit clone https://github.com/opendatalab/UniMERNet.git\n```\n\n```bash\ncd UniMERNet/models\n# Download the model and tokenizer individually or use git-lfs\ngit lfs install\ngit clone https://huggingface.co/wanderkid/unimernet\n```\n\n### Installation\n\n``` bash \nconda create -n unimernet python=3.10\n\nconda activate unimernet\n\npip install unimernet\n```\n\n### Run the demo\n> Input an image and predict the LaTeX code\n\n```bash\npython demo.py\n```\n> Input an image and obtain the LaTeX code and corresponding rendered image\n\n```bash\njupyter-lab ./demo.ipynb\n```\n\n\n## Performance Comparison (BLEU) with SOTA Methods.\n> UniMERNet significantly outperforms mainstream models in recognizing real-world mathematical expressions, demonstrating superior performance across Simple Printed Expressions (SPE), Complex Printed Expressions (CPE), Screen-Captured Expressions (SCE), and Handwritten Expressions (HWE), as evidenced by the comparative BLEU Score evaluation. \n\n\n\n## Visualization Result with Different Methods.\n> UniMERNet excels in visual recognition of challenging samples, outperforming other methods. \n\n\n\n\n## TODO\n\n- [x] Release inference code and checkpoints of UniMERNet.\n- [ ] Release UniMER-1M and UniMER-Test.\n- [ ] Release the training code for UniMERNet.\n\n## Citation\nIf you find our models / code / papers useful in your research, please consider giving us a star \u2b50 and citing our work \ud83d\udcdd, thank you :)\n```bibtex\n@misc{wang2024unimernet,\n title={UniMERNet: A Universal Network for Real-World Mathematical Expression Recognition}, \n author={Bin Wang and Zhuangcheng Gu and Chao Xu and Bo Zhang and Botian Shi and Conghui He},\n year={2024},\n eprint={2404.15254},\n archivePrefix={arXiv},\n primaryClass={cs.CV}\n}\n```\n\n## Acknowledgements\n- [VIGC](https://github.com/opendatalab/VIGC). The model framework is dependent on VIGC.\n- [Texify](https://github.com/VikParuchuri/texify). A mainstream MER algorithm, UniMERNet data processing refers to Texify.\n- [Latex-OCR](https://github.com/lukas-blecher/LaTeX-OCR). Another mainstream MER algorithm.\n- [Donut](https://huggingface.co/naver-clova-ix/donut-base). The UniMERNet's Transformer Encoder-Decoder are referenced from Donut.\n- [Nougat](https://github.com/facebookresearch/nougat). The tokenizer uses Nougat.\n\n## Contact Us\nIf you have any questions, comments, or suggestions, please do not hesitate to contact us at wangbin@pjlab.org.cn.\n\n## License\n[Apache License 2.0](LICENSE)\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "UniMERNet: A Universal Network for Real-World Mathematical Expression Recognition",
"version": "0.0.9",
"project_urls": {
"Homepage": "https://github.com/opendatalab/UniMERNet",
"Repository": "https://github.com/opendatalab/UniMERNet"
},
"split_keywords": [
"mer",
" latex",
" markdown",
" pdf"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "df6c9a11bd14d21b8c19f0ea2ae7c598fd4fcd5c034455fd9ff913f45b8c4878",
"md5": "dcf15393cdcd4acc780a16910cbc9025",
"sha256": "f8f9ab6c6c90b9537fac455fb28fcfa724ae43ec148fbd3ed1c30f99dbf3dbb4"
},
"downloads": -1,
"filename": "unimernet-0.0.9-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "dcf15393cdcd4acc780a16910cbc9025",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 2223230,
"upload_time": "2024-05-06T07:58:20",
"upload_time_iso_8601": "2024-05-06T07:58:20.887193Z",
"url": "https://files.pythonhosted.org/packages/df/6c/9a11bd14d21b8c19f0ea2ae7c598fd4fcd5c034455fd9ff913f45b8c4878/unimernet-0.0.9-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-06 07:58:20",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "opendatalab",
"github_project": "UniMERNet",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "unimernet"
}
JSON
