# SciPDF Parser
A Python parser for scientific PDF based on [GROBID](https://github.com/kermitt2/grobid).
## Installation
Use `pip` to install from this Github repository
```bash
pip install git+https://github.com/titipata/scipdf_parser
```
**Note**
* We also need an `en_core_web_sm` model for spacy, where you can run `python -m spacy download en_core_web_sm` to download it
* You can change GROBID version in `serve_grobid.sh` to test the parser on a new GROBID version
## Usage
Run the GROBID using the given bash script before parsing PDF
```bash
bash serve_grobid.sh
```
This script will download GROBID and run the service at default port 8070 (see more [here](https://grobid.readthedocs.io/en/latest/Grobid-service/)).
To parse a PDF provided in `example_data` folder or direct URL, use the following function:
```python
import scipdf
article_dict = scipdf.parse_pdf_to_dict('example_data/futoma2017improved.pdf') # return dictionary
# option to parse directly from URL to PDF, if as_list is set to True, output 'text' of parsed section will be in a list of paragraphs instead
article_dict = scipdf.parse_pdf_to_dict('https://www.biorxiv.org/content/biorxiv/early/2018/11/20/463760.full.pdf', as_list=False)
# output example
>> {
'title': 'Proceedings of Machine Learning for Healthcare',
'abstract': '...',
'sections': [
{'heading': '...', 'text': '...'},
{'heading': '...', 'text': '...'},
...
],
'references': [
{'title': '...', 'year': '...', 'journal': '...', 'author': '...'},
...
],
'figures': [
{'figure_label': '...', 'figure_type': '...', 'figure_id': '...', 'figure_caption': '...', 'figure_data': '...'},
...
],
'doi': '...'
}
xml = scipdf.parse_pdf('example_data/futoma2017improved.pdf', soup=True) # option to parse full XML from GROBID
```
To parse figures from PDF using [pdffigures2](https://github.com/allenai/pdffigures2), you can run
```python
scipdf.parse_figures('example_data', output_folder='figures') # folder should contain only PDF files
```
You can see example output figures in `figures` folder.
Raw data
{
"_id": null,
"home_page": "https://github.com/lynner-haode/pdf_parser",
"name": "levis-pdfparse",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "SCI,PDF parser,GROBID,Python PDF parser",
"author": "Levis",
"author_email": "lynner.haode@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/c1/a1/e2b44246a6beefc42c5007cf9625ea71aca2d277f3632141712760a74ec2/levis_pdfparse-0.1.0.tar.gz",
"platform": null,
"description": "# SciPDF Parser\n\nA Python parser for scientific PDF based on [GROBID](https://github.com/kermitt2/grobid).\n\n## Installation\n\nUse `pip` to install from this Github repository\n\n```bash\npip install git+https://github.com/titipata/scipdf_parser\n```\n\n**Note**\n* We also need an `en_core_web_sm` model for spacy, where you can run `python -m spacy download en_core_web_sm` to download it\n* You can change GROBID version in `serve_grobid.sh` to test the parser on a new GROBID version\n\n## Usage\n\nRun the GROBID using the given bash script before parsing PDF\n\n```bash\nbash serve_grobid.sh\n```\n\nThis script will download GROBID and run the service at default port 8070 (see more [here](https://grobid.readthedocs.io/en/latest/Grobid-service/)).\nTo parse a PDF provided in `example_data` folder or direct URL, use the following function:\n\n```python\nimport scipdf\narticle_dict = scipdf.parse_pdf_to_dict('example_data/futoma2017improved.pdf') # return dictionary\n \n# option to parse directly from URL to PDF, if as_list is set to True, output 'text' of parsed section will be in a list of paragraphs instead\narticle_dict = scipdf.parse_pdf_to_dict('https://www.biorxiv.org/content/biorxiv/early/2018/11/20/463760.full.pdf', as_list=False)\n\n# output example\n>> {\n 'title': 'Proceedings of Machine Learning for Healthcare',\n 'abstract': '...',\n 'sections': [\n {'heading': '...', 'text': '...'},\n {'heading': '...', 'text': '...'},\n ...\n ],\n 'references': [\n {'title': '...', 'year': '...', 'journal': '...', 'author': '...'},\n ...\n ],\n 'figures': [\n {'figure_label': '...', 'figure_type': '...', 'figure_id': '...', 'figure_caption': '...', 'figure_data': '...'},\n ...\n ],\n 'doi': '...'\n}\n\nxml = scipdf.parse_pdf('example_data/futoma2017improved.pdf', soup=True) # option to parse full XML from GROBID\n```\n\nTo parse figures from PDF using [pdffigures2](https://github.com/allenai/pdffigures2), you can run\n\n```python\nscipdf.parse_figures('example_data', output_folder='figures') # folder should contain only PDF files\n```\n\nYou can see example output figures in `figures` folder.\n",
"bugtrack_url": null,
"license": "(c) MIT License 2023 Levis",
"summary": "Python parser for scientific PDF based on GROBID.",
"version": "0.1.0",
"split_keywords": [
"sci",
"pdf parser",
"grobid",
"python pdf parser"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b8f821a139072b7fce90a389164d29ccc598c8bea3e0c7abe7445ef3a848a4b5",
"md5": "e4d9aec817a3b0fefadb59df637ed30c",
"sha256": "ea19a1fadf318f59ad4321aca40f6fa419b6c053a7a92792128f12d4f4f8c080"
},
"downloads": -1,
"filename": "levis_pdfparse-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e4d9aec817a3b0fefadb59df637ed30c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 30169092,
"upload_time": "2023-03-22T09:37:48",
"upload_time_iso_8601": "2023-03-22T09:37:48.679403Z",
"url": "https://files.pythonhosted.org/packages/b8/f8/21a139072b7fce90a389164d29ccc598c8bea3e0c7abe7445ef3a848a4b5/levis_pdfparse-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c1a1e2b44246a6beefc42c5007cf9625ea71aca2d277f3632141712760a74ec2",
"md5": "589078d267731f34faefd05add875edf",
"sha256": "6a4059e12389dc295cc36ca5def32ecda886f3eb43c0874a39de1e2afd4f9772"
},
"downloads": -1,
"filename": "levis_pdfparse-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "589078d267731f34faefd05add875edf",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 30154891,
"upload_time": "2023-03-22T09:38:15",
"upload_time_iso_8601": "2023-03-22T09:38:15.160497Z",
"url": "https://files.pythonhosted.org/packages/c1/a1/e2b44246a6beefc42c5007cf9625ea71aca2d277f3632141712760a74ec2/levis_pdfparse-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-03-22 09:38:15",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "lynner-haode",
"github_project": "pdf_parser",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "levis-pdfparse"
}
JSON
