# mizgra
[](https://pypi.org/project/mizgra/)  [](https://hub.docker.com/r/lszeremeta/mizgra)
Converts Mizar ESX MML mathematical data to property graph formats - GraphML, YARS-PG. Produce files ready to import to various graph databases, including Neo4j. Supports external RDF data in various formats.
To save output to file, add `> /path/to/output.graphml` to the end of the command, e.g.:
```shell
mizgra /path/to/esx_mml /path/to/mml.lar -m /path/to/metadata.xml -c /path/to/CSVs -cv > mmlkg.graphml
```
Read more about the MMLKG project at the [MMLKG website](https://mmlkg.uwb.edu.pl).
## Run mizgra
There are several ways to run mizgra. You can install it from PyPI, run it using Docker, or run it from the source.
Choose the most convenient way for you.
### PyPI
1. Install mizgra:
```shell
pip install mizgra
```
2. Run mizgra:
```shell
mizgra [ESXMML] [MMLLAR]
```
### Docker from Docker Hub
1. Pull Docker image from Docker Hub:
```shell
docker pull lszeremeta/mizgra
```
2. Run Docker container:
```shell
docker run --rm -v /path/to/esx/and/mml.lar:/app/input lszeremeta/mizgra /app/input/esx_mml /app/input/mml.lar
```
### Docker from source
1. Build Docker image:
```shell
docker build -t mizgra .
```
2. Run Docker container:
```shell
docker run --rm -v /path/to/esx/and/mml.lar:/app/input mizgra /app/input/esx_mml /app/input/mml.lar
```
### Python from source
1. Clone this repository and go to its directory:
```shell
git clone git@github.com:lszeremeta/mizgra.git
cd mizgra
```
2. Install requirements:
```shell
pip install -r requirements.txt
```
3. Run mizgra:
```shell
python -m mizgra [ESXMML] [MMLLAR]
```
## Recommended usage
Mizgra requires two arguments:
- `ESXMML`: Path to the ESX MML files directory. You can [download](https://github.com/arturkornilowicz/esx_files/archive/refs/heads/main.zip) it from [esx_files](https://github.com/arturkornilowicz/esx_files/) repository.
- `MMLLAR`: Path to the mml.lar file. You can [download](https://github.com/arturkornilowicz/esx_files/archive/refs/heads/main.zip) it from [esx_files](https://github.com/arturkornilowicz/esx_files/) repository.
Metadata and CSV relations are recommended. To include them, use `-m` and `-c` arguments.
You can generate metadata with statistics using [metadata_gen](https://github.com/domel/metadata_gen)
and CSV relations using [csvrelgen](https://github.com/arturkornilowicz/csvrelgen).
To save the output to a file, add `> /path/to/output.graphml` to the end of the command.
Example:
```shell
mizgra /path/to/esx_mml /path/to/mml.lar -m /path/to/metadata.xml -c /path/to/CSVs -cv > mmlkg.graphml
```
See [Usage](#usage) for more details.
## Usage
```shell
usage: mizgra [-h] [-v] [-o OUTPUT]
[-s {filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} [{filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} ...]]
[-d {filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} [{filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} ...]]
[-m METADATA] [-c CSV [CSV ...]] [-cv] [-r RDF] [-rf {json-ld,hext,n3,nquads,nt,trix,turtle,xml}]
ESXMML MMLLAR
```
To see all options in your version use `-h` or `--help`:
```shell
mizgra -h
```
### Informative arguments
- `-h`, `--help`: Show help message and exit.
- `-v`, `--version`: Show program version and exit.
### Required arguments
- `ESXMML`: Path to the ESX MML files directory. You can [download](https://github.com/arturkornilowicz/esx_files/archive/refs/heads/main.zip) it from [esx_files](https://github.com/arturkornilowicz/esx_files/) repository.
- `MMLLAR`: Path to the mml.lar file. You can [download](https://github.com/arturkornilowicz/esx_files/archive/refs/heads/main.zip) it from [esx_files](https://github.com/arturkornilowicz/esx_files/) repository.
### Output preference arguments
- `-o OUTPUT`, `--output OUTPUT`: Output format (`graphml`, `yarspg`; default: `graphml`).
- `-s {filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} [{filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} ...], --show {filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} [{filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} ...]`:
Show only selected outputs; by default, all possible results are shown, excluding those that are disabled by default (
local-ref-relations, usages-relations, and broader-relations; handled by CSV relations). To show disabled
outputs, specify them directly with -s/--show option. It's recommended to use CSV relations.
- `-d {filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} [{filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} ...], --disable {filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} [{filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} ...]`:
Disable selected outputs (local-ref-relations, usages-relations, and broader-relations are disabled by default; handled
by CSV relations).
### External data arguments
- `-m METADATA`, `--metadata METADATA`: Path to the metadata file. If not specified, metadata without dataset statistics will
be included. You can generate a full metadata file using: https://github.com/domel/metadata_gen
- `-c CSV`, `--csv CSV`: Paths to the CSV files/directories with relations. You can generate it using: https://github.com/arturkornilowicz/csvrelgen
- `-cv`, `--csv-validate`: Validates CSV files by several checks before reading to minimize the possibility of CSV errors;
turned off by default.
- `-r RDF`, `--rdf RDF`: Path to the RDF data file; for URLs or files with not standard extensions may be needed to
specify format using `-rf`.
- `-rf {json-ld,hext,n3,nquads,nt,trix,turtle,xml}`, `--rdf-format {json-ld,hext,n3,nquads,nt,trix,turtle,xml}`: RDF
data file format, specify if not detected automatically
## Examples
Convert ESX MML data to GraphML format:
```shell
mizgra /path/to/esx_mml /path/to/mml.lar > graph.graphml
```
Convert ESX MML data to YARS-PG format:
```shell
mizgra /path/to/esx_mml /path/to/mml.lar -o yarspg > graph.yarspg
```
Convert ESX MML data to GraphML format, showing only metadata and member-relations:
```shell
mizgra /path/to/esx_mml /path/to/mml.lar -s metadata member-relations > graph.graphml
```
Convert ESX MML data to GraphML format and include external metadata:
```shell
mizgra /path/to/esx_mml /path/to/mml.lar -m /path/to/metadata.xml > graph.graphml
```
Convert ESX MML data to GraphML format and include RDF data in Turtle format:
```shell
mizgra /path/to/esx_mml /path/to/mml.lar -r /path/to/external.ttl -rf turtle > graph.graphml
```
Convert ESX MML data to GraphML format, include CSV data from selected directory, validate CSV files:
```shell
mizgra /path/to/esx_mml /path/to/mml.lar -c /path/to/external -cv > graph.graphml
```
Convert ESX MML data to GraphML format and include CSV data:
```shell
mizgra /path/to/esx_mml /path/to/mml.lar -c /path/to/external.csv > graph.graphml
```
Convert ESX MML data to GraphML format and include CSV data from multiple files and directories:
```shell
mizgra /path/to/esx_mml /path/to/mml.lar -c /path/to/external1.csv /path/to/external2 /path/to/external3.csv > graph.graphml
```
## Contribution
Would you like to improve this project? Great! We are waiting for your help and suggestions. If you are new to open
source contributions, read [How to Contribute to Open Source](https://opensource.guide/how-to-contribute/).
## License
Distributed under [MIT License](https://github.com/lszeremeta/mizgra/blob/main/LICENSE).
This project contains a sample external RDF data file ([rdf_data.nt](https://github.com/lszeremeta/mizgra/blob/main/mizgra/resources/rdf_data.nt)), prepared by Dominik Tomaszuk ([@domel](https://github.com/domel)) and available under the [CC-BY 4.0 License](https://creativecommons.org/licenses/by/4.0/). The base for the query was developed using [Wikidata](https://www.wikidata.org/), which is available under the [Creative Commons CC0 License](https://creativecommons.org/publicdomain/zero/1.0/). While it also includes references to [DBpedia](https://wiki.dbpedia.org/) and [YAGO](https://yago-knowledge.org/) (whose data are on [Creative Commons Attribution-ShareAlike 3.0 License](https://creativecommons.org/licenses/by-sa/3.0/)), Dominik Tomaszuk only retrieved non-data links from these sources. The structure of this file is entirely different than the above datasets.
Raw data
{
"_id": null,
"home_page": "https://github.com/lszeremeta/mizgra",
"name": "mizgra",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "mizgra,mathematics,parser,CLI,XML,ESX,RDF,MML,Mizar,Mizar Mathematical Library,Mizar Mathematical Library Knowledge Graph,MMLKG,CSV,converter,GraphML,YARS-PG,property graph,graph,graph database,graph data,Neo4j,mathematical data",
"author": "\u0141ukasz Szeremeta",
"author_email": "l.szeremeta.dev+mizgra@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/8a/fb/98b75e7d28dbaed1d7eb26139007ff337ee7f4a6222e5f994431227602e0/mizgra-1.0.1.tar.gz",
"platform": null,
"description": "# mizgra\n\n[](https://pypi.org/project/mizgra/)  [](https://hub.docker.com/r/lszeremeta/mizgra)\n\nConverts Mizar ESX MML mathematical data to property graph formats - GraphML, YARS-PG. Produce files ready to import to various graph databases, including Neo4j. Supports external RDF data in various formats.\n\nTo save output to file, add `> /path/to/output.graphml` to the end of the command, e.g.:\n\n```shell\nmizgra /path/to/esx_mml /path/to/mml.lar -m /path/to/metadata.xml -c /path/to/CSVs -cv > mmlkg.graphml\n```\n\nRead more about the MMLKG project at the [MMLKG website](https://mmlkg.uwb.edu.pl).\n\n## Run mizgra\n\nThere are several ways to run mizgra. You can install it from PyPI, run it using Docker, or run it from the source.\nChoose the most convenient way for you.\n\n### PyPI\n\n1. Install mizgra:\n\n```shell\npip install mizgra\n```\n\n2. Run mizgra:\n\n```shell\nmizgra [ESXMML] [MMLLAR]\n```\n\n### Docker from Docker Hub\n\n1. Pull Docker image from Docker Hub:\n\n```shell\ndocker pull lszeremeta/mizgra\n```\n\n2. Run Docker container:\n\n```shell\ndocker run --rm -v /path/to/esx/and/mml.lar:/app/input lszeremeta/mizgra /app/input/esx_mml /app/input/mml.lar\n```\n\n### Docker from source\n\n1. Build Docker image:\n\n```shell\ndocker build -t mizgra .\n```\n\n2. Run Docker container:\n\n```shell\ndocker run --rm -v /path/to/esx/and/mml.lar:/app/input mizgra /app/input/esx_mml /app/input/mml.lar\n```\n\n### Python from source\n\n1. Clone this repository and go to its directory:\n\n```shell\ngit clone git@github.com:lszeremeta/mizgra.git\ncd mizgra\n```\n\n2. Install requirements:\n\n```shell\npip install -r requirements.txt\n```\n\n3. Run mizgra:\n\n```shell\npython -m mizgra [ESXMML] [MMLLAR]\n```\n\n## Recommended usage\n\nMizgra requires two arguments:\n\n- `ESXMML`: Path to the ESX MML files directory. You can [download](https://github.com/arturkornilowicz/esx_files/archive/refs/heads/main.zip) it from [esx_files](https://github.com/arturkornilowicz/esx_files/) repository.\n- `MMLLAR`: Path to the mml.lar file. You can [download](https://github.com/arturkornilowicz/esx_files/archive/refs/heads/main.zip) it from [esx_files](https://github.com/arturkornilowicz/esx_files/) repository.\n\nMetadata and CSV relations are recommended. To include them, use `-m` and `-c` arguments.\nYou can generate metadata with statistics using [metadata_gen](https://github.com/domel/metadata_gen)\nand CSV relations using [csvrelgen](https://github.com/arturkornilowicz/csvrelgen).\n\nTo save the output to a file, add `> /path/to/output.graphml` to the end of the command.\n\nExample:\n\n```shell\nmizgra /path/to/esx_mml /path/to/mml.lar -m /path/to/metadata.xml -c /path/to/CSVs -cv > mmlkg.graphml\n```\n\nSee [Usage](#usage) for more details.\n\n## Usage\n\n```shell\nusage: mizgra [-h] [-v] [-o OUTPUT]\n [-s {filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} [{filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} ...]]\n [-d {filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} [{filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} ...]]\n [-m METADATA] [-c CSV [CSV ...]] [-cv] [-r RDF] [-rf {json-ld,hext,n3,nquads,nt,trix,turtle,xml}]\n ESXMML MMLLAR\n```\n\nTo see all options in your version use `-h` or `--help`:\n\n```shell\nmizgra -h\n```\n\n### Informative arguments\n\n- `-h`, `--help`: Show help message and exit.\n- `-v`, `--version`: Show program version and exit.\n\n### Required arguments\n\n- `ESXMML`: Path to the ESX MML files directory. You can [download](https://github.com/arturkornilowicz/esx_files/archive/refs/heads/main.zip) it from [esx_files](https://github.com/arturkornilowicz/esx_files/) repository.\n- `MMLLAR`: Path to the mml.lar file. You can [download](https://github.com/arturkornilowicz/esx_files/archive/refs/heads/main.zip) it from [esx_files](https://github.com/arturkornilowicz/esx_files/) repository.\n\n### Output preference arguments\n\n- `-o OUTPUT`, `--output OUTPUT`: Output format (`graphml`, `yarspg`; default: `graphml`).\n- `-s {filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} [{filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} ...], --show {filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} [{filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} ...]`:\n Show only selected outputs; by default, all possible results are shown, excluding those that are disabled by default (\n local-ref-relations, usages-relations, and broader-relations; handled by CSV relations). To show disabled\n outputs, specify them directly with -s/--show option. It's recommended to use CSV relations.\n- `-d {filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} [{filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} ...], --disable {filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} [{filenames,nodes,metadata,member-relations,local-ref-relations,usages-relations,broader-relations,csv-relations,rdf-relations} ...]`:\n Disable selected outputs (local-ref-relations, usages-relations, and broader-relations are disabled by default; handled\n by CSV relations).\n\n### External data arguments\n\n- `-m METADATA`, `--metadata METADATA`: Path to the metadata file. If not specified, metadata without dataset statistics will\n be included. You can generate a full metadata file using: https://github.com/domel/metadata_gen\n- `-c CSV`, `--csv CSV`: Paths to the CSV files/directories with relations. You can generate it using: https://github.com/arturkornilowicz/csvrelgen\n- `-cv`, `--csv-validate`: Validates CSV files by several checks before reading to minimize the possibility of CSV errors;\n turned off by default.\n- `-r RDF`, `--rdf RDF`: Path to the RDF data file; for URLs or files with not standard extensions may be needed to\n specify format using `-rf`.\n- `-rf {json-ld,hext,n3,nquads,nt,trix,turtle,xml}`, `--rdf-format {json-ld,hext,n3,nquads,nt,trix,turtle,xml}`: RDF\n data file format, specify if not detected automatically\n\n## Examples\n\nConvert ESX MML data to GraphML format:\n\n```shell\nmizgra /path/to/esx_mml /path/to/mml.lar > graph.graphml\n```\n\nConvert ESX MML data to YARS-PG format:\n\n```shell\nmizgra /path/to/esx_mml /path/to/mml.lar -o yarspg > graph.yarspg\n```\n\nConvert ESX MML data to GraphML format, showing only metadata and member-relations:\n\n```shell\nmizgra /path/to/esx_mml /path/to/mml.lar -s metadata member-relations > graph.graphml\n```\n\nConvert ESX MML data to GraphML format and include external metadata:\n\n```shell\nmizgra /path/to/esx_mml /path/to/mml.lar -m /path/to/metadata.xml > graph.graphml\n```\n\nConvert ESX MML data to GraphML format and include RDF data in Turtle format:\n\n```shell\nmizgra /path/to/esx_mml /path/to/mml.lar -r /path/to/external.ttl -rf turtle > graph.graphml\n```\n\nConvert ESX MML data to GraphML format, include CSV data from selected directory, validate CSV files:\n\n```shell\nmizgra /path/to/esx_mml /path/to/mml.lar -c /path/to/external -cv > graph.graphml\n```\n\nConvert ESX MML data to GraphML format and include CSV data:\n\n```shell\nmizgra /path/to/esx_mml /path/to/mml.lar -c /path/to/external.csv > graph.graphml\n```\n\nConvert ESX MML data to GraphML format and include CSV data from multiple files and directories:\n\n```shell\nmizgra /path/to/esx_mml /path/to/mml.lar -c /path/to/external1.csv /path/to/external2 /path/to/external3.csv > graph.graphml\n```\n\n## Contribution\n\nWould you like to improve this project? Great! We are waiting for your help and suggestions. If you are new to open\nsource contributions, read [How to Contribute to Open Source](https://opensource.guide/how-to-contribute/).\n\n## License\n\nDistributed under [MIT License](https://github.com/lszeremeta/mizgra/blob/main/LICENSE).\n\nThis project contains a sample external RDF data file ([rdf_data.nt](https://github.com/lszeremeta/mizgra/blob/main/mizgra/resources/rdf_data.nt)), prepared by Dominik Tomaszuk ([@domel](https://github.com/domel)) and available under the [CC-BY 4.0 License](https://creativecommons.org/licenses/by/4.0/). The base for the query was developed using [Wikidata](https://www.wikidata.org/), which is available under the [Creative Commons CC0 License](https://creativecommons.org/publicdomain/zero/1.0/). While it also includes references to [DBpedia](https://wiki.dbpedia.org/) and [YAGO](https://yago-knowledge.org/) (whose data are on [Creative Commons Attribution-ShareAlike 3.0 License](https://creativecommons.org/licenses/by-sa/3.0/)), Dominik Tomaszuk only retrieved non-data links from these sources. The structure of this file is entirely different than the above datasets.\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "Converts Mizar ESX MML mathematical data to property graph formats - GraphML, YARS-PG for Neo4j, and other graph databases",
"version": "1.0.1",
"project_urls": {
"Homepage": "https://github.com/lszeremeta/mizgra"
},
"split_keywords": [
"mizgra",
"mathematics",
"parser",
"cli",
"xml",
"esx",
"rdf",
"mml",
"mizar",
"mizar mathematical library",
"mizar mathematical library knowledge graph",
"mmlkg",
"csv",
"converter",
"graphml",
"yars-pg",
"property graph",
"graph",
"graph database",
"graph data",
"neo4j",
"mathematical data"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e0de70efb5262877634a135f7be08d35222e38eaa01a285153bad9970ee0ea7d",
"md5": "77228ac542cb5d45074508048e111fec",
"sha256": "9f246bbdb65ab894b729d52800b5dc3438373bac630703ea4385038bf4701716"
},
"downloads": -1,
"filename": "mizgra-1.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "77228ac542cb5d45074508048e111fec",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 189675,
"upload_time": "2023-11-13T16:22:29",
"upload_time_iso_8601": "2023-11-13T16:22:29.100082Z",
"url": "https://files.pythonhosted.org/packages/e0/de/70efb5262877634a135f7be08d35222e38eaa01a285153bad9970ee0ea7d/mizgra-1.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8afb98b75e7d28dbaed1d7eb26139007ff337ee7f4a6222e5f994431227602e0",
"md5": "95158cb158859bbfd307022183a742a7",
"sha256": "4003f0eb36e8902fabb3d3026474d94a18094c4cabac147a77b5069494067a4a"
},
"downloads": -1,
"filename": "mizgra-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "95158cb158859bbfd307022183a742a7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 185123,
"upload_time": "2023-11-13T16:22:31",
"upload_time_iso_8601": "2023-11-13T16:22:31.192170Z",
"url": "https://files.pythonhosted.org/packages/8a/fb/98b75e7d28dbaed1d7eb26139007ff337ee7f4a6222e5f994431227602e0/mizgra-1.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-13 16:22:31",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "lszeremeta",
"github_project": "mizgra",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "mizgra"
}
JSON
