OpenTrepWrapper
===============
# Table of Content (ToC)
<small><i><a href='http://ecotrust-canada.github.io/markdown-toc/'>Table of contents generated with markdown-toc</a></i></small>
# Overview
[OpenTREP](https://github.com/trep/opentrep) aims at providing a clean API,
its corresponding C++ implementation and Python extension, for parsing
travel-/transport-focused requests. It powers the
https://transport-search.org Web site (as well
as its newer version, https://www2.transport-search.org).
As part of the [OpenTREP releases](https://github.com/trep/opentrep/releases),
there is a Python extension, named
[`opentrep` and published on PyPi](https://pypi.org/project/opentrep/).
That Python extension comes as a so-called binary wheel only for MacOS
(which makes it very easy to install on that platform, then).
[Contributions](https://github.com/trep/opentrep/pulls) are welcome
to also build wheels for other platforms (_e.g._, Linux, including WSL
on MS Windows). So far, for those other platforms, the Python extension
has to be built from the source distribution, also
[available on PyPi in the same place](https://pypi.org/project/opentrep/).
[OpenTrepWrapper (this project)](https://github.com/trep/wrapper) provides
a simple Python wrapper around the OpenTREP Python extension, and is also
[released on PyPi](https://pypi.org/project/OpenTrepWrapper/).
# Refefences
* OpenTREP project: https://github.com/trep/opentrep
* This GitHub project: https://github.com/trep/wrapper
* PyPi artifacts:
+ OpenTREP Python extension: https://github.com/trep/opentrep/pulls
+ OpenTREP simple Python wrapper: https://pypi.org/project/OpenTrepWrapper/
* OpenTravelData (OPTD) project: https://github.com/opentraveldata/opentraveldata
* [How to install `pyenv` and `pipenv`](https://github.com/machine-learning-helpers/induction-python/tree/master/installation/virtual-env)
* How to setup, on your favorite Linux distribution (_e.g._, Ubuntu LTS,
Debian or CentOS), C++-, Python- and database-related development tools:
https://github.com/cpp-projects-showcase/docker-images
* [Twine utility](https://github.com/pypa/twine)
# Configuration
## Installation of `pyenv` and `pipenv`
* See https://github.com/machine-learning-helpers/induction-python/tree/master/installation/virtual-env
for the details.
* Install Python:
```bash
$ pushd ~/.pyenv && git pull && popd # if pyenv is in ~/.pyenv
$ pyenv install 3.8.5 && pyenv global 3.8.5 && \
pip install -U pip pipenv && pyenv global system
```
* Clone this Git repository:
```bash
$ mkdir -p ~/dev/geo/trep && \
git clone https://github.com/trep/wrapper.git ~/dev/geo/trep-wrapper
```
* Install the Python virtual environment:
```bash
$ cd ~/dev/geo/trep-wrapper
$ pipenv --rm && rm -f Pipfile.lock && \
pipenv install && pipenv install --dev
```
## Installation of OpenTREP
This module does not install OpenTREP itself; the overview gives the general
procedure on how to do it.
Also, on RedHat/CentOS/Fedora, OpenTREP is packaged and can therefore easily
be installed with the native packager manager (`dnf` or `yum`).
Alternatively, on most of the platforms, it can be installed by following
the instructions in the
[OpenTREP `README.md` file](https://github.com/trep/opentrep/tree/master/README.md).
* Basically, a few C++-, Python- and database-related development tools
have to be installed. The
[Docker images for C++ projects GitHub repository](https://github.com/cpp-projects-showcase/docker-images)
gives all the details in the `Dockerfile` of the corresponding Linux
distribution folders, _e.g._,
[`ubuntu2004/Dockerfile`](https://github.com/cpp-projects-showcase/docker-images/blob/master/ubuntu2004/Dockerfile)
for Ubuntu 20.04 LTS (Focal Fossa).
* And then the OpenTREP Python extension may simply be installed with `pip`
(either invoked through the native Python interpreter or through `pipenv`
with a Python interpreter installed by `pyenv`).
* Note that the OpenTREP Python extension is not referred to in the
[`Pipfile` file](https://github.com/trep/wrapper/blob/master/Pipfile),
in order to have a minimum viable Python virtual environment installed
by `pipenv`. If `opentrep` were referred to in the `Pipfile` file,
the installation of the Python virtual environment may just stop in
error after an attempt to compile the OpenTREP Python extension,
and it would be hard to debug, without evenhaving a minimal viable
Python virtual environment.
* Once the Python virtual environment has been installed with `pipenv`,
The installation of the OpenTREP Python extension should go something like:
```bash
user@laptop$ pipenv shell
Launching subshell in virtual environment…
(trep-wrapper-FkiBIXof) ✔ python -V
Python 3.8.5
(trep-wrapper-FkiBIXof) ✔ pip install -U opentrep
Collecting opentrep
Using cached opentrep-0.7.7.post6-cp38-cp38-macosx_10_15_x86_64.whl (10.3 MB)
Installing collected packages: opentrep
Successfully installed opentrep-0.7.7.post6
(trep-wrapper-FkiBIXof) ✔ exit
```
* Check that the OpenTREP Python extension (`opentrep`) has been installed
properly:
```bash
user@laptop$ pipenv run python -mpip freeze|grep opentrep
opentrep==0.7.7.post6
$ pipenv run python -mpip show opentrep
Name: opentrep
Version: 0.7.7.post6
Summary: Simple Python wrapper for OpenTREP
Home-page: https://github.com/trep/opentrep
Location: ~/.local/share/virtualenvs/trep-wrapper-FkiBIXof/lib/python3.8/site-packages
```
* Set a few environment variables accordingly:
```bash
$ export TREP_VENV="${HOME}/.local/share/virtualenvs/trep-wrapper-FkiBIXof"
$ export PYTHONPATH="${TREP_VENV}/lib:${TREP_VENV}/lib/python3.8/site-packages/pyopentrep"
$ export LD_LIBRARY_PATH="${TREP_VENV}/lib"
$ export PATH="${TREP_VENV}/bin:${PATH}"
```
* Alternatively, if OpenTREP, including its Python extension, has been
installed from the sources, the environment variables would rather be:
```bash
$ export TREP_DIR="${HOME}/dev/deliveries/opentrep-latest"
$ export PYTHONPATH="${TREP_DIR}/lib/python3.8/site-packages/pyopentrep:${TREP_DIR}/lib"
$ export LD_LIBRARY_PATH="${TREP_DIR}/lib"
$ export PATH="${TREP_DIR}/bin:${PATH}"
```
* As a reminder, on MacOS,
+ OpenTREP may be installed with `pip`, without any virtual environment
(as it does not play well with the MacOS Python so-called framework):
```bash
$ python3 -mpip uninstall -y opentrep
$ python3 -mpip install -U opentrep
```
+ The envrionment variables become:
```bash
$ export PYTHONPATH=/usr/local/lib/python3.8/site-packages/pyopentrep:/usr/local/lib
$ export DYLD_LIBRARY_PATH=/usr/local/lib
```
* For the remaining of this document, it is assumed that OpenTREP has been
installed by `pip`, either with the native Python framework (MacOS)
or with `pipenv` (Linux, including WSL on MS Windows)
### Download the OpenTravelData (OPTD) data and index them
* Download the OpenTravelData:
```python
>>> import opentraveldata
>>> myOPTD = opentraveldata.OpenTravelData()
>>> myOPTD.downloadFilesIfNeeded()
>>> myOPTD
OpenTravelData:
Local IATA/ICAO POR file: /tmp/opentraveldata/optd_por_public_all.csv
Local UN/LOCODE POR file: /tmp/opentraveldata/optd_por_unlc.csv
>>> myOPTD.fileSizes()
(44079255, 4888752)
>>>
```
* Initialize the Xapian index with the `-i` option of `pyopentrep.py`,
so as to index the full OpenTravelData (OPTD) POR (points of reference)
data file
+ On Linux:
```bash
$ pipenv run python ${TREP_VENV}/lib/python3.8/site-packages/pyopentrep/pyopentrep.py -p /tmp/opentraveldata/optd_por_public_all.csv -i
```
+ On MacOS:
```bash
$ DYLD_INSERT_LIBRARIES=/Library/Developer/CommandLineTools/usr/lib/clang/11.0.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib ASAN_OPTIONS=detect_container_overflow=0 /usr/local/Cellar/python\@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/Resources/Python.app/Contents/MacOS/Python /usr/local/lib/python3.8/site-packages/pyopentrep/pyopentrep.py -p /tmp/opentraveldata/optd_por_public_all.csv -i
```
```bash
OPTD-maintained list of POR (points of reference): '/tmp/opentraveldata/optd_por_public_all.csv'
Xapian-based travel database/index: '/tmp/opentrep/xapian_traveldb0'
SQLite database: '/tmp/opentrep/sqlite_travel.db'
Perform the indexation of the (Xapian-based) travel database.
That operation may take several minutes on some slow machines.
It takes less than 20 seconds on fast ones...
Number of actually parsed records: 1,000, out of 103,394 records in the POR data file so far
...
Number of actually parsed records: 20,000, out of 122,394 records in the POR data file so far
Done. Indexed 20335 POR (points of reference)
```
### Test of the OpenTREP Python extension
* Start the Python interpreter on Linux:
```bash
$ pipenv run python
```
* Start the Python interpreter on MacOS:
```bash
$ DYLD_INSERT_LIBRARIES=/Library/Developer/CommandLineTools/usr/lib/clang/11.0.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib ASAN_OPTIONS=detect_container_overflow=0 /usr/local/Cellar/python\@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/Resources/Python.app/Contents/MacOS/Python
```
* Test the OpenTREP Python extension:
```python
>>> import pyopentrep
>>> openTrepLibrary = pyopentrep.OpenTrepSearcher()
>>> initOK = openTrepLibrary.init ('/tmp/opentraveldata/optd_por_public.csv', '/tmp/opentrep/xapian_traveldb', 'sqlite', '/tmp/opentrep/sqlite_travel.db', 0, False, True, True, 'pyopentrep.log')
>>> initOK
True
>>> openTrepLibrary.search('S', 'nce sfo')
'NCE/0,SFO/0'
```
## Test the wrapper application
* In the following Python examples, it is assumed that an interactive
Python Shell has been launched:
```bash
$ pipenv run python
Python 3.8.5 (default, Jul 24 2020, 13:35:18)
>>>
```
* Or, on MacOS:
```bash
$ ASAN_OPTIONS=detect_container_overflow=0 \
DYLD_INSERT_LIBRARIES=/Library/Developer/CommandLineTools/usr/lib/clang/11.0.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib \
/usr/local/Cellar/python\@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/Resources/Python.app/Contents/MacOS/Python
Python 3.8.5 (default, Jul 24 2020, 13:35:18)
>>>
```
* Import the module and initialize the Python extension:
```python
>>> import OpenTrepWrapper
>>> otp = OpenTrepWrapper.OpenTrepLib()
>>> otp.init_cpp_extension(por_path=None,
xapian_index_path="/tmp/opentrep/xapian_traveldb",
sql_db_type="nodb",
sql_db_conn_str=None,
deployment_nb=0,
log_path="test_OpenTrepWrapper.log",
log_level=5)
>>>
```
* Index the OPTD data file:
```python
>>> otp.index()
```
* Search
+ Short output format:
```python
>>> otp.search(search_string="nce sfo", outputFormat="S")
([(89.8466, 'NCE'), (357.45599999999996, 'SFO')], '')
------------------
```
+ Full output format:
```python
>>> otp.search(search_string="nce sfo", outputFormat="F")
search_string: nce sfo
Raw result from the OpenTrep library:
1. NCE-A-6299418, 8.16788%, Nice Côte d'Azur International Airport, Nice Cote d'Azur International Airport, LFMN, , FRNCE, , 0, 1970-Jan-01, 2999-Dec-31, , NCE|2990440|Nice|Nice|FR|PAC, PAC, FR, , France, 427, France, EUR, NA, Europe, 43.6584, 7.21587, S, AIRP, 93, Provence-Alpes-Côte d'Azur, Provence-Alpes-Cote d'Azur, 06, Alpes-Maritimes, Alpes-Maritimes, 062, 06088, 0, 3, 5, Europe/Paris, 1, 2, 1, 2018-Dec-05, , https://en.wikipedia.org/wiki/Nice_C%C3%B4te_d%27Azur_Airport, 43.6627, 7.20787, nce, nce, 8984.66%, 0, 0
2. SFO-C-5391959, 32.496%, San Francisco, San Francisco, , , USSFO, , 0, 1970-Jan-01, 2999-Dec-31, , SFO|5391959|San Francisco|San Francisco|US|CA, CA, US, , United States, 91, California, USD, NA, North America, 37.7749, -122.419, P, PPLA2, CA, California, California, 075, City and County of San Francisco, City and County of San Francisco, Z, , 864816, 16, 28, America/Los_Angeles, -8, -7, -8, 2019-Sep-05, SFO, https://en.wikipedia.org/wiki/San_Francisco, 0, 0, sfo, sfo, 35745.6%, 0, 0
------------------
```
+ JSON output format:
```python
>>> otp.search(search_string="nce sfo", outputFormat="J")
search_string: nce sfo
Raw (JSON) result from the OpenTrep library:
{
"locations": [
{
"iata_code": "NCE",
"icao_code": "LFMN",
"geonames_id": "6299418",
"feature_class": "S",
"feature_code": "AIRP",
...
},
{
"iata_code": "SFO",
"icao_code": "",
"geonames_id": "5391959",
"feature_class": "P",
"feature_code": "PPLA2",
...
}
]
}
------------------
```
+ Interpreted format from JSON:
```python
>>> otp.search(search_string="nce sfo", outputFormat="I")
search_string: nce sfo
JSON format => recognised place (city/airport) codes:
NCE-LFMN-6299418 (8.1678768123135352%) / NCE: 43.658411000000001 7.2158720000000001; SFO--5391959 (32.496021550940505%) / SFO: 37.774929999999998 -122.41942;
------------------
```
* End the Python session:
```python
>>> quit()
```
* On MacOS, if there is an issue with the interceptors:
```bash
$ ASAN_OPTIONS=detect_container_overflow=0 \
DYLD_INSERT_LIBRARIES=/Library/Developer/CommandLineTools/usr/lib/clang/11.0.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib \
/usr/local/Cellar/python\@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/Resources/Python.app/Contents/MacOS/Python -mpytest test_OpenTrepWrapper.py
====================== test session starts =====================
platform darwin -- Python 3.8.5, pytest-5.4.3, py-1.8.1, pluggy-0.13.1
rootdir: /Users/darnaud/dev/geo/trep-wrapper
plugins: dash-1.12.0
collected 3 items
test_OpenTrepWrapper.py ... [100%]
====================== 3 passed in 1.35s =======================
```
# Release OpenTrepWrapper to PyPi
* Build the Python artifacts for OpenTrepWrapper:
```bash
$ rm -rf dist build */*.egg-info *.egg-info .tox MANIFEST
$ pipenv run python setup.py sdist bdist_wheel
$ ls -lFh dist
total 56
-rw-r--r-- 1 user staff 7.7K Mar 2 11:14 OpenTrepWrapper-0.7.7.post1-py3-none-any.whl
-rw-r--r-- 1 user staff 7.3K Mar 2 11:14 OpenTrepWrapper-0.7.7.post1-py3.8.egg
-rw-r--r-- 1 user staff 8.4K Mar 2 11:14 OpenTrepWrapper-0.7.7.post1.tar.gz
```
* Publish to PyPi:
```bash
$ pipenv run twine upload -u __token__ dist/*
```
Raw data
{
"_id": null,
"home_page": "https://github.com/trep/wrapper",
"name": "OpenTrepWrapper",
"maintainer": "",
"docs_url": null,
"requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*",
"maintainer_email": "",
"keywords": "data,trep,request,parser,travel,transport,search,wrapper,optd,opentraveldata,opentrep",
"author": "Denis Arnaud",
"author_email": "Denis Arnaud <denis.arnaud_fedora@m4x.org>",
"download_url": "https://files.pythonhosted.org/packages/c7/1f/aea05f89d1df3f0a1921a0db79341a01b63584ebd435760cd819a1271dc9/OpenTrepWrapper-0.7.7.post3.tar.gz",
"platform": null,
"description": "OpenTrepWrapper\n===============\n\n# Table of Content (ToC)\n<small><i><a href='http://ecotrust-canada.github.io/markdown-toc/'>Table of contents generated with markdown-toc</a></i></small>\n\n# Overview\n[OpenTREP](https://github.com/trep/opentrep) aims at providing a clean API,\nits corresponding C++ implementation and Python extension, for parsing\ntravel-/transport-focused requests. It powers the\nhttps://transport-search.org Web site (as well\nas its newer version, https://www2.transport-search.org).\n\nAs part of the [OpenTREP releases](https://github.com/trep/opentrep/releases),\nthere is a Python extension, named\n[`opentrep` and published on PyPi](https://pypi.org/project/opentrep/).\nThat Python extension comes as a so-called binary wheel only for MacOS\n(which makes it very easy to install on that platform, then).\n[Contributions](https://github.com/trep/opentrep/pulls) are welcome\nto also build wheels for other platforms (_e.g._, Linux, including WSL\non MS Windows). So far, for those other platforms, the Python extension\nhas to be built from the source distribution, also\n[available on PyPi in the same place](https://pypi.org/project/opentrep/).\n\n[OpenTrepWrapper (this project)](https://github.com/trep/wrapper) provides\na simple Python wrapper around the OpenTREP Python extension, and is also\n[released on PyPi](https://pypi.org/project/OpenTrepWrapper/).\n\n# Refefences\n* OpenTREP project: https://github.com/trep/opentrep\n* This GitHub project: https://github.com/trep/wrapper\n* PyPi artifacts:\n + OpenTREP Python extension: https://github.com/trep/opentrep/pulls\n + OpenTREP simple Python wrapper: https://pypi.org/project/OpenTrepWrapper/\n* OpenTravelData (OPTD) project: https://github.com/opentraveldata/opentraveldata\n* [How to install `pyenv` and `pipenv`](https://github.com/machine-learning-helpers/induction-python/tree/master/installation/virtual-env)\n* How to setup, on your favorite Linux distribution (_e.g._, Ubuntu LTS,\n Debian or CentOS), C++-, Python- and database-related development tools:\n https://github.com/cpp-projects-showcase/docker-images\n* [Twine utility](https://github.com/pypa/twine)\n\n# Configuration\n\n## Installation of `pyenv` and `pipenv`\n* See https://github.com/machine-learning-helpers/induction-python/tree/master/installation/virtual-env\n for the details.\n\n* Install Python:\n```bash\n$ pushd ~/.pyenv && git pull && popd # if pyenv is in ~/.pyenv\n$ pyenv install 3.8.5 && pyenv global 3.8.5 && \\\n pip install -U pip pipenv && pyenv global system\n```\n\n* Clone this Git repository:\n```bash\n$ mkdir -p ~/dev/geo/trep && \\\n git clone https://github.com/trep/wrapper.git ~/dev/geo/trep-wrapper\n```\n\n* Install the Python virtual environment:\n```bash\n$ cd ~/dev/geo/trep-wrapper\n$ pipenv --rm && rm -f Pipfile.lock && \\\n pipenv install && pipenv install --dev\n```\n\n## Installation of OpenTREP\nThis module does not install OpenTREP itself; the overview gives the general\nprocedure on how to do it.\n\nAlso, on RedHat/CentOS/Fedora, OpenTREP is packaged and can therefore easily\nbe installed with the native packager manager (`dnf` or `yum`).\n\nAlternatively, on most of the platforms, it can be installed by following\nthe instructions in the\n[OpenTREP `README.md` file](https://github.com/trep/opentrep/tree/master/README.md).\n* Basically, a few C++-, Python- and database-related development tools\n have to be installed. The\n [Docker images for C++ projects GitHub repository](https://github.com/cpp-projects-showcase/docker-images)\n gives all the details in the `Dockerfile` of the corresponding Linux\n distribution folders, _e.g._,\n [`ubuntu2004/Dockerfile`](https://github.com/cpp-projects-showcase/docker-images/blob/master/ubuntu2004/Dockerfile)\n for Ubuntu 20.04 LTS (Focal Fossa).\n\n* And then the OpenTREP Python extension may simply be installed with `pip`\n (either invoked through the native Python interpreter or through `pipenv`\n with a Python interpreter installed by `pyenv`).\n\n* Note that the OpenTREP Python extension is not referred to in the\n [`Pipfile` file](https://github.com/trep/wrapper/blob/master/Pipfile),\n in order to have a minimum viable Python virtual environment installed\n by `pipenv`. If `opentrep` were referred to in the `Pipfile` file,\n the installation of the Python virtual environment may just stop in\n error after an attempt to compile the OpenTREP Python extension,\n and it would be hard to debug, without evenhaving a minimal viable\n Python virtual environment.\n\n* Once the Python virtual environment has been installed with `pipenv`,\n The installation of the OpenTREP Python extension should go something like:\n```bash\nuser@laptop$ pipenv shell\nLaunching subshell in virtual environment\u2026\n(trep-wrapper-FkiBIXof) \u2714 python -V\nPython 3.8.5\n(trep-wrapper-FkiBIXof) \u2714 pip install -U opentrep\nCollecting opentrep\n Using cached opentrep-0.7.7.post6-cp38-cp38-macosx_10_15_x86_64.whl (10.3 MB)\nInstalling collected packages: opentrep\nSuccessfully installed opentrep-0.7.7.post6\n(trep-wrapper-FkiBIXof) \u2714 exit\n```\n\n* Check that the OpenTREP Python extension (`opentrep`) has been installed\n properly: \n```bash\nuser@laptop$ pipenv run python -mpip freeze|grep opentrep\nopentrep==0.7.7.post6\n$ pipenv run python -mpip show opentrep\nName: opentrep\nVersion: 0.7.7.post6\nSummary: Simple Python wrapper for OpenTREP\nHome-page: https://github.com/trep/opentrep\nLocation: ~/.local/share/virtualenvs/trep-wrapper-FkiBIXof/lib/python3.8/site-packages\n```\n\n* Set a few environment variables accordingly:\n```bash\n$ export TREP_VENV=\"${HOME}/.local/share/virtualenvs/trep-wrapper-FkiBIXof\"\n$ export PYTHONPATH=\"${TREP_VENV}/lib:${TREP_VENV}/lib/python3.8/site-packages/pyopentrep\"\n$ export LD_LIBRARY_PATH=\"${TREP_VENV}/lib\"\n$ export PATH=\"${TREP_VENV}/bin:${PATH}\"\n```\n\n* Alternatively, if OpenTREP, including its Python extension, has been\n installed from the sources, the environment variables would rather be:\n```bash\n$ export TREP_DIR=\"${HOME}/dev/deliveries/opentrep-latest\"\n$ export PYTHONPATH=\"${TREP_DIR}/lib/python3.8/site-packages/pyopentrep:${TREP_DIR}/lib\"\n$ export LD_LIBRARY_PATH=\"${TREP_DIR}/lib\"\n$ export PATH=\"${TREP_DIR}/bin:${PATH}\"\n```\n\n* As a reminder, on MacOS,\n + OpenTREP may be installed with `pip`, without any virtual environment\n (as it does not play well with the MacOS Python so-called framework):\n```bash\n$ python3 -mpip uninstall -y opentrep\n$ python3 -mpip install -U opentrep\n```\n + The envrionment variables become:\n```bash\n$ export PYTHONPATH=/usr/local/lib/python3.8/site-packages/pyopentrep:/usr/local/lib\n$ export DYLD_LIBRARY_PATH=/usr/local/lib\n```\n\n* For the remaining of this document, it is assumed that OpenTREP has been\n installed by `pip`, either with the native Python framework (MacOS)\n or with `pipenv` (Linux, including WSL on MS Windows)\n\n### Download the OpenTravelData (OPTD) data and index them\n* Download the OpenTravelData:\n```python\n>>> import opentraveldata\n>>> myOPTD = opentraveldata.OpenTravelData()\n>>> myOPTD.downloadFilesIfNeeded()\n>>> myOPTD\nOpenTravelData:\n\tLocal IATA/ICAO POR file: /tmp/opentraveldata/optd_por_public_all.csv\n\tLocal UN/LOCODE POR file: /tmp/opentraveldata/optd_por_unlc.csv\n>>> myOPTD.fileSizes()\n(44079255, 4888752)\n>>> \n```\n\n* Initialize the Xapian index with the `-i` option of `pyopentrep.py`,\n so as to index the full OpenTravelData (OPTD) POR (points of reference)\n data file\n + On Linux:\n```bash\n$ pipenv run python ${TREP_VENV}/lib/python3.8/site-packages/pyopentrep/pyopentrep.py -p /tmp/opentraveldata/optd_por_public_all.csv -i\n```\n + On MacOS:\n```bash\n$ DYLD_INSERT_LIBRARIES=/Library/Developer/CommandLineTools/usr/lib/clang/11.0.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib ASAN_OPTIONS=detect_container_overflow=0 /usr/local/Cellar/python\\@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/Resources/Python.app/Contents/MacOS/Python /usr/local/lib/python3.8/site-packages/pyopentrep/pyopentrep.py -p /tmp/opentraveldata/optd_por_public_all.csv -i\n```\n```bash\nOPTD-maintained list of POR (points of reference): '/tmp/opentraveldata/optd_por_public_all.csv'\nXapian-based travel database/index: '/tmp/opentrep/xapian_traveldb0'\nSQLite database: '/tmp/opentrep/sqlite_travel.db'\nPerform the indexation of the (Xapian-based) travel database.\nThat operation may take several minutes on some slow machines.\nIt takes less than 20 seconds on fast ones...\nNumber of actually parsed records: 1,000, out of 103,394 records in the POR data file so far\n...\nNumber of actually parsed records: 20,000, out of 122,394 records in the POR data file so far\nDone. Indexed 20335 POR (points of reference)\n```\n\n### Test of the OpenTREP Python extension\n* Start the Python interpreter on Linux:\n```bash\n$ pipenv run python\n```\n\n* Start the Python interpreter on MacOS:\n```bash\n$ DYLD_INSERT_LIBRARIES=/Library/Developer/CommandLineTools/usr/lib/clang/11.0.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib ASAN_OPTIONS=detect_container_overflow=0 /usr/local/Cellar/python\\@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/Resources/Python.app/Contents/MacOS/Python\n```\n\n* Test the OpenTREP Python extension:\n```python\n>>> import pyopentrep\n>>> openTrepLibrary = pyopentrep.OpenTrepSearcher()\n>>> initOK = openTrepLibrary.init ('/tmp/opentraveldata/optd_por_public.csv', '/tmp/opentrep/xapian_traveldb', 'sqlite', '/tmp/opentrep/sqlite_travel.db', 0, False, True, True, 'pyopentrep.log')\n>>> initOK\nTrue\n>>> openTrepLibrary.search('S', 'nce sfo')\n'NCE/0,SFO/0'\n```\n\n## Test the wrapper application\n* In the following Python examples, it is assumed that an interactive\n Python Shell has been launched:\n```bash\n$ pipenv run python\nPython 3.8.5 (default, Jul 24 2020, 13:35:18) \n>>> \n```\n\n* Or, on MacOS:\n```bash\n$ ASAN_OPTIONS=detect_container_overflow=0 \\\n DYLD_INSERT_LIBRARIES=/Library/Developer/CommandLineTools/usr/lib/clang/11.0.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib \\\n /usr/local/Cellar/python\\@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/Resources/Python.app/Contents/MacOS/Python\nPython 3.8.5 (default, Jul 24 2020, 13:35:18) \n>>> \n```\n\n* Import the module and initialize the Python extension:\n```python\n>>> import OpenTrepWrapper\n>>> otp = OpenTrepWrapper.OpenTrepLib()\n>>> otp.init_cpp_extension(por_path=None,\n xapian_index_path=\"/tmp/opentrep/xapian_traveldb\",\n sql_db_type=\"nodb\",\n sql_db_conn_str=None,\n deployment_nb=0,\n log_path=\"test_OpenTrepWrapper.log\",\n log_level=5)\n>>>\n```\n\n* Index the OPTD data file:\n```python\n>>> otp.index()\n```\n\n* Search\n + Short output format:\n```python\n>>> otp.search(search_string=\"nce sfo\", outputFormat=\"S\")\n([(89.8466, 'NCE'), (357.45599999999996, 'SFO')], '')\n------------------\n```\n + Full output format:\n```python\n>>> otp.search(search_string=\"nce sfo\", outputFormat=\"F\")\nsearch_string: nce sfo\nRaw result from the OpenTrep library:\n1. NCE-A-6299418, 8.16788%, Nice C\u00f4te d'Azur International Airport, Nice Cote d'Azur International Airport, LFMN, , FRNCE, , 0, 1970-Jan-01, 2999-Dec-31, , NCE|2990440|Nice|Nice|FR|PAC, PAC, FR, , France, 427, France, EUR, NA, Europe, 43.6584, 7.21587, S, AIRP, 93, Provence-Alpes-C\u00f4te d'Azur, Provence-Alpes-Cote d'Azur, 06, Alpes-Maritimes, Alpes-Maritimes, 062, 06088, 0, 3, 5, Europe/Paris, 1, 2, 1, 2018-Dec-05, , https://en.wikipedia.org/wiki/Nice_C%C3%B4te_d%27Azur_Airport, 43.6627, 7.20787, nce, nce, 8984.66%, 0, 0\n2. SFO-C-5391959, 32.496%, San Francisco, San Francisco, , , USSFO, , 0, 1970-Jan-01, 2999-Dec-31, , SFO|5391959|San Francisco|San Francisco|US|CA, CA, US, , United States, 91, California, USD, NA, North America, 37.7749, -122.419, P, PPLA2, CA, California, California, 075, City and County of San Francisco, City and County of San Francisco, Z, , 864816, 16, 28, America/Los_Angeles, -8, -7, -8, 2019-Sep-05, SFO, https://en.wikipedia.org/wiki/San_Francisco, 0, 0, sfo, sfo, 35745.6%, 0, 0\n\n------------------\n```\n + JSON output format:\n```python\n>>> otp.search(search_string=\"nce sfo\", outputFormat=\"J\")\nsearch_string: nce sfo\nRaw (JSON) result from the OpenTrep library:\n{\n \"locations\": [\n {\n \"iata_code\": \"NCE\",\n \"icao_code\": \"LFMN\",\n \"geonames_id\": \"6299418\",\n \"feature_class\": \"S\",\n \"feature_code\": \"AIRP\",\n ...\n },\n {\n \"iata_code\": \"SFO\",\n \"icao_code\": \"\",\n \"geonames_id\": \"5391959\",\n \"feature_class\": \"P\",\n \"feature_code\": \"PPLA2\",\n ...\n }\n ]\n}\n\n------------------\n```\n + Interpreted format from JSON:\n```python\n>>> otp.search(search_string=\"nce sfo\", outputFormat=\"I\")\nsearch_string: nce sfo\nJSON format => recognised place (city/airport) codes:\nNCE-LFMN-6299418 (8.1678768123135352%) / NCE: 43.658411000000001 7.2158720000000001; SFO--5391959 (32.496021550940505%) / SFO: 37.774929999999998 -122.41942; \n------------------\n```\n\n* End the Python session:\n```python\n>>> quit()\n```\n\n* On MacOS, if there is an issue with the interceptors:\n```bash\n$ ASAN_OPTIONS=detect_container_overflow=0 \\\n DYLD_INSERT_LIBRARIES=/Library/Developer/CommandLineTools/usr/lib/clang/11.0.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib \\\n /usr/local/Cellar/python\\@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/Resources/Python.app/Contents/MacOS/Python -mpytest test_OpenTrepWrapper.py\n====================== test session starts =====================\nplatform darwin -- Python 3.8.5, pytest-5.4.3, py-1.8.1, pluggy-0.13.1\nrootdir: /Users/darnaud/dev/geo/trep-wrapper\nplugins: dash-1.12.0\ncollected 3 items\ntest_OpenTrepWrapper.py ... [100%]\n\n====================== 3 passed in 1.35s =======================\n```\n\n# Release OpenTrepWrapper to PyPi\n* Build the Python artifacts for OpenTrepWrapper:\n```bash\n$ rm -rf dist build */*.egg-info *.egg-info .tox MANIFEST\n$ pipenv run python setup.py sdist bdist_wheel\n$ ls -lFh dist\ntotal 56\n-rw-r--r-- 1 user staff 7.7K Mar 2 11:14 OpenTrepWrapper-0.7.7.post1-py3-none-any.whl\n-rw-r--r-- 1 user staff 7.3K Mar 2 11:14 OpenTrepWrapper-0.7.7.post1-py3.8.egg\n-rw-r--r-- 1 user staff 8.4K Mar 2 11:14 OpenTrepWrapper-0.7.7.post1.tar.gz\n```\n\n* Publish to PyPi:\n```bash\n$ pipenv run twine upload -u __token__ dist/*\n```\n\n\n",
"bugtrack_url": null,
"license": "",
"summary": "A Python wrapper module for OpenTrep",
"version": "0.7.7.post3",
"project_urls": {
"Homepage": "https://github.com/trep/wrapper",
"changelog": "https://opentrep.readthedocs.io/en/latest/opentrep.html",
"documentation": "https://opentrep.readthedocs.io/en/latest/"
},
"split_keywords": [
"data",
"trep",
"request",
"parser",
"travel",
"transport",
"search",
"wrapper",
"optd",
"opentraveldata",
"opentrep"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "4bf6ffa91fecfad819600ce1a9544e2f1656ea5599bdfbd2baa5e08516d3dba6",
"md5": "8f2f98b84f4aabc6de446030b7a1e035",
"sha256": "b9049647b26fd15d6e64e70c5f20b724f1ab410354e66d8635e5a8ebca0ebaf5"
},
"downloads": -1,
"filename": "OpenTrepWrapper-0.7.7.post3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8f2f98b84f4aabc6de446030b7a1e035",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*",
"size": 13982,
"upload_time": "2023-09-22T16:44:52",
"upload_time_iso_8601": "2023-09-22T16:44:52.955663Z",
"url": "https://files.pythonhosted.org/packages/4b/f6/ffa91fecfad819600ce1a9544e2f1656ea5599bdfbd2baa5e08516d3dba6/OpenTrepWrapper-0.7.7.post3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c71faea05f89d1df3f0a1921a0db79341a01b63584ebd435760cd819a1271dc9",
"md5": "5149ec332e4ccc3b8c5712fbb3047b12",
"sha256": "11e9276d8593c245eb5c8f2c6ed16f1158a6d5e077841894f832c7c363effe46"
},
"downloads": -1,
"filename": "OpenTrepWrapper-0.7.7.post3.tar.gz",
"has_sig": false,
"md5_digest": "5149ec332e4ccc3b8c5712fbb3047b12",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*",
"size": 13858,
"upload_time": "2023-09-22T16:44:54",
"upload_time_iso_8601": "2023-09-22T16:44:54.732121Z",
"url": "https://files.pythonhosted.org/packages/c7/1f/aea05f89d1df3f0a1921a0db79341a01b63584ebd435760cd819a1271dc9/OpenTrepWrapper-0.7.7.post3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-09-22 16:44:54",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "trep",
"github_project": "wrapper",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "opentrepwrapper"
}
JSON
