# LIBKDV - A Versatile Kernel Density Visualization Library for Geospatial Analytics
Kernel Density Visualization (KDV) has been extensively used for many geospatial analysis tasks. Some representative examples include traffic accident hotspot detection, crime hotspot detection, and disease outbreak detection. Although many scientific software packages, including Scipy and Scikit-learn, geographical software packages, including QGIS and ArcGIS, and visualization software packages, including Deck.gl and KDV-Explorer, can also support KDV, none of these tools, to the best of our knowledge, can be scalable to high resolution size (e.g., 2560 x 1920) and large-scale datasets (e.g., one million data points). Therefore, the huge computational cost limits the applicability of using the off-the-shelf software tools to support advanced (or more complex) geospatial analytics, e.g., bandwidth-tuning analysis and spatiotemporal analysis, which involves computing multiple KDVs in one batch.
# Introduction:
To overcome the above issue, we develop the first versatile programming library (LIBKDV), by combining our recent studies (SLAM [1] and SWS [2]), which can reduce the worst-case time complexity for supporting different types of KDV-based geospatial analytics, including:
(1) Bandwidth-tuning analysis (cf. Figure 1): Domain experts can first set multiple bandwidths in a batch, and then generate multiple KDVs with respect to these bandwidths.
(2) Spatiotemporal analysis (cf. Figure 2): Domain experts can leverage a more complex spatiotemporal kernel density function to generate time-dependent hotspot maps that correspond to different timestamps.
To further enhance the efficiency for these two tasks, we fully parallelize our methods, SLAM and SWS.
<!-- <img width="849" alt="03e58de5950a5d503b73952e8a3bbd1" src="https://user-images.githubusercontent.com/96175669/146165826-eef5f116-3e37-4bec-91dc-899af61fed18.png">-->
# Installation Guidelines:
1. First, build the virtual environment in the Anaconda (recommended Python 3.9)
```
conda create -n libkdv python=3.9
```
2. Activate the virtual environment
```
conda activate libkdv
```
3. Install the dependencies and the library
```
conda install -c conda-forge geopandas keplergl notebook
pip install libkdv
```
4. Anticipated problem(s) and possible solution(s)
> *OSError: could not find or load spatialindex_c-64.dll*
```
pip install rtree==0.9.3
```
# How to Use:
1. Import LIBKDV and Pandas in your code
```
import libkdv
import pandas as pd
```
2. Create the LIBKDV object and compute the heatmap
```
libkdv_obj = kdv(dataset, KDV_type,
GPS=true,
bandwidth_s=1000, row_pixels=800, col_pixels=640,
bandwidth_t=6, t_pixels=32,
num_threads=8)
libkdv_obj.compute()
```
Required arguments
> dataset: **Pandas object**, *the dataset.* (for preparation, please refer to the steps in data_processing.ipynb)<br />
> KDV_type: **String**, "KDV" *- single KDV* or "STKDV" *- Spatio-Temporal KDV.*<br />
Optional arguments
> GPS: **Boolean**, **true** *- use geographic coordinate system * or false *- use simple (X, Y) coordinates (evaluation.ipynb).*<br />
> bandwidth_s: **Float**, *the spatial bandwidth (in terms of meters), default is **1000**.*<br />
> row_pixels: **Integer**, *the number of grids in the x-axis, default is **800**.*<br />
> col_pixels: **Integer**, *the number of grids in the y-axis, default is **640**.*<br />
> bandwidth_t: **Float**, *the temporal bandwidth (in terms of days), default is **6**. **REQUIRED** if KDV_type="STKDV".*<br />
> t_pixels: **Integer**, *the number of grids in the t-axis, default is **32**. **REQUIRED** if KDV_type="STKDV".*<br />
> num_threads: **Integer**, *the number of threads, default is **8**.*<br />
Example for computing a single KDV:<br />
```
NewYork = pd.read_csv('./Datasets/New_York.csv')
traffic_kdv = kdv(NewYork,KDV_type="KDV",bandwidth_s=1000)
traffic_kdv.compute()
```
Example for supporting the bandwidth-tuning analysis task:<br />
```
bandwidths_traffic_kdv = [500,700,900,1100,1300,1500,1700,1900,2100,2300] #Set the bandwidths
result_traffic_kdv = [] #Stores the final results
traffic_kdv = kdv(NewYork,KDV_type="KDV")
for band in bandwidths_traffic_kdv:
kdv_traffic_kdv.bandwidth_s = band
result_traffic_kdv.append(traffic_kdv.compute())
```
Example for supporting the spatiotemporal analysis task:<br />
```
NewYork = pd.read_csv('./Datasets/New_York.csv')
traffic_kdv = kdv(NewYork,KDV_type="STKDV",bandwidth_s=1000,bandwidth_t=10)
traffic_kdv.compute()
```
3. Show the heatmaps by [KerplerGL](https://kepler.gl/)
To generate a single KDV or support the spatiotemporal analysis task, you can use the following code.
```
from keplergl import KeplerGl
map_traffic_kdv = KeplerGl(height=600, data={"data_1": traffic_kdv.result})
map_traffic_kdv
```
To support the bandwidth-tuning analysis task, you can use the following code.
```
from keplergl import KeplerGl
map_traffic_kdv_bands = KeplerGl(height=500)
for i in range(len(bandwidths_traffic_kdv)):
map_traffic_kdv_bands.add_data(data=result_traffic_kdv[i], name='data_%d'%(i+1))
map_traffic_kdv_bands
```
# Sample datasets:
We offer three sample datasets for testing, which are (1) Seattle crime dataset [a], (2) New York traffic accident dataset [b], and (3) Ontario COVID-19 dataset [c]. The python code (data_processing.py) and the Jupyter notebook (data_processing.ipynb) for extracting these datasets are provided in this Github link.
[a] Seattle Open Data. https://data.seattle.gov/Public-Safety/SPD-Crime-Data-2008-Present/tazs-3rd5. <br />
[b] NYC Open Data. https://data.cityofnewyork.us/Public-Safety/Motor-Vehicle-Collisions-Crashes/h9gi-nx95. <br />
[c] Ontario Open Data. https://data.ontario.ca/dataset/confirmed-positive-cases-of-covid-19-in-ontario. <br />
# Advantages:
There are three main advantages for using our LIBKDV.<br />
***Easy-to-use software package:*** Domain experts only need to write a few lines of python codes for using our LIBKDV, which is as easy as using other python packages, including Scikit-learn and Scipy.<br />
***High efficiency:*** LIBKDV is the first library that can reduce the worst-case time complexity for generating KDV, which cannot be achieved by other software tools. Therefore, instead of calling the KDV function in other python packages (Scikit-learn and Scipy), domain experts can call our efficient KDV function in LIBKDV.<br />
***High versatility:*** Due to the high efficiency of LIBKDV, our library can support more KDV-based geospatial analysis tasks, including bandwidth-tuning analysis and spatiotemporal analysis, which cannot be natively and feasibly supported by other software tools.<br />
<!--Add Practical Performance -->
<!-- LIBKDV is superior to those KDV solutions in scientific library, scikit-learn and scipy, in terms of response time and functionality, espeically for high resolution (1k monitor) and large data cardinality (millions objects). As shown in our jupyter notebook example, LIBKDV is XX times faster than scipy and yyy times faster than scikit-learn on our testing workstation.-->
# Project Members:
[Prof. (Edison) Tsz Nam Chan](https://www.comp.hkbu.edu.hk/~edisonchan/), Hong Kong Baptist University<br />
Mr. Pak Lon Ip, Universiy of Macau<br />
Mr. Kaiyan Zhao, Universiy of Macau<br />
[Prof. (Ryan) Leong Hou U](https://www.fst.um.edu.mo/personal/ryanlhu/), Universiy of Macau<br />
[Prof. Byron Choi](https://www.comp.hkbu.edu.hk/~bchoi/), Hong Kong Baptist University<br />
[Prof. Jianliang Xu](https://www.comp.hkbu.edu.hk/~xujl/), Hong Kong Baptist University<br />
Raw data
{
"_id": null,
"home_page": "https://github.com/libkdv",
"name": "libkdv",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "pip,KDE,heatmap,KDV,KeplerGL",
"author": "Tsz Nam Chan,PakLon Ip, Ryan Leong Hou U",
"author_email": "www.1nick@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/77/34/a65411138c6a795da0176bf0dfbe25d68482650fd6657fa2700e35a28ada/libkdv-1.1.20.tar.gz",
"platform": "any",
"description": "# LIBKDV - A Versatile Kernel Density Visualization Library for Geospatial Analytics\nKernel Density Visualization (KDV) has been extensively used for many geospatial analysis tasks. Some representative examples include traffic accident hotspot detection, crime hotspot detection, and disease outbreak detection. Although many scientific software packages, including Scipy and Scikit-learn, geographical software packages, including QGIS and ArcGIS, and visualization software packages, including Deck.gl and KDV-Explorer, can also support KDV, none of these tools, to the best of our knowledge, can be scalable to high resolution size (e.g., 2560 x 1920) and large-scale datasets (e.g., one million data points). Therefore, the huge computational cost limits the applicability of using the off-the-shelf software tools to support advanced (or more complex) geospatial analytics, e.g., bandwidth-tuning analysis and spatiotemporal analysis, which involves computing multiple KDVs in one batch.\n\n# Introduction:\nTo overcome the above issue, we develop the first versatile programming library (LIBKDV), by combining our recent studies (SLAM [1] and SWS [2]), which can reduce the worst-case time complexity for supporting different types of KDV-based geospatial analytics, including:\n\n(1)\tBandwidth-tuning analysis (cf. Figure 1): Domain experts can first set multiple bandwidths in a batch, and then generate multiple KDVs with respect to these bandwidths.\n\n\n\n\n(2)\tSpatiotemporal analysis (cf. Figure 2): Domain experts can leverage a more complex spatiotemporal kernel density function to generate time-dependent hotspot maps that correspond to different timestamps.\n\n\n\nTo further enhance the efficiency for these two tasks, we fully parallelize our methods, SLAM and SWS.\n\n<!-- <img width=\"849\" alt=\"03e58de5950a5d503b73952e8a3bbd1\" src=\"https://user-images.githubusercontent.com/96175669/146165826-eef5f116-3e37-4bec-91dc-899af61fed18.png\">-->\n\n# Installation Guidelines:\n\n1. First, build the virtual environment in the Anaconda (recommended Python 3.9)\n```\nconda create -n libkdv python=3.9\n```\n2. Activate the virtual environment\n```\nconda activate libkdv\n```\n3. Install the dependencies and the library\n``` \nconda install -c conda-forge geopandas keplergl notebook\npip install libkdv\n```\n4. Anticipated problem(s) and possible solution(s)\n> *OSError: could not find or load spatialindex_c-64.dll*\n```\npip install rtree==0.9.3\n```\n\n# How to Use:\n\n1. Import LIBKDV and Pandas in your code\n```\nimport libkdv\nimport pandas as pd\n```\n\n2. Create the LIBKDV object and compute the heatmap\n```\nlibkdv_obj = kdv(dataset, KDV_type,\n GPS=true, \n bandwidth_s=1000, row_pixels=800, col_pixels=640, \n bandwidth_t=6, t_pixels=32,\n num_threads=8)\nlibkdv_obj.compute()\n```\nRequired arguments\n> dataset: **Pandas object**, *the dataset.* (for preparation, please refer to the steps in data_processing.ipynb)<br />\n> KDV_type: **String**, \"KDV\" *- single KDV* or \"STKDV\" *- Spatio-Temporal KDV.*<br />\n\nOptional arguments\n> GPS: **Boolean**, **true** *- use geographic coordinate system * or false *- use simple (X, Y) coordinates (evaluation.ipynb).*<br />\n> bandwidth_s: **Float**, *the spatial bandwidth (in terms of meters), default is **1000**.*<br />\n> row_pixels: **Integer**, *the number of grids in the x-axis, default is **800**.*<br />\n> col_pixels: **Integer**, *the number of grids in the y-axis, default is **640**.*<br />\n> bandwidth_t: **Float**, *the temporal bandwidth (in terms of days), default is **6**. **REQUIRED** if KDV_type=\"STKDV\".*<br />\n> t_pixels: **Integer**, *the number of grids in the t-axis, default is **32**. **REQUIRED** if KDV_type=\"STKDV\".*<br />\n> num_threads: **Integer**, *the number of threads, default is **8**.*<br />\n\nExample for computing a single KDV:<br />\n```\nNewYork = pd.read_csv('./Datasets/New_York.csv')\ntraffic_kdv = kdv(NewYork,KDV_type=\"KDV\",bandwidth_s=1000)\ntraffic_kdv.compute()\n```\nExample for supporting the bandwidth-tuning analysis task:<br />\n```\nbandwidths_traffic_kdv = [500,700,900,1100,1300,1500,1700,1900,2100,2300] #Set the bandwidths\nresult_traffic_kdv = [] #Stores the final results\ntraffic_kdv = kdv(NewYork,KDV_type=\"KDV\")\nfor band in bandwidths_traffic_kdv:\n kdv_traffic_kdv.bandwidth_s = band\n result_traffic_kdv.append(traffic_kdv.compute())\n```\nExample for supporting the spatiotemporal analysis task:<br />\n```\nNewYork = pd.read_csv('./Datasets/New_York.csv')\ntraffic_kdv = kdv(NewYork,KDV_type=\"STKDV\",bandwidth_s=1000,bandwidth_t=10)\ntraffic_kdv.compute()\n```\n\n3. Show the heatmaps by [KerplerGL](https://kepler.gl/)\n\nTo generate a single KDV or support the spatiotemporal analysis task, you can use the following code.\n```\nfrom keplergl import KeplerGl\nmap_traffic_kdv = KeplerGl(height=600, data={\"data_1\": traffic_kdv.result})\nmap_traffic_kdv\n```\n\nTo support the bandwidth-tuning analysis task, you can use the following code.\n```\nfrom keplergl import KeplerGl\nmap_traffic_kdv_bands = KeplerGl(height=500)\n\nfor i in range(len(bandwidths_traffic_kdv)):\n map_traffic_kdv_bands.add_data(data=result_traffic_kdv[i], name='data_%d'%(i+1))\nmap_traffic_kdv_bands\n```\n\n# Sample datasets:\nWe offer three sample datasets for testing, which are (1) Seattle crime dataset [a], (2) New York traffic accident dataset [b], and (3) Ontario COVID-19 dataset [c]. The python code (data_processing.py) and the Jupyter notebook (data_processing.ipynb) for extracting these datasets are provided in this Github link.\n\n[a] Seattle Open Data. https://data.seattle.gov/Public-Safety/SPD-Crime-Data-2008-Present/tazs-3rd5. <br />\n[b] NYC Open Data. https://data.cityofnewyork.us/Public-Safety/Motor-Vehicle-Collisions-Crashes/h9gi-nx95. <br />\n[c] Ontario Open Data. https://data.ontario.ca/dataset/confirmed-positive-cases-of-covid-19-in-ontario. <br />\n\n# Advantages:\nThere are three main advantages for using our LIBKDV.<br />\n***Easy-to-use software package:*** Domain experts only need to write a few lines of python codes for using our LIBKDV, which is as easy as using other python packages, including Scikit-learn and Scipy.<br />\n***High efficiency:*** LIBKDV is the first library that can reduce the worst-case time complexity for generating KDV, which cannot be achieved by other software tools. Therefore, instead of calling the KDV function in other python packages (Scikit-learn and Scipy), domain experts can call our efficient KDV function in LIBKDV.<br />\n***High versatility:*** Due to the high efficiency of LIBKDV, our library can support more KDV-based geospatial analysis tasks, including bandwidth-tuning analysis and spatiotemporal analysis, which cannot be natively and feasibly supported by other software tools.<br />\n\n<!--Add Practical Performance -->\n<!-- LIBKDV is superior to those KDV solutions in scientific library, scikit-learn and scipy, in terms of response time and functionality, espeically for high resolution (1k monitor) and large data cardinality (millions objects). As shown in our jupyter notebook example, LIBKDV is XX times faster than scipy and yyy times faster than scikit-learn on our testing workstation.-->\n\n\n# Project Members:\n[Prof. (Edison) Tsz Nam Chan](https://www.comp.hkbu.edu.hk/~edisonchan/), Hong Kong Baptist University<br />\nMr. Pak Lon Ip, Universiy of Macau<br />\nMr. Kaiyan Zhao, Universiy of Macau<br />\n[Prof. (Ryan) Leong Hou U](https://www.fst.um.edu.mo/personal/ryanlhu/), Universiy of Macau<br />\n[Prof. Byron Choi](https://www.comp.hkbu.edu.hk/~bchoi/), Hong Kong Baptist University<br />\n[Prof. Jianliang Xu](https://www.comp.hkbu.edu.hk/~xujl/), Hong Kong Baptist University<br />\n\n\n",
"bugtrack_url": null,
"license": "MIT Licence",
"summary": "A library of feature heatmap algorithm",
"version": "1.1.20",
"project_urls": {
"Homepage": "https://github.com/libkdv"
},
"split_keywords": [
"pip",
"kde",
"heatmap",
"kdv",
"keplergl"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3d9769971c7b4645ec63f75959ad054209435f3ff2b13b08be0159a6c726f93b",
"md5": "a4d576a01ba36f3034787f2e831ae58d",
"sha256": "3d32d1ae27264495a37453347f809d7bb6fde8535ab29a956925086664f7b9e4"
},
"downloads": -1,
"filename": "libkdv-1.1.20-py3-none-any.whl",
"has_sig": false,
"md5_digest": "a4d576a01ba36f3034787f2e831ae58d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 2180813,
"upload_time": "2023-11-27T13:17:25",
"upload_time_iso_8601": "2023-11-27T13:17:25.506026Z",
"url": "https://files.pythonhosted.org/packages/3d/97/69971c7b4645ec63f75959ad054209435f3ff2b13b08be0159a6c726f93b/libkdv-1.1.20-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7734a65411138c6a795da0176bf0dfbe25d68482650fd6657fa2700e35a28ada",
"md5": "0bb4a5d4f28c64f72f4d901fad123ff7",
"sha256": "ac6d720863f03cd9da1a40d95905445ec9d8edb5c9b18e2699a083813a3c020d"
},
"downloads": -1,
"filename": "libkdv-1.1.20.tar.gz",
"has_sig": false,
"md5_digest": "0bb4a5d4f28c64f72f4d901fad123ff7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 2138506,
"upload_time": "2023-11-27T13:17:30",
"upload_time_iso_8601": "2023-11-27T13:17:30.825588Z",
"url": "https://files.pythonhosted.org/packages/77/34/a65411138c6a795da0176bf0dfbe25d68482650fd6657fa2700e35a28ada/libkdv-1.1.20.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-27 13:17:30",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "libkdv"
}
JSON