<p align="center">
<img src="./assets/logo.png" alt="LENS Logo" width="600"/>
</p>
[](https://lens-vpr.readthedocs.io/en/latest/?badge=latest)
[](https://creativecommons.org/licenses/by-nc-sa/4.0/)
[](https://qcr.ai)
[](https://github.com/AdamDHines/LENS/stargazers)
[](https://pepy.tech/project/lens-vpr)
[](https://pixi.sh)
[](https://anaconda.org/conda-forge/lens-vpr)
[](./README.md)
This repository contains code for **LENS** - **L**ocational **E**ncoding with **N**euromorphic **S**ystems. LENS combines neuromorphic algorithms, sensors, and hardware to perform accurate, real-time robotic localization using visual place recognition (VPR).
LENS performs VPR with the SynSense [Speck<sup>TM</sup>](https://www.synsense.ai/products/speck-2/) development kits, featuring a combination of a dynamic vision sensor and neuromorphic System-on-Chip processor for real-time, energy-efficient localization.
LENS can also be used with conventional CPU, GPU, and Apple Silicon (MPS) devices to perform event-based VPR thanks to the [Sinabs](https://sinabs.readthedocs.io/en/v2.0.0/) spiking network architecture.
_For more information, please visit the [LENS Documentation](https://lens-vpr.readthedocs.io/en/latest/)_.
## Getting started
For reproducibility and simplicity, we use [pixi](https://prefix.dev/) for package management and installation. If not already installed, please run the following command in your terminal:
```console
curl -fsSL https://pixi.sh/install.sh | bash
```
_You will be prompted to restart your terminal once installed. For more information, please refer to the [pixi documentation](https://pixi.sh/latest/)._
Run the following in your terminal to clone the LENS repository and navigate to the project directory:
```console
git clone git@github.com:AdamDHines/LENS.git
cd ~/LENS
```
_For alternative package and dependency installation, please see the [LENS documentation](https://lens-vpr.readthedocs.io/en/latest/installation.html#conda)._
## Quick demo
Get started using our demo dataset and pre-trained model to evaluate the system. Run the following in your command terminal to see the demo:
```console
pixi run demo
```
### Train and evaluate new model
Test out training and evaluating a new model with our ultra-fast learning method using our provided demo dataset by running the following in your command terminal:
```console
pixi run train
pixi run evaluate
```
_For a full guide on training and evaluating your own datasets, please visit the [LENS documentation](https://lens-vpr.readthedocs.io/en/latest/train_setup.html)._
### Optimize network hyperparameters
To get the best localization performance on benchmark or custom datasets, you can tune your network hyperparameters using [Weights & Biases](https://wandb.ai/site) through our convenient optimizer script:
```console
pixi run optimizer
```
_For detailed instructions on setting up Weights & Biases and the optimizer, please refer to the [LENS documentation](https://lens-vpr.readthedocs.io/en/latest/optimizer_setup.html)._
### Deployment on neuromorphic hardware
LENS was developed using a SynSense Speck2fDevKit. If you have one of these kits, deploying to it is simple. Try out LENS using our pre-trained model and datasets by deploying simulated event streams on-chip:
```console
pixi run sim-speck
```
Additionally, models can be deployed onto the Speck2fDevKit for low-latency and energy efficient VPR with sequence matching in real-time:
```console
pixi run on-speck
```
_For more details on deployment to the Speck2fDevKit, please visit the [LENS documentation](https://lens-vpr.readthedocs.io/en/latest/sp_overview.html)._
## Dataset
For all data relating to our manuscript, we have a dedicated permanent repository at https://zenodo.org/records/15392412, as well as including all data in this repository, which can found in the [./lens/data](./lens/data) folder.
We acknowledge the Brisbane-Event-VPR dataset from https://zenodo.org/records/4302805.
## License and citation
This repository is licensed under the permissive [MIT License](./LICENSE). If you use our code, please cite our [paper](https://www.science.org/doi/10.1126/scirobotics.ads3968):
```
@article{HinesLENS2025,
author = {Adam D. Hines and Michael Milford and Tobias Fischer },
title = {A compact neuromorphic system for ultra–energy-efficient, on-device robot localization},
journal = {Science Robotics},
volume = {10},
number = {103},
pages = {eads3968},
year = {2025},
doi = {10.1126/scirobotics.ads3968},
URL = {https://www.science.org/doi/abs/10.1126/scirobotics.ads3968}
}
```
## Issues, bugs, and feature requests
If you encounter problems whilst running the code or if you have a suggestion for a feature or improvement, please report it as an [issue](https://github.com/AdamDHines/VPRTempoNeuro/issues).
Raw data
{
"_id": null,
"home_page": "https://github.com/AdamDHines/LENS",
"name": "lens-vpr",
"maintainer": null,
"docs_url": null,
"requires_python": "!=3.12.*,>=3.6",
"maintainer_email": null,
"keywords": "robotics, visual-place-recognition, neuromorphic-computing, spiking-neural-network, dynamic-vision-sensors",
"author": "Adam D Hines, Michael Milford and Tobias Fischer",
"author_email": "adam.hines@qut.edu.au",
"download_url": "https://files.pythonhosted.org/packages/7d/7c/a45c8718fb40f68028ee9baab109dfe31509078594c57f6b8f8f5eef54a9/lens-vpr-0.1.5.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n <img src=\"./assets/logo.png\" alt=\"LENS Logo\" width=\"600\"/>\n</p>\n\n\n[](https://lens-vpr.readthedocs.io/en/latest/?badge=latest)\n[](https://creativecommons.org/licenses/by-nc-sa/4.0/)\n[](https://qcr.ai)\n[](https://github.com/AdamDHines/LENS/stargazers)\n[](https://pepy.tech/project/lens-vpr)\n[](https://pixi.sh)\n[](https://anaconda.org/conda-forge/lens-vpr)\n\n[](./README.md)\n\nThis repository contains code for **LENS** - **L**ocational **E**ncoding with **N**euromorphic **S**ystems. LENS combines neuromorphic algorithms, sensors, and hardware to perform accurate, real-time robotic localization using visual place recognition (VPR). \n\nLENS performs VPR with the SynSense [Speck<sup>TM</sup>](https://www.synsense.ai/products/speck-2/) development kits, featuring a combination of a dynamic vision sensor and neuromorphic System-on-Chip processor for real-time, energy-efficient localization. \n\nLENS can also be used with conventional CPU, GPU, and Apple Silicon (MPS) devices to perform event-based VPR thanks to the [Sinabs](https://sinabs.readthedocs.io/en/v2.0.0/) spiking network architecture.\n\n_For more information, please visit the [LENS Documentation](https://lens-vpr.readthedocs.io/en/latest/)_.\n\n## Getting started\nFor reproducibility and simplicity, we use [pixi](https://prefix.dev/) for package management and installation. If not already installed, please run the following command in your terminal:\n\n```console\ncurl -fsSL https://pixi.sh/install.sh | bash\n```\n\n_You will be prompted to restart your terminal once installed. For more information, please refer to the [pixi documentation](https://pixi.sh/latest/)._ \n\nRun the following in your terminal to clone the LENS repository and navigate to the project directory:\n```console\ngit clone git@github.com:AdamDHines/LENS.git\ncd ~/LENS\n```\n\n_For alternative package and dependency installation, please see the [LENS documentation](https://lens-vpr.readthedocs.io/en/latest/installation.html#conda)._\n\n## Quick demo\nGet started using our demo dataset and pre-trained model to evaluate the system. Run the following in your command terminal to see the demo:\n\n```console\npixi run demo\n```\n\n### Train and evaluate new model\nTest out training and evaluating a new model with our ultra-fast learning method using our provided demo dataset by running the following in your command terminal:\n\n```console\npixi run train\npixi run evaluate\n```\n\n\n\n_For a full guide on training and evaluating your own datasets, please visit the [LENS documentation](https://lens-vpr.readthedocs.io/en/latest/train_setup.html)._\n\n### Optimize network hyperparameters\nTo get the best localization performance on benchmark or custom datasets, you can tune your network hyperparameters using [Weights & Biases](https://wandb.ai/site) through our convenient optimizer script: \n\n```console\npixi run optimizer\n```\n\n_For detailed instructions on setting up Weights & Biases and the optimizer, please refer to the [LENS documentation](https://lens-vpr.readthedocs.io/en/latest/optimizer_setup.html)._\n\n### Deployment on neuromorphic hardware\nLENS was developed using a SynSense Speck2fDevKit. If you have one of these kits, deploying to it is simple. Try out LENS using our pre-trained model and datasets by deploying simulated event streams on-chip:\n\n```console\npixi run sim-speck\n```\n\nAdditionally, models can be deployed onto the Speck2fDevKit for low-latency and energy efficient VPR with sequence matching in real-time:\n```console\npixi run on-speck\n```\n\n_For more details on deployment to the Speck2fDevKit, please visit the [LENS documentation](https://lens-vpr.readthedocs.io/en/latest/sp_overview.html)._\n\n## Dataset\nFor all data relating to our manuscript, we have a dedicated permanent repository at https://zenodo.org/records/15392412, as well as including all data in this repository, which can found in the [./lens/data](./lens/data) folder.\n\nWe acknowledge the Brisbane-Event-VPR dataset from https://zenodo.org/records/4302805.\n\n## License and citation\nThis repository is licensed under the permissive [MIT License](./LICENSE). If you use our code, please cite our [paper](https://www.science.org/doi/10.1126/scirobotics.ads3968):\n\n```\n@article{HinesLENS2025,\n author = {Adam D. Hines and Michael Milford and Tobias Fischer },\n title = {A compact neuromorphic system for ultra\u2013energy-efficient, on-device robot localization},\n journal = {Science Robotics},\n volume = {10},\n number = {103},\n pages = {eads3968},\n year = {2025},\n doi = {10.1126/scirobotics.ads3968},\n URL = {https://www.science.org/doi/abs/10.1126/scirobotics.ads3968}\n}\n```\n\n## Issues, bugs, and feature requests\nIf you encounter problems whilst running the code or if you have a suggestion for a feature or improvement, please report it as an [issue](https://github.com/AdamDHines/VPRTempoNeuro/issues).\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "LENS: Locational Encoding with Neuromorphic Systems",
"version": "0.1.5",
"project_urls": {
"Homepage": "https://github.com/AdamDHines/LENS"
},
"split_keywords": [
"robotics",
" visual-place-recognition",
" neuromorphic-computing",
" spiking-neural-network",
" dynamic-vision-sensors"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "e3a02fca0c380a44f3649e21912f07954276acddd31fb0437009c376dc8cbd35",
"md5": "9925e447c6b75bb64a3ef270fe114c62",
"sha256": "716da07383fd23d6f0e795c54352e3810185d6c8ac73e18b346c1737f2c08081"
},
"downloads": -1,
"filename": "lens_vpr-0.1.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9925e447c6b75bb64a3ef270fe114c62",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "!=3.12.*,>=3.6",
"size": 61186,
"upload_time": "2025-08-03T23:45:21",
"upload_time_iso_8601": "2025-08-03T23:45:21.100726Z",
"url": "https://files.pythonhosted.org/packages/e3/a0/2fca0c380a44f3649e21912f07954276acddd31fb0437009c376dc8cbd35/lens_vpr-0.1.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "7d7ca45c8718fb40f68028ee9baab109dfe31509078594c57f6b8f8f5eef54a9",
"md5": "7d85f096c94757fec9fedcdb3ba73fb5",
"sha256": "7cee40574c29762d709687f2e995e0d71c55d29055ea45025d18b7afa2cfa1de"
},
"downloads": -1,
"filename": "lens-vpr-0.1.5.tar.gz",
"has_sig": false,
"md5_digest": "7d85f096c94757fec9fedcdb3ba73fb5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "!=3.12.*,>=3.6",
"size": 49811,
"upload_time": "2025-08-03T23:45:22",
"upload_time_iso_8601": "2025-08-03T23:45:22.341024Z",
"url": "https://files.pythonhosted.org/packages/7d/7c/a45c8718fb40f68028ee9baab109dfe31509078594c57f6b8f8f5eef54a9/lens-vpr-0.1.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-03 23:45:22",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "AdamDHines",
"github_project": "LENS",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "torch",
"specs": [
[
">=",
"2.1.1"
]
]
},
{
"name": "torchvision",
"specs": [
[
">=",
"0.16.1"
]
]
},
{
"name": "numpy",
"specs": [
[
">=",
"1.26.2"
]
]
},
{
"name": "pandas",
"specs": [
[
">=",
"2.1.1"
]
]
},
{
"name": "tqdm",
"specs": [
[
">=",
"4.65.0"
]
]
},
{
"name": "prettytable",
"specs": [
[
">=",
"3.5.0"
]
]
},
{
"name": "scikit-learn",
"specs": [
[
">=",
"1.2.2"
]
]
},
{
"name": "sinabs",
"specs": [
[
">=",
"3.0.1"
]
]
},
{
"name": "h5py",
"specs": [
[
">=",
"3.10.0"
]
]
},
{
"name": "imageio",
"specs": [
[
">=",
"2.34.1"
]
]
},
{
"name": "matplotlib",
"specs": [
[
">=",
"3.8.2"
]
]
},
{
"name": "pynmea2",
"specs": [
[
">=",
"1.19.0"
]
]
},
{
"name": "scipy",
"specs": [
[
">=",
"1.11.4"
]
]
},
{
"name": "wandb",
"specs": [
[
">=",
"0.16.2"
]
]
}
],
"lcname": "lens-vpr"
}
JSON
