# <img src="assets/logo.svg" width="25"> NAVER: A Neuro-Symbolic Compositional Automaton for Visual Grounding with Explicit Logic Reasoning
<div align="center">
<img src="assets/teaser.svg">
<p></p>
</div>
<div align="center">
<a href="https://github.com/ControlNet/NAVER/issues">
<img src="https://img.shields.io/github/issues/ControlNet/NAVER?style=flat-square">
</a>
<a href="https://github.com/ControlNet/NAVER/network/members">
<img src="https://img.shields.io/github/forks/ControlNet/NAVER?style=flat-square">
</a>
<a href="https://github.com/ControlNet/NAVER/stargazers">
<img src="https://img.shields.io/github/stars/ControlNet/NAVER?style=flat-square">
</a>
<a href="https://github.com/ControlNet/NAVER/blob/master/LICENSE">
<img src="https://img.shields.io/github/license/ControlNet/NAVER?style=flat-square">
</a>
<a href="https://arxiv.org/abs/2502.00372">
<img src="https://img.shields.io/badge/arXiv-2502.00372-b31b1b.svg?style=flat-square">
</a>
</div>
<div align="center">
<a href="https://pypi.org/project/naver/">
<img src="https://img.shields.io/pypi/v/naver?style=flat-square">
</a>
<a href="https://pypi.org/project/naver/">
<img src="https://img.shields.io/pypi/dm/naver?style=flat-square">
</a>
<a href="https://www.python.org/"><img src="https://img.shields.io/pypi/pyversions/naver?style=flat-square"></a>
</div>
**This repo is the official implementation for the paper [NAVER: A Neuro-Symbolic Compositional Automaton for Visual Grounding with Explicit Logic Reasoning](https://arxiv.org/abs/2502.00372) in ICCV 2025.**
## Release
- [2025/06/28] 🔥 **NAVER** code is open sourced in GitHub.
- [2025/06/25] 🎉 **NAVER** paper is accepted by ICCV 2025.
## TODOs
We're working on the following TODOs:
- [x] GUI demo.
- [ ] Support more LLMs.
- [ ] Video demo & slides presentation.
## Docker (GUI Demo)
We provide a Docker image for the GUI demo.
```bash
docker run --runtime=nvidia --gpus=all -p <GUI-PORT>:8000 -e OPENAI_API_KEY=<OPENAI-API-KEY> -e AZURE_OPENAI_URL=<AZURE-OPENAI-URL> controlnet/naver:latest
```
The GUI will be available at `http://0.0.0.0:<GUI-PORT>`. See details below for more information.
## Installation
### Requirements
- Python >= 3.10
- conda
Please follow the instructions below to install the required packages and set up the environment.
### 1. Clone this repository.
```Bash
git clone https://github.com/ControlNet/NAVER
```
### 2. Setup conda environment and install dependencies.
Option 1: Using [pixi](https://prefix.dev/) (recommended):
```Bash
pixi install
pixi shell
```
Option 2: Building from source (You may need to setup the CUDA and PyTorch manually):
```Bash
conda install conda-forge/label/rust_dev::rust=1.78 -c conda-forge -y
pip install "git+https://github.com/scallop-lang/scallop.git@f8fac18#egg=scallopy&subdirectory=etc/scallopy"
pip install -e .
```
### 3. Configure the environments
Edit the file `.env` or setup in CLI to configure the environment variables.
```
OPENAI_API_KEY=your-api-key # if you want to use OpenAI LLMs
AZURE_OPENAI_URL= # if you want to use Azure OpenAI LLMs
OLLAMA_HOST=http://ollama.server:11434 # if you want to use your OLLaMA server for llama or deepseek
# do not change this TORCH_HOME variable
TORCH_HOME=./pretrained_models
```
### 4. Download the pretrained models
Run the scripts to download the pretrained models to the `./pretrained_models` directory.
```Bash
python -m hydra_vl4ai.download_model --base_config config/refcoco.yaml --model_config config/model_config.yaml --extra_packages naver.tool
```
## Inference
You may need 28GB vRAM to run NAVER. Consider editing the file in `./config/model_config.yaml` to load the models in multiple GPUs.
### Inference with GUI
You need `nodejs` and `npm` to run the GUI demo. It will automatically compile and build the frontend.
The GUI will be available at `http://0.0.0.0:8000`.
```Bash
python demo_gui.py \
--base_config <YOUR-CONFIG-DIR> \
--model_config <MODEL-CONFIG-PATH>
```
### Inference with given one image and query
```Bash
python demo_cli.py \
--image <IMAGE_PATH> \
--query <QUERY> \
--base_config <YOUR-CONFIG-DIR> \
--model_config <MODEL-CONFIG-PATH>
```
The result will be printed in the console.
### Inference dataset
```Bash
python main.py \
--data_root <YOUR-DATA-ROOT> \
--base_config <YOUR-CONFIG-DIR> \
--model_config <MODEL-CONFIG-PATH>
```
Then the inference results are saved in the `./result` directory for evaluation.
## Evaluation
```Bash
python evaluate.py --input <RESULT_JSONL_PATH>
```
The evaluation results will be printed in the console. Note the output from LLM is random, so the evaluation results may be slightly different from the paper.
## Citation
If you find this work useful for your research, please consider citing it.
```bibtex
@article{cai2025naver,
title = {NAVER: A Neuro-Symbolic Compositional Automaton for Visual Grounding with Explicit Logic Reasoning},
author = {Cai, Zhixi and Ke, Fucai and Jahangard, Simindokht and Garcia de la Banda, Maria and Haffari, Reza and Stuckey, Peter J. and Rezatofighi, Hamid},
journal = {arXiv preprint arXiv:2502.00372},
year = {2025},
}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/ControlNet/NAVER",
"name": "naver",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "deep learning, pytorch, AI",
"author": "ControlNet",
"author_email": "smczx@hotmail.com",
"download_url": "https://files.pythonhosted.org/packages/90/61/caa284b476e82b7ebf1da12d1f27e153f075f0b24ca7d3e6ec35b608b741/naver-0.0.2.tar.gz",
"platform": null,
"description": "# <img src=\"assets/logo.svg\" width=\"25\"> NAVER: A Neuro-Symbolic Compositional Automaton for Visual Grounding with Explicit Logic Reasoning\n\n<div align=\"center\">\n <img src=\"assets/teaser.svg\">\n <p></p>\n</div>\n\n<div align=\"center\">\n <a href=\"https://github.com/ControlNet/NAVER/issues\">\n <img src=\"https://img.shields.io/github/issues/ControlNet/NAVER?style=flat-square\">\n </a>\n <a href=\"https://github.com/ControlNet/NAVER/network/members\">\n <img src=\"https://img.shields.io/github/forks/ControlNet/NAVER?style=flat-square\">\n </a>\n <a href=\"https://github.com/ControlNet/NAVER/stargazers\">\n <img src=\"https://img.shields.io/github/stars/ControlNet/NAVER?style=flat-square\">\n </a>\n <a href=\"https://github.com/ControlNet/NAVER/blob/master/LICENSE\">\n <img src=\"https://img.shields.io/github/license/ControlNet/NAVER?style=flat-square\">\n </a>\n <a href=\"https://arxiv.org/abs/2502.00372\">\n <img src=\"https://img.shields.io/badge/arXiv-2502.00372-b31b1b.svg?style=flat-square\">\n </a>\n</div>\n\n<div align=\"center\"> \n <a href=\"https://pypi.org/project/naver/\">\n <img src=\"https://img.shields.io/pypi/v/naver?style=flat-square\">\n </a>\n <a href=\"https://pypi.org/project/naver/\">\n <img src=\"https://img.shields.io/pypi/dm/naver?style=flat-square\">\n </a>\n <a href=\"https://www.python.org/\"><img src=\"https://img.shields.io/pypi/pyversions/naver?style=flat-square\"></a>\n</div>\n\n\n**This repo is the official implementation for the paper [NAVER: A Neuro-Symbolic Compositional Automaton for Visual Grounding with Explicit Logic Reasoning](https://arxiv.org/abs/2502.00372) in ICCV 2025.**\n\n\n## Release\n\n- [2025/06/28] \ud83d\udd25 **NAVER** code is open sourced in GitHub.\n- [2025/06/25] \ud83c\udf89 **NAVER** paper is accepted by ICCV 2025.\n\n## TODOs\n\nWe're working on the following TODOs:\n- [x] GUI demo.\n- [ ] Support more LLMs.\n- [ ] Video demo & slides presentation.\n\n## Docker (GUI Demo)\n\nWe provide a Docker image for the GUI demo.\n\n```bash\ndocker run --runtime=nvidia --gpus=all -p <GUI-PORT>:8000 -e OPENAI_API_KEY=<OPENAI-API-KEY> -e AZURE_OPENAI_URL=<AZURE-OPENAI-URL> controlnet/naver:latest\n```\n\nThe GUI will be available at `http://0.0.0.0:<GUI-PORT>`. See details below for more information.\n\n## Installation\n\n### Requirements\n\n- Python >= 3.10\n- conda\n\nPlease follow the instructions below to install the required packages and set up the environment.\n\n### 1. Clone this repository.\n```Bash\ngit clone https://github.com/ControlNet/NAVER\n```\n\n### 2. Setup conda environment and install dependencies. \n\nOption 1: Using [pixi](https://prefix.dev/) (recommended):\n```Bash\npixi install\npixi shell\n```\n\nOption 2: Building from source (You may need to setup the CUDA and PyTorch manually):\n```Bash\nconda install conda-forge/label/rust_dev::rust=1.78 -c conda-forge -y\npip install \"git+https://github.com/scallop-lang/scallop.git@f8fac18#egg=scallopy&subdirectory=etc/scallopy\"\npip install -e .\n```\n\n### 3. Configure the environments\n\nEdit the file `.env` or setup in CLI to configure the environment variables.\n\n```\nOPENAI_API_KEY=your-api-key # if you want to use OpenAI LLMs\nAZURE_OPENAI_URL= # if you want to use Azure OpenAI LLMs\nOLLAMA_HOST=http://ollama.server:11434 # if you want to use your OLLaMA server for llama or deepseek\n# do not change this TORCH_HOME variable\nTORCH_HOME=./pretrained_models\n```\n\n### 4. Download the pretrained models\nRun the scripts to download the pretrained models to the `./pretrained_models` directory. \n\n```Bash\npython -m hydra_vl4ai.download_model --base_config config/refcoco.yaml --model_config config/model_config.yaml --extra_packages naver.tool\n```\n\n## Inference\n\nYou may need 28GB vRAM to run NAVER. Consider editing the file in `./config/model_config.yaml` to load the models in multiple GPUs.\n\n### Inference with GUI\n\nYou need `nodejs` and `npm` to run the GUI demo. It will automatically compile and build the frontend.\n\nThe GUI will be available at `http://0.0.0.0:8000`.\n\n```Bash\npython demo_gui.py \\\n --base_config <YOUR-CONFIG-DIR> \\\n --model_config <MODEL-CONFIG-PATH>\n```\n\n\n\n### Inference with given one image and query\n```Bash\npython demo_cli.py \\\n --image <IMAGE_PATH> \\\n --query <QUERY> \\\n --base_config <YOUR-CONFIG-DIR> \\\n --model_config <MODEL-CONFIG-PATH>\n```\n\nThe result will be printed in the console.\n\n### Inference dataset\n\n```Bash\npython main.py \\\n --data_root <YOUR-DATA-ROOT> \\\n --base_config <YOUR-CONFIG-DIR> \\\n --model_config <MODEL-CONFIG-PATH>\n```\n\nThen the inference results are saved in the `./result` directory for evaluation.\n\n## Evaluation\n\n```Bash\npython evaluate.py --input <RESULT_JSONL_PATH>\n```\n\nThe evaluation results will be printed in the console. Note the output from LLM is random, so the evaluation results may be slightly different from the paper.\n\n## Citation\nIf you find this work useful for your research, please consider citing it.\n```bibtex\n@article{cai2025naver,\n title = {NAVER: A Neuro-Symbolic Compositional Automaton for Visual Grounding with Explicit Logic Reasoning},\n author = {Cai, Zhixi and Ke, Fucai and Jahangard, Simindokht and Garcia de la Banda, Maria and Haffari, Reza and Stuckey, Peter J. and Rezatofighi, Hamid},\n journal = {arXiv preprint arXiv:2502.00372},\n year = {2025},\n}\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "Official implementation of NAVER.",
"version": "0.0.2",
"project_urls": {
"Bug Tracker": "https://github.com/ControlNet/NAVER/issues",
"Homepage": "https://github.com/ControlNet/NAVER",
"Repository": "https://github.com/ControlNet/NAVER"
},
"split_keywords": [
"deep learning",
" pytorch",
" ai"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "0721619f0d86a4f7a42ed1c26ce7c5f71e8beabefca6c0082c4751f9ba3abf15",
"md5": "82934b7755f6544f9e7634702ac52b80",
"sha256": "39cf0688098da0e849b8b425a6c8d6c519dcbacd02355504334e288d25d02b3d"
},
"downloads": -1,
"filename": "naver-0.0.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "82934b7755f6544f9e7634702ac52b80",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 36799,
"upload_time": "2025-09-07T12:34:44",
"upload_time_iso_8601": "2025-09-07T12:34:44.969827Z",
"url": "https://files.pythonhosted.org/packages/07/21/619f0d86a4f7a42ed1c26ce7c5f71e8beabefca6c0082c4751f9ba3abf15/naver-0.0.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "9061caa284b476e82b7ebf1da12d1f27e153f075f0b24ca7d3e6ec35b608b741",
"md5": "0291713d56e09f5d5ab2262666166a51",
"sha256": "9ccf61d5c0a7af34781e655539fbb6c9790e822a214916dd6d93f17ce8e441a0"
},
"downloads": -1,
"filename": "naver-0.0.2.tar.gz",
"has_sig": false,
"md5_digest": "0291713d56e09f5d5ab2262666166a51",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 29610,
"upload_time": "2025-09-07T12:34:46",
"upload_time_iso_8601": "2025-09-07T12:34:46.451837Z",
"url": "https://files.pythonhosted.org/packages/90/61/caa284b476e82b7ebf1da12d1f27e153f075f0b24ca7d3e6ec35b608b741/naver-0.0.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-09-07 12:34:46",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ControlNet",
"github_project": "NAVER",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "naver"
}
JSON
