<h1 align="center">
equilib
</h1>
<h4 align="center">
Processing Equirectangular Images with Python
</h4>
<div align="center">
<a href="https://badge.fury.io/py/pyequilib"><img src="https://badge.fury.io/py/pyequilib.svg" alt="PyPI version"></a>
<a href="https://pypi.org/project/pyequilib"><img src="https://img.shields.io/pypi/pyversions/pyequilib"></a>
<a href="https://github.com/haruishi43/equilib/actions"><img src="https://github.com/haruishi43/equilib/workflows/ci/badge.svg"></a>
<a href="https://github.com/haruishi43/equilib/blob/master/LICENSE"><img alt="GitHub license" src="https://img.shields.io/github/license/haruishi43/equilib"></a>
</div>
<img src=".img/equilib.png" alt="equilib" width="720"/>
- A library for processing equirectangular image that runs on Python.
- Developed using Python>=3.6 (`c++` is WIP).
- Compatible with `cuda` tensors for faster processing.
- No need for other dependencies except for `numpy` and `torch`.
- Added functionality like creating rotation matrices, batched processing, and automatic type detection.
- Highly modular
If you found this module helpful to your project, please site this repository:
```
@software{pyequilib2021github,
author = {Haruya Ishikawa},
title = {PyEquilib: Processing Equirectangular Images with Python},
url = {http://github.com/haruishi43/equilib},
version = {0.5.0},
year = {2021},
}
```
## Installation:
Prerequisites:
- Python (>=3.6)
- Pytorch (tested on 1.12)
```Bash
pip install pyequilib
```
For developing, use:
```Bash
git clone --recursive https://github.com/haruishi43/equilib.git
cd equilib
pip install -r requirements.txt
pip install -e .
# or
python setup.py develop
```
__NOTE__: might not work for PyTorch>=2.0. If you have any issues, please open an issue.
## Basic Usage:
`equilib` has different transforms of equirectangular (or cubemap) images (note each transform has `class` and `func` APIs):
- `Cube2Equi`/`cube2equi`: cubemap to equirectangular transform
- `Equi2Cube`/`equi2cube`: equirectangular to cubemap transform
- `Equi2Equi`/`equi2equi`: equirectangular transform
- `Equi2Pers`/`equi2pers`: equirectangular to perspective transform
There are no _real_ differences in `class` or `func` APIs:
- `class` APIs will allow instantiating a class which you can call many times without having to specify configurations (`class` APIs call the `func` API)
- `func` APIs are useful when there are no repetitive calls
- both `class` and `func` APIs are extensible, so you can extend them to your use-cases or create a method that's more optimized (pull requests are welcome btw)
Each API automatically detects the input type (`numpy.ndarray` or `torch.Tensor`), and outputs are the same type.
An example for `Equi2Pers`/`equi2pers`:
<table>
<tr>
<td><pre>Equi2Pers</pre></td>
<td><pre>equi2pers</pre></td>
</tr>
<tr>
<td>
<pre>
```Python
import numpy as np
from PIL import Image
from equilib import Equi2Pers
# Input equirectangular image
equi_img = Image.open("./some_image.jpg")
equi_img = np.asarray(equi_img)
equi_img = np.transpose(equi_img, (2, 0, 1))
# rotations
rots = {
'roll': 0.,
'pitch': np.pi/4, # rotate vertical
'yaw': np.pi/4, # rotate horizontal
}
# Intialize equi2pers
equi2pers = Equi2Pers(
height=480,
width=640,
fov_x=90.0,
mode="bilinear",
)
# obtain perspective image
pers_img = equi2pers(
equi=equi_img,
rots=rots,
)
```
</pre>
</td>
<td>
<pre>
```Python
import numpy as np
from PIL import Image
from equilib import equi2pers
# Input equirectangular image
equi_img = Image.open("./some_image.jpg")
equi_img = np.asarray(equi_img)
equi_img = np.transpose(equi_img, (2, 0, 1))
# rotations
rots = {
'roll': 0.,
'pitch': np.pi/4, # rotate vertical
'yaw': np.pi/4, # rotate horizontal
}
# Run equi2pers
pers_img = equi2pers(
equi=equi_img,
rots=rots,
height=480,
width=640,
fov_x=90.0,
mode="bilinear",
)
```
</pre>
</td>
</table>
For more information about how each APIs work, take a look in [.readme](.readme/) or go through example codes in the `tests` or `scripts`.
### Coordinate System:
__Right-handed rule XYZ global coordinate system__. `x-axis` faces forward and `z-axis` faces up.
- `roll`: counter-clockwise rotation about the `x-axis`
- `pitch`: counter-clockwise rotation about the `y-axis`
- `yaw`: counter-clockwise rotation about the `z-axis`
You can chnage the right-handed coordinate system so that the `z-axis` faces down by adding `z_down=True` as a parameter.
See demo scripts under `scripts`.
## Grid Sampling
To process equirectangular images fast, whether to crop perspective images from the equirectangular image, the library takes advantage of grid sampling techniques.
Some sampling techniques are already implemented, such as `scipy.ndimage.map_coordiantes` and `cv2.remap`.
This project's goal was to reduce these dependencies and use `cuda` and batch processing with `torch` and `c++` for a faster processing of equirectangular images.
There were not many projects online for these purposes.
In this library, we implement varieties of methods using `c++`, `numpy`, and `torch`.
This part of the code needs `cuda` acceleration because grid sampling is parallelizable.
For `torch`, the built-in `torch.nn.functional.grid_sample` function is very fast and reliable.
I have implemented a _pure_ `torch` implementation of `grid_sample` which is very customizable (might not be fast as the native function).
For `numpy`, I have implemented grid sampling methods that are faster than `scipy` and more robust than `cv2.remap`.
Just like with this implementation of `torch`, `numpy` implementation is just as customizable.
It is also possible to pass the `scipy` and `cv2`'s grid sampling function through the use of `override_func` argument in `grid_sample`.
Developing _faster_ approaches and `c++` methods are __WIP__.
See [here](equilib/grid_sample/README.md) for more info on implementations.
Some notes:
- By default, `numpy`'s [`grid_sample`](equilib/grid_sample/numpy/) will use pure `numpy` implementation. It is possible to override this implementation with `scipy` and `cv2`'s implementation using [`override_func`](tests/equi2pers/numpy_run_baselines.py).
- By default, `torch`'s [`grid_sample`](equilib/grid_sample/torch/) will use the official implementation.
- Benchmarking codes are stored in `tests/`. For example, benchmarking codes for `numpy`'s `equi2pers` is located in [`tests/equi2pers/numpy_run_baselines.py`](tests/equi2pers/numpy_run_baselines.py) and you can benchmark the runtime performance using different parameters against `scipy` and `cv2`.
## Develop:
Test files for `equilib` are included under `tests`.
Running tests:
```Bash
pytest tests
```
Note that I have added codes to benchmark every step of the process so that it is possible to optimize the code.
If you find there are optimal ways of the implementation or bugs, all pull requests and issues are welcome.
Check [CONTRIBUTING.md](./CONTRIBUTING.md) for more information
### TODO:
- [ ] Documentations for each transform
- [x] Add table and statistics for speed improvements
- [x] Batch processing for `numpy`
- [x] Mixed precision for `torch`
- [ ] `c++` version of grid sampling
- [ ] More accurate intrinsic matrix formulation using vertial FOV for `equi2pers`
- [ ] Multiprocessing support (slow when running on `torch.distributed`)
## Acknowledgements:
- [py360convert](https://github.com/sunset1995/py360convert)
- [Perspective-and-Equirectangular](https://github.com/timy90022/Perspective-and-Equirectangular)
Raw data
{
"_id": null,
"home_page": "https://github.com/haruishi43/equilib",
"name": "pyequilib",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "Equirectangular,Computer Vision",
"author": "Haruya Ishikawa",
"author_email": "haru.ishi43@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/e5/a3/6b39bb3286212f79e89c1f5bb85096a05c0c5d5feefd99b50eb56ad33aa0/pyequilib-0.5.8.tar.gz",
"platform": null,
"description": "<h1 align=\"center\">\n equilib\n</h1>\n\n<h4 align=\"center\">\n Processing Equirectangular Images with Python\n</h4>\n\n<div align=\"center\">\n <a href=\"https://badge.fury.io/py/pyequilib\"><img src=\"https://badge.fury.io/py/pyequilib.svg\" alt=\"PyPI version\"></a>\n <a href=\"https://pypi.org/project/pyequilib\"><img src=\"https://img.shields.io/pypi/pyversions/pyequilib\"></a>\n <a href=\"https://github.com/haruishi43/equilib/actions\"><img src=\"https://github.com/haruishi43/equilib/workflows/ci/badge.svg\"></a>\n <a href=\"https://github.com/haruishi43/equilib/blob/master/LICENSE\"><img alt=\"GitHub license\" src=\"https://img.shields.io/github/license/haruishi43/equilib\"></a>\n</div>\n\n<img src=\".img/equilib.png\" alt=\"equilib\" width=\"720\"/>\n\n- A library for processing equirectangular image that runs on Python.\n- Developed using Python>=3.6 (`c++` is WIP).\n- Compatible with `cuda` tensors for faster processing.\n- No need for other dependencies except for `numpy` and `torch`.\n- Added functionality like creating rotation matrices, batched processing, and automatic type detection.\n- Highly modular\n\nIf you found this module helpful to your project, please site this repository:\n```\n@software{pyequilib2021github,\n author = {Haruya Ishikawa},\n title = {PyEquilib: Processing Equirectangular Images with Python},\n url = {http://github.com/haruishi43/equilib},\n version = {0.5.0},\n year = {2021},\n}\n```\n\n## Installation:\n\nPrerequisites:\n- Python (>=3.6)\n- Pytorch (tested on 1.12)\n\n```Bash\npip install pyequilib\n```\n\nFor developing, use:\n\n```Bash\ngit clone --recursive https://github.com/haruishi43/equilib.git\ncd equilib\n\npip install -r requirements.txt\n\npip install -e .\n# or\npython setup.py develop\n```\n\n__NOTE__: might not work for PyTorch>=2.0. If you have any issues, please open an issue.\n\n## Basic Usage:\n\n`equilib` has different transforms of equirectangular (or cubemap) images (note each transform has `class` and `func` APIs):\n- `Cube2Equi`/`cube2equi`: cubemap to equirectangular transform\n- `Equi2Cube`/`equi2cube`: equirectangular to cubemap transform\n- `Equi2Equi`/`equi2equi`: equirectangular transform\n- `Equi2Pers`/`equi2pers`: equirectangular to perspective transform\n\nThere are no _real_ differences in `class` or `func` APIs:\n- `class` APIs will allow instantiating a class which you can call many times without having to specify configurations (`class` APIs call the `func` API)\n- `func` APIs are useful when there are no repetitive calls\n- both `class` and `func` APIs are extensible, so you can extend them to your use-cases or create a method that's more optimized (pull requests are welcome btw)\n\nEach API automatically detects the input type (`numpy.ndarray` or `torch.Tensor`), and outputs are the same type.\n\nAn example for `Equi2Pers`/`equi2pers`:\n\n<table>\n<tr>\n<td><pre>Equi2Pers</pre></td>\n<td><pre>equi2pers</pre></td>\n</tr>\n\n<tr>\n<td>\n<pre>\n\n```Python\nimport numpy as np\nfrom PIL import Image\nfrom equilib import Equi2Pers\n\n# Input equirectangular image\nequi_img = Image.open(\"./some_image.jpg\")\nequi_img = np.asarray(equi_img)\nequi_img = np.transpose(equi_img, (2, 0, 1))\n\n# rotations\nrots = {\n 'roll': 0.,\n 'pitch': np.pi/4, # rotate vertical\n 'yaw': np.pi/4, # rotate horizontal\n}\n\n# Intialize equi2pers\nequi2pers = Equi2Pers(\n height=480,\n width=640,\n fov_x=90.0,\n mode=\"bilinear\",\n)\n\n# obtain perspective image\npers_img = equi2pers(\n equi=equi_img,\n rots=rots,\n)\n```\n\n</pre>\n</td>\n\n<td>\n<pre>\n\n```Python\nimport numpy as np\nfrom PIL import Image\nfrom equilib import equi2pers\n\n# Input equirectangular image\nequi_img = Image.open(\"./some_image.jpg\")\nequi_img = np.asarray(equi_img)\nequi_img = np.transpose(equi_img, (2, 0, 1))\n\n# rotations\nrots = {\n 'roll': 0.,\n 'pitch': np.pi/4, # rotate vertical\n 'yaw': np.pi/4, # rotate horizontal\n}\n\n# Run equi2pers\npers_img = equi2pers(\n equi=equi_img,\n rots=rots,\n height=480,\n width=640,\n fov_x=90.0,\n mode=\"bilinear\",\n)\n```\n\n</pre>\n</td>\n</table>\n\nFor more information about how each APIs work, take a look in [.readme](.readme/) or go through example codes in the `tests` or `scripts`.\n\n\n### Coordinate System:\n\n__Right-handed rule XYZ global coordinate system__. `x-axis` faces forward and `z-axis` faces up.\n- `roll`: counter-clockwise rotation about the `x-axis`\n- `pitch`: counter-clockwise rotation about the `y-axis`\n- `yaw`: counter-clockwise rotation about the `z-axis`\n\nYou can chnage the right-handed coordinate system so that the `z-axis` faces down by adding `z_down=True` as a parameter.\n\nSee demo scripts under `scripts`.\n\n\n## Grid Sampling\n\nTo process equirectangular images fast, whether to crop perspective images from the equirectangular image, the library takes advantage of grid sampling techniques.\nSome sampling techniques are already implemented, such as `scipy.ndimage.map_coordiantes` and `cv2.remap`.\nThis project's goal was to reduce these dependencies and use `cuda` and batch processing with `torch` and `c++` for a faster processing of equirectangular images.\nThere were not many projects online for these purposes.\nIn this library, we implement varieties of methods using `c++`, `numpy`, and `torch`.\nThis part of the code needs `cuda` acceleration because grid sampling is parallelizable.\nFor `torch`, the built-in `torch.nn.functional.grid_sample` function is very fast and reliable.\nI have implemented a _pure_ `torch` implementation of `grid_sample` which is very customizable (might not be fast as the native function).\nFor `numpy`, I have implemented grid sampling methods that are faster than `scipy` and more robust than `cv2.remap`.\nJust like with this implementation of `torch`, `numpy` implementation is just as customizable.\nIt is also possible to pass the `scipy` and `cv2`'s grid sampling function through the use of `override_func` argument in `grid_sample`.\nDeveloping _faster_ approaches and `c++` methods are __WIP__.\nSee [here](equilib/grid_sample/README.md) for more info on implementations.\n\nSome notes:\n\n- By default, `numpy`'s [`grid_sample`](equilib/grid_sample/numpy/) will use pure `numpy` implementation. It is possible to override this implementation with `scipy` and `cv2`'s implementation using [`override_func`](tests/equi2pers/numpy_run_baselines.py).\n- By default, `torch`'s [`grid_sample`](equilib/grid_sample/torch/) will use the official implementation.\n- Benchmarking codes are stored in `tests/`. For example, benchmarking codes for `numpy`'s `equi2pers` is located in [`tests/equi2pers/numpy_run_baselines.py`](tests/equi2pers/numpy_run_baselines.py) and you can benchmark the runtime performance using different parameters against `scipy` and `cv2`.\n\n## Develop:\n\nTest files for `equilib` are included under `tests`.\n\nRunning tests:\n```Bash\npytest tests\n```\n\nNote that I have added codes to benchmark every step of the process so that it is possible to optimize the code.\nIf you find there are optimal ways of the implementation or bugs, all pull requests and issues are welcome.\n\nCheck [CONTRIBUTING.md](./CONTRIBUTING.md) for more information\n\n### TODO:\n\n- [ ] Documentations for each transform\n- [x] Add table and statistics for speed improvements\n- [x] Batch processing for `numpy`\n- [x] Mixed precision for `torch`\n- [ ] `c++` version of grid sampling\n- [ ] More accurate intrinsic matrix formulation using vertial FOV for `equi2pers`\n- [ ] Multiprocessing support (slow when running on `torch.distributed`)\n\n## Acknowledgements:\n\n- [py360convert](https://github.com/sunset1995/py360convert)\n- [Perspective-and-Equirectangular](https://github.com/timy90022/Perspective-and-Equirectangular)\n",
"bugtrack_url": null,
"license": "Apache License 2.0",
"summary": "equirectangular image processing with python using minimum dependencies",
"version": "0.5.8",
"project_urls": {
"Homepage": "https://github.com/haruishi43/equilib"
},
"split_keywords": [
"equirectangular",
"computer vision"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e760b0a3eae47431fd1f9912a94ae9ce5e5afc17436bfeb74a082c11d94543f8",
"md5": "43d2b72400eda8f712e51d38098609f3",
"sha256": "645cdb71d2432800df60f3176df6baf033cdfedbdadceb8bb2349ed70afb338e"
},
"downloads": -1,
"filename": "pyequilib-0.5.8-py3-none-any.whl",
"has_sig": false,
"md5_digest": "43d2b72400eda8f712e51d38098609f3",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 129341,
"upload_time": "2024-01-17T03:31:40",
"upload_time_iso_8601": "2024-01-17T03:31:40.018432Z",
"url": "https://files.pythonhosted.org/packages/e7/60/b0a3eae47431fd1f9912a94ae9ce5e5afc17436bfeb74a082c11d94543f8/pyequilib-0.5.8-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e5a36b39bb3286212f79e89c1f5bb85096a05c0c5d5feefd99b50eb56ad33aa0",
"md5": "f1cd2093a75db0a23be5a550b27017b7",
"sha256": "fa34947162583bdf235d94e6b9d2d32847575ba1357ced9eeb4ec91e5650b592"
},
"downloads": -1,
"filename": "pyequilib-0.5.8.tar.gz",
"has_sig": false,
"md5_digest": "f1cd2093a75db0a23be5a550b27017b7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 71122,
"upload_time": "2024-01-17T03:31:41",
"upload_time_iso_8601": "2024-01-17T03:31:41.995199Z",
"url": "https://files.pythonhosted.org/packages/e5/a3/6b39bb3286212f79e89c1f5bb85096a05c0c5d5feefd99b50eb56ad33aa0/pyequilib-0.5.8.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-17 03:31:41",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "haruishi43",
"github_project": "equilib",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "pyequilib"
}
JSON
