<p align="center">
<picture>
<img alt="logo" src="assets/threefiner_icon.png" width="20%">
</picture>
</br>
<b>Threefiner</b>
</p>
An interface for text-guided mesh refinement.
https://github.com/3DTopia/threefiner/assets/25863658/a4abe725-b542-4a4a-a6d4-e4c4821f7d96
### Features
* **Mesh in, mesh out**: we support `ply` with vertex colors, `obj`, and single object `glb/gltf` with textures!
* **Easy to use**: both a CLI and a GUI is available.
* **Performant**: Refine your texture in 1 minute with Deepfloyd-IF-II.
### Install
We rely on `torch` and several CUDA extensions, please make sure you install them correctly first!
```bash
# tiny-cuda-nn
pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
# nvdiffrast
pip install git+https://github.com/NVlabs/nvdiffrast
# [optional, will use pysdf if unavailable] cubvh:
pip install git+https://github.com/ashawkey/cubvh
```
To use [Deepfloyd-IF](https://github.com/deep-floyd/IF) models, please log in to your huggingface and accept the [license](https://huggingface.co/DeepFloyd/IF-I-XL-v1.0).
To install this package:
```bash
# install from pypi
pip install threefiner
# install from github
pip install git+https://github.com/3DTopia/threefiner
# local install
git clone https://github.com/3DTopia/threefiner
cd threefiner
pip install .
```
### Usage
```bash
### command line interface
threefiner --help
# this is short for
python -m threefiner.cli --help
### refine a coarse mesh ('input.obj') using Stable-diffusion and save to 'logs/hamburger.glb'
threefiner sd --mesh input.obj --prompt 'a hamburger' --outdir logs --save hamburger.glb
### if the initial texture is good, we recommend using IF2 for refinement.
# by default, it will save to './name_fine.glb'
threefiner if2 --mesh name.glb --prompt 'description'
### if the initial texture is not good, we recommend using SD or IF first.
threefiner sd --mesh name.glb --prompt 'description'
threefiner if --mesh name.glb --prompt 'description'
### if the initial geometry is good, you can fix the geometry.
threefiner sd_fixgeo --mesh name.glb --prompt 'description'
threefiner if_fixgeo --mesh name.glb --prompt 'description'
threefiner if2_fixgeo --mesh name.glb --prompt 'description'
### advanced
# directional text prompt (append front/side/back view in text prompt)
# you need to know the mesh's front facing direction and specify it by '--front_dir'
# we use the OpenGL coordinate system, i.e., +x is right, +y is up, +z is front (more details: https://kit.kiui.moe/camera/)
# clock-wise rotation can be specified per 90 degree, e.g., +z1, -y2
threefiner if2 --mesh input.glb --prompt 'description' --text_dir --front_dir='+z'
# adjust training iterations
threefiner if2 --mesh input.glb --prompt 'description' --iters 1000
# explicitly fix the geometry and only refine texture
threefiner if2 --fix-geo --geom_mode mesh --mesh input.glb --prompt 'description' # equals if2_fixgeo
# open a GUI to visualize the training progress (needs a desktop)
threefiner if2 --mesh input.glb --prompt 'description' --gui
```
Gradio demo:
```bash
# requires gradio 4
python gradio_app.py if2
```
For more examples, please see [scripts](./scripts/).
### Q&A
* **How to make sure `--front_dir` for your model?**
You may first visualize it in a 3D viewer that follows OpenGL coordinate system:
<p align="center">
<picture>
<img alt="example_front_dir" src="assets/coord.jpg" width="50%">
</picture>
</p>
The chair is facing down the Y axis (Green), so we can use `--front_dir="-y"` to rectify it to face +Z axis (Blue).
* **fatal error: EGL/egl.h: No such file or directory**
By default, we use the OpenGL rasterizer. This error means there is no OpenGL installation, which is often the case for headless servers.
It's recommended to install OpenGL (along with NVIDIA driver) as it brings better performance.
Otherwise, you can append `--force_cuda_rast` to use the CUDA rasterizer instead.
## Acknowledgement
This work is built on many amazing research works and open-source projects, thanks a lot to all the authors for sharing!
- SDS `guidance` classes are based on [diffusers](https://github.com/huggingface/diffusers).
- `diffmc` geometry is based on [diso](https://github.com/SarahWeiii/diso).
- `mesh` geometry is based on [nerf2mesh](https://github.com/ashawkey/nerf2mesh).
- Texture encoding is based on [tinycudann](https://github.com/NVlabs/tiny-cuda-nn).
- Mesh renderer is based on [nvdiffrast](https://github.com/NVlabs/nvdiffrast).
- GUI is based on [dearpygui](https://github.com/hoffstadt/DearPyGui).
- The coarse models used in demo are generated by [Genie](https://lumalabs.ai/genie?view=create) and [3DTopia](https://github.com/3DTopia/3DTopia).
Raw data
{
"_id": null,
"home_page": "https://github.com/3DTopia/threefiner",
"name": "threefiner",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "generative mesh refinement",
"author": "kiui",
"author_email": "ashawkey1999@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/11/67/92efd4bea693e126b6638a1a26a19b89b10f6fc98c3f5a0793c1f3bf07ad/threefiner-0.1.2.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n <picture>\n <img alt=\"logo\" src=\"assets/threefiner_icon.png\" width=\"20%\">\n </picture>\n </br>\n <b>Threefiner</b>\n</p>\n\nAn interface for text-guided mesh refinement.\n\nhttps://github.com/3DTopia/threefiner/assets/25863658/a4abe725-b542-4a4a-a6d4-e4c4821f7d96\n\n### Features\n* **Mesh in, mesh out**: we support `ply` with vertex colors, `obj`, and single object `glb/gltf` with textures!\n* **Easy to use**: both a CLI and a GUI is available.\n* **Performant**: Refine your texture in 1 minute with Deepfloyd-IF-II.\n\n### Install\n\nWe rely on `torch` and several CUDA extensions, please make sure you install them correctly first!\n```bash\n# tiny-cuda-nn\npip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch\n\n# nvdiffrast\npip install git+https://github.com/NVlabs/nvdiffrast\n\n# [optional, will use pysdf if unavailable] cubvh:\npip install git+https://github.com/ashawkey/cubvh\n```\n\nTo use [Deepfloyd-IF](https://github.com/deep-floyd/IF) models, please log in to your huggingface and accept the [license](https://huggingface.co/DeepFloyd/IF-I-XL-v1.0).\n\nTo install this package:\n```bash\n# install from pypi\npip install threefiner\n\n# install from github\npip install git+https://github.com/3DTopia/threefiner\n\n# local install\ngit clone https://github.com/3DTopia/threefiner\ncd threefiner\npip install .\n```\n\n### Usage\n\n```bash\n### command line interface\nthreefiner --help\n# this is short for\npython -m threefiner.cli --help\n\n### refine a coarse mesh ('input.obj') using Stable-diffusion and save to 'logs/hamburger.glb'\nthreefiner sd --mesh input.obj --prompt 'a hamburger' --outdir logs --save hamburger.glb\n\n### if the initial texture is good, we recommend using IF2 for refinement.\n# by default, it will save to './name_fine.glb'\nthreefiner if2 --mesh name.glb --prompt 'description'\n\n### if the initial texture is not good, we recommend using SD or IF first.\nthreefiner sd --mesh name.glb --prompt 'description'\nthreefiner if --mesh name.glb --prompt 'description'\n\n### if the initial geometry is good, you can fix the geometry.\nthreefiner sd_fixgeo --mesh name.glb --prompt 'description'\nthreefiner if_fixgeo --mesh name.glb --prompt 'description'\nthreefiner if2_fixgeo --mesh name.glb --prompt 'description'\n\n### advanced\n# directional text prompt (append front/side/back view in text prompt)\n# you need to know the mesh's front facing direction and specify it by '--front_dir'\n# we use the OpenGL coordinate system, i.e., +x is right, +y is up, +z is front (more details: https://kit.kiui.moe/camera/)\n# clock-wise rotation can be specified per 90 degree, e.g., +z1, -y2\nthreefiner if2 --mesh input.glb --prompt 'description' --text_dir --front_dir='+z'\n\n# adjust training iterations\nthreefiner if2 --mesh input.glb --prompt 'description' --iters 1000\n\n# explicitly fix the geometry and only refine texture\nthreefiner if2 --fix-geo --geom_mode mesh --mesh input.glb --prompt 'description' # equals if2_fixgeo\n\n# open a GUI to visualize the training progress (needs a desktop)\nthreefiner if2 --mesh input.glb --prompt 'description' --gui\n```\n\nGradio demo:\n```bash\n# requires gradio 4\npython gradio_app.py if2\n```\n\nFor more examples, please see [scripts](./scripts/).\n\n### Q&A\n\n* **How to make sure `--front_dir` for your model?**\n \n You may first visualize it in a 3D viewer that follows OpenGL coordinate system:\n <p align=\"center\">\n <picture>\n <img alt=\"example_front_dir\" src=\"assets/coord.jpg\" width=\"50%\">\n </picture>\n </p>\n The chair is facing down the Y axis (Green), so we can use `--front_dir=\"-y\"` to rectify it to face +Z axis (Blue).\n\n* **fatal error: EGL/egl.h: No such file or directory**\n\n By default, we use the OpenGL rasterizer. This error means there is no OpenGL installation, which is often the case for headless servers.\n It's recommended to install OpenGL (along with NVIDIA driver) as it brings better performance.\n Otherwise, you can append `--force_cuda_rast` to use the CUDA rasterizer instead.\n\n## Acknowledgement\n\nThis work is built on many amazing research works and open-source projects, thanks a lot to all the authors for sharing!\n\n- SDS `guidance` classes are based on [diffusers](https://github.com/huggingface/diffusers).\n- `diffmc` geometry is based on [diso](https://github.com/SarahWeiii/diso).\n- `mesh` geometry is based on [nerf2mesh](https://github.com/ashawkey/nerf2mesh).\n- Texture encoding is based on [tinycudann](https://github.com/NVlabs/tiny-cuda-nn).\n- Mesh renderer is based on [nvdiffrast](https://github.com/NVlabs/nvdiffrast).\n- GUI is based on [dearpygui](https://github.com/hoffstadt/DearPyGui).\n- The coarse models used in demo are generated by [Genie](https://lumalabs.ai/genie?view=create) and [3DTopia](https://github.com/3DTopia/3DTopia).\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Threefiner: a text-guided mesh refiner",
"version": "0.1.2",
"project_urls": {
"Homepage": "https://github.com/3DTopia/threefiner"
},
"split_keywords": [
"generative",
"mesh",
"refinement"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6550a8a7f7325dc84d89bfc641ef453c8515a09db139916f929b654992bd3c6d",
"md5": "8f0ece93a9ed2078b8cb77a615cb70c4",
"sha256": "58a7183ae9dd2b562da4acb6bc1873fb183888b1c15d4758a6eb9a81513fb62d"
},
"downloads": -1,
"filename": "threefiner-0.1.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8f0ece93a9ed2078b8cb77a615cb70c4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 1144540,
"upload_time": "2024-01-27T04:35:38",
"upload_time_iso_8601": "2024-01-27T04:35:38.806386Z",
"url": "https://files.pythonhosted.org/packages/65/50/a8a7f7325dc84d89bfc641ef453c8515a09db139916f929b654992bd3c6d/threefiner-0.1.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "116792efd4bea693e126b6638a1a26a19b89b10f6fc98c3f5a0793c1f3bf07ad",
"md5": "c7fe89815f9cbe591ed56faeab50ee76",
"sha256": "1fb73f2b6ecc888133ab34668cef873312b8988f1d638e52e2d035d06a9b1dc2"
},
"downloads": -1,
"filename": "threefiner-0.1.2.tar.gz",
"has_sig": false,
"md5_digest": "c7fe89815f9cbe591ed56faeab50ee76",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 1118336,
"upload_time": "2024-01-27T04:35:46",
"upload_time_iso_8601": "2024-01-27T04:35:46.058779Z",
"url": "https://files.pythonhosted.org/packages/11/67/92efd4bea693e126b6638a1a26a19b89b10f6fc98c3f5a0793c1f3bf07ad/threefiner-0.1.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-27 04:35:46",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "3DTopia",
"github_project": "threefiner",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "threefiner"
}
JSON
