<div align="center">
# IGN LiDAR HD Processing Library
[](https://badge.fury.io/py/ign-lidar-hd)
[](https://pypi.org/project/ign-lidar-hd/)
[](https://www.python.org/downloads/)
[](https://opensource.org/licenses/MIT)
[](tests/)
[](https://sducournau.github.io/IGN_LIDAR_HD_DATASET/)
**Version 1.7.6** | [📚 Documentation](https://sducournau.github.io/IGN_LIDAR_HD_DATASET/)
<img src="https://github.com/sducournau/IGN_LIDAR_HD_DATASET/blob/main/website/static/img/lod3.png?raw=true" alt="LoD3 Building Model Icon" width="600">
**A comprehensive Python library for processing IGN LiDAR HD data into machine learning-ready datasets for Building Level of Detail (LOD) classification tasks.**
[Quick Start](#-quick-start) • [Features](#-key-features) • [Documentation](https://sducournau.github.io/IGN_LIDAR_HD_DATASET/) • [Examples](#-examples) • [API Reference](#-api-reference)
</div>
---
## 📋 Table of Contents
- [Overview](#-overview)
- [What's New](#-whats-new)
- [Quick Start](#-quick-start)
- [Key Features](#-key-features)
- [Usage Guide](#-usage-guide)
- [Library Architecture](#-library-architecture)
- [Output Format](#-output-format)
- [Examples](#-examples)
- [Documentation](#-documentation--resources)
- [Development](#-development)
- [Requirements](#-requirements)
- [License & Support](#-license--support)
---
## 📊 Overview
This library transforms raw IGN (Institut National de l'Information Géographique et Forestière) LiDAR HD point clouds into structured datasets ready for machine learning applications. Built specifically for building classification tasks, it handles the complete pipeline from data acquisition to training-ready patches.
### 📺 Video Demo
<div align="center">
<a href="https://www.youtube.com/watch?v=ksBWEhkVqQI" target="_blank">
<img src="https://github.com/sducournau/IGN_LIDAR_HD_DATASET/blob/v1.6.3/website/static/img/aerial.png?raw=true" alt="IGN LiDAR HD Processing Demo" width="800">
</a>
<p><em>▶️ Click to watch: Learn how to process LiDAR data for machine learning applications</em></p>
</div>
### 🔄 Processing Workflow
```mermaid
flowchart TD
A[IGN LiDAR HD Data] --> B[Download Tiles]
B --> C[Enrich with Features]
C --> D[Create Training Patches]
D --> E[ML-Ready Dataset]
B --> B1[Smart Skip Detection]
C --> C1[GPU/CPU Processing]
C --> C2[Geometric Features]
C --> C3[Data Augmentation]
C --> C4[RGB + Infrared NIR]
C --> C5[NDVI Calculation]
D --> D1[LOD Classification]
style A fill:#e1f5fe
style E fill:#e8f5e8
style B1 fill:#fff3e0
style C1 fill:#fff3e0
style C3 fill:#fff3e0
style C4 fill:#c8e6c9
style C5 fill:#c8e6c9
```
### 📈 Project Stats
- 🏗️ **14 core modules** - Comprehensive processing toolkit
- 📝 **10 example scripts** - From basic usage to advanced workflows
- 🧪 **Comprehensive test suite** - Ensuring reliability and performance
- 🌍 **50+ curated tiles** - Covering diverse French territories
- ⚡ **GPU & CPU support** - Flexible computation backends
- 🔄 **Smart resumability** - Never reprocess existing data
---
## ✨ What's New
### Version 1.7.6 - Critical Fix & Verification 🔧
**Highlights:**
- 🐛 **Critical Fix**: Verticality computation in GPU chunked processing
- Fixed: Verticality was all zeros in large files (>5M points) with `--use-gpu`
- Impact: Wall detection now working correctly
- Result: ~20% smaller files (removed uncomputed features)
- 🔍 **New Feature Verification System**
- Command: `ign-lidar-hd verify --input-dir <path>`
- Validates RGB, NIR, geometric features in enriched files
- Comprehensive anomaly detection and statistics
### Version 1.7.5 - Performance Breakthrough 🚀
**Highlights:**
- 🚀 **100-200x faster** feature computation through vectorized operations
- 💯 **100% GPU utilization** - Fixed efficiency bottlenecks
- 💾 **50-60% memory reduction** - Per-chunk architecture for all modes
- ⏱️ **Real-world impact**: 18M points in ~64 seconds (GPU+cuML) vs 14+ minutes before
- 🧠 **Intelligent auto-scaling** - Adaptive parameters based on hardware
**Performance Modes:**
- 🖥️ **CPU-only**: 60 min/tile (baseline) - 1.8GB RAM
- ⚡ **Hybrid GPU** (CuPy): 7-10 min/tile (6-8x speedup) - 2.8GB VRAM
- 🚀 **Full GPU** (RAPIDS cuML): 3-5 min/tile (12-20x speedup) - 3.4GB VRAM
📖 [Full Release Notes](https://sducournau.github.io/IGN_LIDAR_HD_DATASET/release-notes/) | [📋 Changelog](CHANGELOG.md)
### Recent Features
- **v1.7.6**: Feature verification system, verticality fix
- **v1.7.4**: GPU acceleration with RAPIDS cuML, WSL2 support
- **v1.7.3**: Infrared (NIR) augmentation for NDVI calculation
- **v1.7.1**: Auto-parameter analysis for optimal processing
- **v1.7.0**: Artifact mitigation preprocessing (60-80% reduction)
---
## 🚀 Quick Start
### Installation
#### Standard Installation (CPU Only)
```bash
pip install ign-lidar-hd
ign-lidar-hd --version # Verify installation
```
#### GPU Acceleration (Optional - 6-20x Speedup)
For optimal performance, install with GPU support:
```bash
# Quick install using provided script
./install_cuml.sh
# Or manual installation
# Prerequisites: NVIDIA GPU (4GB+ VRAM), CUDA 12.0+, Miniconda/Anaconda
conda create -n ign_gpu python=3.12 -y
conda activate ign_gpu
conda install -c rapidsai -c conda-forge -c nvidia cuml=24.10 cupy cuda-version=12.5 -y
pip install ign-lidar-hd
# Verify GPU setup
python scripts/verify_gpu_setup.py
```
**📚 Detailed Installation Guides:**
- [Standard Installation](https://sducournau.github.io/IGN_LIDAR_HD_DATASET/installation/quick-start)
- [GPU Setup Guide](https://sducournau.github.io/IGN_LIDAR_HD_DATASET/guides/gpu-acceleration)
- [Troubleshooting](https://sducournau.github.io/IGN_LIDAR_HD_DATASET/guides/troubleshooting)
### Quick Example
```bash
# 1. Download sample data
ign-lidar-hd download --bbox 2.3,48.8,2.4,48.9 --output data/ --max-tiles 5
# 2. Enrich with features (GPU accelerated if available)
ign-lidar-hd enrich --input-dir data/ --output enriched/ --mode full --use-gpu
# 3. Create training patches
ign-lidar-hd patch --input-dir enriched/ --output patches/ --lod-level LOD2
```
**Python API:**
```python
from ign_lidar import LiDARProcessor
# Initialize processor
processor = LiDARProcessor(lod_level="LOD2")
# Process a single tile
patches = processor.process_tile("data.laz", "output/")
# Process multiple files
patches = processor.process_directory("data/", "output/", num_workers=4)
```
---
## 📋 Key Features
### 🏗️ Core Processing
- **Pure LiDAR processing** - Geometric analysis without RGB dependencies
- **RGB & Infrared augmentation** - Optional color and Near-Infrared (NIR) from IGN orthophotos
- **NDVI-ready datasets** - Automatic vegetation index calculation (RGB + NIR)
- **Multi-level classification** - LOD2 (15 classes) and LOD3 (30+ classes) support
- **Rich features** - Surface normals, curvature, planarity, verticality, local density
- **Architectural styles** - Automatic building style inference
- **Preprocessing** - Artifact mitigation (60-80% scan line reduction)
- **Auto-parameters** - Intelligent tile analysis for optimal processing
### ⚡ Performance & Optimization
- **GPU acceleration** - CUDA-accelerated with RAPIDS cuML (6-20x speedup)
- **Parallel processing** - Multi-worker support with CPU core detection
- **Memory optimization** - Per-chunk architecture, 50-60% memory reduction
- **Smart skip detection** - Resume interrupted workflows automatically
- **Batch operations** - Process hundreds of tiles efficiently
- **Scalable** - Tested up to 1B+ points
### 🔧 Workflow Automation
- **Pipeline configuration** - YAML-based declarative workflows
- **Integrated downloader** - IGN WFS tile discovery and batch downloading
- **Format flexibility** - LAZ 1.4 (full features) or QGIS-compatible output
- **Unified CLI** - Single `ign-lidar-hd` command with intuitive subcommands
- **Idempotent operations** - Safe to restart, never reprocesses existing data
### 🌍 Geographic Intelligence
- **Strategic locations** - Pre-configured urban, coastal, and rural areas
- **Bounding box filtering** - Spatial subsetting for targeted analysis
- **Coordinate handling** - Automatic Lambert93 ↔ WGS84 transformations
- **Tile management** - Curated collection of 50+ test tiles across France
---
## 📖 Usage Guide
### Command Line Interface
The library provides a unified `ign-lidar-hd` command with four main subcommands:
#### 1. Download Command
Download LiDAR tiles from IGN:
```bash
# Download by bounding box
ign-lidar-hd download --bbox 2.3,48.8,2.4,48.9 --output data/ --max-tiles 10
# Download specific tiles
ign-lidar-hd download --tiles tile1.laz tile2.laz --output data/
```
#### 2. Enrich Command
Enrich LAZ files with geometric features:
```bash
# Basic enrichment
ign-lidar-hd enrich --input-dir data/ --output enriched/ --mode full
# GPU-accelerated enrichment
ign-lidar-hd enrich --input-dir data/ --output enriched/ --use-gpu
# Full-featured enrichment (recommended)
ign-lidar-hd enrich \
--input-dir data/ \
--output enriched/ \
--mode full \
--use-gpu \
--auto-params \
--preprocess \
--add-rgb --rgb-cache-dir cache/rgb \
--add-infrared --infrared-cache-dir cache/infrared
# Custom preprocessing
ign-lidar-hd enrich \
--input-dir data/ \
--output enriched/ \
--preprocess \
--sor-k 15 --sor-std 2.5 \
--ror-radius 1.0 --ror-neighbors 4 \
--voxel-size 0.5
```
**Preprocessing Options:**
- `--preprocess` - Enable artifact mitigation
- `--sor-k` - Statistical outlier removal: number of neighbors (default: 12)
- `--sor-std` - SOR: std deviation multiplier (default: 2.0)
- `--ror-radius` - Radius outlier removal: search radius in meters (default: 1.0)
- `--ror-neighbors` - ROR: minimum neighbors required (default: 4)
- `--voxel-size` - Voxel downsampling size in meters (optional)
**Augmentation Options:**
- `--add-rgb` - Add RGB colors from IGN orthophotos
- `--add-infrared` - Add NIR values from IGN IRC orthophotos
- `--augment` - Enable geometric augmentation (disabled by default)
- `--num-augmentations` - Number of augmented versions (default: 3)
#### 3. Patch Command
Create training patches from enriched files:
```bash
# Create patches
ign-lidar-hd patch \
--input-dir enriched/ \
--output patches/ \
--lod-level LOD2 \
--patch-size 150.0 \
--num-workers 4
```
#### 4. Verify Command
Verify features in enriched LAZ files:
```bash
# Verify a single file
ign-lidar-hd verify --input enriched/file.laz
# Verify all files in a directory
ign-lidar-hd verify --input-dir enriched/
# Quick check with sample display
ign-lidar-hd verify --input enriched/file.laz --show-samples
# Batch verification (first 10 files)
ign-lidar-hd verify --input-dir enriched/ --max-files 10
# Quiet mode (summary only)
ign-lidar-hd verify --input-dir enriched/ --quiet
```
**Verification Features:**
- ✅ RGB values (presence, ranges, diversity)
- ✅ NIR/infrared values
- ✅ Geometric features (linearity, planarity, sphericity, anisotropy, roughness)
- ✅ Value range validation [0, 1]
- ✅ Anomaly detection (default values, out-of-range)
- ✅ Statistical distributions and sample point display
#### 5. Pipeline Command (Recommended)
Execute complete workflows using YAML configuration:
```bash
# Create example configuration
ign-lidar-hd pipeline config.yaml --create-example full
# Run configured pipeline
ign-lidar-hd pipeline config.yaml
```
**Example YAML Configuration:**
```yaml
global:
num_workers: 4
download:
bbox: "2.3, 48.8, 2.4, 48.9"
output: "data/raw"
max_tiles: 10
enrich:
input_dir: "data/raw"
output: "data/enriched"
mode: "full"
use_gpu: true
auto_params: true
preprocess: true
add_rgb: true
add_infrared: true
rgb_cache_dir: "cache/rgb"
infrared_cache_dir: "cache/infrared"
patch:
input_dir: "data/enriched"
output: "data/patches"
lod_level: "LOD2"
num_points: 16384
```
### Python API
#### Basic Usage
```python
from ign_lidar import LiDARProcessor
# Initialize processor
processor = LiDARProcessor(
lod_level="LOD2",
patch_size=150.0,
patch_overlap=0.1
)
# Process single tile
patches = processor.process_tile("input.laz", "output/")
# Process directory
patches = processor.process_directory("input_dir/", "output_dir/", num_workers=4)
```
#### Batch Download
```python
from ign_lidar import IGNLiDARDownloader
# Initialize downloader
downloader = IGNLiDARDownloader("downloads/")
# Download by bounding box (WGS84)
tiles = downloader.download_by_bbox(
bbox=(-2.0, 47.0, -1.0, 48.0),
max_tiles=10
)
# Download specific tiles
tile_names = ["LHD_FXX_0186_6834_PTS_C_LAMB93_IGN69"]
downloader.download_tiles(tile_names)
```
#### Configuration
```python
# LOD Levels
processor = LiDARProcessor(lod_level="LOD2") # 15 classes
processor = LiDARProcessor(lod_level="LOD3") # 30+ classes
# Processing Options
processor = LiDARProcessor(
lod_level="LOD2",
patch_size=150.0, # Patch size in meters
patch_overlap=0.1, # 10% overlap
bbox=[xmin, ymin, xmax, ymax] # Spatial filter
)
```
---
## 🏗️ Library Architecture
### Component Architecture
```mermaid
graph TB
subgraph "Core Processing"
P[processor.py<br/>🔧 Main Engine]
F[features.py<br/>⚡ Feature Extraction]
GPU[features_gpu.py<br/>🖥️ GPU Acceleration]
end
subgraph "Data Management"
D[downloader.py<br/>📥 IGN WFS Integration]
TL[tile_list.py<br/>📂 Tile Management]
SL[strategic_locations.py<br/>🗺️ Geographic Zones]
MD[metadata.py<br/>📊 Dataset Metadata]
end
subgraph "Classification & Styles"
C[classes.py<br/>🏢 LOD2/LOD3 Schemas]
AS[architectural_styles.py<br/>🎨 Style Inference]
end
subgraph "Integration & Config"
CLI[cli.py<br/>🖱️ Command Interface]
CFG[config.py<br/>⚙️ Configuration]
QGIS[qgis_converter.py<br/>🔄 QGIS Compatibility]
U[utils.py<br/>🛠️ Core Utilities]
end
CLI --> P
CLI --> D
P --> F
P --> GPU
P --> C
F --> AS
D --> TL
D --> SL
P --> MD
style P fill:#e3f2fd
style F fill:#e8f5e8
style D fill:#fff3e0
style CLI fill:#f3e5f5
```
### Module Responsibilities
| Module | Purpose | Key Features |
| ---------------------------- | ---------------------- | ---------------------------------------------------------- |
| 🔧 `processor.py` | Main processing engine | Patch creation, LOD classification, workflow orchestration |
| 📥 `downloader.py` | IGN WFS integration | Tile discovery, batch download, smart skip detection |
| ⚡ `features.py` | Feature extraction | Normals, curvature, geometric properties |
| 🖥️ `features_gpu.py` | GPU acceleration | CUDA-optimized feature computation |
| 🏢 `classes.py` | Classification schemas | LOD2/LOD3 building taxonomies |
| 🎨 `architectural_styles.py` | Style inference | Building architecture classification |
### Example Workflows
```text
examples/
├── 🚀 basic_usage.py # Getting started
├── 🏙️ example_urban_simple.py # Urban processing
├── ⚡ parallel_processing_example.py # Performance optimization
├── 🔄 full_workflow_example.py # End-to-end pipeline
├── 🎨 multistyle_processing.py # Architecture analysis
├── 🧠 pytorch_dataloader.py # ML integration
├── 🆕 pipeline_example.py # YAML pipeline usage
├── 🆕 enrich_with_rgb.py # RGB augmentation
├── 🆕 demo_infrared_augmentation.py # Infrared augmentation
└── workflows/ # Production pipelines
config_examples/
├── 🆕 pipeline_full.yaml # Complete workflow
├── 🆕 pipeline_enrich.yaml # Enrich-only
└── 🆕 pipeline_patch.yaml # Patch-only
```
---
## 📦 Output Format
### Data Structure
```mermaid
graph TB
subgraph "Raw Input"
LAZ[LAZ Point Cloud<br/>XYZ + Intensity<br/>Classification]
end
subgraph "Enriched Data"
ELAZ[Enhanced LAZ<br/>+ 30 Features<br/>+ Building Labels]
end
subgraph "ML Dataset"
NPZ[NPZ Patches<br/>16K points each<br/>Ready for Training]
end
subgraph "NPZ Contents"
COORD[Coordinates<br/>X, Y, Z]
GEOM[Geometric Features<br/>Normals, Curvature]
SEMANTIC[Semantic Features<br/>Planarity, Verticality]
META[Metadata<br/>Intensity, Return#]
LABELS[Building Labels<br/>LOD2/LOD3 Classes]
end
LAZ --> ELAZ
ELAZ --> NPZ
NPZ --> COORD
NPZ --> GEOM
NPZ --> SEMANTIC
NPZ --> META
NPZ --> LABELS
style LAZ fill:#ffebee
style ELAZ fill:#e3f2fd
style NPZ fill:#e8f5e8
```
### NPZ File Structure
Each patch is saved as an NPZ file containing:
```python
{
'points': np.ndarray, # [N, 3] XYZ coordinates
'normals': np.ndarray, # [N, 3] surface normals
'curvature': np.ndarray, # [N] principal curvature
'intensity': np.ndarray, # [N] normalized intensity
'return_number': np.ndarray, # [N] return number
'height': np.ndarray, # [N] height above ground
'planarity': np.ndarray, # [N] planarity measure
'verticality': np.ndarray, # [N] verticality measure
'horizontality': np.ndarray, # [N] horizontality measure
'density': np.ndarray, # [N] local point density
'labels': np.ndarray, # [N] building class labels
# Optional (with augmentation):
'red': np.ndarray, # [N] RGB red channel
'green': np.ndarray, # [N] RGB green channel
'blue': np.ndarray, # [N] RGB blue channel
'infrared': np.ndarray, # [N] NIR values
}
```
### Data Dimensions
| Component | Shape | Data Type | Description |
| ---------- | -------- | --------- | -------------------------- |
| `points` | [N, 3] | `float32` | 3D coordinates (X, Y, Z) |
| `normals` | [N, 3] | `float32` | Surface normal vectors |
| `features` | [N, 27+] | `float32` | Geometric feature matrix |
| `labels` | [N] | `uint8` | Building component classes |
| `metadata` | [4] | `object` | Patch info (bbox, tile_id) |
> **📦 Typical patch**: 16,384 points, ~2.5MB compressed, ~8MB in memory
---
## 📝 Examples
### Urban Processing
```python
# High-detail urban processing
from ign_lidar import LiDARProcessor
processor = LiDARProcessor(lod_level="LOD3", num_augmentations=5)
patches = processor.process_tile("urban_area.laz", "output/urban/")
```
### Rural Processing
```python
# Simplified rural processing
processor = LiDARProcessor(lod_level="LOD2", num_augmentations=2)
patches = processor.process_tile("rural_area.laz", "output/rural/")
```
### Batch Processing
```python
from ign_lidar import WORKING_TILES, get_tiles_by_environment
# Get coastal tiles
coastal_tiles = get_tiles_by_environment("coastal")
# Process all coastal areas
for tile_info in coastal_tiles:
patches = processor.process_tile(
f"data/{tile_info['tile_name']}.laz",
f"output/coastal/{tile_info['tile_name']}/"
)
```
### PyTorch Integration
```python
from torch.utils.data import Dataset, DataLoader
import numpy as np
import glob
class LiDARPatchDataset(Dataset):
def __init__(self, patch_dir):
self.patch_files = glob.glob(f"{patch_dir}/**/*.npz", recursive=True)
def __len__(self):
return len(self.patch_files)
def __getitem__(self, idx):
data = np.load(self.patch_files[idx])
points = data['points']
features = np.concatenate([
data['normals'],
data['curvature'][:, None],
data['intensity'][:, None]
], axis=1)
labels = data['labels']
return points, features, labels
# Create dataloader
dataset = LiDARPatchDataset("patches/")
dataloader = DataLoader(dataset, batch_size=32, shuffle=True)
```
---
## 📚 Documentation & Resources
### Complete Documentation Hub
For comprehensive documentation, see the **[Documentation Hub](docs/README.md)**:
- **[📖 User Guides](docs/guides/)** - Quick start guides, QGIS integration, troubleshooting
- **[⚡ Features](docs/features/)** - Smart skip detection, format preferences, workflow optimization
- **[🔧 Technical Reference](docs/reference/)** - Memory optimization, performance tuning
- **[📦 Archive](docs/archive/)** - Bug fixes history, release notes, migration guides
### Essential Quick Links
- **[🎯 Quick Reference Card](QUICK_REFERENCE.md)** - Fast reference for all commands
- **[⚡ Smart Skip Features](docs/features/SMART_SKIP_SUMMARY.md)** - Resume workflows efficiently
- **[🗺️ QGIS Integration](docs/guides/QUICK_START_QGIS.md)** - GIS compatibility guide
- **[⚙️ Memory Optimization](docs/reference/MEMORY_OPTIMIZATION.md)** - Performance tuning
- **[📋 Output Formats](docs/features/OUTPUT_FORMAT_PREFERENCES.md)** - LAZ 1.4 vs QGIS formats
### Examples & Workflows
- **[Basic Usage](examples/basic_usage.py)** - Simple processing examples
- **[Urban Processing](examples/example_urban_simple.py)** - City-specific workflows
- **[Parallel Processing](examples/parallel_processing_example.py)** - Multi-worker optimization
- **[Full Workflow](examples/full_workflow_example.py)** - End-to-end pipeline
- **[Pipeline Configuration](examples/pipeline_example.py)** - YAML-based workflows
- **[RGB Augmentation](examples/enrich_with_rgb.py)** - Orthophoto integration
- **[PyTorch Integration](examples/pytorch_dataloader.py)** - ML training setup
---
## 🛠️ Development
### Setup Development Environment
```bash
git clone https://github.com/sducournau/IGN_LIDAR_HD_DATASET
cd IGN_LIDAR_HD_DATASET
pip install -e ".[dev]"
```
### Run Tests
```bash
pytest tests/
```
### Code Formatting
```bash
black ign_lidar/
flake8 ign_lidar/
```
---
## 🔗 Requirements
- Python 3.8+
- NumPy >= 1.21.0
- laspy >= 2.3.0
- scikit-learn >= 1.0.0
- tqdm >= 4.60.0
- requests >= 2.25.0
- PyYAML >= 6.0 (for pipeline configuration)
- Pillow >= 9.0.0 (for RGB augmentation)
**Optional (for GPU acceleration):**
- CUDA >= 12.0
- CuPy >= 12.0.0
- RAPIDS cuML >= 24.10 (recommended for best performance)
---
## 📚 API Reference
### Core Classes
- **`LiDARProcessor`**: Main processing engine for tile and directory processing
- **`IGNLiDARDownloader`**: Batch download functionality from IGN WFS service
- **`LOD2_CLASSES`**, **`LOD3_CLASSES`**: Classification taxonomies
### Utility Functions
- **`compute_normals()`**: Surface normal computation
- **`compute_curvature()`**: Principal curvature calculation
- **`extract_geometric_features()`**: Comprehensive feature extraction
- **`get_tiles_by_environment()`**: Filter tiles by environment type
---
## 📄 License & Support
### License
MIT License - see [LICENSE](LICENSE) file for details.
### Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
### Support
For issues and questions, please use the [GitHub Issues](https://github.com/sducournau/IGN_LIDAR_HD_DATASET/issues) page.
---
<div align="center">
**Made with ❤️ for the LiDAR and Machine Learning communities**
[⬆ Back to top](#ign-lidar-hd-processing-library)
</div>
Raw data
{
"_id": null,
"home_page": null,
"name": "ign-lidar-hd",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "lidar, ign, machine-learning, gis, building-classification",
"author": null,
"author_email": "imagodata <simon.ducournau@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/f5/ab/e36e06e54eb6143a23f80079f8b28ee38543b56adb4dfc2850ebbb187512/ign_lidar_hd-1.7.6.tar.gz",
"platform": null,
"description": "<div align=\"center\">\n\n# IGN LiDAR HD Processing Library\n\n[](https://badge.fury.io/py/ign-lidar-hd)\n[](https://pypi.org/project/ign-lidar-hd/)\n[](https://www.python.org/downloads/)\n[](https://opensource.org/licenses/MIT)\n[](tests/)\n[](https://sducournau.github.io/IGN_LIDAR_HD_DATASET/)\n\n**Version 1.7.6** | [\ud83d\udcda Documentation](https://sducournau.github.io/IGN_LIDAR_HD_DATASET/)\n\n<img src=\"https://github.com/sducournau/IGN_LIDAR_HD_DATASET/blob/main/website/static/img/lod3.png?raw=true\" alt=\"LoD3 Building Model Icon\" width=\"600\">\n\n**A comprehensive Python library for processing IGN LiDAR HD data into machine learning-ready datasets for Building Level of Detail (LOD) classification tasks.**\n\n[Quick Start](#-quick-start) \u2022 [Features](#-key-features) \u2022 [Documentation](https://sducournau.github.io/IGN_LIDAR_HD_DATASET/) \u2022 [Examples](#-examples) \u2022 [API Reference](#-api-reference)\n\n</div>\n\n---\n\n## \ud83d\udccb Table of Contents\n\n- [Overview](#-overview)\n- [What's New](#-whats-new)\n- [Quick Start](#-quick-start)\n- [Key Features](#-key-features)\n- [Usage Guide](#-usage-guide)\n- [Library Architecture](#-library-architecture)\n- [Output Format](#-output-format)\n- [Examples](#-examples)\n- [Documentation](#-documentation--resources)\n- [Development](#-development)\n- [Requirements](#-requirements)\n- [License & Support](#-license--support)\n\n---\n\n## \ud83d\udcca Overview\n\nThis library transforms raw IGN (Institut National de l'Information G\u00e9ographique et Foresti\u00e8re) LiDAR HD point clouds into structured datasets ready for machine learning applications. Built specifically for building classification tasks, it handles the complete pipeline from data acquisition to training-ready patches.\n\n### \ud83d\udcfa Video Demo\n\n<div align=\"center\">\n <a href=\"https://www.youtube.com/watch?v=ksBWEhkVqQI\" target=\"_blank\">\n <img src=\"https://github.com/sducournau/IGN_LIDAR_HD_DATASET/blob/v1.6.3/website/static/img/aerial.png?raw=true\" alt=\"IGN LiDAR HD Processing Demo\" width=\"800\">\n </a>\n <p><em>\u25b6\ufe0f Click to watch: Learn how to process LiDAR data for machine learning applications</em></p>\n</div>\n\n### \ud83d\udd04 Processing Workflow\n\n```mermaid\nflowchart TD\n A[IGN LiDAR HD Data] --> B[Download Tiles]\n B --> C[Enrich with Features]\n C --> D[Create Training Patches]\n D --> E[ML-Ready Dataset]\n\n B --> B1[Smart Skip Detection]\n C --> C1[GPU/CPU Processing]\n C --> C2[Geometric Features]\n C --> C3[Data Augmentation]\n C --> C4[RGB + Infrared NIR]\n C --> C5[NDVI Calculation]\n D --> D1[LOD Classification]\n\n style A fill:#e1f5fe\n style E fill:#e8f5e8\n style B1 fill:#fff3e0\n style C1 fill:#fff3e0\n style C3 fill:#fff3e0\n style C4 fill:#c8e6c9\n style C5 fill:#c8e6c9\n```\n\n### \ud83d\udcc8 Project Stats\n\n- \ud83c\udfd7\ufe0f **14 core modules** - Comprehensive processing toolkit\n- \ud83d\udcdd **10 example scripts** - From basic usage to advanced workflows\n- \ud83e\uddea **Comprehensive test suite** - Ensuring reliability and performance\n- \ud83c\udf0d **50+ curated tiles** - Covering diverse French territories\n- \u26a1 **GPU & CPU support** - Flexible computation backends\n- \ud83d\udd04 **Smart resumability** - Never reprocess existing data\n\n---\n\n## \u2728 What's New\n\n### Version 1.7.6 - Critical Fix & Verification \ud83d\udd27\n\n**Highlights:**\n\n- \ud83d\udc1b **Critical Fix**: Verticality computation in GPU chunked processing\n - Fixed: Verticality was all zeros in large files (>5M points) with `--use-gpu`\n - Impact: Wall detection now working correctly\n - Result: ~20% smaller files (removed uncomputed features)\n- \ud83d\udd0d **New Feature Verification System**\n - Command: `ign-lidar-hd verify --input-dir <path>`\n - Validates RGB, NIR, geometric features in enriched files\n - Comprehensive anomaly detection and statistics\n\n### Version 1.7.5 - Performance Breakthrough \ud83d\ude80\n\n**Highlights:**\n\n- \ud83d\ude80 **100-200x faster** feature computation through vectorized operations\n- \ud83d\udcaf **100% GPU utilization** - Fixed efficiency bottlenecks\n- \ud83d\udcbe **50-60% memory reduction** - Per-chunk architecture for all modes\n- \u23f1\ufe0f **Real-world impact**: 18M points in ~64 seconds (GPU+cuML) vs 14+ minutes before\n- \ud83e\udde0 **Intelligent auto-scaling** - Adaptive parameters based on hardware\n\n**Performance Modes:**\n\n- \ud83d\udda5\ufe0f **CPU-only**: 60 min/tile (baseline) - 1.8GB RAM\n- \u26a1 **Hybrid GPU** (CuPy): 7-10 min/tile (6-8x speedup) - 2.8GB VRAM\n- \ud83d\ude80 **Full GPU** (RAPIDS cuML): 3-5 min/tile (12-20x speedup) - 3.4GB VRAM\n\n\ud83d\udcd6 [Full Release Notes](https://sducournau.github.io/IGN_LIDAR_HD_DATASET/release-notes/) | [\ud83d\udccb Changelog](CHANGELOG.md)\n\n### Recent Features\n\n- **v1.7.6**: Feature verification system, verticality fix\n- **v1.7.4**: GPU acceleration with RAPIDS cuML, WSL2 support\n- **v1.7.3**: Infrared (NIR) augmentation for NDVI calculation\n- **v1.7.1**: Auto-parameter analysis for optimal processing\n- **v1.7.0**: Artifact mitigation preprocessing (60-80% reduction)\n\n---\n\n## \ud83d\ude80 Quick Start\n\n### Installation\n\n#### Standard Installation (CPU Only)\n\n```bash\npip install ign-lidar-hd\nign-lidar-hd --version # Verify installation\n```\n\n#### GPU Acceleration (Optional - 6-20x Speedup)\n\nFor optimal performance, install with GPU support:\n\n```bash\n# Quick install using provided script\n./install_cuml.sh\n\n# Or manual installation\n# Prerequisites: NVIDIA GPU (4GB+ VRAM), CUDA 12.0+, Miniconda/Anaconda\nconda create -n ign_gpu python=3.12 -y\nconda activate ign_gpu\nconda install -c rapidsai -c conda-forge -c nvidia cuml=24.10 cupy cuda-version=12.5 -y\npip install ign-lidar-hd\n\n# Verify GPU setup\npython scripts/verify_gpu_setup.py\n```\n\n**\ud83d\udcda Detailed Installation Guides:**\n\n- [Standard Installation](https://sducournau.github.io/IGN_LIDAR_HD_DATASET/installation/quick-start)\n- [GPU Setup Guide](https://sducournau.github.io/IGN_LIDAR_HD_DATASET/guides/gpu-acceleration)\n- [Troubleshooting](https://sducournau.github.io/IGN_LIDAR_HD_DATASET/guides/troubleshooting)\n\n### Quick Example\n\n```bash\n# 1. Download sample data\nign-lidar-hd download --bbox 2.3,48.8,2.4,48.9 --output data/ --max-tiles 5\n\n# 2. Enrich with features (GPU accelerated if available)\nign-lidar-hd enrich --input-dir data/ --output enriched/ --mode full --use-gpu\n\n# 3. Create training patches\nign-lidar-hd patch --input-dir enriched/ --output patches/ --lod-level LOD2\n```\n\n**Python API:**\n\n```python\nfrom ign_lidar import LiDARProcessor\n\n# Initialize processor\nprocessor = LiDARProcessor(lod_level=\"LOD2\")\n\n# Process a single tile\npatches = processor.process_tile(\"data.laz\", \"output/\")\n\n# Process multiple files\npatches = processor.process_directory(\"data/\", \"output/\", num_workers=4)\n```\n\n---\n\n## \ud83d\udccb Key Features\n\n### \ud83c\udfd7\ufe0f Core Processing\n\n- **Pure LiDAR processing** - Geometric analysis without RGB dependencies\n- **RGB & Infrared augmentation** - Optional color and Near-Infrared (NIR) from IGN orthophotos\n- **NDVI-ready datasets** - Automatic vegetation index calculation (RGB + NIR)\n- **Multi-level classification** - LOD2 (15 classes) and LOD3 (30+ classes) support\n- **Rich features** - Surface normals, curvature, planarity, verticality, local density\n- **Architectural styles** - Automatic building style inference\n- **Preprocessing** - Artifact mitigation (60-80% scan line reduction)\n- **Auto-parameters** - Intelligent tile analysis for optimal processing\n\n### \u26a1 Performance & Optimization\n\n- **GPU acceleration** - CUDA-accelerated with RAPIDS cuML (6-20x speedup)\n- **Parallel processing** - Multi-worker support with CPU core detection\n- **Memory optimization** - Per-chunk architecture, 50-60% memory reduction\n- **Smart skip detection** - Resume interrupted workflows automatically\n- **Batch operations** - Process hundreds of tiles efficiently\n- **Scalable** - Tested up to 1B+ points\n\n### \ud83d\udd27 Workflow Automation\n\n- **Pipeline configuration** - YAML-based declarative workflows\n- **Integrated downloader** - IGN WFS tile discovery and batch downloading\n- **Format flexibility** - LAZ 1.4 (full features) or QGIS-compatible output\n- **Unified CLI** - Single `ign-lidar-hd` command with intuitive subcommands\n- **Idempotent operations** - Safe to restart, never reprocesses existing data\n\n### \ud83c\udf0d Geographic Intelligence\n\n- **Strategic locations** - Pre-configured urban, coastal, and rural areas\n- **Bounding box filtering** - Spatial subsetting for targeted analysis\n- **Coordinate handling** - Automatic Lambert93 \u2194 WGS84 transformations\n- **Tile management** - Curated collection of 50+ test tiles across France\n\n---\n\n## \ud83d\udcd6 Usage Guide\n\n### Command Line Interface\n\nThe library provides a unified `ign-lidar-hd` command with four main subcommands:\n\n#### 1. Download Command\n\nDownload LiDAR tiles from IGN:\n\n```bash\n# Download by bounding box\nign-lidar-hd download --bbox 2.3,48.8,2.4,48.9 --output data/ --max-tiles 10\n\n# Download specific tiles\nign-lidar-hd download --tiles tile1.laz tile2.laz --output data/\n```\n\n#### 2. Enrich Command\n\nEnrich LAZ files with geometric features:\n\n```bash\n# Basic enrichment\nign-lidar-hd enrich --input-dir data/ --output enriched/ --mode full\n\n# GPU-accelerated enrichment\nign-lidar-hd enrich --input-dir data/ --output enriched/ --use-gpu\n\n# Full-featured enrichment (recommended)\nign-lidar-hd enrich \\\n --input-dir data/ \\\n --output enriched/ \\\n --mode full \\\n --use-gpu \\\n --auto-params \\\n --preprocess \\\n --add-rgb --rgb-cache-dir cache/rgb \\\n --add-infrared --infrared-cache-dir cache/infrared\n\n# Custom preprocessing\nign-lidar-hd enrich \\\n --input-dir data/ \\\n --output enriched/ \\\n --preprocess \\\n --sor-k 15 --sor-std 2.5 \\\n --ror-radius 1.0 --ror-neighbors 4 \\\n --voxel-size 0.5\n```\n\n**Preprocessing Options:**\n\n- `--preprocess` - Enable artifact mitigation\n- `--sor-k` - Statistical outlier removal: number of neighbors (default: 12)\n- `--sor-std` - SOR: std deviation multiplier (default: 2.0)\n- `--ror-radius` - Radius outlier removal: search radius in meters (default: 1.0)\n- `--ror-neighbors` - ROR: minimum neighbors required (default: 4)\n- `--voxel-size` - Voxel downsampling size in meters (optional)\n\n**Augmentation Options:**\n\n- `--add-rgb` - Add RGB colors from IGN orthophotos\n- `--add-infrared` - Add NIR values from IGN IRC orthophotos\n- `--augment` - Enable geometric augmentation (disabled by default)\n- `--num-augmentations` - Number of augmented versions (default: 3)\n\n#### 3. Patch Command\n\nCreate training patches from enriched files:\n\n```bash\n# Create patches\nign-lidar-hd patch \\\n --input-dir enriched/ \\\n --output patches/ \\\n --lod-level LOD2 \\\n --patch-size 150.0 \\\n --num-workers 4\n```\n\n#### 4. Verify Command\n\nVerify features in enriched LAZ files:\n\n```bash\n# Verify a single file\nign-lidar-hd verify --input enriched/file.laz\n\n# Verify all files in a directory\nign-lidar-hd verify --input-dir enriched/\n\n# Quick check with sample display\nign-lidar-hd verify --input enriched/file.laz --show-samples\n\n# Batch verification (first 10 files)\nign-lidar-hd verify --input-dir enriched/ --max-files 10\n\n# Quiet mode (summary only)\nign-lidar-hd verify --input-dir enriched/ --quiet\n```\n\n**Verification Features:**\n\n- \u2705 RGB values (presence, ranges, diversity)\n- \u2705 NIR/infrared values\n- \u2705 Geometric features (linearity, planarity, sphericity, anisotropy, roughness)\n- \u2705 Value range validation [0, 1]\n- \u2705 Anomaly detection (default values, out-of-range)\n- \u2705 Statistical distributions and sample point display\n\n#### 5. Pipeline Command (Recommended)\n\nExecute complete workflows using YAML configuration:\n\n```bash\n# Create example configuration\nign-lidar-hd pipeline config.yaml --create-example full\n\n# Run configured pipeline\nign-lidar-hd pipeline config.yaml\n```\n\n**Example YAML Configuration:**\n\n```yaml\nglobal:\n num_workers: 4\n\ndownload:\n bbox: \"2.3, 48.8, 2.4, 48.9\"\n output: \"data/raw\"\n max_tiles: 10\n\nenrich:\n input_dir: \"data/raw\"\n output: \"data/enriched\"\n mode: \"full\"\n use_gpu: true\n auto_params: true\n preprocess: true\n add_rgb: true\n add_infrared: true\n rgb_cache_dir: \"cache/rgb\"\n infrared_cache_dir: \"cache/infrared\"\n\npatch:\n input_dir: \"data/enriched\"\n output: \"data/patches\"\n lod_level: \"LOD2\"\n num_points: 16384\n```\n\n### Python API\n\n#### Basic Usage\n\n```python\nfrom ign_lidar import LiDARProcessor\n\n# Initialize processor\nprocessor = LiDARProcessor(\n lod_level=\"LOD2\",\n patch_size=150.0,\n patch_overlap=0.1\n)\n\n# Process single tile\npatches = processor.process_tile(\"input.laz\", \"output/\")\n\n# Process directory\npatches = processor.process_directory(\"input_dir/\", \"output_dir/\", num_workers=4)\n```\n\n#### Batch Download\n\n```python\nfrom ign_lidar import IGNLiDARDownloader\n\n# Initialize downloader\ndownloader = IGNLiDARDownloader(\"downloads/\")\n\n# Download by bounding box (WGS84)\ntiles = downloader.download_by_bbox(\n bbox=(-2.0, 47.0, -1.0, 48.0),\n max_tiles=10\n)\n\n# Download specific tiles\ntile_names = [\"LHD_FXX_0186_6834_PTS_C_LAMB93_IGN69\"]\ndownloader.download_tiles(tile_names)\n```\n\n#### Configuration\n\n```python\n# LOD Levels\nprocessor = LiDARProcessor(lod_level=\"LOD2\") # 15 classes\nprocessor = LiDARProcessor(lod_level=\"LOD3\") # 30+ classes\n\n# Processing Options\nprocessor = LiDARProcessor(\n lod_level=\"LOD2\",\n patch_size=150.0, # Patch size in meters\n patch_overlap=0.1, # 10% overlap\n bbox=[xmin, ymin, xmax, ymax] # Spatial filter\n)\n```\n\n---\n\n## \ud83c\udfd7\ufe0f Library Architecture\n\n### Component Architecture\n\n```mermaid\ngraph TB\n subgraph \"Core Processing\"\n P[processor.py<br/>\ud83d\udd27 Main Engine]\n F[features.py<br/>\u26a1 Feature Extraction]\n GPU[features_gpu.py<br/>\ud83d\udda5\ufe0f GPU Acceleration]\n end\n\n subgraph \"Data Management\"\n D[downloader.py<br/>\ud83d\udce5 IGN WFS Integration]\n TL[tile_list.py<br/>\ud83d\udcc2 Tile Management]\n SL[strategic_locations.py<br/>\ud83d\uddfa\ufe0f Geographic Zones]\n MD[metadata.py<br/>\ud83d\udcca Dataset Metadata]\n end\n\n subgraph \"Classification & Styles\"\n C[classes.py<br/>\ud83c\udfe2 LOD2/LOD3 Schemas]\n AS[architectural_styles.py<br/>\ud83c\udfa8 Style Inference]\n end\n\n subgraph \"Integration & Config\"\n CLI[cli.py<br/>\ud83d\uddb1\ufe0f Command Interface]\n CFG[config.py<br/>\u2699\ufe0f Configuration]\n QGIS[qgis_converter.py<br/>\ud83d\udd04 QGIS Compatibility]\n U[utils.py<br/>\ud83d\udee0\ufe0f Core Utilities]\n end\n\n CLI --> P\n CLI --> D\n P --> F\n P --> GPU\n P --> C\n F --> AS\n D --> TL\n D --> SL\n P --> MD\n\n style P fill:#e3f2fd\n style F fill:#e8f5e8\n style D fill:#fff3e0\n style CLI fill:#f3e5f5\n```\n\n### Module Responsibilities\n\n| Module | Purpose | Key Features |\n| ---------------------------- | ---------------------- | ---------------------------------------------------------- |\n| \ud83d\udd27 `processor.py` | Main processing engine | Patch creation, LOD classification, workflow orchestration |\n| \ud83d\udce5 `downloader.py` | IGN WFS integration | Tile discovery, batch download, smart skip detection |\n| \u26a1 `features.py` | Feature extraction | Normals, curvature, geometric properties |\n| \ud83d\udda5\ufe0f `features_gpu.py` | GPU acceleration | CUDA-optimized feature computation |\n| \ud83c\udfe2 `classes.py` | Classification schemas | LOD2/LOD3 building taxonomies |\n| \ud83c\udfa8 `architectural_styles.py` | Style inference | Building architecture classification |\n\n### Example Workflows\n\n```text\nexamples/\n\u251c\u2500\u2500 \ud83d\ude80 basic_usage.py # Getting started\n\u251c\u2500\u2500 \ud83c\udfd9\ufe0f example_urban_simple.py # Urban processing\n\u251c\u2500\u2500 \u26a1 parallel_processing_example.py # Performance optimization\n\u251c\u2500\u2500 \ud83d\udd04 full_workflow_example.py # End-to-end pipeline\n\u251c\u2500\u2500 \ud83c\udfa8 multistyle_processing.py # Architecture analysis\n\u251c\u2500\u2500 \ud83e\udde0 pytorch_dataloader.py # ML integration\n\u251c\u2500\u2500 \ud83c\udd95 pipeline_example.py # YAML pipeline usage\n\u251c\u2500\u2500 \ud83c\udd95 enrich_with_rgb.py # RGB augmentation\n\u251c\u2500\u2500 \ud83c\udd95 demo_infrared_augmentation.py # Infrared augmentation\n\u2514\u2500\u2500 workflows/ # Production pipelines\n\nconfig_examples/\n\u251c\u2500\u2500 \ud83c\udd95 pipeline_full.yaml # Complete workflow\n\u251c\u2500\u2500 \ud83c\udd95 pipeline_enrich.yaml # Enrich-only\n\u2514\u2500\u2500 \ud83c\udd95 pipeline_patch.yaml # Patch-only\n```\n\n---\n\n## \ud83d\udce6 Output Format\n\n### Data Structure\n\n```mermaid\ngraph TB\n subgraph \"Raw Input\"\n LAZ[LAZ Point Cloud<br/>XYZ + Intensity<br/>Classification]\n end\n\n subgraph \"Enriched Data\"\n ELAZ[Enhanced LAZ<br/>+ 30 Features<br/>+ Building Labels]\n end\n\n subgraph \"ML Dataset\"\n NPZ[NPZ Patches<br/>16K points each<br/>Ready for Training]\n end\n\n subgraph \"NPZ Contents\"\n COORD[Coordinates<br/>X, Y, Z]\n GEOM[Geometric Features<br/>Normals, Curvature]\n SEMANTIC[Semantic Features<br/>Planarity, Verticality]\n META[Metadata<br/>Intensity, Return#]\n LABELS[Building Labels<br/>LOD2/LOD3 Classes]\n end\n\n LAZ --> ELAZ\n ELAZ --> NPZ\n NPZ --> COORD\n NPZ --> GEOM\n NPZ --> SEMANTIC\n NPZ --> META\n NPZ --> LABELS\n\n style LAZ fill:#ffebee\n style ELAZ fill:#e3f2fd\n style NPZ fill:#e8f5e8\n```\n\n### NPZ File Structure\n\nEach patch is saved as an NPZ file containing:\n\n```python\n{\n 'points': np.ndarray, # [N, 3] XYZ coordinates\n 'normals': np.ndarray, # [N, 3] surface normals\n 'curvature': np.ndarray, # [N] principal curvature\n 'intensity': np.ndarray, # [N] normalized intensity\n 'return_number': np.ndarray, # [N] return number\n 'height': np.ndarray, # [N] height above ground\n 'planarity': np.ndarray, # [N] planarity measure\n 'verticality': np.ndarray, # [N] verticality measure\n 'horizontality': np.ndarray, # [N] horizontality measure\n 'density': np.ndarray, # [N] local point density\n 'labels': np.ndarray, # [N] building class labels\n # Optional (with augmentation):\n 'red': np.ndarray, # [N] RGB red channel\n 'green': np.ndarray, # [N] RGB green channel\n 'blue': np.ndarray, # [N] RGB blue channel\n 'infrared': np.ndarray, # [N] NIR values\n}\n```\n\n### Data Dimensions\n\n| Component | Shape | Data Type | Description |\n| ---------- | -------- | --------- | -------------------------- |\n| `points` | [N, 3] | `float32` | 3D coordinates (X, Y, Z) |\n| `normals` | [N, 3] | `float32` | Surface normal vectors |\n| `features` | [N, 27+] | `float32` | Geometric feature matrix |\n| `labels` | [N] | `uint8` | Building component classes |\n| `metadata` | [4] | `object` | Patch info (bbox, tile_id) |\n\n> **\ud83d\udce6 Typical patch**: 16,384 points, ~2.5MB compressed, ~8MB in memory\n\n---\n\n## \ud83d\udcdd Examples\n\n### Urban Processing\n\n```python\n# High-detail urban processing\nfrom ign_lidar import LiDARProcessor\n\nprocessor = LiDARProcessor(lod_level=\"LOD3\", num_augmentations=5)\npatches = processor.process_tile(\"urban_area.laz\", \"output/urban/\")\n```\n\n### Rural Processing\n\n```python\n# Simplified rural processing\nprocessor = LiDARProcessor(lod_level=\"LOD2\", num_augmentations=2)\npatches = processor.process_tile(\"rural_area.laz\", \"output/rural/\")\n```\n\n### Batch Processing\n\n```python\nfrom ign_lidar import WORKING_TILES, get_tiles_by_environment\n\n# Get coastal tiles\ncoastal_tiles = get_tiles_by_environment(\"coastal\")\n\n# Process all coastal areas\nfor tile_info in coastal_tiles:\n patches = processor.process_tile(\n f\"data/{tile_info['tile_name']}.laz\",\n f\"output/coastal/{tile_info['tile_name']}/\"\n )\n```\n\n### PyTorch Integration\n\n```python\nfrom torch.utils.data import Dataset, DataLoader\nimport numpy as np\nimport glob\n\nclass LiDARPatchDataset(Dataset):\n def __init__(self, patch_dir):\n self.patch_files = glob.glob(f\"{patch_dir}/**/*.npz\", recursive=True)\n\n def __len__(self):\n return len(self.patch_files)\n\n def __getitem__(self, idx):\n data = np.load(self.patch_files[idx])\n points = data['points']\n features = np.concatenate([\n data['normals'],\n data['curvature'][:, None],\n data['intensity'][:, None]\n ], axis=1)\n labels = data['labels']\n return points, features, labels\n\n# Create dataloader\ndataset = LiDARPatchDataset(\"patches/\")\ndataloader = DataLoader(dataset, batch_size=32, shuffle=True)\n```\n\n---\n\n## \ud83d\udcda Documentation & Resources\n\n### Complete Documentation Hub\n\nFor comprehensive documentation, see the **[Documentation Hub](docs/README.md)**:\n\n- **[\ud83d\udcd6 User Guides](docs/guides/)** - Quick start guides, QGIS integration, troubleshooting\n- **[\u26a1 Features](docs/features/)** - Smart skip detection, format preferences, workflow optimization\n- **[\ud83d\udd27 Technical Reference](docs/reference/)** - Memory optimization, performance tuning\n- **[\ud83d\udce6 Archive](docs/archive/)** - Bug fixes history, release notes, migration guides\n\n### Essential Quick Links\n\n- **[\ud83c\udfaf Quick Reference Card](QUICK_REFERENCE.md)** - Fast reference for all commands\n- **[\u26a1 Smart Skip Features](docs/features/SMART_SKIP_SUMMARY.md)** - Resume workflows efficiently\n- **[\ud83d\uddfa\ufe0f QGIS Integration](docs/guides/QUICK_START_QGIS.md)** - GIS compatibility guide\n- **[\u2699\ufe0f Memory Optimization](docs/reference/MEMORY_OPTIMIZATION.md)** - Performance tuning\n- **[\ud83d\udccb Output Formats](docs/features/OUTPUT_FORMAT_PREFERENCES.md)** - LAZ 1.4 vs QGIS formats\n\n### Examples & Workflows\n\n- **[Basic Usage](examples/basic_usage.py)** - Simple processing examples\n- **[Urban Processing](examples/example_urban_simple.py)** - City-specific workflows\n- **[Parallel Processing](examples/parallel_processing_example.py)** - Multi-worker optimization\n- **[Full Workflow](examples/full_workflow_example.py)** - End-to-end pipeline\n- **[Pipeline Configuration](examples/pipeline_example.py)** - YAML-based workflows\n- **[RGB Augmentation](examples/enrich_with_rgb.py)** - Orthophoto integration\n- **[PyTorch Integration](examples/pytorch_dataloader.py)** - ML training setup\n\n---\n\n## \ud83d\udee0\ufe0f Development\n\n### Setup Development Environment\n\n```bash\ngit clone https://github.com/sducournau/IGN_LIDAR_HD_DATASET\ncd IGN_LIDAR_HD_DATASET\npip install -e \".[dev]\"\n```\n\n### Run Tests\n\n```bash\npytest tests/\n```\n\n### Code Formatting\n\n```bash\nblack ign_lidar/\nflake8 ign_lidar/\n```\n\n---\n\n## \ud83d\udd17 Requirements\n\n- Python 3.8+\n- NumPy >= 1.21.0\n- laspy >= 2.3.0\n- scikit-learn >= 1.0.0\n- tqdm >= 4.60.0\n- requests >= 2.25.0\n- PyYAML >= 6.0 (for pipeline configuration)\n- Pillow >= 9.0.0 (for RGB augmentation)\n\n**Optional (for GPU acceleration):**\n\n- CUDA >= 12.0\n- CuPy >= 12.0.0\n- RAPIDS cuML >= 24.10 (recommended for best performance)\n\n---\n\n## \ud83d\udcda API Reference\n\n### Core Classes\n\n- **`LiDARProcessor`**: Main processing engine for tile and directory processing\n- **`IGNLiDARDownloader`**: Batch download functionality from IGN WFS service\n- **`LOD2_CLASSES`**, **`LOD3_CLASSES`**: Classification taxonomies\n\n### Utility Functions\n\n- **`compute_normals()`**: Surface normal computation\n- **`compute_curvature()`**: Principal curvature calculation\n- **`extract_geometric_features()`**: Comprehensive feature extraction\n- **`get_tiles_by_environment()`**: Filter tiles by environment type\n\n---\n\n## \ud83d\udcc4 License & Support\n\n### License\n\nMIT License - see [LICENSE](LICENSE) file for details.\n\n### Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n### Support\n\nFor issues and questions, please use the [GitHub Issues](https://github.com/sducournau/IGN_LIDAR_HD_DATASET/issues) page.\n\n---\n\n<div align=\"center\">\n\n**Made with \u2764\ufe0f for the LiDAR and Machine Learning communities**\n\n[\u2b06 Back to top](#ign-lidar-hd-processing-library)\n\n</div>\n",
"bugtrack_url": null,
"license": null,
"summary": "IGN LiDAR HD Dataset Processing Library for Building LOD Classification",
"version": "1.7.6",
"project_urls": {
"Documentation": "https://github.com/sducournau/IGN_LIDAR_HD_DATASET#readme",
"Homepage": "https://github.com/sducournau/IGN_LIDAR_HD_DATASET",
"Issues": "https://github.com/sducournau/IGN_LIDAR_HD_DATASET/issues",
"Repository": "https://github.com/sducournau/IGN_LIDAR_HD_DATASET"
},
"split_keywords": [
"lidar",
" ign",
" machine-learning",
" gis",
" building-classification"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "f5eccb8f0bd00177e3727c2b5e198e48611d3e41638b77cc2b7fc593e6272dfe",
"md5": "f30f3977df09a1a6d6688a044e7aac81",
"sha256": "50cfba48eeb385a12678b8b9e1e239f7dcd34b14a3f34b34dd2aa9338902264a"
},
"downloads": -1,
"filename": "ign_lidar_hd-1.7.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f30f3977df09a1a6d6688a044e7aac81",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 1902053,
"upload_time": "2025-10-06T18:40:37",
"upload_time_iso_8601": "2025-10-06T18:40:37.814076Z",
"url": "https://files.pythonhosted.org/packages/f5/ec/cb8f0bd00177e3727c2b5e198e48611d3e41638b77cc2b7fc593e6272dfe/ign_lidar_hd-1.7.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "f5abe36e06e54eb6143a23f80079f8b28ee38543b56adb4dfc2850ebbb187512",
"md5": "8e239c5bde9a9f5397f218d2b698dc13",
"sha256": "47c1cba1f5d207cdf8d5630f1cc482e70860d2b687530474f1cc6e4cebba9a7f"
},
"downloads": -1,
"filename": "ign_lidar_hd-1.7.6.tar.gz",
"has_sig": false,
"md5_digest": "8e239c5bde9a9f5397f218d2b698dc13",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 1456252,
"upload_time": "2025-10-06T18:40:39",
"upload_time_iso_8601": "2025-10-06T18:40:39.395244Z",
"url": "https://files.pythonhosted.org/packages/f5/ab/e36e06e54eb6143a23f80079f8b28ee38543b56adb4dfc2850ebbb187512/ign_lidar_hd-1.7.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-06 18:40:39",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "sducournau",
"github_project": "IGN_LIDAR_HD_DATASET#readme",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "numpy",
"specs": [
[
">=",
"1.21.0"
]
]
},
{
"name": "laspy",
"specs": [
[
">=",
"2.3.0"
]
]
},
{
"name": "lazrs",
"specs": [
[
">=",
"0.5.0"
]
]
},
{
"name": "scikit-learn",
"specs": [
[
">=",
"1.0.0"
]
]
},
{
"name": "tqdm",
"specs": [
[
">=",
"4.60.0"
]
]
},
{
"name": "requests",
"specs": [
[
">=",
"2.25.0"
]
]
},
{
"name": "click",
"specs": [
[
">=",
"8.0.0"
]
]
},
{
"name": "psutil",
"specs": [
[
">=",
"5.8.0"
]
]
},
{
"name": "PyYAML",
"specs": [
[
">=",
"6.0"
]
]
},
{
"name": "Pillow",
"specs": [
[
">=",
"9.0.0"
]
]
},
{
"name": "h5py",
"specs": [
[
">=",
"3.0.0"
]
]
}
],
"lcname": "ign-lidar-hd"
}
JSON
