[](https://swarmsort.readthedocs.io/en/latest/)
[](https://pypi.org/project/swarmsort/)
[](https://pypi.org/project/swarmsort/)
[](https://github.com/cfosseprez/swarmsort/actions/workflows/test.yml)
[](https://github.com/cfosseprez/swarmsort/blob/main/LICENSE)
[](https://doi.org/10.5281/zenodo.17051857)
# SwarmSort
**Reliable multi-object tracking-by-detection: fast, accurate, and easy â perfect for top-view microscopy with hundreds of objects** đŻ
<div align="center">
<table>
<tr>
<td align="center">
<img src="https://github.com/user-attachments/assets/f67f8cb0-4d57-407c-9723-6dc7e5037a2c" style="width:80%;" alt="Detection Demo">
<br><b>Real-time tracking of 150 paramecia at 80 FPS</b>
</td>
<td align="center">
<img src="https://github.com/user-attachments/assets/b1a95557-a1db-4328-9442-d85c41d82e7c" style="width:90%;" alt="Tracking Demo">
<br><b>Real time performances for up to 500 individuals</b>
</td>
</tr>
</table>
</div>
## Core Capabilities
SwarmSort solves the data association problem in multi-object tracking by:
- **Maintaining temporal consistency** of object identities across frames using motion prediction, appearance and uncertainty
- **Handling occlusions and collisions** through re-identification with visual embeddings
- **optional lightweight gpu based embedding integrated** for more accuracy and speed
- **Preventing ID switches** in dense scenarios using uncertainty-aware cost computation and embedding freezing
- **Fast!** The library achieves real-time performance (80-100 FPS for 100 objects) through Numba JIT compilation, vectorized operations, and optional GPU acceleration.
## Key Features
### Advanced Tracking Algorithms
- **Uncertainty-based cost system** - Adaptive association costs based on track age, local density, and detection reliability
- **Smart collision handling** - Density-based embedding freezing prevents ID switches in crowded scenarios
- **Re-identification capability** - Recovers lost tracks using visual embeddings and motion prediction
- **Hybrid assignment strategy** - Combines greedy matching for obvious associations with Hungarian algorithm for complex cases
- **Dual Kalman filter options** - Simple constant velocity or OC-SORT style acceleration model
- **Occlusion handling** - Maintains tracks through temporary occlusions using motion prediction
### Real-time performance
- **Numba JIT compilation** - Critical mathematical functions compiled to machine code
- **Vectorized operations** - Batch processing using NumPy for efficient computation
- **GPU acceleration** - Optional CUDA support via CuPy for embedding extraction
- **Memory efficient** - Bounded memory usage with automatic cleanup of stale tracks
### Flexible Integration
- **Detector agnostic** - Works with any object detection source (YOLO, Detectron2, custom detectors)
- **Configurable parameters** - Fine-tune behavior for specific species (microscopy, crowds ..)
- **Multiple embedding methods** - Support for various visual feature extractors
- **Comprehensive API** - Access to track states, lifecycle management, and detailed statistics
### Production Ready
- **Extensive test coverage** - 200+ unit tests covering edge cases and error conditions
- **Cross-platform support** - Tested on Linux, Windows, macOS
- **Detailed documentation** - Complete API reference with practical examples
## Citation
If you use SwarmSort in your research, please cite:
```bibtex
@software{swarmsort,
title={SwarmSort: High-Performance Multi-Object Tracking},
author={Charles Fosseprez},
year={2025},
url={https://github.com/cfosseprez/swarmsort},
doi={10.5281/zenodo.17051857}
}
```
## đ Documentation
**[Full Documentation](https://swarmsort.readthedocs.io/en/latest/)**
## đŚ Installation
```bash
# Option 1: Install from PyPI
pip install swarmsort
# Option 2: Install from PyPI with gpu embedding support
pip install swarmsort[gpu]
# Option 3: Install from GitHub
pip install git+https://github.com/cfosseprez/swarmsort.git
```
## Quick Start
### Your First Tracker in 30 Seconds
```python
import numpy as np
from swarmsort import SwarmSortTracker, Detection
# Step 1: Create a tracker (it's that simple!)
tracker = SwarmSortTracker()
# Step 2: Tell the tracker what you detected this frame
# In real use, these would come from your object detector (YOLO, etc.)
detections = [
Detection(position=[100, 200], confidence=0.9), # A person at position (100, 200)
Detection(position=[300, 400], confidence=0.8), # Another person at (300, 400)
]
# Step 3: Get tracking results - SwarmSort handles all the complexity!
tracked_objects = tracker.update(detections)
# Step 4: Use the results - each object has a unique ID that persists across frames
for obj in tracked_objects:
print(f"Person {obj.id} is at position {obj.position} with {obj.confidence:.0%} confidence")
# Output: Person 1 is at position [100. 200.] with 90% confidence
```
### Real-World Example: Tracking Paramecia in Video
```python
import cv2
from swarmsort import SwarmSortTracker, Detection
tracker = SwarmSortTracker()
# Process a video file
video = cv2.VideoCapture('microscopy.mp4')
while True:
ret, frame = video.read()
if not ret:
break
# Get detections from your favorite detector
# For this example, let's say we detected 2 people:
detections = [
Detection(
position=[320, 240], # Center of bounding box
confidence=0.95,
bbox=[300, 220, 340, 260] # x1, y1, x2, y2
),
Detection(
position=[150, 180],
confidence=0.87,
bbox=[130, 160, 170, 200]
)
]
# SwarmSort assigns consistent IDs across frames
tracked = tracker.update(detections)
# Draw results on frame
for person in tracked:
if person.bbox is not None:
x1, y1, x2, y2 = person.bbox.astype(int)
# Each Paramecium keeps the same ID and color throughout the video!
color = (0, 255, 0) # Green for tracked objects
cv2.rectangle(frame, (x1, y1), (x2, y2), color, 2)
cv2.putText(frame, f"ID: {person.id}", (x1, y1-10),
cv2.FONT_HERSHEY_SIMPLEX, 0.5, color, 2)
cv2.imshow('Tracking Results', frame)
if cv2.waitKey(1) == ord('q'):
break
```
### đ Direct Integration with YOLO and Other Detectors
SwarmSort seamlessly integrates with popular object detectors through optimized conversion utilities:
#### **YOLO v8/v11 Integration**
```python
from ultralytics import YOLO
from swarmsort import yolo_to_detections, SwarmSortTracker, SwarmSortConfig
# Initialize YOLO detector
model = YOLO('yolov8n.pt') # or yolov11n.pt, yolov8x.pt, etc.
# Initialize SwarmSort tracker
tracker = SwarmSortTracker(SwarmSortConfig(
max_distance=150,
uncertainty_weight=0.3
))
# Process video stream
cap = cv2.VideoCapture('video.mp4')
while True:
ret, frame = cap.read()
if not ret:
break
# Detect objects with YOLO
results = model.predict(frame, conf=0.5)
# Convert YOLO output to SwarmSort format (optimized, zero-copy when possible)
detections = yolo_to_detections(
results[0],
confidence_threshold=0.5,
class_filter=[0, 1, 2] # Only track persons, bicycles, cars
)
# Track objects
tracked_objects = tracker.update(detections)
# Draw results
for obj in tracked_objects:
if obj.bbox is not None:
x1, y1, x2, y2 = obj.bbox.astype(int)
cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 255, 0), 2)
cv2.putText(frame, f"ID:{obj.id}", (x1, y1-10),
cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)
```
#### **Custom Detector Integration**
```python
from swarmsort import numpy_to_detections, prepare_detections
# If your detector outputs numpy arrays
boxes = np.array([[100, 100, 200, 200], # [x1, y1, x2, y2] format
[300, 150, 400, 250]])
confidences = np.array([0.9, 0.85])
# Convert to SwarmSort format
detections = numpy_to_detections(
boxes=boxes,
confidences=confidences,
format='xyxy' # Supports: 'xyxy', 'xywh', 'cxcywh'
)
# Or with embeddings from your own feature extractor
embeddings = your_feature_extractor(image_patches) # Shape: (N, embedding_dim)
detections = numpy_to_detections(boxes, confidences, embeddings=embeddings)
```
#### **Universal Auto-Conversion**
```python
from swarmsort import prepare_detections
# Automatically detects format and converts + verifies
detections = prepare_detections(
any_detection_data, # YOLO results, numpy arrays, or Detection objects
source_format='auto', # Auto-detects format
confidence_threshold=0.5,
auto_fix=True # Fixes common issues (clips bounds, normalizes, etc.)
)
# The prepare_detections function:
# â Auto-detects input format (YOLO, numpy, etc.)
# â Converts to SwarmSort Detection format
# â Validates all inputs
# â Auto-fixes common issues (out-of-bounds, NaN values, etc.)
# â Optimized with vectorized operations
```
#### **Batch Processing for Maximum Speed**
```python
# Process entire video in batches (useful for offline analysis)
from swarmsort import yolo_to_detections_batch
# Get all YOLO predictions at once
results = model.predict('video.mp4', stream=True)
all_detections = yolo_to_detections_batch(list(results))
# Track through all frames
all_tracks = []
for frame_detections in all_detections:
tracked = tracker.update(frame_detections)
all_tracks.append(tracked)
```
### Using Visual Features (Embeddings) for Better Tracking
Embeddings help the tracker recognize objects by their appearance, not just position. This is super useful when:
- Objects move quickly or unpredictably
- Multiple similar objects are close together
- Objects temporarily disappear and reappear
SwarmSort can use your GPU for the integrated default fast lightweight embedding:
```python
from swarmsort import SwarmSortTracker, SwarmSortConfig, Detection
import numpy as np
# Enable appearance-based tracking
config = SwarmSortConfig(
do_embeddings=True, # Use visual features for matching
embedding_weight=1.0, # How much to trust appearance vs motion
)
tracker = SwarmSortTracker(config)
```
Or you can use a personalized embedding, and pass it directly the Detection.
```python
# In practice, embeddings come from a feature extractor (ResNet, etc.)
# Here's a simple example:
def get_embedding_from_image(image_patch):
"""Your feature extractor - could be a neural network"""
# This would be your CNN/feature extractor
# Returns a N-dimensional feature vector
return np.random.randn(128).astype(np.float32)
# Create detection with visual features
person_image = frame[160:200, 130:170] # Crop person from frame
embedding = get_embedding_from_image(person_image)
detection = Detection(
position=[150, 180], # Center position
confidence=0.9,
embedding=embedding, # Visual features help maintain ID
bbox=[130, 160, 170, 200] # Bounding box
)
# The tracker now uses BOTH motion AND appearance for matching!
tracked_objects = tracker.update([detection])
```
## âď¸ Configuration Made Easy
### Understanding the max_distance Parameter
The `max_distance` parameter is the foundation of SwarmSort's configuration. **Important**: Set it **1.5-2x higher** than the expected maximum pixel movement between frames because:
- The actual matching uses a **combination** of spatial distance, embedding similarity, and uncertainty penalties
- With embeddings enabled, the effective matching distance is reduced by visual similarity
- Uncertainty penalties further modify the association costs
- Example: If objects move up to 100 pixels between frames, set `max_distance=150-200`
Many other parameters **automatically scale** with `max_distance`:
```python
# When you set max_distance=150, these defaults are automatically set:
local_density_radius = 150 # Same as max_distance
greedy_threshold = 30 # max_distance / 5
reid_max_distance = 150 # Same as max_distance
```
### All Parameters
| Parameter | Default | Description |
|----------------------------|--------------------|---------------------------------------------------------------------------------------|
| **Core Tracking** | | |
| `max_distance` | 150.0 | Maximum distance for detection-track association |
| `detection_conf_threshold` | 0.0 | Minimum confidence for detections |
| `max_track_age` | 30 | Maximum frames to keep track alive without detections |
| **Motion prediction** | | |
| `kalman_type` | 'simple' | Kalman filter type: 'simple' or 'oc' (OC-SORT style) |
| **Uncertainty System** | | |
| `uncertainty_weight` | 0.33 | Weight for uncertainty penalties (0 = disabled) |
| `local_density_radius` | max_distance | Radius for computing local track density (defaults to max_distance) |
| **Embeddings** | | |
| `do_embeddings` | True | Whether to use embedding features |
| `embedding_function` | 'cupytexture' | Integrated embedding function: "cupytexture", "cupytexture_color", "mega_cupytexture" |
| `embedding_weight` | 1.0 | Weight for embedding similarity in cost function |
| `max_embeddings_per_track` | 15 | Maximum embeddings stored per track |
| `embedding_matching_method` | 'weighted_average' | Method for multi-embedding matching |
| **Collision Handling** | | |
| `collision_freeze_embeddings` | True | Freeze embedding updates in dense areas |
| `embedding_freeze_density` | 1 | Freeze when âĽN tracks within radius |
| **Assignment Strategy** | | |
| `assignment_strategy` | 'hybrid' | Assignment method: 'hungarian', 'greedy', or 'hybrid' |
| `greedy_threshold` | max_distance/5 | Distance threshold for greedy assignment |
| **Track Initialization** | | |
| `min_consecutive_detections` | 6 | Minimum consecutive detections to create track |
| `max_detection_gap` | 2 | Maximum gap between detections |
| `pending_detection_distance` | max_distance | Distance threshold for pending detection matching |
| **Re-identification** | | |
| `reid_enabled` | True | Enable re-identification of lost tracks |
| `reid_max_distance` | max_distance*1.5 | Maximum distance for ReID |
| `reid_embedding_threshold` | 0.3 | Embedding threshold for ReID |
| **Experimental** | | |
| `use_probabilistic_costs` | False | Use gaussian fusion for cost computation |
### đŻ Preset Configurations for Common Scenarios
Best Settings for Performance:
For good balance (speed + accuracy): up to 300 individuals
```python
config = SwarmSortConfig(
kalman_type="simple", # Fast but accurate enough
assignment_strategy="hybrid", # Good balance
uncertainty_weight=0.33, # Some uncertainty handling
do_embeddings=True, # Use embeddings if available
reid_enabled=False, # Skip for speed
)
```
For maximum speed across all scales: 300+ individuals
```python
config = SwarmSortConfig(
kalman_type="simple", # Fastest Kalman filter
assignment_strategy="greedy", # Fastest assignment
uncertainty_weight=0.0, # Disable uncertainty
do_embeddings=False, # No embeddings
reid_enabled=False, # No re-identification
)
```
### đ§ Understanding Key Parameters
```python
# The most important parameters to tune:
config = SwarmSortConfig(
# 1. How far can an object move between frames?
max_distance=150.0, # Increase for fast objects, decrease for slow
# 2. How many frames to confirm a new track?
min_consecutive_detections=6, # Lower = faster response, more false positives
# Higher = slower response, fewer false positives
# 3. How long to keep lost tracks?
max_track_age=30, # At 30 FPS, this is 1 second of "memory"
# 4. Use appearance features?
do_embeddings=True, # True if objects look different from each other
# False if all objects look the same (e.g., identical boxes)
# 5. How to handle crowded scenes?
collision_freeze_embeddings=True, # Prevents ID switches when objects touch
uncertainty_weight=0.33, # Higher = more conservative in uncertain situations
)
```
## Advanced Usage
### Different Configuration Methods
```python
from swarmsort import SwarmSortTracker, SwarmSortConfig
# Default tracker
tracker = SwarmSortTracker()
# With configuration object
config = SwarmSortConfig(max_distance=100.0, do_embeddings=True)
tracker = SwarmSortTracker(config)
# With dictionary config
tracker = SwarmSortTracker({'max_distance': 100.0, 'do_embeddings': True})
```
### Basic Standalone Usage
```python
from swarmsort import SwarmSortTracker, SwarmSortConfig
# SwarmSort is a standalone tracker - no special integration needed
tracker = SwarmSortTracker()
# Configure for specific use cases
config = SwarmSortConfig(
do_embeddings=True,
reid_enabled=True,
max_distance=100.0,
assignment_strategy='hybrid', # Use hybrid assignment strategy
uncertainty_weight=0.33 # Enable uncertainty-based costs
)
tracker_configured = SwarmSortTracker(config)
```
## đŚ Working with Data
### Input: Detection Objects
Detections are what you feed into the tracker - they represent objects found in the current frame:
```python
from swarmsort import Detection
import numpy as np
# Minimal detection - just position and confidence
simple_detection = Detection(
position=[320, 240], # Center point (x, y)
confidence=0.9 # How sure are we this is real? (0-1)
)
# Full detection with all the bells and whistles
full_detection = Detection(
position=np.array([320, 240]), # Object center
confidence=0.95, # Detection confidence
bbox=np.array([300, 220, 340, 260]), # Bounding box [x1, y1, x2, y2]
embedding=feature_vector, # Visual features (for example from your CNN)
class_id=0, # 0=person, 1=car, etc.
id="yolo_detection_42" # Your detector's ID (optional)
)
# Real-world example: Converting YOLO output to SwarmSort
def yolo_to_swarmsort(yolo_results):
detections = []
for box in yolo_results.boxes:
x1, y1, x2, y2 = box.xyxy[0].numpy()
center_x = (x1 + x2) / 2
center_y = (y1 + y2) / 2
detections.append(Detection(
position=[center_x, center_y],
confidence=box.conf[0].item(),
bbox=[x1, y1, x2, y2],
class_id=int(box.cls[0])
))
return detections
```
### Output: TrackedObject Results
The tracker returns TrackedObject instances with rich information about each tracked object:
```python
# Get tracking results
tracked_objects = tracker.update(detections)
for obj in tracked_objects:
# Identity
print(f"đ Track ID: {obj.id}") # Unique ID that persists across frames
# Location & Motion
print(f"đ Position: {obj.position}") # Current [x, y] position
print(f"âĄď¸ Velocity: {obj.velocity}") # Speed and direction [vx, vy]
# Confidence & Quality
print(f"â
Confidence: {obj.confidence:.1%}") # How confident are we?
print(f"đ Track quality: {obj.hits}/{obj.age}") # Hits/Age ratio
# Track Status
if obj.time_since_update == 0:
print("đ˘ Currently visible")
else:
print(f"đĄ Lost for {obj.time_since_update} frames")
# Bounding Box (if available)
if obj.bbox is not None:
x1, y1, x2, y2 = obj.bbox
width = x2 - x1
height = y2 - y1
print(f"đ Size: {width:.0f}x{height:.0f} pixels")
```
### đ Track Lifecycle Management
SwarmSort provides fine control over track states - perfect for different visualization needs:
```python
# Get only tracks that are currently visible
alive_tracks = tracker.update(detections)
print(f"đď¸ Visible now: {len(alive_tracks)} objects")
# Get tracks that were recently lost (useful for smooth visualization)
recently_lost = tracker.get_recently_lost_tracks(max_frames_lost=5)
print(f"đť Recently lost: {len(recently_lost)} objects")
# Get everything (visible + recently lost)
all_active = tracker.get_all_active_tracks(max_frames_lost=5)
print(f"đ Total active: {len(all_active)} objects")
# Example: Different visualization for different states
for obj in alive_tracks:
draw_solid_box(frame, obj, color='green') # Solid box for visible
for obj in recently_lost:
draw_dashed_box(frame, obj, color='yellow') # Dashed box for lost
```
## ⥠Performance & Optimization
### Why SwarmSort is Fast
SwarmSort is optimized for real-world performance:
- **Numba JIT Compilation**: Math operations run at C speed
- **Vectorized NumPy**: Batch operations instead of loops
- **Smart Caching**: Reuses computed embeddings and distances
- **Memory Pooling**: Reduces allocation overhead
- **Early Exit Logic**: Skips unnecessary computations
## Visualization Example
SwarmSort includes built-in visualization utilities for beautiful tracking displays:
<details>
<summary>Click to expand code example</summary>
```python
import cv2
import numpy as np
from swarmsort import SwarmSortTracker, Detection, SwarmSortConfig
from swarmsort import TrackingVisualizer, VisualizationConfig
# Initialize tracker with your preferred settings
config = SwarmSortConfig(
do_embeddings=True,
embedding_function='cupytexture', # or 'mega_cupytexture' for more features
assignment_strategy='hybrid',
uncertainty_weight=0.33
)
tracker = SwarmSortTracker(config)
# Initialize visualizer with custom settings
vis_config = VisualizationConfig(
show_trails=True,
trail_length=30,
show_ids=True,
show_confidence=True,
show_velocity_vectors=True,
id_font_scale=0.5,
id_thickness=2,
box_thickness=2
)
visualizer = TrackingVisualizer(vis_config)
# Example usage with video
cap = cv2.VideoCapture('video.mp4') # Or use 0 for webcam
while True:
ret, frame = cap.read()
if not ret:
break
# Detect objects (replace with your detector)
# Here's a mock detection for demonstration
detections = [
Detection(
position=np.array([100, 200]),
confidence=0.9,
bbox=np.array([80, 180, 120, 220])
),
Detection(
position=np.array([300, 400]),
confidence=0.85,
bbox=np.array([280, 380, 320, 420])
)
]
# Update tracker
tracked_objects = tracker.update(detections)
# Draw tracking results with built-in visualizer
frame = visualizer.draw_tracks(frame, tracked_objects)
# Optionally show recently lost tracks
recently_lost = tracker.get_recently_lost_tracks(max_frames_lost=5)
frame = visualizer.draw_lost_tracks(frame, recently_lost)
# Display frame
cv2.imshow('SwarmSort Tracking', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
```
</details>
### Quick Visualization (One-liner)
For quick prototyping, use the convenience function:
```python
from swarmsort import quick_visualize
# One line to visualize tracking results!
frame_with_tracks = quick_visualize(frame, tracked_objects, show_trails=True)
```
### Custom Drawing (If you need more control)
If you prefer to implement custom visualization:
<details>
<summary>Click to expand code example</summary>
```python
import cv2
import numpy as np
from swarmsort import SwarmSortTracker, Detection, SwarmSortConfig
# Initialize tracker
tracker = SwarmSortTracker(SwarmSortConfig(do_embeddings=True))
# Function to draw tracking results
def draw_tracks(frame, tracked_objects, show_trails=True):
"""Draw bounding boxes and tracking information on frame."""
# Store trail history (in production, store this outside the function)
if not hasattr(draw_tracks, 'trails'):
draw_tracks.trails = {}
for obj in tracked_objects:
# Get track color (consistent color per ID)
color = np.random.RandomState(obj.id).randint(0, 255, 3).tolist()
# Draw bounding box if available
if obj.bbox is not None:
x1, y1, x2, y2 = obj.bbox.astype(int)
cv2.rectangle(frame, (x1, y1), (x2, y2), color, 2)
# Draw track ID and confidence
label = f"ID:{obj.id} ({obj.confidence:.2f})"
cv2.putText(frame, label, (x1, y1-10),
cv2.FONT_HERSHEY_SIMPLEX, 0.5, color, 2)
# Draw center point
cx, cy = obj.position.astype(int)
cv2.circle(frame, (cx, cy), 5, color, -1)
# Update and draw trail
if show_trails:
if obj.id not in draw_tracks.trails:
draw_tracks.trails[obj.id] = []
draw_tracks.trails[obj.id].append((cx, cy))
# Keep only last 30 points
draw_tracks.trails[obj.id] = draw_tracks.trails[obj.id][-30:]
# Draw trail
points = draw_tracks.trails[obj.id]
for i in range(1, len(points)):
cv2.line(frame, points[i-1], points[i], color, 2)
# Clean up old trails
active_ids = {obj.id for obj in tracked_objects}
draw_tracks.trails = {k: v for k, v in draw_tracks.trails.items()
if k in active_ids}
return frame
```
</details>
### Simple Visualization with Matplotlib
For a simpler visualization or for Jupyter notebooks:
<details>
<summary>Click to expand code example</summary>
```python
import matplotlib.pyplot as plt
import matplotlib.patches as patches
from matplotlib.animation import FuncAnimation
import numpy as np
from swarmsort import SwarmSortTracker, Detection
# Create figure
fig, ax = plt.subplots(figsize=(10, 8))
ax.set_xlim(0, 640)
ax.set_ylim(480, 0) # Invert y-axis for image coordinates
ax.set_aspect('equal')
ax.set_title('SwarmSort Multi-Object Tracking')
tracker = SwarmSortTracker()
track_history = {}
def update_plot(frame_num):
ax.clear()
ax.set_xlim(0, 640)
ax.set_ylim(480, 0)
ax.set_title(f'Frame {frame_num}')
# Generate mock detections (replace with real detections)
detections = [
Detection(
position=np.array([320 + 100*np.sin(frame_num/10), 240]),
confidence=0.9,
bbox=np.array([300 + 100*np.sin(frame_num/10), 220,
340 + 100*np.sin(frame_num/10), 260])
),
Detection(
position=np.array([200, 240 + 100*np.cos(frame_num/10)]),
confidence=0.85,
bbox=np.array([180, 220 + 100*np.cos(frame_num/10),
220, 260 + 100*np.cos(frame_num/10)])
)
]
# Update tracker
tracked_objects = tracker.update(detections)
# Plot tracked objects
for obj in tracked_objects:
# Get consistent color for track ID
np.random.seed(obj.id)
color = np.random.rand(3)
# Draw bounding box
if obj.bbox is not None:
x1, y1, x2, y2 = obj.bbox
rect = patches.Rectangle((x1, y1), x2-x1, y2-y1,
linewidth=2, edgecolor=color,
facecolor='none')
ax.add_patch(rect)
# Draw center point
ax.scatter(obj.position[0], obj.position[1],
c=[color], s=100, marker='o')
# Add ID label
ax.text(obj.position[0], obj.position[1]-20, f'ID:{obj.id}',
color=color, fontsize=12, ha='center', weight='bold')
# Update history
if obj.id not in track_history:
track_history[obj.id] = []
track_history[obj.id].append(obj.position.copy())
# Draw trail
if len(track_history[obj.id]) > 1:
trail = np.array(track_history[obj.id])
ax.plot(trail[:, 0], trail[:, 1], color=color,
linewidth=2, alpha=0.5)
# Clean old tracks
active_ids = {obj.id for obj in tracked_objects}
for track_id in list(track_history.keys()):
if track_id not in active_ids:
if len(track_history[track_id]) > 50: # Remove very old tracks
del track_history[track_id]
# Create animation
anim = FuncAnimation(fig, update_plot, frames=200,
interval=50, repeat=True)
plt.show()
# To save as video:
# anim.save('tracking_visualization.mp4', writer='ffmpeg', fps=20)
```
</details>
## đ Advanced Features
### đ§ How SwarmSort Thinks: The Intelligence Behind the Tracking
#### **Uncertainty-Aware Tracking**
SwarmSort knows when it's confident and when it's not:
```python
# The tracker automatically adjusts behavior based on uncertainty:
# - New tracks: "I'm not sure yet, let me observe more"
# - Established tracks: "I know this object well"
# - Crowded areas: "Need to be extra careful here"
config = SwarmSortConfig(
uncertainty_weight=0.33, # How much to consider uncertainty
# 0.0 = Ignore uncertainty (aggressive)
# 0.5 = Balanced approach
# 1.0 = Very conservative
)
# Example: High uncertainty for drone tracking (unpredictable motion)
drone_config = SwarmSortConfig(
uncertainty_weight=0.6, # Be more careful with uncertain tracks
kalman_type='oc', # Better motion model for erratic movement
)
```
#### **Smart Collision Prevention**
Prevents ID switches when objects get close:
```python
# Scenario: Tracking dancers who frequently cross paths
dance_config = SwarmSortConfig(
collision_freeze_embeddings=True, # Lock visual features when close
embedding_freeze_density=1, # Freeze when anyone is within...
local_density_radius=100.0, # ...100 pixels
)
# What happens:
# 1. Two Paramecium approach each other
# 2. SwarmSort detects they're getting close
# 3. Visual features are "frozen" - relies on motion only
# 4. Prevents mixing up their identities
# 5. Once separated, visual matching resumes
```
#### **Hybrid Assignment Strategy**
Combines the best of both worlds:
```python
config = SwarmSortConfig(
assignment_strategy='hybrid', # Smart mode (default)
greedy_threshold=30.0, # Fast matching for obvious cases
)
# How it works:
# 1. Obvious matches (very close): Uses fast greedy assignment
# 2. Ambiguous cases: Falls back to optimal Hungarian algorithm
# 3. Best of both: Fast AND accurate
config.assignment_strategy = 'hybrid' # optimal matching
```
### đ Re-Identification: Bringing Lost Objects Back
Perfect for scenarios where objects temporarily disappear:
```python
# Example: Security camera at a store entrance
config = SwarmSortConfig(
reid_enabled=True, # Enable re-identification
reid_max_distance=200.0, # Search this far for lost tracks
reid_embedding_threshold=0.25, # How similar must appearances be?
)
# What happens:
# 1. Person walks behind a pillar (track lost)
# 2. Person reappears on the other side
# 3. SwarmSort compares appearance with recently lost tracks
# 4. Same person? Same ID! Tracking continues seamlessly
# Real-world usage:
tracker = SwarmSortTracker(config)
results = tracker.update(detections)
# The person who disappeared at frame 100 and reappeared at frame 120
# will have the SAME track ID - perfect for counting and analytics!
```
## đ§ Troubleshooting & FAQ
### Common Issues and Solutions
**Q: My tracks keep switching IDs when objects cross paths**
```python
# Solution: Enable collision handling
config = SwarmSortConfig(
collision_freeze_embeddings=True, # Prevent ID switches
embedding_freeze_density=1, # Freeze when objects are close
do_embeddings=True, # Use visual features
embedding_weight=1.5, # Trust appearance more
)
```
**Q: New tracks take too long to appear**
```python
# Solution: Reduce initialization requirements
config = SwarmSortConfig(
min_consecutive_detections=2, # Was 6, now faster
init_conf_threshold=0.3, # Accept lower confidence
)
```
**Q: Too many false tracks from noise**
```python
# Solution: Be more strict about track creation
config = SwarmSortConfig(
min_consecutive_detections=8, # Require more detections
init_conf_threshold=0.7, # Higher confidence needed
detection_conf_threshold=0.5, # Filter out weak detections
)
```
**Q: Tracks disappear too quickly**
```python
# Solution: Keep tracks alive longer
config = SwarmSortConfig(
max_track_age=30, # Keep for 2 seconds at 30 FPS (was 30)
reid_enabled=True, # Try to re-identify lost tracks
)
```
**Q: Performance is too slow**
Consider processing every other frame
## Examples
See the `examples/` directory for comprehensive usage examples.
## Testing
```bash
# Run tests
poetry run pytest
# Run tests with coverage
poetry run pytest --cov=swarmsort --cov-report=html
# Run specific test
poetry run pytest tests/test_basic.py::test_basic_tracking
```
## Development
### Development Setup
Want to contribute or modify SwarmSort? Here's how to set up a development environment:
```bash
# Clone the repository
git clone https://github.com/cfosseprez/swarmsort.git
cd swarmsort
# Install with Poetry (recommended for development)
poetry install --with dev
# Or use pip in editable mode
pip install -e ".[dev]"
```
## Benchmarking
```bash
# Run benchmarks
poetry run pytest tests/ --benchmark-only
```
## License
GPL 3.0 or later - see LICENSE file for details.
## Contributing
1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request
### Development Workflow
```bash
# Install development dependencies
poetry install --with dev
# Run linting
poetry run black swarmsort/
poetry run flake8 swarmsort/
# Run type checking
poetry run mypy swarmsort/
```
Raw data
{
"_id": null,
"home_page": "https://github.com/cfosseprez/swarmsort",
"name": "swarmsort",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.12,>=3.9",
"maintainer_email": null,
"keywords": "tracking, computer-vision, multi-object-tracking, embeddings, real-time",
"author": "Charles Fosseprez",
"author_email": "charles.fosseprez.pro@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/e4/89/ac044bd5a6580c9602110c73d71dc69e6cb8c720fed1d0e11c0df16af48f/swarmsort-1.1.0.tar.gz",
"platform": null,
"description": "[](https://swarmsort.readthedocs.io/en/latest/)\n[](https://pypi.org/project/swarmsort/)\n[](https://pypi.org/project/swarmsort/)\n[](https://github.com/cfosseprez/swarmsort/actions/workflows/test.yml)\n[](https://github.com/cfosseprez/swarmsort/blob/main/LICENSE)\n[](https://doi.org/10.5281/zenodo.17051857)\n\n\n\n\n\n# SwarmSort\n\n**Reliable multi-object tracking-by-detection: fast, accurate, and easy \u2014 perfect for top-view microscopy with hundreds of objects** \ud83c\udfaf\n\n\n<div align=\"center\">\n <table>\n <tr>\n <td align=\"center\">\n <img src=\"https://github.com/user-attachments/assets/f67f8cb0-4d57-407c-9723-6dc7e5037a2c\" style=\"width:80%;\" alt=\"Detection Demo\">\n <br><b>Real-time tracking of 150 paramecia at 80 FPS</b>\n </td>\n <td align=\"center\">\n <img src=\"https://github.com/user-attachments/assets/b1a95557-a1db-4328-9442-d85c41d82e7c\" style=\"width:90%;\" alt=\"Tracking Demo\">\n <br><b>Real time performances for up to 500 individuals</b>\n </td>\n </tr>\n </table>\n</div>\n\n## Core Capabilities\n\nSwarmSort solves the data association problem in multi-object tracking by:\n- **Maintaining temporal consistency** of object identities across frames using motion prediction, appearance and uncertainty\n- **Handling occlusions and collisions** through re-identification with visual embeddings \n- **optional lightweight gpu based embedding integrated** for more accuracy and speed\n- **Preventing ID switches** in dense scenarios using uncertainty-aware cost computation and embedding freezing\n- **Fast!** The library achieves real-time performance (80-100 FPS for 100 objects) through Numba JIT compilation, vectorized operations, and optional GPU acceleration.\n\n\n## Key Features\n\n### Advanced Tracking Algorithms\n- **Uncertainty-based cost system** - Adaptive association costs based on track age, local density, and detection reliability\n- **Smart collision handling** - Density-based embedding freezing prevents ID switches in crowded scenarios\n- **Re-identification capability** - Recovers lost tracks using visual embeddings and motion prediction\n- **Hybrid assignment strategy** - Combines greedy matching for obvious associations with Hungarian algorithm for complex cases\n- **Dual Kalman filter options** - Simple constant velocity or OC-SORT style acceleration model\n- **Occlusion handling** - Maintains tracks through temporary occlusions using motion prediction\n\n### Real-time performance\n- **Numba JIT compilation** - Critical mathematical functions compiled to machine code\n- **Vectorized operations** - Batch processing using NumPy for efficient computation\n- **GPU acceleration** - Optional CUDA support via CuPy for embedding extraction\n- **Memory efficient** - Bounded memory usage with automatic cleanup of stale tracks\n\n### Flexible Integration\n- **Detector agnostic** - Works with any object detection source (YOLO, Detectron2, custom detectors)\n- **Configurable parameters** - Fine-tune behavior for specific species (microscopy, crowds ..)\n- **Multiple embedding methods** - Support for various visual feature extractors\n- **Comprehensive API** - Access to track states, lifecycle management, and detailed statistics\n\n### Production Ready\n- **Extensive test coverage** - 200+ unit tests covering edge cases and error conditions\n- **Cross-platform support** - Tested on Linux, Windows, macOS\n- **Detailed documentation** - Complete API reference with practical examples\n\n## Citation\n\nIf you use SwarmSort in your research, please cite:\n\n```bibtex\n@software{swarmsort,\n title={SwarmSort: High-Performance Multi-Object Tracking},\n author={Charles Fosseprez},\n year={2025},\n url={https://github.com/cfosseprez/swarmsort},\n doi={10.5281/zenodo.17051857}\n}\n```\n## \ud83d\udcd6 Documentation\n\n**[Full Documentation](https://swarmsort.readthedocs.io/en/latest/)**\n\n## \ud83d\udce6 Installation\n\n\n```bash\n# Option 1: Install from PyPI \npip install swarmsort\n\n# Option 2: Install from PyPI with gpu embedding support\npip install swarmsort[gpu]\n\n# Option 3: Install from GitHub\npip install git+https://github.com/cfosseprez/swarmsort.git\n```\n\n\n\n## Quick Start\n\n### Your First Tracker in 30 Seconds\n\n```python\nimport numpy as np\nfrom swarmsort import SwarmSortTracker, Detection\n\n# Step 1: Create a tracker (it's that simple!)\ntracker = SwarmSortTracker()\n\n# Step 2: Tell the tracker what you detected this frame\n# In real use, these would come from your object detector (YOLO, etc.)\ndetections = [\n Detection(position=[100, 200], confidence=0.9), # A person at position (100, 200)\n Detection(position=[300, 400], confidence=0.8), # Another person at (300, 400)\n]\n\n# Step 3: Get tracking results - SwarmSort handles all the complexity!\ntracked_objects = tracker.update(detections)\n\n# Step 4: Use the results - each object has a unique ID that persists across frames\nfor obj in tracked_objects:\n print(f\"Person {obj.id} is at position {obj.position} with {obj.confidence:.0%} confidence\")\n # Output: Person 1 is at position [100. 200.] with 90% confidence\n```\n\n### Real-World Example: Tracking Paramecia in Video\n\n```python\nimport cv2\nfrom swarmsort import SwarmSortTracker, Detection\n\ntracker = SwarmSortTracker()\n\n# Process a video file\nvideo = cv2.VideoCapture('microscopy.mp4')\n\nwhile True:\n ret, frame = video.read()\n if not ret:\n break\n \n # Get detections from your favorite detector\n # For this example, let's say we detected 2 people:\n detections = [\n Detection(\n position=[320, 240], # Center of bounding box\n confidence=0.95,\n bbox=[300, 220, 340, 260] # x1, y1, x2, y2\n ),\n Detection(\n position=[150, 180],\n confidence=0.87,\n bbox=[130, 160, 170, 200]\n )\n ]\n \n # SwarmSort assigns consistent IDs across frames\n tracked = tracker.update(detections)\n \n # Draw results on frame\n for person in tracked:\n if person.bbox is not None:\n x1, y1, x2, y2 = person.bbox.astype(int)\n # Each Paramecium keeps the same ID and color throughout the video!\n color = (0, 255, 0) # Green for tracked objects\n cv2.rectangle(frame, (x1, y1), (x2, y2), color, 2)\n cv2.putText(frame, f\"ID: {person.id}\", (x1, y1-10),\n cv2.FONT_HERSHEY_SIMPLEX, 0.5, color, 2)\n \n cv2.imshow('Tracking Results', frame)\n if cv2.waitKey(1) == ord('q'):\n break\n```\n\n### \ud83d\udd0c Direct Integration with YOLO and Other Detectors\n\nSwarmSort seamlessly integrates with popular object detectors through optimized conversion utilities:\n\n#### **YOLO v8/v11 Integration**\n```python\nfrom ultralytics import YOLO\nfrom swarmsort import yolo_to_detections, SwarmSortTracker, SwarmSortConfig\n\n# Initialize YOLO detector\nmodel = YOLO('yolov8n.pt') # or yolov11n.pt, yolov8x.pt, etc.\n\n# Initialize SwarmSort tracker\ntracker = SwarmSortTracker(SwarmSortConfig(\n max_distance=150,\n uncertainty_weight=0.3\n))\n\n# Process video stream\ncap = cv2.VideoCapture('video.mp4')\nwhile True:\n ret, frame = cap.read()\n if not ret:\n break\n \n # Detect objects with YOLO\n results = model.predict(frame, conf=0.5)\n \n # Convert YOLO output to SwarmSort format (optimized, zero-copy when possible)\n detections = yolo_to_detections(\n results[0], \n confidence_threshold=0.5,\n class_filter=[0, 1, 2] # Only track persons, bicycles, cars\n )\n \n # Track objects\n tracked_objects = tracker.update(detections)\n \n # Draw results\n for obj in tracked_objects:\n if obj.bbox is not None:\n x1, y1, x2, y2 = obj.bbox.astype(int)\n cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 255, 0), 2)\n cv2.putText(frame, f\"ID:{obj.id}\", (x1, y1-10),\n cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)\n```\n\n#### **Custom Detector Integration**\n```python\nfrom swarmsort import numpy_to_detections, prepare_detections\n\n# If your detector outputs numpy arrays\nboxes = np.array([[100, 100, 200, 200], # [x1, y1, x2, y2] format\n [300, 150, 400, 250]])\nconfidences = np.array([0.9, 0.85])\n\n# Convert to SwarmSort format\ndetections = numpy_to_detections(\n boxes=boxes,\n confidences=confidences,\n format='xyxy' # Supports: 'xyxy', 'xywh', 'cxcywh'\n)\n\n# Or with embeddings from your own feature extractor\nembeddings = your_feature_extractor(image_patches) # Shape: (N, embedding_dim)\ndetections = numpy_to_detections(boxes, confidences, embeddings=embeddings)\n```\n\n#### **Universal Auto-Conversion**\n```python\nfrom swarmsort import prepare_detections\n\n# Automatically detects format and converts + verifies\ndetections = prepare_detections(\n any_detection_data, # YOLO results, numpy arrays, or Detection objects\n source_format='auto', # Auto-detects format\n confidence_threshold=0.5,\n auto_fix=True # Fixes common issues (clips bounds, normalizes, etc.)\n)\n\n# The prepare_detections function:\n# \u2713 Auto-detects input format (YOLO, numpy, etc.)\n# \u2713 Converts to SwarmSort Detection format\n# \u2713 Validates all inputs\n# \u2713 Auto-fixes common issues (out-of-bounds, NaN values, etc.)\n# \u2713 Optimized with vectorized operations\n```\n\n#### **Batch Processing for Maximum Speed**\n```python\n# Process entire video in batches (useful for offline analysis)\nfrom swarmsort import yolo_to_detections_batch\n\n# Get all YOLO predictions at once\nresults = model.predict('video.mp4', stream=True)\nall_detections = yolo_to_detections_batch(list(results))\n\n# Track through all frames\nall_tracks = []\nfor frame_detections in all_detections:\n tracked = tracker.update(frame_detections)\n all_tracks.append(tracked)\n```\n\n\n### Using Visual Features (Embeddings) for Better Tracking\n\nEmbeddings help the tracker recognize objects by their appearance, not just position. This is super useful when:\n- Objects move quickly or unpredictably\n- Multiple similar objects are close together\n- Objects temporarily disappear and reappear\n\n\nSwarmSort can use your GPU for the integrated default fast lightweight embedding:\n\n```python\nfrom swarmsort import SwarmSortTracker, SwarmSortConfig, Detection\nimport numpy as np\n\n# Enable appearance-based tracking\nconfig = SwarmSortConfig(\n do_embeddings=True, # Use visual features for matching\n embedding_weight=1.0, # How much to trust appearance vs motion\n)\ntracker = SwarmSortTracker(config)\n```\n\nOr you can use a personalized embedding, and pass it directly the Detection.\n\n```python\n# In practice, embeddings come from a feature extractor (ResNet, etc.)\n# Here's a simple example:\ndef get_embedding_from_image(image_patch):\n \"\"\"Your feature extractor - could be a neural network\"\"\"\n # This would be your CNN/feature extractor\n # Returns a N-dimensional feature vector\n return np.random.randn(128).astype(np.float32)\n\n# Create detection with visual features\nperson_image = frame[160:200, 130:170] # Crop person from frame\nembedding = get_embedding_from_image(person_image)\n\ndetection = Detection(\n position=[150, 180], # Center position\n confidence=0.9,\n embedding=embedding, # Visual features help maintain ID\n bbox=[130, 160, 170, 200] # Bounding box\n)\n\n# The tracker now uses BOTH motion AND appearance for matching!\ntracked_objects = tracker.update([detection])\n```\n\n## \u2699\ufe0f Configuration Made Easy\n\n### Understanding the max_distance Parameter\n\nThe `max_distance` parameter is the foundation of SwarmSort's configuration. **Important**: Set it **1.5-2x higher** than the expected maximum pixel movement between frames because:\n\n- The actual matching uses a **combination** of spatial distance, embedding similarity, and uncertainty penalties\n- With embeddings enabled, the effective matching distance is reduced by visual similarity \n- Uncertainty penalties further modify the association costs\n- Example: If objects move up to 100 pixels between frames, set `max_distance=150-200`\n\nMany other parameters **automatically scale** with `max_distance`:\n```python\n# When you set max_distance=150, these defaults are automatically set:\nlocal_density_radius = 150 # Same as max_distance\ngreedy_threshold = 30 # max_distance / 5\nreid_max_distance = 150 # Same as max_distance\n```\n\n### All Parameters\n\n| Parameter | Default | Description |\n|----------------------------|--------------------|---------------------------------------------------------------------------------------|\n| **Core Tracking** | | |\n| `max_distance` | 150.0 | Maximum distance for detection-track association |\n| `detection_conf_threshold` | 0.0 | Minimum confidence for detections |\n| `max_track_age` | 30 | Maximum frames to keep track alive without detections |\n| **Motion prediction** | | |\n| `kalman_type` | 'simple' | Kalman filter type: 'simple' or 'oc' (OC-SORT style) |\n| **Uncertainty System** | | |\n| `uncertainty_weight` | 0.33 | Weight for uncertainty penalties (0 = disabled) |\n| `local_density_radius` | max_distance | Radius for computing local track density (defaults to max_distance) |\n| **Embeddings** | | |\n| `do_embeddings` | True | Whether to use embedding features |\n| `embedding_function` | 'cupytexture' | Integrated embedding function: \"cupytexture\", \"cupytexture_color\", \"mega_cupytexture\" |\n| `embedding_weight` | 1.0 | Weight for embedding similarity in cost function |\n| `max_embeddings_per_track` | 15 | Maximum embeddings stored per track |\n| `embedding_matching_method` | 'weighted_average' | Method for multi-embedding matching |\n| **Collision Handling** | | |\n| `collision_freeze_embeddings` | True | Freeze embedding updates in dense areas |\n| `embedding_freeze_density` | 1 | Freeze when \u2265N tracks within radius |\n| **Assignment Strategy** | | |\n| `assignment_strategy` | 'hybrid' | Assignment method: 'hungarian', 'greedy', or 'hybrid' |\n| `greedy_threshold` | max_distance/5 | Distance threshold for greedy assignment |\n| **Track Initialization** | | |\n| `min_consecutive_detections` | 6 | Minimum consecutive detections to create track |\n| `max_detection_gap` | 2 | Maximum gap between detections |\n| `pending_detection_distance` | max_distance | Distance threshold for pending detection matching |\n| **Re-identification** | | |\n| `reid_enabled` | True | Enable re-identification of lost tracks |\n| `reid_max_distance` | max_distance*1.5 | Maximum distance for ReID |\n| `reid_embedding_threshold` | 0.3 | Embedding threshold for ReID |\n| **Experimental** | | |\n| `use_probabilistic_costs` | False | Use gaussian fusion for cost computation |\n\n\n### \ud83c\udfaf Preset Configurations for Common Scenarios\n\nBest Settings for Performance:\n\n\n For good balance (speed + accuracy): up to 300 individuals\n```python\n config = SwarmSortConfig(\n kalman_type=\"simple\", # Fast but accurate enough \n assignment_strategy=\"hybrid\", # Good balance \n uncertainty_weight=0.33, # Some uncertainty handling \n do_embeddings=True, # Use embeddings if available \n reid_enabled=False, # Skip for speed \n )\n```\n For maximum speed across all scales: 300+ individuals\n```python\n config = SwarmSortConfig(\n kalman_type=\"simple\", # Fastest Kalman filter \n assignment_strategy=\"greedy\", # Fastest assignment \n uncertainty_weight=0.0, # Disable uncertainty \n do_embeddings=False, # No embeddings \n reid_enabled=False, # No re-identification \n )\n```\n\n### \ud83d\udd27 Understanding Key Parameters\n\n```python\n# The most important parameters to tune:\n\nconfig = SwarmSortConfig(\n # 1. How far can an object move between frames?\n max_distance=150.0, # Increase for fast objects, decrease for slow\n \n # 2. How many frames to confirm a new track?\n min_consecutive_detections=6, # Lower = faster response, more false positives\n # Higher = slower response, fewer false positives\n \n # 3. How long to keep lost tracks?\n max_track_age=30, # At 30 FPS, this is 1 second of \"memory\"\n \n # 4. Use appearance features?\n do_embeddings=True, # True if objects look different from each other\n # False if all objects look the same (e.g., identical boxes)\n \n # 5. How to handle crowded scenes?\n collision_freeze_embeddings=True, # Prevents ID switches when objects touch\n uncertainty_weight=0.33, # Higher = more conservative in uncertain situations\n)\n```\n\n## Advanced Usage\n\n### Different Configuration Methods\n\n```python\nfrom swarmsort import SwarmSortTracker, SwarmSortConfig\n\n# Default tracker\ntracker = SwarmSortTracker()\n\n# With configuration object\nconfig = SwarmSortConfig(max_distance=100.0, do_embeddings=True)\ntracker = SwarmSortTracker(config)\n\n# With dictionary config\ntracker = SwarmSortTracker({'max_distance': 100.0, 'do_embeddings': True})\n```\n\n### Basic Standalone Usage\n\n```python\nfrom swarmsort import SwarmSortTracker, SwarmSortConfig\n\n# SwarmSort is a standalone tracker - no special integration needed\ntracker = SwarmSortTracker()\n\n# Configure for specific use cases\nconfig = SwarmSortConfig(\n do_embeddings=True,\n reid_enabled=True,\n max_distance=100.0,\n assignment_strategy='hybrid', # Use hybrid assignment strategy\n uncertainty_weight=0.33 # Enable uncertainty-based costs\n)\ntracker_configured = SwarmSortTracker(config)\n```\n\n## \ud83d\udce6 Working with Data\n\n### Input: Detection Objects\n\nDetections are what you feed into the tracker - they represent objects found in the current frame:\n\n```python\nfrom swarmsort import Detection\nimport numpy as np\n\n# Minimal detection - just position and confidence\nsimple_detection = Detection(\n position=[320, 240], # Center point (x, y)\n confidence=0.9 # How sure are we this is real? (0-1)\n)\n\n# Full detection with all the bells and whistles\nfull_detection = Detection(\n position=np.array([320, 240]), # Object center\n confidence=0.95, # Detection confidence\n bbox=np.array([300, 220, 340, 260]), # Bounding box [x1, y1, x2, y2]\n embedding=feature_vector, # Visual features (for example from your CNN)\n class_id=0, # 0=person, 1=car, etc.\n id=\"yolo_detection_42\" # Your detector's ID (optional)\n)\n\n# Real-world example: Converting YOLO output to SwarmSort\ndef yolo_to_swarmsort(yolo_results):\n detections = []\n for box in yolo_results.boxes:\n x1, y1, x2, y2 = box.xyxy[0].numpy()\n center_x = (x1 + x2) / 2\n center_y = (y1 + y2) / 2\n \n detections.append(Detection(\n position=[center_x, center_y],\n confidence=box.conf[0].item(),\n bbox=[x1, y1, x2, y2],\n class_id=int(box.cls[0])\n ))\n return detections\n```\n\n### Output: TrackedObject Results\n\nThe tracker returns TrackedObject instances with rich information about each tracked object:\n\n```python\n# Get tracking results\ntracked_objects = tracker.update(detections)\n\nfor obj in tracked_objects:\n # Identity\n print(f\"\ud83c\udd94 Track ID: {obj.id}\") # Unique ID that persists across frames\n \n # Location & Motion\n print(f\"\ud83d\udccd Position: {obj.position}\") # Current [x, y] position\n print(f\"\u27a1\ufe0f Velocity: {obj.velocity}\") # Speed and direction [vx, vy]\n \n # Confidence & Quality\n print(f\"\u2705 Confidence: {obj.confidence:.1%}\") # How confident are we?\n print(f\"\ud83d\udcca Track quality: {obj.hits}/{obj.age}\") # Hits/Age ratio\n \n # Track Status\n if obj.time_since_update == 0:\n print(\"\ud83d\udfe2 Currently visible\")\n else:\n print(f\"\ud83d\udfe1 Lost for {obj.time_since_update} frames\")\n \n # Bounding Box (if available)\n if obj.bbox is not None:\n x1, y1, x2, y2 = obj.bbox\n width = x2 - x1\n height = y2 - y1\n print(f\"\ud83d\udcd0 Size: {width:.0f}x{height:.0f} pixels\")\n```\n\n### \ud83d\udd04 Track Lifecycle Management\n\nSwarmSort provides fine control over track states - perfect for different visualization needs:\n\n```python\n# Get only tracks that are currently visible\nalive_tracks = tracker.update(detections)\nprint(f\"\ud83d\udc41\ufe0f Visible now: {len(alive_tracks)} objects\")\n\n# Get tracks that were recently lost (useful for smooth visualization)\nrecently_lost = tracker.get_recently_lost_tracks(max_frames_lost=5)\nprint(f\"\ud83d\udc7b Recently lost: {len(recently_lost)} objects\")\n\n# Get everything (visible + recently lost)\nall_active = tracker.get_all_active_tracks(max_frames_lost=5)\nprint(f\"\ud83d\udcca Total active: {len(all_active)} objects\")\n\n# Example: Different visualization for different states\nfor obj in alive_tracks:\n draw_solid_box(frame, obj, color='green') # Solid box for visible\n \nfor obj in recently_lost:\n draw_dashed_box(frame, obj, color='yellow') # Dashed box for lost\n```\n\n\n## \u26a1 Performance & Optimization\n\n### Why SwarmSort is Fast\n\nSwarmSort is optimized for real-world performance:\n\n- **Numba JIT Compilation**: Math operations run at C speed\n- **Vectorized NumPy**: Batch operations instead of loops\n- **Smart Caching**: Reuses computed embeddings and distances\n- **Memory Pooling**: Reduces allocation overhead\n- **Early Exit Logic**: Skips unnecessary computations\n\n\n## Visualization Example\n\nSwarmSort includes built-in visualization utilities for beautiful tracking displays:\n<details>\n <summary>Click to expand code example</summary>\n\n```python\nimport cv2\nimport numpy as np\nfrom swarmsort import SwarmSortTracker, Detection, SwarmSortConfig\nfrom swarmsort import TrackingVisualizer, VisualizationConfig\n\n# Initialize tracker with your preferred settings\nconfig = SwarmSortConfig(\n do_embeddings=True,\n embedding_function='cupytexture', # or 'mega_cupytexture' for more features\n assignment_strategy='hybrid',\n uncertainty_weight=0.33\n)\ntracker = SwarmSortTracker(config)\n\n# Initialize visualizer with custom settings\nvis_config = VisualizationConfig(\n show_trails=True,\n trail_length=30,\n show_ids=True,\n show_confidence=True,\n show_velocity_vectors=True,\n id_font_scale=0.5,\n id_thickness=2,\n box_thickness=2\n)\nvisualizer = TrackingVisualizer(vis_config)\n\n# Example usage with video\ncap = cv2.VideoCapture('video.mp4') # Or use 0 for webcam\n\nwhile True:\n ret, frame = cap.read()\n if not ret:\n break\n \n # Detect objects (replace with your detector)\n # Here's a mock detection for demonstration\n detections = [\n Detection(\n position=np.array([100, 200]),\n confidence=0.9,\n bbox=np.array([80, 180, 120, 220])\n ),\n Detection(\n position=np.array([300, 400]),\n confidence=0.85,\n bbox=np.array([280, 380, 320, 420])\n )\n ]\n \n # Update tracker\n tracked_objects = tracker.update(detections)\n \n # Draw tracking results with built-in visualizer\n frame = visualizer.draw_tracks(frame, tracked_objects)\n \n # Optionally show recently lost tracks\n recently_lost = tracker.get_recently_lost_tracks(max_frames_lost=5)\n frame = visualizer.draw_lost_tracks(frame, recently_lost)\n \n # Display frame\n cv2.imshow('SwarmSort Tracking', frame)\n if cv2.waitKey(1) & 0xFF == ord('q'):\n break\n\ncap.release()\ncv2.destroyAllWindows()\n```\n\n</details>\n\n### Quick Visualization (One-liner)\n\nFor quick prototyping, use the convenience function:\n\n```python\nfrom swarmsort import quick_visualize\n\n# One line to visualize tracking results!\nframe_with_tracks = quick_visualize(frame, tracked_objects, show_trails=True)\n```\n\n### Custom Drawing (If you need more control)\n\nIf you prefer to implement custom visualization:\n\n<details>\n <summary>Click to expand code example</summary>\n\n```python\nimport cv2\nimport numpy as np\nfrom swarmsort import SwarmSortTracker, Detection, SwarmSortConfig\n\n# Initialize tracker\ntracker = SwarmSortTracker(SwarmSortConfig(do_embeddings=True))\n\n# Function to draw tracking results\ndef draw_tracks(frame, tracked_objects, show_trails=True):\n \"\"\"Draw bounding boxes and tracking information on frame.\"\"\"\n # Store trail history (in production, store this outside the function)\n if not hasattr(draw_tracks, 'trails'):\n draw_tracks.trails = {}\n \n for obj in tracked_objects:\n # Get track color (consistent color per ID)\n color = np.random.RandomState(obj.id).randint(0, 255, 3).tolist()\n \n # Draw bounding box if available\n if obj.bbox is not None:\n x1, y1, x2, y2 = obj.bbox.astype(int)\n cv2.rectangle(frame, (x1, y1), (x2, y2), color, 2)\n \n # Draw track ID and confidence\n label = f\"ID:{obj.id} ({obj.confidence:.2f})\"\n cv2.putText(frame, label, (x1, y1-10), \n cv2.FONT_HERSHEY_SIMPLEX, 0.5, color, 2)\n \n # Draw center point\n cx, cy = obj.position.astype(int)\n cv2.circle(frame, (cx, cy), 5, color, -1)\n \n # Update and draw trail\n if show_trails:\n if obj.id not in draw_tracks.trails:\n draw_tracks.trails[obj.id] = []\n draw_tracks.trails[obj.id].append((cx, cy))\n \n # Keep only last 30 points\n draw_tracks.trails[obj.id] = draw_tracks.trails[obj.id][-30:]\n \n # Draw trail\n points = draw_tracks.trails[obj.id]\n for i in range(1, len(points)):\n cv2.line(frame, points[i-1], points[i], color, 2)\n \n # Clean up old trails\n active_ids = {obj.id for obj in tracked_objects}\n draw_tracks.trails = {k: v for k, v in draw_tracks.trails.items() \n if k in active_ids}\n \n return frame\n```\n\n</details>\n\n### Simple Visualization with Matplotlib\n\nFor a simpler visualization or for Jupyter notebooks:\n\n<details>\n <summary>Click to expand code example</summary>\n\n```python\nimport matplotlib.pyplot as plt\nimport matplotlib.patches as patches\nfrom matplotlib.animation import FuncAnimation\nimport numpy as np\nfrom swarmsort import SwarmSortTracker, Detection\n\n# Create figure\nfig, ax = plt.subplots(figsize=(10, 8))\nax.set_xlim(0, 640)\nax.set_ylim(480, 0) # Invert y-axis for image coordinates\nax.set_aspect('equal')\nax.set_title('SwarmSort Multi-Object Tracking')\n\ntracker = SwarmSortTracker()\ntrack_history = {}\n\ndef update_plot(frame_num):\n ax.clear()\n ax.set_xlim(0, 640)\n ax.set_ylim(480, 0)\n ax.set_title(f'Frame {frame_num}')\n \n # Generate mock detections (replace with real detections)\n detections = [\n Detection(\n position=np.array([320 + 100*np.sin(frame_num/10), 240]),\n confidence=0.9,\n bbox=np.array([300 + 100*np.sin(frame_num/10), 220, \n 340 + 100*np.sin(frame_num/10), 260])\n ),\n Detection(\n position=np.array([200, 240 + 100*np.cos(frame_num/10)]),\n confidence=0.85,\n bbox=np.array([180, 220 + 100*np.cos(frame_num/10),\n 220, 260 + 100*np.cos(frame_num/10)])\n )\n ]\n \n # Update tracker\n tracked_objects = tracker.update(detections)\n \n # Plot tracked objects\n for obj in tracked_objects:\n # Get consistent color for track ID\n np.random.seed(obj.id)\n color = np.random.rand(3)\n \n # Draw bounding box\n if obj.bbox is not None:\n x1, y1, x2, y2 = obj.bbox\n rect = patches.Rectangle((x1, y1), x2-x1, y2-y1,\n linewidth=2, edgecolor=color, \n facecolor='none')\n ax.add_patch(rect)\n \n # Draw center point\n ax.scatter(obj.position[0], obj.position[1], \n c=[color], s=100, marker='o')\n \n # Add ID label\n ax.text(obj.position[0], obj.position[1]-20, f'ID:{obj.id}',\n color=color, fontsize=12, ha='center', weight='bold')\n \n # Update history\n if obj.id not in track_history:\n track_history[obj.id] = []\n track_history[obj.id].append(obj.position.copy())\n \n # Draw trail\n if len(track_history[obj.id]) > 1:\n trail = np.array(track_history[obj.id])\n ax.plot(trail[:, 0], trail[:, 1], color=color, \n linewidth=2, alpha=0.5)\n \n # Clean old tracks\n active_ids = {obj.id for obj in tracked_objects}\n for track_id in list(track_history.keys()):\n if track_id not in active_ids:\n if len(track_history[track_id]) > 50: # Remove very old tracks\n del track_history[track_id]\n\n# Create animation\nanim = FuncAnimation(fig, update_plot, frames=200, \n interval=50, repeat=True)\nplt.show()\n\n# To save as video:\n# anim.save('tracking_visualization.mp4', writer='ffmpeg', fps=20)\n```\n\n</details>\n\n## \ud83d\ude80 Advanced Features\n\n### \ud83e\udde0 How SwarmSort Thinks: The Intelligence Behind the Tracking\n\n#### **Uncertainty-Aware Tracking**\nSwarmSort knows when it's confident and when it's not:\n\n```python\n# The tracker automatically adjusts behavior based on uncertainty:\n# - New tracks: \"I'm not sure yet, let me observe more\"\n# - Established tracks: \"I know this object well\"\n# - Crowded areas: \"Need to be extra careful here\"\n\nconfig = SwarmSortConfig(\n uncertainty_weight=0.33, # How much to consider uncertainty\n # 0.0 = Ignore uncertainty (aggressive)\n # 0.5 = Balanced approach\n # 1.0 = Very conservative\n)\n\n# Example: High uncertainty for drone tracking (unpredictable motion)\ndrone_config = SwarmSortConfig(\n uncertainty_weight=0.6, # Be more careful with uncertain tracks\n kalman_type='oc', # Better motion model for erratic movement\n)\n```\n\n#### **Smart Collision Prevention**\nPrevents ID switches when objects get close:\n\n```python\n# Scenario: Tracking dancers who frequently cross paths\ndance_config = SwarmSortConfig(\n collision_freeze_embeddings=True, # Lock visual features when close\n embedding_freeze_density=1, # Freeze when anyone is within...\n local_density_radius=100.0, # ...100 pixels\n)\n\n# What happens:\n# 1. Two Paramecium approach each other\n# 2. SwarmSort detects they're getting close\n# 3. Visual features are \"frozen\" - relies on motion only\n# 4. Prevents mixing up their identities\n# 5. Once separated, visual matching resumes\n```\n\n#### **Hybrid Assignment Strategy**\nCombines the best of both worlds:\n\n```python\nconfig = SwarmSortConfig(\n assignment_strategy='hybrid', # Smart mode (default)\n greedy_threshold=30.0, # Fast matching for obvious cases\n)\n\n# How it works:\n# 1. Obvious matches (very close): Uses fast greedy assignment\n# 2. Ambiguous cases: Falls back to optimal Hungarian algorithm\n# 3. Best of both: Fast AND accurate\n\n\nconfig.assignment_strategy = 'hybrid' # optimal matching\n```\n\n### \ud83d\udd0d Re-Identification: Bringing Lost Objects Back\n\nPerfect for scenarios where objects temporarily disappear:\n\n```python\n# Example: Security camera at a store entrance\nconfig = SwarmSortConfig(\n reid_enabled=True, # Enable re-identification\n reid_max_distance=200.0, # Search this far for lost tracks\n reid_embedding_threshold=0.25, # How similar must appearances be?\n)\n\n# What happens:\n# 1. Person walks behind a pillar (track lost)\n# 2. Person reappears on the other side\n# 3. SwarmSort compares appearance with recently lost tracks\n# 4. Same person? Same ID! Tracking continues seamlessly\n\n# Real-world usage:\ntracker = SwarmSortTracker(config)\nresults = tracker.update(detections)\n\n# The person who disappeared at frame 100 and reappeared at frame 120\n# will have the SAME track ID - perfect for counting and analytics!\n```\n\n## \ud83d\udd27 Troubleshooting & FAQ\n\n### Common Issues and Solutions\n\n**Q: My tracks keep switching IDs when objects cross paths**\n```python\n# Solution: Enable collision handling\nconfig = SwarmSortConfig(\n collision_freeze_embeddings=True, # Prevent ID switches\n embedding_freeze_density=1, # Freeze when objects are close\n do_embeddings=True, # Use visual features\n embedding_weight=1.5, # Trust appearance more\n)\n```\n\n**Q: New tracks take too long to appear**\n```python\n# Solution: Reduce initialization requirements\nconfig = SwarmSortConfig(\n min_consecutive_detections=2, # Was 6, now faster\n init_conf_threshold=0.3, # Accept lower confidence\n)\n```\n\n**Q: Too many false tracks from noise**\n```python\n# Solution: Be more strict about track creation\nconfig = SwarmSortConfig(\n min_consecutive_detections=8, # Require more detections\n init_conf_threshold=0.7, # Higher confidence needed\n detection_conf_threshold=0.5, # Filter out weak detections\n)\n```\n\n**Q: Tracks disappear too quickly**\n```python\n# Solution: Keep tracks alive longer\nconfig = SwarmSortConfig(\n max_track_age=30, # Keep for 2 seconds at 30 FPS (was 30)\n reid_enabled=True, # Try to re-identify lost tracks\n)\n```\n\n**Q: Performance is too slow**\nConsider processing every other frame\n\n## Examples\n\nSee the `examples/` directory for comprehensive usage examples.\n\n## Testing\n\n```bash\n# Run tests\npoetry run pytest\n\n# Run tests with coverage\npoetry run pytest --cov=swarmsort --cov-report=html\n\n# Run specific test\npoetry run pytest tests/test_basic.py::test_basic_tracking\n```\n\n## Development\n\n### Development Setup\n\nWant to contribute or modify SwarmSort? Here's how to set up a development environment:\n\n```bash\n# Clone the repository\ngit clone https://github.com/cfosseprez/swarmsort.git\ncd swarmsort\n\n# Install with Poetry (recommended for development)\npoetry install --with dev\n\n# Or use pip in editable mode\npip install -e \".[dev]\"\n```\n\n## Benchmarking\n\n```bash\n# Run benchmarks\npoetry run pytest tests/ --benchmark-only\n```\n\n## License\n\nGPL 3.0 or later - see LICENSE file for details.\n\n## Contributing\n\n1. Fork the repository\n2. Create a feature branch (`git checkout -b feature/amazing-feature`)\n3. Commit your changes (`git commit -m 'Add amazing feature'`)\n4. Push to the branch (`git push origin feature/amazing-feature`)\n5. Open a Pull Request\n\n### Development Workflow\n\n\n```bash\n# Install development dependencies\npoetry install --with dev\n\n# Run linting\npoetry run black swarmsort/\npoetry run flake8 swarmsort/\n\n# Run type checking\npoetry run mypy swarmsort/\n```\n\n",
"bugtrack_url": null,
"license": "GPL-3.0-or-later",
"summary": "Standalone SwarmSort real-time multi-object tracker with integrated lightweight embeddings",
"version": "1.1.0",
"project_urls": {
"Homepage": "https://github.com/cfosseprez/swarmsort",
"Repository": "https://github.com/cfosseprez/swarmsort"
},
"split_keywords": [
"tracking",
" computer-vision",
" multi-object-tracking",
" embeddings",
" real-time"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "86f63b7af98abd7d55570d8bd825a98544d9f87f5c216483d68656e0bb06d9cb",
"md5": "02561135f47121eea9ae39eb64bb752e",
"sha256": "8b22545827d8d90f09676632ca338efaa6742bc7f1a6ac55b2f348a527d383fa"
},
"downloads": -1,
"filename": "swarmsort-1.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "02561135f47121eea9ae39eb64bb752e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.12,>=3.9",
"size": 140639,
"upload_time": "2025-09-09T04:25:53",
"upload_time_iso_8601": "2025-09-09T04:25:53.205881Z",
"url": "https://files.pythonhosted.org/packages/86/f6/3b7af98abd7d55570d8bd825a98544d9f87f5c216483d68656e0bb06d9cb/swarmsort-1.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e489ac044bd5a6580c9602110c73d71dc69e6cb8c720fed1d0e11c0df16af48f",
"md5": "427bffe3232e1662170b9e1341e21179",
"sha256": "02d48ecb1680dda3f4f2ccd7d07419d2f1c8de17fcf6a055f62ffd0dc1fa4706"
},
"downloads": -1,
"filename": "swarmsort-1.1.0.tar.gz",
"has_sig": false,
"md5_digest": "427bffe3232e1662170b9e1341e21179",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.12,>=3.9",
"size": 143360,
"upload_time": "2025-09-09T04:25:55",
"upload_time_iso_8601": "2025-09-09T04:25:55.173297Z",
"url": "https://files.pythonhosted.org/packages/e4/89/ac044bd5a6580c9602110c73d71dc69e6cb8c720fed1d0e11c0df16af48f/swarmsort-1.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-09-09 04:25:55",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cfosseprez",
"github_project": "swarmsort",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "swarmsort"
}
JSON
