# SNIC: Sparse Network of Idempotent Comparisons
A high-performance Python library for efficient ranking of large datasets using sparse comparison networks.
## What is SNIC?
SNIC is a novel ranking algorithm that dramatically reduces the number of comparisons needed to rank large sets of elements. Instead of requiring O(n²) pairwise comparisons like traditional methods, SNIC achieves O(n log n) efficiency through:
- **Sparse Networks**: Structured comparison networks that capture ranking information with minimal comparisons
- **Base-sized Matchups**: Fixed-size groups that maximize information extraction per comparison
- **GBER Foundation**: Mathematical framework using Generalized Base Exponential Representation for optimal network structure
## Key Benefits
- **Scalable**: Efficiently handles datasets from hundreds to millions of elements
- **Sparse**: Requires far fewer comparisons than traditional ranking methods
- **Structured**: Deterministic, mathematically-grounded approach to comparison generation
- **Fast**: High-performance Rust implementation with Python bindings
## Quick Start
```bash
pip install snic
```
```python
import snic
# Generate matchups for ranking 100 elements with base-3 comparisons
matchups = snic.stream_matches_from(100, 3)
# Each matchup contains 3 elements to be ranked
print(f"Generated {len(matchups)} matchups")
print(f"First matchup: {matchups[0]}") # e.g., [0, 33, 67]
# After ranking each matchup externally (by human judgment, ML model, etc.)
# Convert ranked matchups back to final ranking
ranked_matchups = [
[1, 0, 2], # Example: element 1 ranked first, 0 second, 2 third
[4, 3, 5], # Continue for all matchups...
# ...
]
final_ranking = snic.stream_rankings_from(ranked_matchups)
print(f"Final ranking: {final_ranking}")
```
## How It Works
1. **Decomposition**: SNIC uses GBER to break down your dataset size into optimal subnetworks
2. **Matchup Generation**: Creates structured groups of elements for comparison
3. **Sparse Comparisons**: Each element participates in logarithmically few comparisons
4. **Ranking Synthesis**: Combines local rankings into a global result
## Use Cases
- **Large-scale surveys**: Rank thousands of items with minimal human effort
- **Content recommendation**: Efficiently determine user preferences
- **Tournament systems**: Fair bracket generation for competitions
- **Data analysis**: Rank features, samples, or model outputs at scale
## Development
### Local Setup
```bash
# Clone the repository
git clone https://github.com/ryzhakar/snic-rs
cd snic-rs
# Install development dependencies
pip install maturin
# Build and install in development mode
maturin develop
# Run tests
python -m pytest
```
### Building
```bash
# Build wheel
maturin build --release
# Build and publish to PyPI
maturin publish
```
## Algorithm Details
SNIC is based on rigorous mathematical foundations:
- **GBER (Generalized Base Exponential Representation)**: Decomposes dataset sizes into optimal subnetwork structures
- **Stride-based Matchup Generation**: Ensures comprehensive coverage with minimal redundancy
- **Hub-and-Spoke Connectivity**: Connects subnetworks for global ranking consistency
For detailed algorithm documentation, see the [wiki](wiki/).
## Performance
SNIC's efficiency scales logarithmically:
| Dataset Size | Traditional Comparisons | SNIC Comparisons | Reduction |
|-------------|------------------------|------------------|-----------|
| 1,000 | 499,500 | ~3,000 | 99.4% |
| 10,000 | 49,995,000 | ~40,000 | 99.9% |
| 100,000 | 4,999,950,000 | ~500,000 | 99.99% |
## License
AGPLv3 License - see [LICENSE](LICENSE) for details.
Raw data
{
"_id": null,
"home_page": null,
"name": "snic",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "ranking, comparison, sparse-network, algorithm, data-analysis, machine-learning",
"author": null,
"author_email": "Arthur Ryzhak <ryzhakar@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/8d/65/6bd343623f587bb2f329099220bf3993bd4e13a13b6004f2c1529843a315/snic-0.1.0.tar.gz",
"platform": null,
"description": "# SNIC: Sparse Network of Idempotent Comparisons\n\nA high-performance Python library for efficient ranking of large datasets using sparse comparison networks.\n\n## What is SNIC?\n\nSNIC is a novel ranking algorithm that dramatically reduces the number of comparisons needed to rank large sets of elements. Instead of requiring O(n\u00b2) pairwise comparisons like traditional methods, SNIC achieves O(n log n) efficiency through:\n\n- **Sparse Networks**: Structured comparison networks that capture ranking information with minimal comparisons\n- **Base-sized Matchups**: Fixed-size groups that maximize information extraction per comparison\n- **GBER Foundation**: Mathematical framework using Generalized Base Exponential Representation for optimal network structure\n\n## Key Benefits\n\n- **Scalable**: Efficiently handles datasets from hundreds to millions of elements\n- **Sparse**: Requires far fewer comparisons than traditional ranking methods\n- **Structured**: Deterministic, mathematically-grounded approach to comparison generation\n- **Fast**: High-performance Rust implementation with Python bindings\n\n## Quick Start\n\n```bash\npip install snic\n```\n\n```python\nimport snic\n\n# Generate matchups for ranking 100 elements with base-3 comparisons\nmatchups = snic.stream_matches_from(100, 3)\n\n# Each matchup contains 3 elements to be ranked\nprint(f\"Generated {len(matchups)} matchups\")\nprint(f\"First matchup: {matchups[0]}\") # e.g., [0, 33, 67]\n\n# After ranking each matchup externally (by human judgment, ML model, etc.)\n# Convert ranked matchups back to final ranking\nranked_matchups = [\n [1, 0, 2], # Example: element 1 ranked first, 0 second, 2 third\n [4, 3, 5], # Continue for all matchups...\n # ... \n]\n\nfinal_ranking = snic.stream_rankings_from(ranked_matchups)\nprint(f\"Final ranking: {final_ranking}\")\n```\n\n## How It Works\n\n1. **Decomposition**: SNIC uses GBER to break down your dataset size into optimal subnetworks\n2. **Matchup Generation**: Creates structured groups of elements for comparison\n3. **Sparse Comparisons**: Each element participates in logarithmically few comparisons\n4. **Ranking Synthesis**: Combines local rankings into a global result\n\n## Use Cases\n\n- **Large-scale surveys**: Rank thousands of items with minimal human effort\n- **Content recommendation**: Efficiently determine user preferences\n- **Tournament systems**: Fair bracket generation for competitions\n- **Data analysis**: Rank features, samples, or model outputs at scale\n\n## Development\n\n### Local Setup\n\n```bash\n# Clone the repository\ngit clone https://github.com/ryzhakar/snic-rs\ncd snic-rs\n\n# Install development dependencies\npip install maturin\n\n# Build and install in development mode\nmaturin develop\n\n# Run tests\npython -m pytest\n```\n\n### Building\n\n```bash\n# Build wheel\nmaturin build --release\n\n# Build and publish to PyPI\nmaturin publish\n```\n\n## Algorithm Details\n\nSNIC is based on rigorous mathematical foundations:\n\n- **GBER (Generalized Base Exponential Representation)**: Decomposes dataset sizes into optimal subnetwork structures\n- **Stride-based Matchup Generation**: Ensures comprehensive coverage with minimal redundancy \n- **Hub-and-Spoke Connectivity**: Connects subnetworks for global ranking consistency\n\nFor detailed algorithm documentation, see the [wiki](wiki/).\n\n## Performance\n\nSNIC's efficiency scales logarithmically:\n\n| Dataset Size | Traditional Comparisons | SNIC Comparisons | Reduction |\n|-------------|------------------------|------------------|-----------|\n| 1,000 | 499,500 | ~3,000 | 99.4% |\n| 10,000 | 49,995,000 | ~40,000 | 99.9% |\n| 100,000 | 4,999,950,000 | ~500,000 | 99.99% |\n\n## License\n\nAGPLv3 License - see [LICENSE](LICENSE) for details.\n\n",
"bugtrack_url": null,
"license": "AGPLv3",
"summary": "High-performance library for efficient ranking of large datasets using sparse comparison networks",
"version": "0.1.0",
"project_urls": {
"Bug Tracker": "https://github.com/ryzhakar/snic-rs/issues",
"Documentation": "https://github.com/ryzhakar/snic-rs/wiki",
"Homepage": "https://github.com/ryzhakar/snic-rs",
"Repository": "https://github.com/ryzhakar/snic-rs.git"
},
"split_keywords": [
"ranking",
" comparison",
" sparse-network",
" algorithm",
" data-analysis",
" machine-learning"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "c1c83c0f835a04e7e23ca1b2d9ceb3c15264d000e3cc96ab6556b6d77d1eed55",
"md5": "e2c785520108a5eb44439fd7b0da66e7",
"sha256": "48ce77d862b3bb7d0b13a4bffbfa7e9fed68cfd1e75b3ce68c00410b127d28e2"
},
"downloads": -1,
"filename": "snic-0.1.0-cp38-abi3-macosx_10_12_x86_64.whl",
"has_sig": false,
"md5_digest": "e2c785520108a5eb44439fd7b0da66e7",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.8",
"size": 288095,
"upload_time": "2025-08-15T17:44:22",
"upload_time_iso_8601": "2025-08-15T17:44:22.842879Z",
"url": "https://files.pythonhosted.org/packages/c1/c8/3c0f835a04e7e23ca1b2d9ceb3c15264d000e3cc96ab6556b6d77d1eed55/snic-0.1.0-cp38-abi3-macosx_10_12_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "84337089e1d4d0064273544827eab3c157e4a1025f13b5e9611c526197dd017c",
"md5": "2cb0f5d6dec4b111ac9f54578edc8477",
"sha256": "bcb17960a12cad98221c85d111b0f9e674ee7c3202ad42d66acec914cfb7bf1c"
},
"downloads": -1,
"filename": "snic-0.1.0-cp38-abi3-macosx_11_0_arm64.whl",
"has_sig": false,
"md5_digest": "2cb0f5d6dec4b111ac9f54578edc8477",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.8",
"size": 275330,
"upload_time": "2025-08-15T17:44:24",
"upload_time_iso_8601": "2025-08-15T17:44:24.465686Z",
"url": "https://files.pythonhosted.org/packages/84/33/7089e1d4d0064273544827eab3c157e4a1025f13b5e9611c526197dd017c/snic-0.1.0-cp38-abi3-macosx_11_0_arm64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "1abdeb0bed4fdf25ee3590f0c61d0944dc2ff6f96746234a0b48f33d0964c0df",
"md5": "b09e5eb162f37e8348387419410c25de",
"sha256": "fa45895ce76448fa4cc3ae92fa4b4b0bc0c886ae4df4df9751734cc880b151fd"
},
"downloads": -1,
"filename": "snic-0.1.0-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl",
"has_sig": false,
"md5_digest": "b09e5eb162f37e8348387419410c25de",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.8",
"size": 300196,
"upload_time": "2025-08-15T17:44:25",
"upload_time_iso_8601": "2025-08-15T17:44:25.440668Z",
"url": "https://files.pythonhosted.org/packages/1a/bd/eb0bed4fdf25ee3590f0c61d0944dc2ff6f96746234a0b48f33d0964c0df/snic-0.1.0-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "09ebd2d8a52fdf40fc8fe041fe7c948f7f0059223f2d961421520161d3724d60",
"md5": "3b22072bc4d78c56119e42313fab5eef",
"sha256": "06088101960dc791be3db7007bc509d265ef391f38318e5467c14817c53b85df"
},
"downloads": -1,
"filename": "snic-0.1.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
"has_sig": false,
"md5_digest": "3b22072bc4d78c56119e42313fab5eef",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.8",
"size": 320717,
"upload_time": "2025-08-15T17:44:26",
"upload_time_iso_8601": "2025-08-15T17:44:26.727039Z",
"url": "https://files.pythonhosted.org/packages/09/eb/d2d8a52fdf40fc8fe041fe7c948f7f0059223f2d961421520161d3724d60/snic-0.1.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "de225b7662a2d652f6fc17bcf4321f2b904ca897f203bae2436f068d14458102",
"md5": "45a53c2d1025b73d01f78219f52fac11",
"sha256": "bbd50cea6e74969dbdd7807641904ee92455b0b9028dcedbb84db0dd3d97af4d"
},
"downloads": -1,
"filename": "snic-0.1.0-cp38-abi3-win_amd64.whl",
"has_sig": false,
"md5_digest": "45a53c2d1025b73d01f78219f52fac11",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.8",
"size": 215961,
"upload_time": "2025-08-15T17:44:27",
"upload_time_iso_8601": "2025-08-15T17:44:27.978178Z",
"url": "https://files.pythonhosted.org/packages/de/22/5b7662a2d652f6fc17bcf4321f2b904ca897f203bae2436f068d14458102/snic-0.1.0-cp38-abi3-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "8d656bd343623f587bb2f329099220bf3993bd4e13a13b6004f2c1529843a315",
"md5": "d6a6795e1e6edd11fd41c0d2de02a068",
"sha256": "5b27dd322518d5f32e135824f8cf9adcbf8964a0cc565b9d15aa5f08587b8917"
},
"downloads": -1,
"filename": "snic-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "d6a6795e1e6edd11fd41c0d2de02a068",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 48175,
"upload_time": "2025-08-15T17:44:28",
"upload_time_iso_8601": "2025-08-15T17:44:28.970715Z",
"url": "https://files.pythonhosted.org/packages/8d/65/6bd343623f587bb2f329099220bf3993bd4e13a13b6004f2c1529843a315/snic-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-15 17:44:28",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ryzhakar",
"github_project": "snic-rs",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "snic"
}
JSON
