# PyFunc: Functional Programming Pipeline for Python
PyFunc is a Python library that brings functional programming fluency to Python, enabling chainable, composable, lazy, and debuggable operations on various data structures.
## ✨ Features
- **🔗 Chainable Operations**: Method chaining for readable data transformations
- **🎯 Placeholder Syntax**: Use `_` to create lambda-free expressions
- **⚡ Lazy Evaluation**: Operations computed only when needed
- **🔄 Function Composition**: Compose functions with `>>` and `<<` operators
- **📊 Rich Data Operations**: Works with scalars, lists, dicts, generators
- **🐛 Built-in Debugging**: Debug and trace pipeline execution
- **🔧 Extensible**: Register custom types and extend functionality
- **📝 Type Safe**: Full type hints and generic support
- **🚀 Multi-Backend Performance**: 5 backends for optimal performance
- **Python**: Universal compatibility (always available)
- **C++**: High-performance general operations
- **Rust**: Memory-safe statistical functions
- **Go**: Lightning-fast bitwise operations
- **Zig**: Blazing mathematical computations (236x speedup!)
## 🚀 Quick Start
```bash
pip install pyfunc-pipeline
```
```python
from pyfunc import pipe, _
# Basic pipeline
result = pipe([1, 2, 3, 4]).filter(_ > 2).map(_ * 10).to_list()
# Result: [30, 40]
# String processing
result = pipe("hello world").explode(" ").map(_.capitalize()).implode(" ").get()
# Result: "Hello World"
# Function composition
double = _ * 2
square = _ ** 2
composed = double >> square # square(double(x))
result = pipe(5).apply(composed).get()
# Result: 100
```
### ⚡ **Performance Backends**
```python
from pyfunc import pipe, set_zig_threshold, set_go_threshold
# Configure performance backends
set_zig_threshold(1000) # Use Zig for math operations ≥ 1000 elements
set_go_threshold(500) # Use Go for bitwise operations ≥ 500 elements
# Automatic backend selection
large_data = list(range(5000))
result = pipe(large_data).sum().get() # Uses Zig automatically (blazing fast!)
# Explicit backend control
result = pipe([1, 2, 3, 4, 5]).sum_zig().get() # Force Zig
result = pipe([15, 31, 63]).bitwise_and_go(7).to_list() # Force Go
result = pipe([1, 2, 3, 4, 5]).median_rust().get() # Force Rust
# Batch operations for maximum performance
from pyfunc.backends import get_backend
backend = get_backend()
if backend.zig_backend:
stats = backend.zig_backend.batch_statistics([1, 2, 3, 4, 5])
# Returns: {'sum': 15.0, 'mean': 3.0, 'min': 1.0, 'max': 5.0, 'stdev': 1.414}
```
## 🎯 Core Concepts
### Pipeline Chaining
Every value can be lifted into a pipeline for transformation:
```python
from pyfunc import pipe, _
# Numbers
pipe([1, 2, 3, 4]).filter(_ > 2).map(_ ** 2).sum().get()
# Result: 25
# Strings
pipe(" hello world ").apply(_.strip().title()).explode(" ").to_list()
# Result: ['Hello', 'World']
# Dictionaries
pipe({"a": 1, "b": 2}).map_values(_ * 10).get()
# Result: {"a": 10, "b": 20}
```
### Placeholder Syntax
The `_` placeholder creates reusable, composable expressions:
```python
from pyfunc import _
# Arithmetic operations
double = _ * 2
add_ten = _ + 10
# Method calls
normalize = _.strip().lower()
# Comparisons
is_positive = _ > 0
# Composition
process = double >> add_ten # add_ten(double(x))
```
### Lazy Evaluation
Operations are lazy by default - perfect for large datasets:
```python
# Processes only what's needed from 1 million items
result = pipe(range(1_000_000)).filter(_ > 500_000).take(5).to_list()
```
## 📚 Rich API
### String Operations
```python
pipe("hello,world").explode(",").map(_.capitalize()).implode(" & ").get()
# "Hello & World"
pipe("Hello {name}!").template_fill({"name": "PyFunc"}).get()
# "Hello PyFunc!"
```
### Dictionary Operations
```python
users = {"alice": 25, "bob": 30}
pipe(users).map_values(_ + 5).map_keys(_.title()).get()
# {"Alice": 30, "Bob": 35}
```
### Advanced Transformations
```python
# Group by
data = [{"name": "Alice", "dept": "Eng"}, {"name": "Bob", "dept": "Sales"}]
pipe(data).group_by(_["dept"]).get()
# Sliding windows
pipe([1, 2, 3, 4, 5]).window(3).to_list()
# [[1, 2, 3], [2, 3, 4], [3, 4, 5]]
# Combinations
pipe([1, 2, 3]).combinations(2).to_list()
# [(1, 2), (1, 3), (2, 3)]
```
### Side Effects & Debugging
```python
pipe([1, 2, 3, 4])
.debug("Input")
.filter(_ > 2)
.debug("Filtered")
.map(_ ** 2)
.to_list()
```
## 🌟 Real-World Example
```python
from pyfunc import pipe, _
# E-commerce order processing with template mapping
orders = [
{"id": 1, "customer": "Alice", "items": ["laptop", "mouse"], "total": 1200.50},
{"id": 2, "customer": "Bob", "items": ["keyboard"], "total": 75.00},
{"id": 3, "customer": "Charlie", "items": ["monitor", "stand"], "total": 450.25}
]
# Process orders with dictionary and string templates
result = (
pipe(orders)
.filter(_["total"] > 100) # Filter orders > $100
.map({
"id": _["id"],
"customer": _["customer"],
"discounted_total": _["total"] * 0.9 # 10% discount
})
.map("Order #{id} for {customer}: ${discounted_total:.2f}")
.to_list()
)
print(result)
# ['Order #1 for Alice: $1080.45', 'Order #3 for Charlie: $405.23']
```
### ⚠️ **Important Note**
Use **regular string templates**, not f-strings:
```python
# ❌ Wrong - Don't use f-strings
.map(f"Order #{_['id']}") # This will cause an error!
# ✅ Correct - Use regular string templates
.map("Order #{id}") # PyFunc handles the evaluation
```
## 📖 Documentation
- **[Complete Documentation](DOCUMENTATION.md)** - Full API reference and examples
- **[Examples](examples/)** - Real-world usage examples
- **[Changelog](CHANGELOG.md)** - Version history and updates
### 🚀 Performance Backends
- **[C++ Backend](CPP_BACKEND.md)** - High-performance general operations
- **[Zig Backend](ZIG_BACKEND.md)** - Blazing-fast mathematical operations (236x speedup!)
- **[Rust Backend](examples/rust_threshold_example.py)** - Memory-safe statistical functions
- **[Go Backend](build_go.py)** - Lightning-fast bitwise operations
- **[Backend Control Guide](examples/backend_control_guide.py)** - Complete user control examples
### 🔧 **Backend Installation**
```bash
# Install with all backends (recommended)
pip install pyfunc-pipeline[all]
# Or install specific backends
pip install pyfunc-pipeline[cpp] # C++ backend
pip install pyfunc-pipeline[zig] # Zig backend
pip install pyfunc-pipeline[rust] # Rust backend
pip install pyfunc-pipeline[go] # Go backend
# Build backends from source
python build_zig.py # Build Zig backend
python build_go.py # Build Go backend
python build_cpp.py # Build C++ backend
```
## 🤝 Contributing
Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
## 📄 License
MIT License - see [LICENSE](LICENSE) file for details.
Raw data
{
"_id": null,
"home_page": null,
"name": "pyfunc-pipeline",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "Alex <alex@example.com>",
"keywords": "functional, pipeline, data, transformation, lazy, evaluation, placeholder",
"author": null,
"author_email": "Alex <alex@example.com>",
"download_url": "https://files.pythonhosted.org/packages/0a/68/1e52916c95a80cde4f99e3258ad1b162d5bd1bc6cfa02c3fba81957fb6cd/pyfunc_pipeline-0.4.0.tar.gz",
"platform": null,
"description": "# PyFunc: Functional Programming Pipeline for Python\r\n\r\nPyFunc is a Python library that brings functional programming fluency to Python, enabling chainable, composable, lazy, and debuggable operations on various data structures.\r\n\r\n## \u2728 Features\r\n\r\n- **\ud83d\udd17 Chainable Operations**: Method chaining for readable data transformations\r\n- **\ud83c\udfaf Placeholder Syntax**: Use `_` to create lambda-free expressions \r\n- **\u26a1 Lazy Evaluation**: Operations computed only when needed\r\n- **\ud83d\udd04 Function Composition**: Compose functions with `>>` and `<<` operators\r\n- **\ud83d\udcca Rich Data Operations**: Works with scalars, lists, dicts, generators\r\n- **\ud83d\udc1b Built-in Debugging**: Debug and trace pipeline execution\r\n- **\ud83d\udd27 Extensible**: Register custom types and extend functionality\r\n- **\ud83d\udcdd Type Safe**: Full type hints and generic support\r\n- **\ud83d\ude80 Multi-Backend Performance**: 5 backends for optimal performance\r\n - **Python**: Universal compatibility (always available)\r\n - **C++**: High-performance general operations\r\n - **Rust**: Memory-safe statistical functions \r\n - **Go**: Lightning-fast bitwise operations\r\n - **Zig**: Blazing mathematical computations (236x speedup!)\r\n\r\n## \ud83d\ude80 Quick Start\r\n\r\n```bash\r\npip install pyfunc-pipeline\r\n```\r\n\r\n```python\r\nfrom pyfunc import pipe, _\r\n\r\n# Basic pipeline\r\nresult = pipe([1, 2, 3, 4]).filter(_ > 2).map(_ * 10).to_list()\r\n# Result: [30, 40]\r\n\r\n# String processing \r\nresult = pipe(\"hello world\").explode(\" \").map(_.capitalize()).implode(\" \").get()\r\n# Result: \"Hello World\"\r\n\r\n# Function composition\r\ndouble = _ * 2\r\nsquare = _ ** 2\r\ncomposed = double >> square # square(double(x))\r\n\r\nresult = pipe(5).apply(composed).get()\r\n# Result: 100\r\n```\r\n\r\n### \u26a1 **Performance Backends**\r\n\r\n```python\r\nfrom pyfunc import pipe, set_zig_threshold, set_go_threshold\r\n\r\n# Configure performance backends\r\nset_zig_threshold(1000) # Use Zig for math operations \u2265 1000 elements\r\nset_go_threshold(500) # Use Go for bitwise operations \u2265 500 elements\r\n\r\n# Automatic backend selection\r\nlarge_data = list(range(5000))\r\nresult = pipe(large_data).sum().get() # Uses Zig automatically (blazing fast!)\r\n\r\n# Explicit backend control\r\nresult = pipe([1, 2, 3, 4, 5]).sum_zig().get() # Force Zig\r\nresult = pipe([15, 31, 63]).bitwise_and_go(7).to_list() # Force Go\r\nresult = pipe([1, 2, 3, 4, 5]).median_rust().get() # Force Rust\r\n\r\n# Batch operations for maximum performance\r\nfrom pyfunc.backends import get_backend\r\nbackend = get_backend()\r\nif backend.zig_backend:\r\n stats = backend.zig_backend.batch_statistics([1, 2, 3, 4, 5])\r\n # Returns: {'sum': 15.0, 'mean': 3.0, 'min': 1.0, 'max': 5.0, 'stdev': 1.414}\r\n```\r\n\r\n## \ud83c\udfaf Core Concepts\r\n\r\n### Pipeline Chaining\r\n\r\nEvery value can be lifted into a pipeline for transformation:\r\n\r\n```python\r\nfrom pyfunc import pipe, _\r\n\r\n# Numbers\r\npipe([1, 2, 3, 4]).filter(_ > 2).map(_ ** 2).sum().get()\r\n# Result: 25\r\n\r\n# Strings \r\npipe(\" hello world \").apply(_.strip().title()).explode(\" \").to_list()\r\n# Result: ['Hello', 'World']\r\n\r\n# Dictionaries\r\npipe({\"a\": 1, \"b\": 2}).map_values(_ * 10).get()\r\n# Result: {\"a\": 10, \"b\": 20}\r\n```\r\n\r\n### Placeholder Syntax\r\n\r\nThe `_` placeholder creates reusable, composable expressions:\r\n\r\n```python\r\nfrom pyfunc import _\r\n\r\n# Arithmetic operations\r\ndouble = _ * 2\r\nadd_ten = _ + 10\r\n\r\n# Method calls\r\nnormalize = _.strip().lower()\r\n\r\n# Comparisons \r\nis_positive = _ > 0\r\n\r\n# Composition\r\nprocess = double >> add_ten # add_ten(double(x))\r\n```\r\n\r\n### Lazy Evaluation\r\n\r\nOperations are lazy by default - perfect for large datasets:\r\n\r\n```python\r\n# Processes only what's needed from 1 million items\r\nresult = pipe(range(1_000_000)).filter(_ > 500_000).take(5).to_list()\r\n```\r\n\r\n## \ud83d\udcda Rich API\r\n\r\n### String Operations\r\n```python\r\npipe(\"hello,world\").explode(\",\").map(_.capitalize()).implode(\" & \").get()\r\n# \"Hello & World\"\r\n\r\npipe(\"Hello {name}!\").template_fill({\"name\": \"PyFunc\"}).get() \r\n# \"Hello PyFunc!\"\r\n```\r\n\r\n### Dictionary Operations\r\n```python\r\nusers = {\"alice\": 25, \"bob\": 30}\r\npipe(users).map_values(_ + 5).map_keys(_.title()).get()\r\n# {\"Alice\": 30, \"Bob\": 35}\r\n```\r\n\r\n### Advanced Transformations\r\n```python\r\n# Group by\r\ndata = [{\"name\": \"Alice\", \"dept\": \"Eng\"}, {\"name\": \"Bob\", \"dept\": \"Sales\"}]\r\npipe(data).group_by(_[\"dept\"]).get()\r\n\r\n# Sliding windows\r\npipe([1, 2, 3, 4, 5]).window(3).to_list()\r\n# [[1, 2, 3], [2, 3, 4], [3, 4, 5]]\r\n\r\n# Combinations\r\npipe([1, 2, 3]).combinations(2).to_list() \r\n# [(1, 2), (1, 3), (2, 3)]\r\n```\r\n\r\n### Side Effects & Debugging\r\n```python\r\npipe([1, 2, 3, 4])\r\n .debug(\"Input\")\r\n .filter(_ > 2) \r\n .debug(\"Filtered\")\r\n .map(_ ** 2)\r\n .to_list()\r\n```\r\n\r\n## \ud83c\udf1f Real-World Example\r\n\r\n```python\r\nfrom pyfunc import pipe, _\r\n\r\n# E-commerce order processing with template mapping\r\norders = [\r\n {\"id\": 1, \"customer\": \"Alice\", \"items\": [\"laptop\", \"mouse\"], \"total\": 1200.50},\r\n {\"id\": 2, \"customer\": \"Bob\", \"items\": [\"keyboard\"], \"total\": 75.00},\r\n {\"id\": 3, \"customer\": \"Charlie\", \"items\": [\"monitor\", \"stand\"], \"total\": 450.25}\r\n]\r\n\r\n# Process orders with dictionary and string templates\r\nresult = (\r\n pipe(orders)\r\n .filter(_[\"total\"] > 100) # Filter orders > $100\r\n .map({\r\n \"id\": _[\"id\"],\r\n \"customer\": _[\"customer\"],\r\n \"discounted_total\": _[\"total\"] * 0.9 # 10% discount\r\n })\r\n .map(\"Order #{id} for {customer}: ${discounted_total:.2f}\")\r\n .to_list()\r\n)\r\n\r\nprint(result)\r\n# ['Order #1 for Alice: $1080.45', 'Order #3 for Charlie: $405.23']\r\n```\r\n\r\n### \u26a0\ufe0f **Important Note**\r\nUse **regular string templates**, not f-strings:\r\n```python\r\n# \u274c Wrong - Don't use f-strings \r\n.map(f\"Order #{_['id']}\") # This will cause an error!\r\n\r\n# \u2705 Correct - Use regular string templates\r\n.map(\"Order #{id}\") # PyFunc handles the evaluation\r\n```\r\n\r\n## \ud83d\udcd6 Documentation\r\n\r\n- **[Complete Documentation](DOCUMENTATION.md)** - Full API reference and examples\r\n- **[Examples](examples/)** - Real-world usage examples \r\n- **[Changelog](CHANGELOG.md)** - Version history and updates\r\n\r\n### \ud83d\ude80 Performance Backends\r\n\r\n- **[C++ Backend](CPP_BACKEND.md)** - High-performance general operations\r\n- **[Zig Backend](ZIG_BACKEND.md)** - Blazing-fast mathematical operations (236x speedup!)\r\n- **[Rust Backend](examples/rust_threshold_example.py)** - Memory-safe statistical functions\r\n- **[Go Backend](build_go.py)** - Lightning-fast bitwise operations\r\n- **[Backend Control Guide](examples/backend_control_guide.py)** - Complete user control examples\r\n\r\n### \ud83d\udd27 **Backend Installation**\r\n\r\n```bash\r\n# Install with all backends (recommended)\r\npip install pyfunc-pipeline[all]\r\n\r\n# Or install specific backends\r\npip install pyfunc-pipeline[cpp] # C++ backend\r\npip install pyfunc-pipeline[zig] # Zig backend \r\npip install pyfunc-pipeline[rust] # Rust backend\r\npip install pyfunc-pipeline[go] # Go backend\r\n\r\n# Build backends from source\r\npython build_zig.py # Build Zig backend\r\npython build_go.py # Build Go backend\r\npython build_cpp.py # Build C++ backend\r\n```\r\n\r\n## \ud83e\udd1d Contributing\r\n\r\nContributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.\r\n\r\n## \ud83d\udcc4 License\r\n\r\nMIT License - see [LICENSE](LICENSE) file for details.\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Functional programming pipeline for Python with chainable operations, lazy evaluation, and elegant placeholder syntax",
"version": "0.4.0",
"project_urls": {
"Bug Tracker": "https://github.com/alexzzzs/PyFunc/issues",
"Changelog": "https://github.com/alexzzzs/PyFunc/blob/master/CHANGELOG.md",
"Documentation": "https://github.com/alexzzzs/PyFunc/blob/master/DOCUMENTATION.md",
"Homepage": "https://github.com/alexzzzs/PyFunc",
"Repository": "https://github.com/alexzzzs/PyFunc"
},
"split_keywords": [
"functional",
" pipeline",
" data",
" transformation",
" lazy",
" evaluation",
" placeholder"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "978d720c40d90ac04032b6fce556e78acc8a0f65e88b23d45d6da0b074d0789a",
"md5": "46c96cf74e272ecd3803f5488a29faf7",
"sha256": "41bd07f17c062fbd8d4e5d4d89171b8095487f643eebd4ec451819777aac14ef"
},
"downloads": -1,
"filename": "pyfunc_pipeline-0.4.0-cp313-cp313-win_amd64.whl",
"has_sig": false,
"md5_digest": "46c96cf74e272ecd3803f5488a29faf7",
"packagetype": "bdist_wheel",
"python_version": "cp313",
"requires_python": ">=3.8",
"size": 1088281,
"upload_time": "2025-08-05T11:38:18",
"upload_time_iso_8601": "2025-08-05T11:38:18.035132Z",
"url": "https://files.pythonhosted.org/packages/97/8d/720c40d90ac04032b6fce556e78acc8a0f65e88b23d45d6da0b074d0789a/pyfunc_pipeline-0.4.0-cp313-cp313-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "0a681e52916c95a80cde4f99e3258ad1b162d5bd1bc6cfa02c3fba81957fb6cd",
"md5": "92945d0d653e2492e36c90286de42e29",
"sha256": "dc7058e67c54e951bc61f6830befb6c6d97c406f1c76ff333065f28dd5443def"
},
"downloads": -1,
"filename": "pyfunc_pipeline-0.4.0.tar.gz",
"has_sig": false,
"md5_digest": "92945d0d653e2492e36c90286de42e29",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 1010600,
"upload_time": "2025-08-05T11:38:29",
"upload_time_iso_8601": "2025-08-05T11:38:29.311823Z",
"url": "https://files.pythonhosted.org/packages/0a/68/1e52916c95a80cde4f99e3258ad1b162d5bd1bc6cfa02c3fba81957fb6cd/pyfunc_pipeline-0.4.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-05 11:38:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "alexzzzs",
"github_project": "PyFunc",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "pyfunc-pipeline"
}
JSON
