# pyproc-worker
Python worker implementation for [pyproc](https://github.com/YuminosukeSato/pyproc) - Call Python from Go without CGO or microservices.
## Installation
```bash
pip install pyproc-worker
```
## Quick Start
Create a Python worker with your functions:
```python
from pyproc_worker import expose, run_worker
@expose
def predict(req):
"""Your ML model or Python logic here"""
return {"result": req["value"] * 2}
@expose
def process_data(req):
"""Process data with Python libraries"""
import pandas as pd
df = pd.DataFrame(req["data"])
return df.describe().to_dict()
if __name__ == "__main__":
run_worker()
```
Then call it from Go using pyproc:
```go
pool, _ := pyproc.NewPool(pyproc.PoolOptions{
Config: pyproc.PoolConfig{
Workers: 4,
MaxInFlight: 10,
},
WorkerConfig: pyproc.WorkerConfig{
SocketPath: "/tmp/pyproc.sock",
PythonExec: "python3",
WorkerScript: "worker.py",
},
}, nil)
pool.Start(ctx)
defer pool.Shutdown(ctx)
// Call Python function
input := map[string]interface{}{"value": 42}
var output map[string]interface{}
pool.Call(ctx, "predict", input, &output)
```
## Features
- **Simple decorator-based API** - Just use `@expose` to make functions callable
- **Automatic serialization** - Handles JSON serialization/deserialization
- **Built-in health checks** - Health endpoint automatically exposed
- **Graceful shutdown** - Proper cleanup on exit
- **Logging support** - Structured logging with configurable levels
## API Reference
### `@expose` Decorator
Makes a Python function callable from Go:
```python
@expose
def my_function(req):
# req is a dict containing the request data
# Return a dict that will be sent back to Go
return {"result": "success"}
```
### `run_worker(socket_path=None)`
Starts the worker and listens for requests:
```python
if __name__ == "__main__":
# Socket path from environment or command line
run_worker()
# Or specify explicitly
run_worker("/tmp/my-worker.sock")
```
## Environment Variables
- `PYPROC_SOCKET_PATH` - Unix domain socket path
- `PYPROC_LOG_LEVEL` - Logging level (debug, info, warning, error)
## Examples
### Machine Learning Model
```python
import pickle
from pyproc_worker import expose, run_worker
# Load model at startup
with open("model.pkl", "rb") as f:
model = pickle.load(f)
@expose
def predict(req):
features = req["features"]
prediction = model.predict([features])[0]
return {
"prediction": int(prediction),
"confidence": float(model.predict_proba([features])[0].max())
}
if __name__ == "__main__":
run_worker()
```
### Data Processing
```python
import pandas as pd
from pyproc_worker import expose, run_worker
@expose
def analyze_csv(req):
df = pd.DataFrame(req["data"])
return {
"mean": df.mean().to_dict(),
"std": df.std().to_dict(),
"correlation": df.corr().to_dict()
}
if __name__ == "__main__":
run_worker()
```
### Async Operations
```python
import asyncio
from pyproc_worker import expose, run_worker
@expose
async def fetch_data(req):
url = req["url"]
# Async operations work automatically
async with aiohttp.ClientSession() as session:
async with session.get(url) as response:
data = await response.json()
return {"data": data}
if __name__ == "__main__":
run_worker()
```
## Development
### Running Tests
```bash
# Install dev dependencies
pip install -e .[dev]
# Run tests
pytest
```
### Building from Source
```bash
git clone https://github.com/YuminosukeSato/pyproc
cd pyproc/worker/python
pip install -e .
```
## License
Apache 2.0 - See [LICENSE](https://github.com/YuminosukeSato/pyproc/blob/main/LICENSE) for details.
## Links
- [pyproc Go Library](https://github.com/YuminosukeSato/pyproc)
- [Documentation](https://github.com/YuminosukeSato/pyproc#readme)
- [Examples](https://github.com/YuminosukeSato/pyproc/tree/main/examples)
- [Issue Tracker](https://github.com/YuminosukeSato/pyproc/issues)
Raw data
{
"_id": null,
"home_page": null,
"name": "pyproc-worker",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.13,>=3.9",
"maintainer_email": null,
"keywords": "pyproc, worker, go, python, ipc, rpc, unix-socket",
"author": "YuminosukeSato",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/39/6e/af018d504d52451a7fa141fd400637b03d6b010b11f4c2df185e469d5ac9/pyproc_worker-0.1.0.tar.gz",
"platform": null,
"description": "# pyproc-worker\n\nPython worker implementation for [pyproc](https://github.com/YuminosukeSato/pyproc) - Call Python from Go without CGO or microservices.\n\n## Installation\n\n```bash\npip install pyproc-worker\n```\n\n## Quick Start\n\nCreate a Python worker with your functions:\n\n```python\nfrom pyproc_worker import expose, run_worker\n\n@expose\ndef predict(req):\n \"\"\"Your ML model or Python logic here\"\"\"\n return {\"result\": req[\"value\"] * 2}\n\n@expose\ndef process_data(req):\n \"\"\"Process data with Python libraries\"\"\"\n import pandas as pd\n df = pd.DataFrame(req[\"data\"])\n return df.describe().to_dict()\n\nif __name__ == \"__main__\":\n run_worker()\n```\n\nThen call it from Go using pyproc:\n\n```go\npool, _ := pyproc.NewPool(pyproc.PoolOptions{\n Config: pyproc.PoolConfig{\n Workers: 4,\n MaxInFlight: 10,\n },\n WorkerConfig: pyproc.WorkerConfig{\n SocketPath: \"/tmp/pyproc.sock\",\n PythonExec: \"python3\",\n WorkerScript: \"worker.py\",\n },\n}, nil)\n\npool.Start(ctx)\ndefer pool.Shutdown(ctx)\n\n// Call Python function\ninput := map[string]interface{}{\"value\": 42}\nvar output map[string]interface{}\npool.Call(ctx, \"predict\", input, &output)\n```\n\n## Features\n\n- **Simple decorator-based API** - Just use `@expose` to make functions callable\n- **Automatic serialization** - Handles JSON serialization/deserialization\n- **Built-in health checks** - Health endpoint automatically exposed\n- **Graceful shutdown** - Proper cleanup on exit\n- **Logging support** - Structured logging with configurable levels\n\n## API Reference\n\n### `@expose` Decorator\n\nMakes a Python function callable from Go:\n\n```python\n@expose\ndef my_function(req):\n # req is a dict containing the request data\n # Return a dict that will be sent back to Go\n return {\"result\": \"success\"}\n```\n\n### `run_worker(socket_path=None)`\n\nStarts the worker and listens for requests:\n\n```python\nif __name__ == \"__main__\":\n # Socket path from environment or command line\n run_worker()\n \n # Or specify explicitly\n run_worker(\"/tmp/my-worker.sock\")\n```\n\n## Environment Variables\n\n- `PYPROC_SOCKET_PATH` - Unix domain socket path\n- `PYPROC_LOG_LEVEL` - Logging level (debug, info, warning, error)\n\n## Examples\n\n### Machine Learning Model\n\n```python\nimport pickle\nfrom pyproc_worker import expose, run_worker\n\n# Load model at startup\nwith open(\"model.pkl\", \"rb\") as f:\n model = pickle.load(f)\n\n@expose\ndef predict(req):\n features = req[\"features\"]\n prediction = model.predict([features])[0]\n \n return {\n \"prediction\": int(prediction),\n \"confidence\": float(model.predict_proba([features])[0].max())\n }\n\nif __name__ == \"__main__\":\n run_worker()\n```\n\n### Data Processing\n\n```python\nimport pandas as pd\nfrom pyproc_worker import expose, run_worker\n\n@expose\ndef analyze_csv(req):\n df = pd.DataFrame(req[\"data\"])\n \n return {\n \"mean\": df.mean().to_dict(),\n \"std\": df.std().to_dict(),\n \"correlation\": df.corr().to_dict()\n }\n\nif __name__ == \"__main__\":\n run_worker()\n```\n\n### Async Operations\n\n```python\nimport asyncio\nfrom pyproc_worker import expose, run_worker\n\n@expose\nasync def fetch_data(req):\n url = req[\"url\"]\n # Async operations work automatically\n async with aiohttp.ClientSession() as session:\n async with session.get(url) as response:\n data = await response.json()\n \n return {\"data\": data}\n\nif __name__ == \"__main__\":\n run_worker()\n```\n\n## Development\n\n### Running Tests\n\n```bash\n# Install dev dependencies\npip install -e .[dev]\n\n# Run tests\npytest\n```\n\n### Building from Source\n\n```bash\ngit clone https://github.com/YuminosukeSato/pyproc\ncd pyproc/worker/python\npip install -e .\n```\n\n## License\n\nApache 2.0 - See [LICENSE](https://github.com/YuminosukeSato/pyproc/blob/main/LICENSE) for details.\n\n## Links\n\n- [pyproc Go Library](https://github.com/YuminosukeSato/pyproc)\n- [Documentation](https://github.com/YuminosukeSato/pyproc#readme)\n- [Examples](https://github.com/YuminosukeSato/pyproc/tree/main/examples)\n- [Issue Tracker](https://github.com/YuminosukeSato/pyproc/issues)\n",
"bugtrack_url": null,
"license": null,
"summary": "Python worker for pyproc - Call Python from Go without CGO",
"version": "0.1.0",
"project_urls": {
"Documentation": "https://github.com/YuminosukeSato/pyproc#readme",
"Homepage": "https://github.com/YuminosukeSato/pyproc",
"Issues": "https://github.com/YuminosukeSato/pyproc/issues",
"Repository": "https://github.com/YuminosukeSato/pyproc"
},
"split_keywords": [
"pyproc",
" worker",
" go",
" python",
" ipc",
" rpc",
" unix-socket"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "9ce89a6b9c32215a5dfee01174ddf02ac83ef0b40b0b30a562e744a892165310",
"md5": "f0d3efbd7fe0172726cf2d469b21e17a",
"sha256": "86e0e90f1d7469f0fbd7cce3f593d93ce3fb0b3135585ee25c7e17a178430e38"
},
"downloads": -1,
"filename": "pyproc_worker-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f0d3efbd7fe0172726cf2d469b21e17a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.13,>=3.9",
"size": 6363,
"upload_time": "2025-08-15T06:52:35",
"upload_time_iso_8601": "2025-08-15T06:52:35.001906Z",
"url": "https://files.pythonhosted.org/packages/9c/e8/9a6b9c32215a5dfee01174ddf02ac83ef0b40b0b30a562e744a892165310/pyproc_worker-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "396eaf018d504d52451a7fa141fd400637b03d6b010b11f4c2df185e469d5ac9",
"md5": "abd0b36507747f93bd0c13e0020e788a",
"sha256": "467d548e295889fbd1bad251f0008c55b2f796c05d7fe195a2f516588b0091ec"
},
"downloads": -1,
"filename": "pyproc_worker-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "abd0b36507747f93bd0c13e0020e788a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.13,>=3.9",
"size": 6348,
"upload_time": "2025-08-15T06:52:36",
"upload_time_iso_8601": "2025-08-15T06:52:36.376217Z",
"url": "https://files.pythonhosted.org/packages/39/6e/af018d504d52451a7fa141fd400637b03d6b010b11f4c2df185e469d5ac9/pyproc_worker-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-15 06:52:36",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "YuminosukeSato",
"github_project": "pyproc#readme",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "pyproc-worker"
}
JSON
