# VM on Golem Provider Node
Earn by renting out your machine’s compute — like Airbnb for servers. The Provider service runs VMs for requestors, verifies payments via streaming, and lets you withdraw earnings.
## Quick Start (Host and Earn)
1) Install (Python 3.11+ recommended):
```bash
pip install golem-vm-provider
```
2) Start the provider (testnet by default is fine):
```bash
golem-provider start --network testnet
```
Verify your environment and connectivity anytime:
```bash
golem-provider status
```
This checks Multipass availability, local/external port reachability, and whether an update is available on PyPI.
### Status Command (TTY and JSON)
Use `golem-provider status` to quickly assess health.
TTY output highlights
```
Overall Error | Issues detected | Healthy
Multipass ✅ OK | ❌ Missing
Provider Port 0.0.0.0:7466
Local ✅ service is listening | ❌ port unavailable
External ✅ reachable | ❌ unreachable — <reason>
SSH Ports <start>-<end> — OK | limited — N issue(s) | blocked
Usable free <count> # free AND externally reachable
In use <count>
Issues e.g. "100 not reachable externally" or "3 unreachable, 1 not listening"
```
Severity rules
- Overall is Error when any critical prerequisite fails:
- Provider API port not externally reachable (or external check fails).
- No externally reachable SSH ports in the configured range.
- Multipass missing or provider local port not ready.
- Otherwise it shows Issues detected or Healthy.
Machine‑readable JSON
```bash
golem-provider status --json
```
Key fields:
- `overall.status`: "healthy" | "issues" | "error"
- `overall.issues`: list of concise issue strings
- `ports.provider`:
- `port`: int, `host`: string
- `status`: "reachable" | "unreachable" (external check failures are treated as "unreachable")
- `ports.ssh`:
- `range`: [start, end)
- `status`: "ok" | "limited" | "blocked"
- `usable_free`: integer — free AND externally reachable
- `in_use`: integer
- `issues`: `{ unreachable: int, not_listening: int }`
- `ports`: array of per‑port summaries:
- `{ port: int, status: "reachable" | "unreachable" | "unknown", listening: bool }`
Notes
- The concept of "free" in JSON is replaced by `usable_free` (free + externally reachable) to avoid misleading counts when ports are blocked.
- When the external checker is unavailable, per‑port `status` is `"unknown"` and `listening` still reflects local state.
3) Set pricing in USD (GLM rates auto‑compute):
```bash
golem-provider pricing set --usd-per-core 12 --usd-per-mem 4 --usd-per-disk 0.1
```
4) On testnets, optionally fund gas for withdrawals:
```bash
golem-provider wallet faucet-l2
```
You are now discoverable to requestors and will earn as your VMs run.
## System Architecture
```mermaid
graph TB
API[FastAPI Service] --> VMM[VM Manager]
API --> RT[Resource Tracker]
API --> PV[Port Verifier]
VMM --> MP[Multipass Provider]
VMM --> PM[Proxy Manager]
RT --> RM[Resource Monitor]
RT --> AD[Resource Advertiser]
AD --> DS[Discovery Service]
PM --> SSH[SSH Proxy]
PV --> PM
MP --> VM1[VM 1]
MP --> VM2[VM 2]
```
The Provider Node implements a clean, modular architecture where each component handles a specific responsibility:
## Core Components
### Port Verification
The port verification system ensures proper network connectivity:
```mermaid
sequenceDiagram
participant S as Startup
participant PV as Port Verifier
participant PM as Port Manager
participant D as Display
S->>PV: Initialize
PV->>PV: Check Local Ports
PV->>PV: Verify External Access
PV->>D: Update Status
D-->>S: Show Progress
PV->>PM: Register Verified Ports
PM-->>S: Verification Result
```
- Comprehensive port accessibility verification
- Real-time status display with progress indicators
- Local and external port validation
- Automatic port allocation management
### Future Developments
The current port verification system uses dedicated port check servers to verify external accessibility. In future releases, this functionality will be integrated into the Golem Network's verifier nodes, providing:
- Decentralized port verification through the network
- Increased reliability with multiple verification sources
- Consensus-based verification results
- Reduced dependency on centralized services
- Enhanced security through the network's trust system
This integration aligns with Golem's vision of a fully decentralized computing platform, moving critical infrastructure services like port verification into the network itself.
### Resource Management
The resource management system ensures optimal allocation and utilization of system resources:
- Real-time monitoring of CPU, memory, and storage
- Intelligent resource allocation with minimum requirement enforcement
- Threshold-based resource protection
- Automatic resource reclamation
```mermaid
sequenceDiagram
participant API as API
participant RT as Resource Tracker
participant RM as Resource Monitor
participant AD as Advertiser
API->>RT: Request Resource Allocation
RT->>RM: Check Available Resources
RM-->>RT: Resource Status
RT->>RT: Validate Requirements
RT-->>API: Allocation Result
RT->>AD: Notify Resource Update
AD->>DS: Update Advertisement
```
### VM Management
VM operations are handled through Multipass integration:
```mermaid
sequenceDiagram
participant API as API
participant MP as Multipass
participant CI as Cloud Init
participant VM as Virtual Machine
API->>MP: Create VM Request
MP->>CI: Generate Config
CI-->>MP: SSH Configuration
MP->>VM: Launch Instance
VM-->>MP: Status Update
MP-->>API: VM Info
```
- Automated VM provisioning with cloud-init
- Secure SSH key management
- Status monitoring and health checks
- Automatic cleanup procedures
### Network Proxy System
A pure Python implementation manages SSH connections:
```mermaid
sequenceDiagram
participant C as Client
participant PM as Proxy Manager
participant P as Proxy
participant VM as Virtual Machine
C->>PM: SSH Connection
PM->>P: Create Proxy
P->>VM: Forward Connection
VM-->>P: Response
P-->>C: Forward Response
```
- Dynamic port allocation and management
- Connection state persistence
- Clean connection handling
- Automatic proxy cleanup
## Installation (from source / development)
1. Prerequisites:
- Python 3.11+
- Multipass
- Poetry (for development)
2. Install from source:
```bash
cd provider-server
poetry install
```
3. Local environment (optional):
```bash
cp .env.example .env
# Edit .env to tweak defaults if needed
```
## Configuration
Key configuration options in `.env`:
```bash
# Provider Settings
GOLEM_PROVIDER_ID="your-provider-id"
GOLEM_PROVIDER_NAME="your-provider-name"
GOLEM_PROVIDER_COUNTRY="SE"
# Resource Limits
GOLEM_PROVIDER_MAX_VMS=10
GOLEM_PROVIDER_MIN_CPU_CORES=1
GOLEM_PROVIDER_MIN_MEMORY_GB=1
GOLEM_PROVIDER_MIN_STORAGE_GB=10
# Port Verification Settings
GOLEM_PROVIDER_PORT={provider_port} # Default: 7466
GOLEM_PROVIDER_PORT_CHECK_SERVERS=[
"https://ports1.golem.network",
"https://ports2.golem.network"
]
# Network Settings
GOLEM_PROVIDER_PORT_RANGE_START={start_port} # Default: 50800
GOLEM_PROVIDER_PORT_RANGE_END={end_port} # Default: 50900
GOLEM_PROVIDER_PUBLIC_IP="auto"
# Legacy discovery (optional; not required in normal operation)
# GOLEM_PROVIDER_DISCOVERY_URL="http://discovery.golem.network:9001"
# GOLEM_PROVIDER_ADVERTISEMENT_INTERVAL=240
# Network Selection
# Adds an annotation to on-chain advertisements and can be used by requestors to filter
GOLEM_PROVIDER_NETWORK="testnet" # or "mainnet"
```
### Streaming Payments (Native ETH on L2)
Enable on‑chain stream‑gated rentals funded in native ETH. By default, the provider auto‑loads the StreamPayment contract from `contracts/deployments/l2.json` and enables payments out of the box. Configure/override (env prefix `GOLEM_PROVIDER_`):
- `POLYGON_RPC_URL` — EVM RPC URL (default L2 RPC)
- `STREAM_PAYMENT_ADDRESS` — StreamPayment address (defaults from `contracts/deployments/l2.json`)
- `GLM_TOKEN_ADDRESS` — Token address (defaults from `contracts/deployments/l2.json`; `0x0` means native ETH)
- Optional override of deployments directory: set `GOLEM_DEPLOYMENTS_DIR` to a folder containing `l2.json`.
Optional background automation (all disabled by default):
- `STREAM_MIN_REMAINING_SECONDS` — minimum remaining runway to keep a VM running (default 0)
- `STREAM_MONITOR_ENABLED` — stop VMs when remaining runway < threshold (default true)
- `STREAM_MONITOR_INTERVAL_SECONDS` — how frequently to check runway (default 30)
- `STREAM_WITHDRAW_ENABLED` — periodically withdraw vested funds (default false)
- `STREAM_WITHDRAW_INTERVAL_SECONDS` — how often to attempt withdrawals (default 1800)
- `STREAM_MIN_WITHDRAW_WEI` — only withdraw when >= this amount (gas‑aware)
Implementation notes:
- The provider exposes `GET /api/v1/provider/info` returning `provider_id`, `stream_payment_address`, and `glm_token_address`. Requestors should prefer these values when opening streams.
- On successful VM creation with a valid `stream_id`, the provider persists a VM→stream mapping in `streams.json`. This enables the background monitor to stop VMs with low remaining runway and to withdraw vested funds according to configured intervals.
- When a VM is deleted, the VM→stream mapping is cleaned up.
When enabled, the provider verifies each VM creation request’s `stream_id` and refuses to start the VM if:
- stream recipient != provider’s Ethereum address
- deposit is zero, stream not started, or stream halted
- (Optional) remaining runway < `STREAM_MIN_REMAINING_SECONDS`
## API Reference (for integrators)
### Create VM
```bash
POST /api/v1/vms
```
Request:
```json
{
"name": "my-webserver",
"cpu_cores": 2,
"memory_gb": 4,
"storage_gb": 20,
"stream_id": 123 // required when STREAM_PAYMENT_ADDRESS is set
}
```
Response:
```json
{
"id": "golem-my-webserver-20250219-130424",
"name": "my-webserver",
"status": "running",
"ip_address": "192.168.64.2",
"ssh_port": 50800,
"resources": {
"cpu": 2,
"memory": 4,
"storage": 20
}
}
```
### VM Operations
- List VMs: `GET /api/v1/vms`
- Get VM Status: `GET /api/v1/vms/{vm_id}`
- Delete VM: `DELETE /api/v1/vms/{vm_id}`
- Stop VM: `POST /api/v1/vms/{vm_id}/stop`
- Get Access Info: `GET /api/v1/vms/{vm_id}/access`
### Provider Info
```bash
GET /api/v1/provider/info
```
Response:
```json
{
"provider_id": "0xProviderEthereumAddress",
"stream_payment_address": "0xStreamPayment",
"glm_token_address": "0x0000000000000000000000000000000000000000"
}
```
Use this endpoint to discover the correct recipient for creating a GLM stream.
### Payment Streams
- Get a VM’s stream status: `GET /api/v1/vms/{vm_id}/stream`
- List all mapped streams: `GET /api/v1/payments/streams`
Response (per stream):
```json
{
"vm_id": "golem-my-webserver-20250219-130424",
"stream_id": 123,
"verified": true,
"reason": "ok",
"chain": {
"token": "0x0000000000000000000000000000000000000000",
"sender": "0x...",
"recipient": "0xProviderEthereumAddress",
"startTime": 1700000000,
"stopTime": 1700007200,
"ratePerSecond": 12345,
"deposit": 1000000000000000000,
"withdrawn": 0,
"halted": false
},
"computed": {
"now": 1700003600,
"remaining_seconds": 3600,
"vested_wei": 44442000,
"withdrawable_wei": 44442000
}
}
```
Notes:
- Endpoints return 400 when streaming is disabled (zero `STREAM_PAYMENT_ADDRESS`).
- In development mode (`GOLEM_ENVIRONMENT=development`) additional debug logs are emitted around stream verification and monitor ticks.
## Operations
### Starting the Provider
```bash
# Production mode
golem-provider start
# Development mode with extra logs and reload
# In development, providers advertise on the 'development' network and use local IPs.
GOLEM_ENVIRONMENT=development golem-provider start
```
Run as a background service (no terminal):
```bash
# Start in background and write a PID file
golem-provider start --daemon [--network testnet|mainnet]
# Stop the background process
golem-provider stop
# Check environment and port health (unchanged)
golem-provider status [--json]
```
### Mode vs. Network
- Development Mode (`GOLEM_ENVIRONMENT=development`)
- Optimizes for local iteration: enables reload + debug logging and uses local defaults (e.g., local port check servers). May derive a local/LAN IP automatically and prefix the provider name with `DEVMODE-`.
- Does not decide which chain you target.
- Network Selection (`--network` or `GOLEM_PROVIDER_NETWORK`)
- Chooses the discovery/advertisement scope: providers advertise `golem_network=development|testnet|mainnet` and requestors filter accordingly.
- Pair with appropriate RPC envs (`GOLEM_PROVIDER_GOLEM_BASE_RPC_URL`, `GOLEM_PROVIDER_GOLEM_BASE_WS_URL`).
- In development, you can supply separate dev endpoints via `GOLEM_PROVIDER_GOLEM_BASE_DEV_RPC_URL` / `GOLEM_PROVIDER_GOLEM_BASE_DEV_WS_URL` (or generic `GOLEM_BASE_DEV_*`).
- Does not change dev ergonomics (logging, reload, or port verification behavior).
- Payments Network (`GOLEM_PROVIDER_PAYMENTS_NETWORK`)
- Selects the payments chain profile (e.g., `l2.holesky`, `mainnet`). Determines default payments RPC, faucet enablement, and symbols.
Common setups:
- Local dev (separate network): `GOLEM_ENVIRONMENT=development` (defaults to `network=development`).
- Staging on testnet: keep `ENVIRONMENT=production`, set `--network testnet` and testnet RPCs.
- Production on mainnet: `ENVIRONMENT=production` with `--network mainnet` and mainnet RPCs.
The provider will:
1. Verify port accessibility
- Check discovery port (7466)
- Verify SSH ports (50800-50900)
- Display verification progress
2. Initialize resource monitoring
3. Start the proxy manager
4. Begin resource advertisement
5. Listen for VM requests
Notes:
- Advertisements include both `golem_network` (testnet/mainnet) and `golem_payments_network` (e.g., `l2.holesky`). Requestors default to matching both; they can list all payments networks with a CLI flag.
### Faucet
- L3 (Golem Base adverts): provider auto-requests funds on startup from `FAUCET_URL` (defaults to EthWarsaw Holesky) protected by CAPTCHA at `CAPTCHA_URL/05381a2cef5e`.
- L2 (payments): Use the CLI to request native ETH (enabled only on testnet profiles):
```bash
golem-provider wallet faucet-l2
```
Defaults:
- Faucet URL and enablement come from the active payments profile. On `mainnet` (or other profiles without faucet) the command is disabled.
- CAPTCHA: `https://cap.gobas.me/05381a2cef5e`
- Override with env: `GOLEM_PROVIDER_L2_FAUCET_URL`, `GOLEM_PROVIDER_L2_CAPTCHA_URL`, `GOLEM_PROVIDER_L2_CAPTCHA_API_KEY`.
### Streams (CLI)
- List all mapped streams with computed fields:
```bash
poetry run golem-provider streams list
# or JSON
poetry run golem-provider streams list --json
```
- Show one VM’s stream (VM id = `requestor_name`):
```bash
poetry run golem-provider streams show <vm_id>
```
- Summarize earnings and withdrawable amounts:
```bash
poetry run golem-provider streams earnings
# or JSON
poetry run golem-provider streams earnings --json
```
- Withdraw vested funds:
```bash
# One VM by id
poetry run golem-provider streams withdraw --vm-id <vm_id>
# All mapped streams
poetry run golem-provider streams withdraw --all
```
Note: On testnets, the withdraw command auto-attempts to fund the provider's L2 address via the configured faucet if native gas balance is low.
Configure monitor and withdraw via CLI:
```bash
# Set monitor to require 1h remaining, check every 30s
golem-provider config monitor --enable true --interval 30 --min-remaining 3600
# Enable auto-withdraw every 15 minutes when >= 1e15 wei
golem-provider config withdraw --enable true --interval 900 --min-wei 1000000000000000
```
### Resource Advertisement Flow
```mermaid
sequenceDiagram
participant P as Provider
participant RT as Resource Tracker
participant AD as Advertiser
participant DS as Discovery Service
P->>RT: Initialize
P->>RT: Sync with existing VMs
RT->>AD: Register Callback
loop Every 4 minutes
AD->>RT: Get Resources
RT-->>AD: Available Resources
AD->>DS: Post Advertisement
DS-->>AD: Confirmation
end
```
On startup, the provider syncs the resource tracker with all VMs currently running on the host (via Multipass). This ensures advertisements reflect already-allocated CPU, RAM, and storage after restarts, preventing false “outdated advertisement” updates when existing VMs are consuming resources. The sync is based on actual VMs present, independent of any still-open payment streams.
### Monitoring
The provider includes comprehensive logging:
- Resource allocation events
- VM lifecycle changes
- Network proxy operations
- Discovery service interactions
## Technical Details
### Security
- Resource isolation through Multipass
- Secure SSH key provisioning
- Connection proxying for network isolation
- Rate limiting on API endpoints
### Performance
- Asynchronous operations with FastAPI
- Efficient resource tracking
- Connection pooling for proxy servers
- Optimized VM provisioning
### Resource Protection
- CPU threshold: 90%
- Memory threshold: 85%
- Storage threshold: 90%
- Minimum resource guarantees
## Troubleshooting
Common issues and solutions:
### Port Verification Issues
1. Provider Port ({provider_port}) Issues
- Check if port is already in use
- Verify port forwarding on router
- Check firewall rules
- Ensure provider is accessible to requestors
2. VM Access Port Range ({start_port}-{end_port}) Issues
- Verify port range availability
- Check for port conflicts
- Configure router port forwarding
- Review firewall settings for range
3. External Access Issues
- Verify internet connectivity
- Check port check servers are accessible
- Review router NAT/firewall settings
- Consider using alternative port check servers
### Port Verification Monitoring
The provider includes real-time port verification status:
- Visual progress indicators
- Port accessibility status
- Critical issues detection
- Quick fix suggestions
- Links to troubleshooting documentation
Example status output:
```
Overall Healthy
Provider Port {host}:{provider_port}
Local ✅ service is listening
External ✅ reachable
SSH Ports {start_port}-{end_port_minus_one} — OK
Usable free {usable_free}
In use {in_use}
```
### Resource Allocation Issues
- Check system resource availability
- Verify minimum requirements
- Monitor resource thresholds
- Review resource allocation logs
### Discovery Service Issues
- Check network connectivity
- Verify discovery service URL
- Check advertisement interval
- Monitor advertisement responses
- Verify provider registration status
## Contributing
1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Run the tests
5. Submit a pull request
Raw data
{
"_id": null,
"home_page": "https://github.com/cryptobench/vm-on-golem",
"name": "golem-vm-provider",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.11",
"maintainer_email": null,
"keywords": "golem, vm, provider, cloud, decentralized",
"author": "Phillip Jensen",
"author_email": "phillip+vm-on-golem@golemgrid.com",
"download_url": "https://files.pythonhosted.org/packages/c1/f9/3a671b1b5a77d3db5817024fc4e7c61d887354571bd17a78f59e0f801c85/golem_vm_provider-0.1.63.tar.gz",
"platform": null,
"description": "# VM on Golem Provider Node\n\nEarn by renting out your machine\u2019s compute \u2014 like Airbnb for servers. The Provider service runs VMs for requestors, verifies payments via streaming, and lets you withdraw earnings.\n\n## Quick Start (Host and Earn)\n\n1) Install (Python 3.11+ recommended):\n\n```bash\npip install golem-vm-provider\n```\n\n2) Start the provider (testnet by default is fine):\n\n```bash\ngolem-provider start --network testnet\n```\n\nVerify your environment and connectivity anytime:\n\n```bash\ngolem-provider status\n```\nThis checks Multipass availability, local/external port reachability, and whether an update is available on PyPI.\n\n### Status Command (TTY and JSON)\n\nUse `golem-provider status` to quickly assess health.\n\nTTY output highlights\n\n```\nOverall Error | Issues detected | Healthy\n\nMultipass \u2705 OK | \u274c Missing\n\nProvider Port 0.0.0.0:7466\n Local \u2705 service is listening | \u274c port unavailable\n External \u2705 reachable | \u274c unreachable \u2014 <reason>\n\nSSH Ports <start>-<end> \u2014 OK | limited \u2014 N issue(s) | blocked\n Usable free <count> # free AND externally reachable\n In use <count>\n Issues e.g. \"100 not reachable externally\" or \"3 unreachable, 1 not listening\"\n```\n\nSeverity rules\n\n- Overall is Error when any critical prerequisite fails:\n - Provider API port not externally reachable (or external check fails).\n - No externally reachable SSH ports in the configured range.\n - Multipass missing or provider local port not ready.\n- Otherwise it shows Issues detected or Healthy.\n\nMachine\u2011readable JSON\n\n```bash\ngolem-provider status --json\n```\n\nKey fields:\n\n- `overall.status`: \"healthy\" | \"issues\" | \"error\"\n- `overall.issues`: list of concise issue strings\n- `ports.provider`:\n - `port`: int, `host`: string\n - `status`: \"reachable\" | \"unreachable\" (external check failures are treated as \"unreachable\")\n- `ports.ssh`:\n - `range`: [start, end)\n - `status`: \"ok\" | \"limited\" | \"blocked\"\n - `usable_free`: integer \u2014 free AND externally reachable\n - `in_use`: integer\n - `issues`: `{ unreachable: int, not_listening: int }`\n - `ports`: array of per\u2011port summaries:\n - `{ port: int, status: \"reachable\" | \"unreachable\" | \"unknown\", listening: bool }`\n\nNotes\n\n- The concept of \"free\" in JSON is replaced by `usable_free` (free + externally reachable) to avoid misleading counts when ports are blocked.\n- When the external checker is unavailable, per\u2011port `status` is `\"unknown\"` and `listening` still reflects local state.\n\n3) Set pricing in USD (GLM rates auto\u2011compute):\n\n```bash\ngolem-provider pricing set --usd-per-core 12 --usd-per-mem 4 --usd-per-disk 0.1\n```\n\n4) On testnets, optionally fund gas for withdrawals:\n\n```bash\ngolem-provider wallet faucet-l2\n```\n\nYou are now discoverable to requestors and will earn as your VMs run.\n\n## System Architecture\n\n```mermaid\ngraph TB\n API[FastAPI Service] --> VMM[VM Manager]\n API --> RT[Resource Tracker]\n API --> PV[Port Verifier]\n VMM --> MP[Multipass Provider]\n VMM --> PM[Proxy Manager]\n RT --> RM[Resource Monitor]\n RT --> AD[Resource Advertiser]\n AD --> DS[Discovery Service]\n PM --> SSH[SSH Proxy]\n PV --> PM\n MP --> VM1[VM 1]\n MP --> VM2[VM 2]\n```\n\nThe Provider Node implements a clean, modular architecture where each component handles a specific responsibility:\n\n## Core Components\n\n### Port Verification\n\nThe port verification system ensures proper network connectivity:\n\n```mermaid\nsequenceDiagram\n participant S as Startup\n participant PV as Port Verifier\n participant PM as Port Manager\n participant D as Display\n\n S->>PV: Initialize\n PV->>PV: Check Local Ports\n PV->>PV: Verify External Access\n PV->>D: Update Status\n D-->>S: Show Progress\n PV->>PM: Register Verified Ports\n PM-->>S: Verification Result\n```\n\n- Comprehensive port accessibility verification\n- Real-time status display with progress indicators\n- Local and external port validation\n- Automatic port allocation management\n\n### Future Developments\n\nThe current port verification system uses dedicated port check servers to verify external accessibility. In future releases, this functionality will be integrated into the Golem Network's verifier nodes, providing:\n\n- Decentralized port verification through the network\n- Increased reliability with multiple verification sources\n- Consensus-based verification results\n- Reduced dependency on centralized services\n- Enhanced security through the network's trust system\n\nThis integration aligns with Golem's vision of a fully decentralized computing platform, moving critical infrastructure services like port verification into the network itself.\n\n### Resource Management\n\nThe resource management system ensures optimal allocation and utilization of system resources:\n\n- Real-time monitoring of CPU, memory, and storage\n- Intelligent resource allocation with minimum requirement enforcement\n- Threshold-based resource protection\n- Automatic resource reclamation\n\n```mermaid\nsequenceDiagram\n participant API as API\n participant RT as Resource Tracker\n participant RM as Resource Monitor\n participant AD as Advertiser\n\n API->>RT: Request Resource Allocation\n RT->>RM: Check Available Resources\n RM-->>RT: Resource Status\n RT->>RT: Validate Requirements\n RT-->>API: Allocation Result\n RT->>AD: Notify Resource Update\n AD->>DS: Update Advertisement\n```\n\n### VM Management\n\nVM operations are handled through Multipass integration:\n\n```mermaid\nsequenceDiagram\n participant API as API\n participant MP as Multipass\n participant CI as Cloud Init\n participant VM as Virtual Machine\n\n API->>MP: Create VM Request\n MP->>CI: Generate Config\n CI-->>MP: SSH Configuration\n MP->>VM: Launch Instance\n VM-->>MP: Status Update\n MP-->>API: VM Info\n```\n\n- Automated VM provisioning with cloud-init\n- Secure SSH key management\n- Status monitoring and health checks\n- Automatic cleanup procedures\n\n### Network Proxy System\n\nA pure Python implementation manages SSH connections:\n\n```mermaid\nsequenceDiagram\n participant C as Client\n participant PM as Proxy Manager\n participant P as Proxy\n participant VM as Virtual Machine\n\n C->>PM: SSH Connection\n PM->>P: Create Proxy\n P->>VM: Forward Connection\n VM-->>P: Response\n P-->>C: Forward Response\n```\n\n- Dynamic port allocation and management\n- Connection state persistence\n- Clean connection handling\n- Automatic proxy cleanup\n\n## Installation (from source / development)\n\n1. Prerequisites:\n - Python 3.11+\n - Multipass\n - Poetry (for development)\n\n2. Install from source:\n\n```bash\ncd provider-server\npoetry install\n```\n\n3. Local environment (optional):\n\n```bash\ncp .env.example .env\n# Edit .env to tweak defaults if needed\n```\n\n## Configuration\n\nKey configuration options in `.env`:\n\n```bash\n# Provider Settings\nGOLEM_PROVIDER_ID=\"your-provider-id\"\nGOLEM_PROVIDER_NAME=\"your-provider-name\"\nGOLEM_PROVIDER_COUNTRY=\"SE\"\n\n# Resource Limits\nGOLEM_PROVIDER_MAX_VMS=10\nGOLEM_PROVIDER_MIN_CPU_CORES=1\nGOLEM_PROVIDER_MIN_MEMORY_GB=1\nGOLEM_PROVIDER_MIN_STORAGE_GB=10\n\n# Port Verification Settings\nGOLEM_PROVIDER_PORT={provider_port} # Default: 7466\nGOLEM_PROVIDER_PORT_CHECK_SERVERS=[\n \"https://ports1.golem.network\",\n \"https://ports2.golem.network\"\n]\n\n# Network Settings\nGOLEM_PROVIDER_PORT_RANGE_START={start_port} # Default: 50800\nGOLEM_PROVIDER_PORT_RANGE_END={end_port} # Default: 50900\nGOLEM_PROVIDER_PUBLIC_IP=\"auto\"\n\n# Legacy discovery (optional; not required in normal operation)\n# GOLEM_PROVIDER_DISCOVERY_URL=\"http://discovery.golem.network:9001\"\n# GOLEM_PROVIDER_ADVERTISEMENT_INTERVAL=240\n\n# Network Selection\n# Adds an annotation to on-chain advertisements and can be used by requestors to filter\nGOLEM_PROVIDER_NETWORK=\"testnet\" # or \"mainnet\"\n```\n\n### Streaming Payments (Native ETH on L2)\n\nEnable on\u2011chain stream\u2011gated rentals funded in native ETH. By default, the provider auto\u2011loads the StreamPayment contract from `contracts/deployments/l2.json` and enables payments out of the box. Configure/override (env prefix `GOLEM_PROVIDER_`):\n\n- `POLYGON_RPC_URL` \u2014 EVM RPC URL (default L2 RPC)\n- `STREAM_PAYMENT_ADDRESS` \u2014 StreamPayment address (defaults from `contracts/deployments/l2.json`)\n- `GLM_TOKEN_ADDRESS` \u2014 Token address (defaults from `contracts/deployments/l2.json`; `0x0` means native ETH)\n - Optional override of deployments directory: set `GOLEM_DEPLOYMENTS_DIR` to a folder containing `l2.json`.\n\nOptional background automation (all disabled by default):\n\n- `STREAM_MIN_REMAINING_SECONDS` \u2014 minimum remaining runway to keep a VM running (default 0)\n- `STREAM_MONITOR_ENABLED` \u2014 stop VMs when remaining runway < threshold (default true)\n- `STREAM_MONITOR_INTERVAL_SECONDS` \u2014 how frequently to check runway (default 30)\n- `STREAM_WITHDRAW_ENABLED` \u2014 periodically withdraw vested funds (default false)\n- `STREAM_WITHDRAW_INTERVAL_SECONDS` \u2014 how often to attempt withdrawals (default 1800)\n- `STREAM_MIN_WITHDRAW_WEI` \u2014 only withdraw when >= this amount (gas\u2011aware)\n\nImplementation notes:\n\n- The provider exposes `GET /api/v1/provider/info` returning `provider_id`, `stream_payment_address`, and `glm_token_address`. Requestors should prefer these values when opening streams.\n- On successful VM creation with a valid `stream_id`, the provider persists a VM\u2192stream mapping in `streams.json`. This enables the background monitor to stop VMs with low remaining runway and to withdraw vested funds according to configured intervals.\n- When a VM is deleted, the VM\u2192stream mapping is cleaned up.\n\nWhen enabled, the provider verifies each VM creation request\u2019s `stream_id` and refuses to start the VM if:\n\n- stream recipient != provider\u2019s Ethereum address\n- deposit is zero, stream not started, or stream halted\n- (Optional) remaining runway < `STREAM_MIN_REMAINING_SECONDS`\n\n## API Reference (for integrators)\n\n### Create VM\n\n```bash\nPOST /api/v1/vms\n```\n\nRequest:\n\n```json\n{\n \"name\": \"my-webserver\",\n \"cpu_cores\": 2,\n \"memory_gb\": 4,\n \"storage_gb\": 20,\n \"stream_id\": 123 // required when STREAM_PAYMENT_ADDRESS is set\n}\n```\n\nResponse:\n\n```json\n{\n \"id\": \"golem-my-webserver-20250219-130424\",\n \"name\": \"my-webserver\",\n \"status\": \"running\",\n \"ip_address\": \"192.168.64.2\",\n \"ssh_port\": 50800,\n \"resources\": {\n \"cpu\": 2,\n \"memory\": 4,\n \"storage\": 20\n }\n}\n```\n\n### VM Operations\n\n- List VMs: `GET /api/v1/vms`\n- Get VM Status: `GET /api/v1/vms/{vm_id}`\n- Delete VM: `DELETE /api/v1/vms/{vm_id}`\n- Stop VM: `POST /api/v1/vms/{vm_id}/stop`\n- Get Access Info: `GET /api/v1/vms/{vm_id}/access`\n\n### Provider Info\n\n```bash\nGET /api/v1/provider/info\n```\n\nResponse:\n\n```json\n{\n \"provider_id\": \"0xProviderEthereumAddress\",\n \"stream_payment_address\": \"0xStreamPayment\",\n \"glm_token_address\": \"0x0000000000000000000000000000000000000000\" \n \n}\n```\n\nUse this endpoint to discover the correct recipient for creating a GLM stream.\n\n### Payment Streams\n\n- Get a VM\u2019s stream status: `GET /api/v1/vms/{vm_id}/stream`\n- List all mapped streams: `GET /api/v1/payments/streams`\n\nResponse (per stream):\n\n```json\n{\n \"vm_id\": \"golem-my-webserver-20250219-130424\",\n \"stream_id\": 123,\n \"verified\": true,\n \"reason\": \"ok\",\n \"chain\": {\n \"token\": \"0x0000000000000000000000000000000000000000\",\n \"sender\": \"0x...\",\n \"recipient\": \"0xProviderEthereumAddress\",\n \"startTime\": 1700000000,\n \"stopTime\": 1700007200,\n \"ratePerSecond\": 12345,\n \"deposit\": 1000000000000000000,\n \"withdrawn\": 0,\n \"halted\": false\n },\n \"computed\": {\n \"now\": 1700003600,\n \"remaining_seconds\": 3600,\n \"vested_wei\": 44442000,\n \"withdrawable_wei\": 44442000\n }\n}\n```\n\nNotes:\n- Endpoints return 400 when streaming is disabled (zero `STREAM_PAYMENT_ADDRESS`).\n- In development mode (`GOLEM_ENVIRONMENT=development`) additional debug logs are emitted around stream verification and monitor ticks.\n\n ## Operations\n \n### Starting the Provider\n\n```bash\n# Production mode\ngolem-provider start\n\n# Development mode with extra logs and reload\n# In development, providers advertise on the 'development' network and use local IPs.\nGOLEM_ENVIRONMENT=development golem-provider start\n```\n\nRun as a background service (no terminal):\n\n```bash\n# Start in background and write a PID file\ngolem-provider start --daemon [--network testnet|mainnet]\n\n# Stop the background process\ngolem-provider stop\n\n# Check environment and port health (unchanged)\ngolem-provider status [--json]\n```\n\n### Mode vs. Network\n\n- Development Mode (`GOLEM_ENVIRONMENT=development`)\n - Optimizes for local iteration: enables reload + debug logging and uses local defaults (e.g., local port check servers). May derive a local/LAN IP automatically and prefix the provider name with `DEVMODE-`.\n - Does not decide which chain you target.\n\n- Network Selection (`--network` or `GOLEM_PROVIDER_NETWORK`)\n - Chooses the discovery/advertisement scope: providers advertise `golem_network=development|testnet|mainnet` and requestors filter accordingly.\n - Pair with appropriate RPC envs (`GOLEM_PROVIDER_GOLEM_BASE_RPC_URL`, `GOLEM_PROVIDER_GOLEM_BASE_WS_URL`).\n - In development, you can supply separate dev endpoints via `GOLEM_PROVIDER_GOLEM_BASE_DEV_RPC_URL` / `GOLEM_PROVIDER_GOLEM_BASE_DEV_WS_URL` (or generic `GOLEM_BASE_DEV_*`).\n - Does not change dev ergonomics (logging, reload, or port verification behavior).\n\n- Payments Network (`GOLEM_PROVIDER_PAYMENTS_NETWORK`)\n - Selects the payments chain profile (e.g., `l2.holesky`, `mainnet`). Determines default payments RPC, faucet enablement, and symbols.\n\nCommon setups:\n- Local dev (separate network): `GOLEM_ENVIRONMENT=development` (defaults to `network=development`).\n- Staging on testnet: keep `ENVIRONMENT=production`, set `--network testnet` and testnet RPCs.\n- Production on mainnet: `ENVIRONMENT=production` with `--network mainnet` and mainnet RPCs.\n\nThe provider will:\n\n1. Verify port accessibility\n - Check discovery port (7466)\n - Verify SSH ports (50800-50900)\n - Display verification progress\n2. Initialize resource monitoring\n3. Start the proxy manager\n4. Begin resource advertisement\n5. Listen for VM requests\n\nNotes:\n- Advertisements include both `golem_network` (testnet/mainnet) and `golem_payments_network` (e.g., `l2.holesky`). Requestors default to matching both; they can list all payments networks with a CLI flag.\n\n### Faucet\n\n- L3 (Golem Base adverts): provider auto-requests funds on startup from `FAUCET_URL` (defaults to EthWarsaw Holesky) protected by CAPTCHA at `CAPTCHA_URL/05381a2cef5e`.\n- L2 (payments): Use the CLI to request native ETH (enabled only on testnet profiles):\n\n```bash\ngolem-provider wallet faucet-l2\n```\n\nDefaults:\n- Faucet URL and enablement come from the active payments profile. On `mainnet` (or other profiles without faucet) the command is disabled.\n- CAPTCHA: `https://cap.gobas.me/05381a2cef5e`\n- Override with env: `GOLEM_PROVIDER_L2_FAUCET_URL`, `GOLEM_PROVIDER_L2_CAPTCHA_URL`, `GOLEM_PROVIDER_L2_CAPTCHA_API_KEY`.\n\n### Streams (CLI)\n\n- List all mapped streams with computed fields:\n\n```bash\npoetry run golem-provider streams list\n# or JSON\npoetry run golem-provider streams list --json\n```\n\n- Show one VM\u2019s stream (VM id = `requestor_name`):\n\n```bash\npoetry run golem-provider streams show <vm_id>\n```\n\n- Summarize earnings and withdrawable amounts:\n\n```bash\npoetry run golem-provider streams earnings\n# or JSON\npoetry run golem-provider streams earnings --json\n```\n\n- Withdraw vested funds:\n\n```bash\n# One VM by id\npoetry run golem-provider streams withdraw --vm-id <vm_id>\n\n# All mapped streams\npoetry run golem-provider streams withdraw --all\n```\n\nNote: On testnets, the withdraw command auto-attempts to fund the provider's L2 address via the configured faucet if native gas balance is low.\n\nConfigure monitor and withdraw via CLI:\n\n```bash\n# Set monitor to require 1h remaining, check every 30s\ngolem-provider config monitor --enable true --interval 30 --min-remaining 3600\n\n# Enable auto-withdraw every 15 minutes when >= 1e15 wei\ngolem-provider config withdraw --enable true --interval 900 --min-wei 1000000000000000\n```\n\n### Resource Advertisement Flow\n\n```mermaid\nsequenceDiagram\n participant P as Provider\n participant RT as Resource Tracker\n participant AD as Advertiser\n participant DS as Discovery Service\n\n P->>RT: Initialize\n P->>RT: Sync with existing VMs\n RT->>AD: Register Callback\n loop Every 4 minutes\n AD->>RT: Get Resources\n RT-->>AD: Available Resources\n AD->>DS: Post Advertisement\n DS-->>AD: Confirmation\n end\n```\n\nOn startup, the provider syncs the resource tracker with all VMs currently running on the host (via Multipass). This ensures advertisements reflect already-allocated CPU, RAM, and storage after restarts, preventing false \u201coutdated advertisement\u201d updates when existing VMs are consuming resources. The sync is based on actual VMs present, independent of any still-open payment streams.\n\n### Monitoring\n\nThe provider includes comprehensive logging:\n\n- Resource allocation events\n- VM lifecycle changes\n- Network proxy operations\n- Discovery service interactions\n\n## Technical Details\n\n### Security\n\n- Resource isolation through Multipass\n- Secure SSH key provisioning\n- Connection proxying for network isolation\n- Rate limiting on API endpoints\n\n### Performance\n\n- Asynchronous operations with FastAPI\n- Efficient resource tracking\n- Connection pooling for proxy servers\n- Optimized VM provisioning\n\n### Resource Protection\n\n- CPU threshold: 90%\n- Memory threshold: 85%\n- Storage threshold: 90%\n- Minimum resource guarantees\n\n## Troubleshooting\n\nCommon issues and solutions:\n\n### Port Verification Issues\n\n1. Provider Port ({provider_port}) Issues\n\n - Check if port is already in use\n - Verify port forwarding on router\n - Check firewall rules\n - Ensure provider is accessible to requestors\n\n2. VM Access Port Range ({start_port}-{end_port}) Issues\n\n - Verify port range availability\n - Check for port conflicts\n - Configure router port forwarding\n - Review firewall settings for range\n\n3. External Access Issues\n - Verify internet connectivity\n - Check port check servers are accessible\n - Review router NAT/firewall settings\n - Consider using alternative port check servers\n\n### Port Verification Monitoring\n\nThe provider includes real-time port verification status:\n\n- Visual progress indicators\n- Port accessibility status\n- Critical issues detection\n- Quick fix suggestions\n- Links to troubleshooting documentation\n\nExample status output:\n\n```\nOverall Healthy\n\nProvider Port {host}:{provider_port}\n Local \u2705 service is listening\n External \u2705 reachable\n\nSSH Ports {start_port}-{end_port_minus_one} \u2014 OK\n Usable free {usable_free}\n In use {in_use}\n```\n\n### Resource Allocation Issues\n\n- Check system resource availability\n- Verify minimum requirements\n- Monitor resource thresholds\n- Review resource allocation logs\n\n### Discovery Service Issues\n\n- Check network connectivity\n- Verify discovery service URL\n- Check advertisement interval\n- Monitor advertisement responses\n- Verify provider registration status\n\n## Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Make your changes\n4. Run the tests\n5. Submit a pull request\n",
"bugtrack_url": null,
"license": null,
"summary": "VM on Golem Provider Node - Run your own provider node to offer VMs on the Golem Network",
"version": "0.1.63",
"project_urls": {
"Homepage": "https://github.com/cryptobench/vm-on-golem",
"Repository": "https://github.com/cryptobench/vm-on-golem"
},
"split_keywords": [
"golem",
" vm",
" provider",
" cloud",
" decentralized"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "99271430870671ae79c9ba04a78e19bb7edd0f9f1524ba16c6d2e80a5a18fe6c",
"md5": "6f4f075177284140f29eda4791a2d272",
"sha256": "5be0dec0bce874345436e4ae330d538ceadde67591dceb6a9f6d23dbcb0f01af"
},
"downloads": -1,
"filename": "golem_vm_provider-0.1.63-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6f4f075177284140f29eda4791a2d272",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.11",
"size": 94223,
"upload_time": "2025-09-13T08:08:18",
"upload_time_iso_8601": "2025-09-13T08:08:18.968326Z",
"url": "https://files.pythonhosted.org/packages/99/27/1430870671ae79c9ba04a78e19bb7edd0f9f1524ba16c6d2e80a5a18fe6c/golem_vm_provider-0.1.63-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c1f93a671b1b5a77d3db5817024fc4e7c61d887354571bd17a78f59e0f801c85",
"md5": "eaf1d58ddfbc0c7230aa396ebf515eff",
"sha256": "26ba53f2288974b0e22f6530a083aaa446ddf48c691a195f6e8e521c8a2b7ce8"
},
"downloads": -1,
"filename": "golem_vm_provider-0.1.63.tar.gz",
"has_sig": false,
"md5_digest": "eaf1d58ddfbc0c7230aa396ebf515eff",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.11",
"size": 83189,
"upload_time": "2025-09-13T08:08:20",
"upload_time_iso_8601": "2025-09-13T08:08:20.343317Z",
"url": "https://files.pythonhosted.org/packages/c1/f9/3a671b1b5a77d3db5817024fc4e7c61d887354571bd17a78f59e0f801c85/golem_vm_provider-0.1.63.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-09-13 08:08:20",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cryptobench",
"github_project": "vm-on-golem",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "golem-vm-provider"
}
JSON
