# VM on Golem Provider Node
The Provider Node enables participation in the Golem Network by managing virtual machines and computing resources. It handles VM lifecycle management, resource allocation, network proxying, and automated discovery advertisement.
## System Architecture
```mermaid
graph TB
API[FastAPI Service] --> VMM[VM Manager]
API --> RT[Resource Tracker]
API --> PV[Port Verifier]
VMM --> MP[Multipass Provider]
VMM --> PM[Proxy Manager]
RT --> RM[Resource Monitor]
RT --> AD[Resource Advertiser]
AD --> DS[Discovery Service]
PM --> SSH[SSH Proxy]
PV --> PM
MP --> VM1[VM 1]
MP --> VM2[VM 2]
```
The Provider Node implements a clean, modular architecture where each component handles a specific responsibility:
## Core Components
### Port Verification
The port verification system ensures proper network connectivity:
```mermaid
sequenceDiagram
participant S as Startup
participant PV as Port Verifier
participant PM as Port Manager
participant D as Display
S->>PV: Initialize
PV->>PV: Check Local Ports
PV->>PV: Verify External Access
PV->>D: Update Status
D-->>S: Show Progress
PV->>PM: Register Verified Ports
PM-->>S: Verification Result
```
- Comprehensive port accessibility verification
- Real-time status display with progress indicators
- Local and external port validation
- Automatic port allocation management
### Future Developments
The current port verification system uses dedicated port check servers to verify external accessibility. In future releases, this functionality will be integrated into the Golem Network's verifier nodes, providing:
- Decentralized port verification through the network
- Increased reliability with multiple verification sources
- Consensus-based verification results
- Reduced dependency on centralized services
- Enhanced security through the network's trust system
This integration aligns with Golem's vision of a fully decentralized computing platform, moving critical infrastructure services like port verification into the network itself.
### Resource Management
The resource management system ensures optimal allocation and utilization of system resources:
- Real-time monitoring of CPU, memory, and storage
- Intelligent resource allocation with minimum requirement enforcement
- Threshold-based resource protection
- Automatic resource reclamation
```mermaid
sequenceDiagram
participant API as API
participant RT as Resource Tracker
participant RM as Resource Monitor
participant AD as Advertiser
API->>RT: Request Resource Allocation
RT->>RM: Check Available Resources
RM-->>RT: Resource Status
RT->>RT: Validate Requirements
RT-->>API: Allocation Result
RT->>AD: Notify Resource Update
AD->>DS: Update Advertisement
```
### VM Management
VM operations are handled through Multipass integration:
```mermaid
sequenceDiagram
participant API as API
participant MP as Multipass
participant CI as Cloud Init
participant VM as Virtual Machine
API->>MP: Create VM Request
MP->>CI: Generate Config
CI-->>MP: SSH Configuration
MP->>VM: Launch Instance
VM-->>MP: Status Update
MP-->>API: VM Info
```
- Automated VM provisioning with cloud-init
- Secure SSH key management
- Status monitoring and health checks
- Automatic cleanup procedures
### Network Proxy System
A pure Python implementation manages SSH connections:
```mermaid
sequenceDiagram
participant C as Client
participant PM as Proxy Manager
participant P as Proxy
participant VM as Virtual Machine
C->>PM: SSH Connection
PM->>P: Create Proxy
P->>VM: Forward Connection
VM-->>P: Response
P-->>C: Forward Response
```
- Dynamic port allocation and management
- Connection state persistence
- Clean connection handling
- Automatic proxy cleanup
## Installation
1. Prerequisites:
- Python 3.9+
- Multipass
- Poetry
2. Install dependencies:
```bash
cd provider-server
poetry install
```
3. Configure environment:
```bash
cp .env.example .env
# Edit .env with your settings
```
## Configuration
Key configuration options in `.env`:
```bash
# Provider Settings
GOLEM_PROVIDER_ID="your-provider-id"
GOLEM_PROVIDER_NAME="your-provider-name"
GOLEM_PROVIDER_COUNTRY="SE"
# Resource Limits
GOLEM_PROVIDER_MAX_VMS=10
GOLEM_PROVIDER_MIN_CPU_CORES=1
GOLEM_PROVIDER_MIN_MEMORY_GB=1
GOLEM_PROVIDER_MIN_STORAGE_GB=10
# Port Verification Settings
GOLEM_PROVIDER_PORT={provider_port} # Default: 7466
GOLEM_PROVIDER_PORT_CHECK_SERVERS=[
"https://ports1.golem.network",
"https://ports2.golem.network"
]
# Network Settings
GOLEM_PROVIDER_PORT_RANGE_START={start_port} # Default: 50800
GOLEM_PROVIDER_PORT_RANGE_END={end_port} # Default: 50900
GOLEM_PROVIDER_PUBLIC_IP="auto"
# Discovery Settings
GOLEM_PROVIDER_DISCOVERY_URL="http://discovery.golem.network:9001"
GOLEM_PROVIDER_ADVERTISEMENT_INTERVAL=240
```
## API Reference
### Create VM
```bash
POST /api/v1/vms
```
Request:
```json
{
"name": "my-webserver",
"cpu_cores": 2,
"memory_gb": 4,
"storage_gb": 20
}
```
Response:
```json
{
"id": "golem-my-webserver-20250219-130424",
"name": "my-webserver",
"status": "running",
"ip_address": "192.168.64.2",
"ssh_port": 50800,
"resources": {
"cpu": 2,
"memory": 4,
"storage": 20
}
}
```
### VM Operations
- List VMs: `GET /api/v1/vms`
- Get VM Status: `GET /api/v1/vms/{vm_id}`
- Delete VM: `DELETE /api/v1/vms/{vm_id}`
- Get Access Info: `GET /api/v1/vms/{vm_id}/access`
## Operations
### Starting the Provider
```bash
poetry run python run.py
```
The provider will:
1. Verify port accessibility
- Check discovery port (7466)
- Verify SSH ports (50800-50900)
- Display verification progress
2. Initialize resource monitoring
3. Start the proxy manager
4. Begin resource advertisement
5. Listen for VM requests
### Resource Advertisement Flow
```mermaid
sequenceDiagram
participant P as Provider
participant RT as Resource Tracker
participant AD as Advertiser
participant DS as Discovery Service
P->>RT: Initialize
RT->>AD: Register Callback
loop Every 4 minutes
AD->>RT: Get Resources
RT-->>AD: Available Resources
AD->>DS: Post Advertisement
DS-->>AD: Confirmation
end
```
### Monitoring
The provider includes comprehensive logging:
- Resource allocation events
- VM lifecycle changes
- Network proxy operations
- Discovery service interactions
## Technical Details
### Security
- Resource isolation through Multipass
- Secure SSH key provisioning
- Connection proxying for network isolation
- Rate limiting on API endpoints
### Performance
- Asynchronous operations with FastAPI
- Efficient resource tracking
- Connection pooling for proxy servers
- Optimized VM provisioning
### Resource Protection
- CPU threshold: 90%
- Memory threshold: 85%
- Storage threshold: 90%
- Minimum resource guarantees
## Troubleshooting
Common issues and solutions:
### Port Verification Issues
1. Provider Port ({provider_port}) Issues
- Check if port is already in use
- Verify port forwarding on router
- Check firewall rules
- Ensure provider is accessible to requestors
2. VM Access Port Range ({start_port}-{end_port}) Issues
- Verify port range availability
- Check for port conflicts
- Configure router port forwarding
- Review firewall settings for range
3. External Access Issues
- Verify internet connectivity
- Check port check servers are accessible
- Review router NAT/firewall settings
- Consider using alternative port check servers
### Port Verification Monitoring
The provider includes real-time port verification status:
- Visual progress indicators
- Port accessibility status
- Critical issues detection
- Quick fix suggestions
- Links to troubleshooting documentation
Example status output:
```bash
🌟 Port Verification Status
==========================
[✅] Provider Port {provider_port}: External ✓ | Internal ✓
[✅] VM Access Ports: 3 ports available ({start_port}-{start_port+2})
[✅] Overall Status: Provider Ready
└─ Can handle up to {n} concurrent VMs
```
### Resource Allocation Issues
- Check system resource availability
- Verify minimum requirements
- Monitor resource thresholds
- Review resource allocation logs
### Discovery Service Issues
- Check network connectivity
- Verify discovery service URL
- Check advertisement interval
- Monitor advertisement responses
- Verify provider registration status
## Contributing
1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Run the tests
5. Submit a pull request
Raw data
{
"_id": null,
"home_page": "https://github.com/cryptobench/vm-on-golem",
"name": "golem-vm-provider",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.9",
"maintainer_email": null,
"keywords": "golem, vm, provider, cloud, decentralized",
"author": "Phillip Jensen",
"author_email": "phillip+vm-on-golem@golemgrid.com",
"download_url": "https://files.pythonhosted.org/packages/a3/75/837e072627da3dd6030042d5df0a16af8e86146b96d0e67316302828cf34/golem_vm_provider-0.1.21.tar.gz",
"platform": null,
"description": "# VM on Golem Provider Node\n\nThe Provider Node enables participation in the Golem Network by managing virtual machines and computing resources. It handles VM lifecycle management, resource allocation, network proxying, and automated discovery advertisement.\n\n## System Architecture\n\n```mermaid\ngraph TB\n API[FastAPI Service] --> VMM[VM Manager]\n API --> RT[Resource Tracker]\n API --> PV[Port Verifier]\n VMM --> MP[Multipass Provider]\n VMM --> PM[Proxy Manager]\n RT --> RM[Resource Monitor]\n RT --> AD[Resource Advertiser]\n AD --> DS[Discovery Service]\n PM --> SSH[SSH Proxy]\n PV --> PM\n MP --> VM1[VM 1]\n MP --> VM2[VM 2]\n```\n\nThe Provider Node implements a clean, modular architecture where each component handles a specific responsibility:\n\n## Core Components\n\n### Port Verification\n\nThe port verification system ensures proper network connectivity:\n\n```mermaid\nsequenceDiagram\n participant S as Startup\n participant PV as Port Verifier\n participant PM as Port Manager\n participant D as Display\n\n S->>PV: Initialize\n PV->>PV: Check Local Ports\n PV->>PV: Verify External Access\n PV->>D: Update Status\n D-->>S: Show Progress\n PV->>PM: Register Verified Ports\n PM-->>S: Verification Result\n```\n\n- Comprehensive port accessibility verification\n- Real-time status display with progress indicators\n- Local and external port validation\n- Automatic port allocation management\n\n### Future Developments\n\nThe current port verification system uses dedicated port check servers to verify external accessibility. In future releases, this functionality will be integrated into the Golem Network's verifier nodes, providing:\n\n- Decentralized port verification through the network\n- Increased reliability with multiple verification sources\n- Consensus-based verification results\n- Reduced dependency on centralized services\n- Enhanced security through the network's trust system\n\nThis integration aligns with Golem's vision of a fully decentralized computing platform, moving critical infrastructure services like port verification into the network itself.\n\n### Resource Management\n\nThe resource management system ensures optimal allocation and utilization of system resources:\n\n- Real-time monitoring of CPU, memory, and storage\n- Intelligent resource allocation with minimum requirement enforcement\n- Threshold-based resource protection\n- Automatic resource reclamation\n\n```mermaid\nsequenceDiagram\n participant API as API\n participant RT as Resource Tracker\n participant RM as Resource Monitor\n participant AD as Advertiser\n\n API->>RT: Request Resource Allocation\n RT->>RM: Check Available Resources\n RM-->>RT: Resource Status\n RT->>RT: Validate Requirements\n RT-->>API: Allocation Result\n RT->>AD: Notify Resource Update\n AD->>DS: Update Advertisement\n```\n\n### VM Management\n\nVM operations are handled through Multipass integration:\n\n```mermaid\nsequenceDiagram\n participant API as API\n participant MP as Multipass\n participant CI as Cloud Init\n participant VM as Virtual Machine\n\n API->>MP: Create VM Request\n MP->>CI: Generate Config\n CI-->>MP: SSH Configuration\n MP->>VM: Launch Instance\n VM-->>MP: Status Update\n MP-->>API: VM Info\n```\n\n- Automated VM provisioning with cloud-init\n- Secure SSH key management\n- Status monitoring and health checks\n- Automatic cleanup procedures\n\n### Network Proxy System\n\nA pure Python implementation manages SSH connections:\n\n```mermaid\nsequenceDiagram\n participant C as Client\n participant PM as Proxy Manager\n participant P as Proxy\n participant VM as Virtual Machine\n\n C->>PM: SSH Connection\n PM->>P: Create Proxy\n P->>VM: Forward Connection\n VM-->>P: Response\n P-->>C: Forward Response\n```\n\n- Dynamic port allocation and management\n- Connection state persistence\n- Clean connection handling\n- Automatic proxy cleanup\n\n## Installation\n\n1. Prerequisites:\n\n - Python 3.9+\n - Multipass\n - Poetry\n\n2. Install dependencies:\n\n ```bash\n cd provider-server\n poetry install\n ```\n\n3. Configure environment:\n ```bash\n cp .env.example .env\n # Edit .env with your settings\n ```\n\n## Configuration\n\nKey configuration options in `.env`:\n\n```bash\n# Provider Settings\nGOLEM_PROVIDER_ID=\"your-provider-id\"\nGOLEM_PROVIDER_NAME=\"your-provider-name\"\nGOLEM_PROVIDER_COUNTRY=\"SE\"\n\n# Resource Limits\nGOLEM_PROVIDER_MAX_VMS=10\nGOLEM_PROVIDER_MIN_CPU_CORES=1\nGOLEM_PROVIDER_MIN_MEMORY_GB=1\nGOLEM_PROVIDER_MIN_STORAGE_GB=10\n\n# Port Verification Settings\nGOLEM_PROVIDER_PORT={provider_port} # Default: 7466\nGOLEM_PROVIDER_PORT_CHECK_SERVERS=[\n \"https://ports1.golem.network\",\n \"https://ports2.golem.network\"\n]\n\n# Network Settings\nGOLEM_PROVIDER_PORT_RANGE_START={start_port} # Default: 50800\nGOLEM_PROVIDER_PORT_RANGE_END={end_port} # Default: 50900\nGOLEM_PROVIDER_PUBLIC_IP=\"auto\"\n\n# Discovery Settings\nGOLEM_PROVIDER_DISCOVERY_URL=\"http://discovery.golem.network:9001\"\nGOLEM_PROVIDER_ADVERTISEMENT_INTERVAL=240\n```\n\n## API Reference\n\n### Create VM\n\n```bash\nPOST /api/v1/vms\n```\n\nRequest:\n\n```json\n{\n \"name\": \"my-webserver\",\n \"cpu_cores\": 2,\n \"memory_gb\": 4,\n \"storage_gb\": 20\n}\n```\n\nResponse:\n\n```json\n{\n \"id\": \"golem-my-webserver-20250219-130424\",\n \"name\": \"my-webserver\",\n \"status\": \"running\",\n \"ip_address\": \"192.168.64.2\",\n \"ssh_port\": 50800,\n \"resources\": {\n \"cpu\": 2,\n \"memory\": 4,\n \"storage\": 20\n }\n}\n```\n\n### VM Operations\n\n- List VMs: `GET /api/v1/vms`\n- Get VM Status: `GET /api/v1/vms/{vm_id}`\n- Delete VM: `DELETE /api/v1/vms/{vm_id}`\n- Get Access Info: `GET /api/v1/vms/{vm_id}/access`\n\n## Operations\n\n### Starting the Provider\n\n```bash\npoetry run python run.py\n```\n\nThe provider will:\n\n1. Verify port accessibility\n - Check discovery port (7466)\n - Verify SSH ports (50800-50900)\n - Display verification progress\n2. Initialize resource monitoring\n3. Start the proxy manager\n4. Begin resource advertisement\n5. Listen for VM requests\n\n### Resource Advertisement Flow\n\n```mermaid\nsequenceDiagram\n participant P as Provider\n participant RT as Resource Tracker\n participant AD as Advertiser\n participant DS as Discovery Service\n\n P->>RT: Initialize\n RT->>AD: Register Callback\n loop Every 4 minutes\n AD->>RT: Get Resources\n RT-->>AD: Available Resources\n AD->>DS: Post Advertisement\n DS-->>AD: Confirmation\n end\n```\n\n### Monitoring\n\nThe provider includes comprehensive logging:\n\n- Resource allocation events\n- VM lifecycle changes\n- Network proxy operations\n- Discovery service interactions\n\n## Technical Details\n\n### Security\n\n- Resource isolation through Multipass\n- Secure SSH key provisioning\n- Connection proxying for network isolation\n- Rate limiting on API endpoints\n\n### Performance\n\n- Asynchronous operations with FastAPI\n- Efficient resource tracking\n- Connection pooling for proxy servers\n- Optimized VM provisioning\n\n### Resource Protection\n\n- CPU threshold: 90%\n- Memory threshold: 85%\n- Storage threshold: 90%\n- Minimum resource guarantees\n\n## Troubleshooting\n\nCommon issues and solutions:\n\n### Port Verification Issues\n\n1. Provider Port ({provider_port}) Issues\n\n - Check if port is already in use\n - Verify port forwarding on router\n - Check firewall rules\n - Ensure provider is accessible to requestors\n\n2. VM Access Port Range ({start_port}-{end_port}) Issues\n\n - Verify port range availability\n - Check for port conflicts\n - Configure router port forwarding\n - Review firewall settings for range\n\n3. External Access Issues\n - Verify internet connectivity\n - Check port check servers are accessible\n - Review router NAT/firewall settings\n - Consider using alternative port check servers\n\n### Port Verification Monitoring\n\nThe provider includes real-time port verification status:\n\n- Visual progress indicators\n- Port accessibility status\n- Critical issues detection\n- Quick fix suggestions\n- Links to troubleshooting documentation\n\nExample status output:\n\n```bash\n\ud83c\udf1f Port Verification Status\n==========================\n[\u2705] Provider Port {provider_port}: External \u2713 | Internal \u2713\n[\u2705] VM Access Ports: 3 ports available ({start_port}-{start_port+2})\n[\u2705] Overall Status: Provider Ready\n\u2514\u2500 Can handle up to {n} concurrent VMs\n```\n\n### Resource Allocation Issues\n\n- Check system resource availability\n- Verify minimum requirements\n- Monitor resource thresholds\n- Review resource allocation logs\n\n### Discovery Service Issues\n\n- Check network connectivity\n- Verify discovery service URL\n- Check advertisement interval\n- Monitor advertisement responses\n- Verify provider registration status\n\n## Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Make your changes\n4. Run the tests\n5. Submit a pull request\n",
"bugtrack_url": null,
"license": null,
"summary": "VM on Golem Provider Node - Run your own provider node to offer VMs on the Golem Network",
"version": "0.1.21",
"project_urls": {
"Homepage": "https://github.com/cryptobench/vm-on-golem",
"Repository": "https://github.com/cryptobench/vm-on-golem"
},
"split_keywords": [
"golem",
" vm",
" provider",
" cloud",
" decentralized"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "28cca2ca48b1e807b63f08402e0e63ea1133de1c54027da051426bff0e153984",
"md5": "1965117d61e827de5f16b08c81d207ef",
"sha256": "3ac22deb3fc1cbe62e8799a841ab152fd531aa99024e888d0cc5666537baf2cb"
},
"downloads": -1,
"filename": "golem_vm_provider-0.1.21-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1965117d61e827de5f16b08c81d207ef",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.9",
"size": 45568,
"upload_time": "2025-02-20T23:54:48",
"upload_time_iso_8601": "2025-02-20T23:54:48.490183Z",
"url": "https://files.pythonhosted.org/packages/28/cc/a2ca48b1e807b63f08402e0e63ea1133de1c54027da051426bff0e153984/golem_vm_provider-0.1.21-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a375837e072627da3dd6030042d5df0a16af8e86146b96d0e67316302828cf34",
"md5": "4bbd27b7d788b98eb958ef6b8d34e6b3",
"sha256": "f4d5448d241607304563bb5490ae930cc9482c7a1bc8eae969572e2581f615e0"
},
"downloads": -1,
"filename": "golem_vm_provider-0.1.21.tar.gz",
"has_sig": false,
"md5_digest": "4bbd27b7d788b98eb958ef6b8d34e6b3",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.9",
"size": 38246,
"upload_time": "2025-02-20T23:54:52",
"upload_time_iso_8601": "2025-02-20T23:54:52.850423Z",
"url": "https://files.pythonhosted.org/packages/a3/75/837e072627da3dd6030042d5df0a16af8e86146b96d0e67316302828cf34/golem_vm_provider-0.1.21.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-20 23:54:52",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cryptobench",
"github_project": "vm-on-golem",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "golem-vm-provider"
}
JSON
