# TCP over WebSocket
```
┌───────────────────────────────────────────────────────────────────────────────┐
│ TCP-over-WebSocket Architecture │
└───────────────────────────────────────────────────────────────────────────────┘
┌────────────────┐ ┌─────────────────┐ ┌────────────────┐ ┌────────────────┐
│ External XC1 │ │ External XC2 │ │ External XC3 │ │ External XC4 │
│ Site A │ │ Site A │ │ Site B │ │ Site B │
└────────────────┘ └─────────────────┘ └────────────────┘ └────────────────┘
send/receive send/receive send/receive send/receive
│ ▲ │ ▲
▼ │ ▼ │
---------------------------- Client Services ------------------------------
┌────────────────┐ ┌─────────────────┐ ┌────────────────┐ ┌────────────────┐
│ Tunnel 1 │ │ Tunnel 2 │ │ Tunnel 1 │ │ Tunnel 2 │
│ Listen │ │ Connect │ │ Listen │ │ Connect │
│ Socket │ │ Socket │ │ Socket │ │ Socket │
└────────────────┘ └─────────────────┘ └────────────────┘ └────────────────┘
│ │ │ │
┌────────────────────────────────────┐ ┌───────────────────────────────────┐
│ Client 1 │ │ Client 2 │
│ (Primary) │ │ (Secondary) │
└────────────────────────────────────┘ └───────────────────────────────────┘
--------------------------- Client Services ------------------------------
│ │
▼ ▼
┌──────────────────────────────────┐ ┌──────────────────────────────────┐
│ WebSocket 1 │ │ WebSocket 2 │
│ (Client 1 Connection) │ │ (Client 2 Connection) │
└──────────────────────────────────┘ └──────────────────────────────────┘
│ │
▼ ▼
---------------------------- Server Service ------------------------------
┌──────────────────────────────────────────────────────────────────────────────┐
│ SERVER │
│ (Routes between clients) │
│ Active client failover logic │
└──────────────────────────────────────────────────────────────────────────────┘
│ │
┌──────────────────┐ ┌──────────────────┐
│ Tunnel 1 │ │ Tunnel 2 │
│ Connect │ │ Listen │
│ Socket │ │ Socket │
└──────────────────┘ └──────────────────┘
---------------------------- Server Service ------------------------------
│ ▲
▼ │
send/receive send/receive
┌──────────────────┐ ┌──────────────────┐
│ External XS1 │ │ External XS2 │
│ Other Site │ │ Other Site │
└──────────────────┘ └──────────────────┘
Legend:
────► Direction of Connection
```
TCP over WebSocket is a Python 3.9+ service that provides high-availability TCP tunneling through WebSocket connections. It enables secure, reliable forwarding of TCP traffic through HTTP/HTTPS WebSocket connections with automatic failover between two clients.
## Overview
This package provides TCP-over-WebSocket tunneling with high availability failover:
- **Tunnel TCP connections** through HTTP WebSocket connections
- **Multiplex multiple TCP streams** over a single WebSocket connection
- **OPTIONALLY secured** with HTTPS and mutual TLS client certificate authentication
- **High availability** with automatic failover between two client endpoints
- **Windows service support** for production deployments
External TCP sockets connect to TCP ports on the server, which tunnels the traffic over WebSocket to TCP tunnel endpoints running on clients. TCP tunnels are bidirectional once the WebSocket connection is established.
Each TCP tunnel is defined by a `tunnelName` - the server side listens for TCP connections while the client side connects to external TCP sockets.
## Architecture
The service implements a server + two-client architecture for TCP tunneling over WebSocket with configurable tunnels:
- **Server**: Hosts TCP tunnel listen endpoints where external applications connect to access remote TCP services through WebSocket tunnels
- **Client 1**: Primary client that hosts TCP tunnel connect endpoints to external TCP services
- **Client 2**: Secondary client that provides backup connectivity to the same external TCP services
**Tunnel Configuration**: Each tunnel is defined by a unique `tunnelName` and can be configured in either direction:
- **TCP Listen tunnels**: Server listens on a port, client connects to external service
- **TCP Connect tunnels**: Server connects to external service, client listens on a port
External applications connect to TCP ports on one side, and the service tunnels that traffic over WebSocket connections to TCP endpoints on the other side. Only one client is "active" at any time - when the active client becomes unavailable, automatic failover occurs to the standby client.
The WebSocket connection multiplexes all configured tunnels, with each tunnel identified by its unique name for proper routing.
## Key Features
### High Availability
- Automatic failover between two tunnel endpoints
- Server routes TCP tunnel traffic to the active client
- Graceful failover with configurable socket closure timing
- No single point of failure for TCP socket connectivity
### Security
- Optional SSL/TLS encryption for WebSocket connections
- Mutual TLS (client certificate) authentication
- Certificate-based peer verification
- Support for custom CA certificate chains
### Reliable TCP Tunneling
- Packet sequencing ensures ordered TCP data delivery over WebSocket transport
- Connection multiplexing allows multiple TCP streams over single WebSocket
- Automatic WebSocket reconnection maintains tunnel availability
- Data buffering handles out-of-sequence packets from WebSocket layer
### Platform Support
- Cross-platform (Linux, Windows, macOS)
- Windows service integration with proper service lifecycle
- Docker containerization with comprehensive test suite
- Configurable logging with rotation and syslog support
## Installing
Install with the following command
```
pip install tcp-over-websocket
```
NOTE: On windows, it may help to install some dependencies first, otherwise
pip may try to build them.
```
pip install vcversioner
```
## Running
You need to configure the settings before running tcp-over-websocket, but if
you want to just see if it starts run the command
```
run_tcp_over_websocket_service
```
It will start as a client by default and try to reconnect to nothing.
## Configuration
By default the tcp-over-websocket will create a home directory
~/tcp-over-websocket.home and create a `config.json` file in that directory.
To change the location of this directory, pass the config directory name in
as the first argument of the python script
Here is a windows example:
```
python c:\python\Lib\site-packages\tcp_over_websocket
\run_tcp_over_websocket_service.py c:\Users\meuser\tcp-over-websocket-server.
home
```
## High Availability Configuration
The TCP-over-WebSocket service supports a server with exactly two clients for high availability tunneling. When external sockets connect to TCP ports on the server, the traffic is routed through WebSocket connections to the "active" client, which then connects to the actual external TCP sockets. When the first data is sent to a standby client's listening socket, that client becomes active and takes over all tunnel traffic routing.
## Example Client Configuration
Clients host the TCP tunnel endpoints and connect to the server via WebSocket. Create a directory and place the following contents in a config.json file in that directory. Note the `clientId` field which must be either 1 or 2.
```json
{
"dataExchange": {
"enableMutualTLS": false,
"mutualTLSTrustedCACertificateBundleFilePath": "/Users/jchesney/Downloads/tcp_svr/trusted-ca.pem",
"mutualTLSTrustedPeerCertificateBundleFilePath": "/Users/jchesney/Downloads/tcp_svr/certs-of-peers.pem",
"serverUrl": "http://localhost:8080",
"tlsBundleFilePath": "/Users/jchesney/Downloads/tcp_svr/key-cert-ca-root-chain.pem"
},
"logging": {
"daysToKeep": 14,
"level": "DEBUG",
"logToStdout": true,
"syslog": {
"logToSysloyHost": null
}
},
"tcpTunnelConnects": [
{
"connectToHost": "search.brave.com",
"connectToPort": 80,
"tunnelName": "brave"
},
{
"connectToHost": "127.0.0.1",
"connectToPort": 22,
"tunnelName": "test_ssh"
}
],
"tcpTunnelListens": [
{
"listenBindAddress": "127.0.0.1",
"listenPort": 8091,
"tunnelName": "duckduckgo"
}],
"weAreServer": false,
"clientId": 1
}
```
## Example Second Client Configuration
For the second client, use the same configuration but with `clientId: 2` and
different port numbers to avoid conflicts:
```json
{
"dataExchange": {
"serverUrl": "http://localhost:8080"
},
"tcpTunnelListens": [
{
"listenBindAddress": "127.0.0.1",
"listenPort": 8094,
"tunnelName": "duckduckgo"
}],
"weAreServer": false,
"clientId": 2
}
```
## Example Server Configuration
The server provides TCP tunnel endpoints that external sockets connect to. Traffic is then routed over WebSocket to external TCP sockets accessible through the active client.
```json
{
"dataExchange": {
"enableMutualTLS": false,
"mutualTLSTrustedCACertificateBundleFilePath": "/Users/jchesney/Downloads/tcp_svr/trusted-ca.pem",
"mutualTLSTrustedPeerCertificateBundleFilePath": "/Users/jchesney/Downloads/tcp_svr/certs-of-peers.pem",
"serverUrl": "http://localhost:8080",
"tlsBundleFilePath": "/Users/jchesney/Downloads/tcp_svr/key-cert-ca-root-chain.pem"
},
"logging": {
"daysToKeep": 14,
"level": "DEBUG",
"logToStdout": true,
"syslog": {
"logToSysloyHost": null
}
},
"tcpTunnelConnects": [
{
"connectToHost": "duckduckgo.com",
"connectToPort": 80,
"tunnelName": "duckduckgo"
}
],
"tcpTunnelListens": [
{
"listenBindAddress": "127.0.0.1",
"listenPort": 8092,
"tunnelName": "brave"
},
{
"listenBindAddress": "127.0.0.1",
"listenPort": 8022,
"tunnelName": "test_ssh"
}
],
"weAreServer": true
}
```
## What Can Connect to TCP Tunnels
The TCP-over-WebSocket service works with any TCP socket connections. Examples of what might connect to these external TCP sockets include:
- **Web servers** (HTTP/HTTPS on ports 80/443)
- **Database servers** (MySQL on 3306, PostgreSQL on 5432, etc.)
- **SSH servers** (typically port 22)
- **Application servers** (custom TCP protocols)
- **Message brokers** (MQTT, RabbitMQ, etc.)
- **Remote desktop services** (RDP, VNC)
- **Any TCP-based service** that accepts socket connections
The service treats all connections as raw TCP data streams - it doesn't inspect or modify the content, making it protocol-agnostic.
### Configuration Parameters
| Parameter | Description | Default |
|-----------|-------------|---------|
| `clientId` | Client identifier (1 or 2) for HA setup | 1 |
| `weAreServer` | Whether this instance hosts TCP tunnel endpoints | false |
| `serverUrl` | WebSocket server URL for tunnel transport | http://localhost:8080 |
| `standbySocketCloseDurationSecs` | Socket close duration during failover | 30 |
| `enableMutualTLS` | Enable mutual TLS authentication | false |
## Running the Service
### Command Line
```bash
# Server
run_tcp_over_websocket_service /path/to/server/config
# Client 1
run_tcp_over_websocket_service /path/to/client1/config
# Client 2
run_tcp_over_websocket_service /path/to/client2/config
```
## Windows Services
To install tcp-over-websocket, open a command prompt as administrator, and run
the following command.
```
winsvc_tcp_over_websocket_service --username .\user-svc --password "myPa$$" --startup auto install
```
Use `--username` and `--password` to run the service as a non-privileged user.
---
After registering the above, you must go and re-enter the username and password.
1. Run `servcies.msc`
2. Find the `TCP over Websocket` service
3. Open the service properties
4. Click on the `Lok On` tab
5. Click `Local System account` and click `Apply`
6. Select `This account`
7. Enter your service username and password again.
8. Click `Ok`
9. You will then get an alert saying your service user has been granted
permissions to `log on as a service`
---
Consider making the service restart on failure.
1. Again, Open the properties of the service in `services.msc`
2. Click on the `Recovery` tab
3. Change the `First failure` dropdown box to `Restart the Service`
4. Click `Ok`
## Server Side TLS
Never run this service with out client TLS, and especially not without
server TLS.
NOTE: The following all assumes you have x509 certificates in ascii format.
Prepare the standard server side TLS bundle with a command similar to:
```
# Create the file containing the server or client certificates that either
# will send.
cat Root.crt CA.crt MyServerCert.{crt,key} > key-cert-ca-root-chain.pem
# or
cat Root.crt CA.crt MyClientCert.{crt,key} > key-cert-ca-root-chain.pem
```
--
Configure the server to service on SSL:
1. Update both client and server configurations `serverUrl` to start with
`https`
2. Ensure the `tlsBundleFilePath` setting points to your pem bundle as
prepared in the code block above.
3. Restart both client and server services.
## Mutual TLS
Mutual TLS or Client Certificate Authentication is when the client also
sends certificates to the server, and the server verifies them.
In our case, our client also verifies that the server has provided a
specific trusted certificate.
---
For Mutual TLS / Client Certificate Authentication, ensure you have a
certificate that has Client and Server capabilities.
```
# Run
openssl x509 -in mycert.crt -text | grep Web
# Expect to see
# TLS Web Server Authentication, TLS Web Client Authentication
```
---
Prepare the certificates for mutual TLS, the same commands work for both
sides, however, you put the servers certificates in the clients mutualTLS
config and the clients certificates in the servers mutualTLS config.
```
# Create the file that contains the certificate chain of the trusted
certificates.
cat Root.crt CA.crt > mtls/trusted-ca-chain.pem
# Create the file containing the peer certificates to trust
cat MyServerCert.{crt,key} > certs-of-peers.pem
# or
cat MyClientCert.{crt,key} > certs-of-peers.pem
```
---
To configure Mutual TLS, we will:
* Tell the client to send it's own certificat and chain
* Tell both the client and server what certificates to accept from the other
On the Server
1. Set the `enableMutualTLS` to `true`
2. Set the `mutualTLSTrustedCACertificateBundleFilePath` value to the path
of a file containing the Clients root and certificate authority
certificates.
3. Set the `mutualTLSTrustedPeerCertificateBundleFilePath` to the path of a
file containing the Clients public certificate.
On the Client
1. Set the `tlsBundleFilePath` as per the last section.
2. Set the `enableMutualTLS` to `true`
3. Set the `mutualTLSTrustedCACertificateBundleFilePath` value to the path
of a file containing the Servers root and certificate authority
certificates.
4. Set the `mutualTLSTrustedPeerCertificateBundleFilePath` to the path of a
file containing the Servers public certificate.
## Testing
The TCP over WebSocket service includes a comprehensive test suite that validates functionality across multiple scenarios. There are three different methods for running the unit tests:
### 1. Docker Containers (Recommended for CI/Integration Testing)
The Docker approach provides the most isolated and reproducible testing environment.
**Prerequisites:**
```bash
./docker/build.sh
docker-compose -f docker/docker-compose.yml up -d
```
**Run all tests:**
```bash
docker-compose -f docker/docker-compose.yml --profile test up --build
```
**Run tests with verbose output:**
```bash
docker-compose -f docker/docker-compose.yml --profile test run --rm tests /app/run_tests.sh --verbose
```
**Architecture:**
The Docker setup includes:
- **Server** (`tcp-over-websocket-server`) - Routes traffic between clients
- **Client 1** (`tcp-over-websocket-client1`) - First client with `clientId: 1`
- **Client 2** (`tcp-over-websocket-client2`) - Second client with `clientId: 2`
- **Tests** (`tcp-over-websocket-tests`) - Test runner with comprehensive test suites
**Port Configuration:**
- Server: 38080 (WebSocket), 38001-38002 (server-to-client tunnels)
- Client 1: 38011-38012 (client-to-server tunnels)
- Client 2: 38031-38032 (mapped to 38021-38022 internally)
### 2. run_test_suite_locally.py (Recommended for Full Local Testing)
This method orchestrates all services and tests locally using subprocess management.
**Prerequisites:**
```bash
pip install -r requirements.txt
pip install -r tests/requirements.txt
```
**Run the complete test suite:**
```bash
python tests/run_test_suite_locally.py
```
**Features:**
- Automatically starts and stops all three services (server, client1, client2)
- Runs all test suites sequentially with proper cleanup
- Captures and logs output from all services and tests
- Provides comprehensive test reporting with pass/fail counts
- Handles port conflicts and process cleanup
- Saves detailed logs to `test-logs/` directory
### 3. Running Services and Tests Individually (Best for Development)
This method gives you full control over each component, ideal for development and debugging specific issues.
**Step 1: Start Services Manually**
```bash
# Terminal 1 - Start Server
python tests/run_test_server_service.py
# Terminal 2 - Start Client 1
python tests/run_test_client1_service.py
# Terminal 3 - Start Client 2
python tests/run_test_client2_service.py
```
**Step 2: Run Individual Tests**
```bash
# Run specific test suite
pytest tests/test_suite_1_basic_echo.py -v
# Run specific test
pytest tests/test_suite_1_basic_echo.py::TestBasicEcho::test_1_1_server_to_client_tun1_echo -v
# Run with specific markers
pytest tests/ -m "not slow" -v # Skip slow tests
pytest tests/ -m "failover" -v # Run only failover tests
```
**Step 3: Development Workflow**
```bash
# Run all tests with detailed output
pytest tests/ -v --asyncio-mode=auto --tb=short
# Run failed tests only
pytest tests/ --lf -v
# Run with coverage
pytest tests/ --cov=tcp_over_websocket --cov-report=html
```
### Test Suites
The test suite covers comprehensive functionality validation:
1. **Basic Echo Tests (Suite 1)** - Simple connectivity validation with both clients
2. **Data Quality Tests (Suite 2)** - 100MB transfers with SHA-256 checksum validation
3. **Ping Pong Tests (Suite 3)** - 1000 iterations with 10ms delays and 500-byte packets
4. **Performance Tests (Suite 4)** - 5GB bidirectional transfers with throughput measurement
5. **Concurrent Connection Tests (Suite 5)** - Multiple simultaneous connections and sequential cycles
6. **Failover Impact Tests (Suite 6)** - Behavior validation during client transitions
### Test Configuration
Tests use dedicated configurations in `test_config/`:
- `websocket_server/config.json` - Server test configuration
- `websocket_client_1/config.json` - Client 1 test configuration
- `websocket_client_2/config.json` - Client 2 test configuration
### Interpreting Results
Each test provides:
- **Pass/Fail status** with detailed error messages
- **Performance metrics** (throughput, latency, connection rates)
- **Data integrity validation** via SHA-256 checksums
- **Connection state logging** for debugging failover scenarios
- **Comprehensive logs** saved to `test-logs/` directory
## Package Architecture
### Core Classes
#### **FileConfig** (`config/file_config.py`)
Main configuration class providing typed access to all settings:
- Manages `clientId` (1 or 2) for high availability setup
- Determines if instance is server (`weAreServer` boolean)
- Loads TCP tunnel listen and connect configurations
- Integrates data exchange and logging configurations
#### **ActiveRemoteController** (`controllers/active_remote_controller.py`)
High availability manager that handles client switching:
- Tracks which client (1 or 2) is currently active
- Records tunnel connections to determine active client
- Sends kill signals to inactive client connections
- Manages client online/offline state tracking
- Implements automatic failover when active client disconnects
#### **TcpTunnelABC** (`tcp_tunnel/tcp_tunnel_abc.py`)
Abstract base class that defines the core tunneling protocol:
- Manages WebSocket ↔ TCP data flow with packet sequencing
- Handles connection lifecycle (made/lost/closed) events
- Implements packet ordering and buffering logic
- Provides control message handling (connection status)
### Entry Points
#### **run_tcp_over_websocket_service.py**
Main service entry point:
- Initializes Twisted reactor and WebSocket factories
- Sets up server or client mode based on configuration
- Creates and manages all TCP tunnels
- Handles WebSocket connection establishment and monitoring
#### **winsvc_tcp_over_websocket_service.py**
Windows service wrapper:
- Integrates with Windows Service Control Manager
- Provides service installation, start, stop, and removal
- Handles Windows service lifecycle events
## Python 3.9 Compatibility
This project is designed for Python 3.9+ and uses modern Python features:
- Type hints with `typing` module annotations
- f-string formatting throughout
- `pathlib.Path` for cross-platform file operations
- Modern async/await patterns with Twisted's `inlineCallbacks`
### Dependencies
All dependencies are verified compatible with Python 3.9:
- `twisted[tls]==22.10.0` - Async networking and WebSocket support
- `vortexpy==3.4.3` - Message routing and WebSocket abstraction
- `reactivex==4.0.4` - Reactive observables for event handling
- `json-cfg-rw==0.5.0` - JSON configuration file management
- `txhttputil==1.2.8` - HTTP utilities for Twisted
## Troubleshooting
### Common Issues
**Connection Refused**
- Verify server is running and accessible
- Check firewall settings and port availability
- Validate `serverUrl` in client configuration
**Failover Not Working**
- Ensure both clients have different `clientId` values (1 and 2)
- Check `standbySocketCloseDurationSecs` configuration
- Verify both clients can connect to server
**Certificate Errors**
- Validate certificate file paths exist and are readable
- Check certificate format (PEM) and content
- Ensure certificate chains are complete
**High Memory Usage**
- Monitor connection counts and data buffer sizes
- Check for connection leaks in TCP socket code
- Consider adjusting packet sizes for large transfers
### Test Troubleshooting
If tests fail:
1. **Check service status** (for run_test_suite_locally.py):
```bash
# Check if ports are in use
netstat -tulpn | grep -E "(38080|38001|38002|38011|38012|38021|38022)"
# Kill existing processes
pkill -f run_test
```
2. **Check Docker container status**:
```bash
docker-compose -f docker/docker-compose.yml ps
docker-compose -f docker/docker-compose.yml logs
```
3. **View detailed logs**:
```bash
# Local test logs
ls -la test-logs/
# Docker test logs
docker-compose -f docker/docker-compose.yml logs tests
```
4. **Run specific failing test**:
```bash
pytest tests/test_suite_2_data_quality.py::TestDataQuality::test_2_1_100mb_server_to_client_tun1 -v -s
```
### Performance Tuning
**For High Throughput:**
- Increase OS socket buffers
- Use dedicated network interfaces
- Monitor CPU usage during large transfers
**For Many Connections:**
- Adjust OS file descriptor limits
- Monitor memory usage for connection tracking
- Consider connection pooling in external TCP socket code
## License
MIT License - see LICENSE file for full text.
Raw data
{
"_id": null,
"home_page": "https://github.com/Synerty/tcp-over-websocket",
"name": "tcp-over-websocket",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "TCP, Websocket, MutualTLS, synerty, tunnel, proxy",
"author": "Synerty",
"author_email": "contact@synerty.com",
"download_url": "https://files.pythonhosted.org/packages/b1/39/030fb97008d833f779963632e0a96710649c6f14f916cbf0c8df69f06abf/tcp_over_websocket-1.1.1.tar.gz",
"platform": null,
"description": "# TCP over WebSocket\n\n```\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 TCP-over-WebSocket Architecture \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 External XC1 \u2502 \u2502 External XC2 \u2502 \u2502 External XC3 \u2502 \u2502 External XC4 \u2502\n\u2502 Site A \u2502 \u2502 Site A \u2502 \u2502 Site B \u2502 \u2502 Site B \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n send/receive send/receive send/receive send/receive\n \u2502 \u25b2 \u2502 \u25b2\n \u25bc \u2502 \u25bc \u2502\n ---------------------------- Client Services ------------------------------\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Tunnel 1 \u2502 \u2502 Tunnel 2 \u2502 \u2502 Tunnel 1 \u2502 \u2502 Tunnel 2 \u2502\n\u2502 Listen \u2502 \u2502 Connect \u2502 \u2502 Listen \u2502 \u2502 Connect \u2502\n\u2502 Socket \u2502 \u2502 Socket \u2502 \u2502 Socket \u2502 \u2502 Socket \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502 \u2502 \u2502 \u2502\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Client 1 \u2502 \u2502 Client 2 \u2502\n\u2502 (Primary) \u2502 \u2502 (Secondary) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n --------------------------- Client Services ------------------------------\n \u2502 \u2502\n \u25bc \u25bc\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 WebSocket 1 \u2502 \u2502 WebSocket 2 \u2502\n\u2502 (Client 1 Connection) \u2502 \u2502 (Client 2 Connection) \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502 \u2502\n \u25bc \u25bc\n ---------------------------- Server Service ------------------------------\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 SERVER \u2502\n\u2502 (Routes between clients) \u2502\n\u2502 Active client failover logic \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502 \u2502\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 Tunnel 1 \u2502 \u2502 Tunnel 2 \u2502\n \u2502 Connect \u2502 \u2502 Listen \u2502\n \u2502 Socket \u2502 \u2502 Socket \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n ---------------------------- Server Service ------------------------------\n \u2502 \u25b2\n \u25bc \u2502\n send/receive send/receive\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 External XS1 \u2502 \u2502 External XS2 \u2502\n \u2502 Other Site \u2502 \u2502 Other Site \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n\n\nLegend:\n\u2500\u2500\u2500\u2500\u25ba Direction of Connection \n \n \n```\n\n\n\nTCP over WebSocket is a Python 3.9+ service that provides high-availability TCP tunneling through WebSocket connections. It enables secure, reliable forwarding of TCP traffic through HTTP/HTTPS WebSocket connections with automatic failover between two clients.\n\n## Overview\n\nThis package provides TCP-over-WebSocket tunneling with high availability failover:\n\n- **Tunnel TCP connections** through HTTP WebSocket connections\n- **Multiplex multiple TCP streams** over a single WebSocket connection\n- **OPTIONALLY secured** with HTTPS and mutual TLS client certificate authentication\n- **High availability** with automatic failover between two client endpoints\n- **Windows service support** for production deployments\n\nExternal TCP sockets connect to TCP ports on the server, which tunnels the traffic over WebSocket to TCP tunnel endpoints running on clients. TCP tunnels are bidirectional once the WebSocket connection is established.\n\nEach TCP tunnel is defined by a `tunnelName` - the server side listens for TCP connections while the client side connects to external TCP sockets.\n\n## Architecture\n\nThe service implements a server + two-client architecture for TCP tunneling over WebSocket with configurable tunnels:\n\n- **Server**: Hosts TCP tunnel listen endpoints where external applications connect to access remote TCP services through WebSocket tunnels\n- **Client 1**: Primary client that hosts TCP tunnel connect endpoints to external TCP services \n- **Client 2**: Secondary client that provides backup connectivity to the same external TCP services\n\n**Tunnel Configuration**: Each tunnel is defined by a unique `tunnelName` and can be configured in either direction:\n- **TCP Listen tunnels**: Server listens on a port, client connects to external service\n- **TCP Connect tunnels**: Server connects to external service, client listens on a port\n\nExternal applications connect to TCP ports on one side, and the service tunnels that traffic over WebSocket connections to TCP endpoints on the other side. Only one client is \"active\" at any time - when the active client becomes unavailable, automatic failover occurs to the standby client.\n\nThe WebSocket connection multiplexes all configured tunnels, with each tunnel identified by its unique name for proper routing.\n\n## Key Features\n\n### High Availability\n- Automatic failover between two tunnel endpoints\n- Server routes TCP tunnel traffic to the active client\n- Graceful failover with configurable socket closure timing\n- No single point of failure for TCP socket connectivity\n\n### Security\n- Optional SSL/TLS encryption for WebSocket connections\n- Mutual TLS (client certificate) authentication\n- Certificate-based peer verification\n- Support for custom CA certificate chains\n\n### Reliable TCP Tunneling\n- Packet sequencing ensures ordered TCP data delivery over WebSocket transport\n- Connection multiplexing allows multiple TCP streams over single WebSocket\n- Automatic WebSocket reconnection maintains tunnel availability\n- Data buffering handles out-of-sequence packets from WebSocket layer\n\n### Platform Support\n- Cross-platform (Linux, Windows, macOS)\n- Windows service integration with proper service lifecycle\n- Docker containerization with comprehensive test suite\n- Configurable logging with rotation and syslog support\n\n## Installing\n\nInstall with the following command\n\n```\npip install tcp-over-websocket\n```\n\nNOTE: On windows, it may help to install some dependencies first, otherwise\npip may try to build them.\n\n```\npip install vcversioner\n```\n\n## Running\n\nYou need to configure the settings before running tcp-over-websocket, but if\nyou want to just see if it starts run the command\n\n```\nrun_tcp_over_websocket_service\n```\n\nIt will start as a client by default and try to reconnect to nothing.\n\n## Configuration\n\nBy default the tcp-over-websocket will create a home directory\n~/tcp-over-websocket.home and create a `config.json` file in that directory.\n\nTo change the location of this directory, pass the config directory name in\nas the first argument of the python script\n\nHere is a windows example:\n\n```\npython c:\\python\\Lib\\site-packages\\tcp_over_websocket\n\\run_tcp_over_websocket_service.py c:\\Users\\meuser\\tcp-over-websocket-server.\nhome\n```\n\n## High Availability Configuration\n\nThe TCP-over-WebSocket service supports a server with exactly two clients for high availability tunneling. When external sockets connect to TCP ports on the server, the traffic is routed through WebSocket connections to the \"active\" client, which then connects to the actual external TCP sockets. When the first data is sent to a standby client's listening socket, that client becomes active and takes over all tunnel traffic routing.\n\n## Example Client Configuration\n\nClients host the TCP tunnel endpoints and connect to the server via WebSocket. Create a directory and place the following contents in a config.json file in that directory. Note the `clientId` field which must be either 1 or 2.\n\n```json\n{\n \"dataExchange\": {\n \"enableMutualTLS\": false,\n \"mutualTLSTrustedCACertificateBundleFilePath\": \"/Users/jchesney/Downloads/tcp_svr/trusted-ca.pem\",\n \"mutualTLSTrustedPeerCertificateBundleFilePath\": \"/Users/jchesney/Downloads/tcp_svr/certs-of-peers.pem\",\n \"serverUrl\": \"http://localhost:8080\",\n \"tlsBundleFilePath\": \"/Users/jchesney/Downloads/tcp_svr/key-cert-ca-root-chain.pem\"\n },\n \"logging\": {\n \"daysToKeep\": 14,\n \"level\": \"DEBUG\",\n \"logToStdout\": true,\n \"syslog\": {\n \"logToSysloyHost\": null\n }\n },\n \"tcpTunnelConnects\": [\n {\n \"connectToHost\": \"search.brave.com\",\n \"connectToPort\": 80,\n \"tunnelName\": \"brave\"\n },\n {\n \"connectToHost\": \"127.0.0.1\",\n \"connectToPort\": 22,\n \"tunnelName\": \"test_ssh\"\n }\n ],\n \"tcpTunnelListens\": [\n {\n \"listenBindAddress\": \"127.0.0.1\",\n \"listenPort\": 8091,\n \"tunnelName\": \"duckduckgo\"\n }],\n \"weAreServer\": false,\n \"clientId\": 1\n}\n```\n\n## Example Second Client Configuration\n\nFor the second client, use the same configuration but with `clientId: 2` and\ndifferent port numbers to avoid conflicts:\n\n```json\n{\n \"dataExchange\": {\n \"serverUrl\": \"http://localhost:8080\"\n },\n \"tcpTunnelListens\": [\n {\n \"listenBindAddress\": \"127.0.0.1\",\n \"listenPort\": 8094,\n \"tunnelName\": \"duckduckgo\"\n }],\n \"weAreServer\": false,\n \"clientId\": 2\n}\n```\n\n## Example Server Configuration\n\nThe server provides TCP tunnel endpoints that external sockets connect to. Traffic is then routed over WebSocket to external TCP sockets accessible through the active client.\n\n```json\n{\n \"dataExchange\": {\n \"enableMutualTLS\": false,\n \"mutualTLSTrustedCACertificateBundleFilePath\": \"/Users/jchesney/Downloads/tcp_svr/trusted-ca.pem\",\n \"mutualTLSTrustedPeerCertificateBundleFilePath\": \"/Users/jchesney/Downloads/tcp_svr/certs-of-peers.pem\",\n \"serverUrl\": \"http://localhost:8080\",\n \"tlsBundleFilePath\": \"/Users/jchesney/Downloads/tcp_svr/key-cert-ca-root-chain.pem\"\n },\n \"logging\": {\n \"daysToKeep\": 14,\n \"level\": \"DEBUG\",\n \"logToStdout\": true,\n \"syslog\": {\n \"logToSysloyHost\": null\n }\n },\n \"tcpTunnelConnects\": [\n {\n \"connectToHost\": \"duckduckgo.com\",\n \"connectToPort\": 80,\n \"tunnelName\": \"duckduckgo\"\n }\n ],\n \"tcpTunnelListens\": [\n {\n \"listenBindAddress\": \"127.0.0.1\",\n \"listenPort\": 8092,\n \"tunnelName\": \"brave\"\n },\n {\n \"listenBindAddress\": \"127.0.0.1\",\n \"listenPort\": 8022,\n \"tunnelName\": \"test_ssh\"\n }\n ],\n \"weAreServer\": true\n}\n```\n\n## What Can Connect to TCP Tunnels\n\nThe TCP-over-WebSocket service works with any TCP socket connections. Examples of what might connect to these external TCP sockets include:\n\n- **Web servers** (HTTP/HTTPS on ports 80/443)\n- **Database servers** (MySQL on 3306, PostgreSQL on 5432, etc.) \n- **SSH servers** (typically port 22)\n- **Application servers** (custom TCP protocols)\n- **Message brokers** (MQTT, RabbitMQ, etc.)\n- **Remote desktop services** (RDP, VNC)\n- **Any TCP-based service** that accepts socket connections\n\nThe service treats all connections as raw TCP data streams - it doesn't inspect or modify the content, making it protocol-agnostic.\n\n### Configuration Parameters\n\n| Parameter | Description | Default |\n|-----------|-------------|---------|\n| `clientId` | Client identifier (1 or 2) for HA setup | 1 |\n| `weAreServer` | Whether this instance hosts TCP tunnel endpoints | false |\n| `serverUrl` | WebSocket server URL for tunnel transport | http://localhost:8080 |\n| `standbySocketCloseDurationSecs` | Socket close duration during failover | 30 |\n| `enableMutualTLS` | Enable mutual TLS authentication | false |\n\n## Running the Service\n\n### Command Line\n\n```bash\n# Server\nrun_tcp_over_websocket_service /path/to/server/config\n\n# Client 1 \nrun_tcp_over_websocket_service /path/to/client1/config\n\n# Client 2\nrun_tcp_over_websocket_service /path/to/client2/config\n```\n\n## Windows Services\n\nTo install tcp-over-websocket, open a command prompt as administrator, and run\nthe following command.\n\n```\nwinsvc_tcp_over_websocket_service --username .\\user-svc --password \"myPa$$\" --startup auto install\n \n```\n\nUse `--username` and `--password` to run the service as a non-privileged user.\n\n---\n\nAfter registering the above, you must go and re-enter the username and password.\n\n1. Run `servcies.msc`\n2. Find the `TCP over Websocket` service\n3. Open the service properties\n4. Click on the `Lok On` tab\n5. Click `Local System account` and click `Apply`\n6. Select `This account`\n7. Enter your service username and password again.\n8. Click `Ok`\n9. You will then get an alert saying your service user has been granted\n permissions to `log on as a service`\n\n---\n\nConsider making the service restart on failure.\n\n1. Again, Open the properties of the service in `services.msc`\n2. Click on the `Recovery` tab\n3. Change the `First failure` dropdown box to `Restart the Service`\n4. Click `Ok`\n\n## Server Side TLS\n\nNever run this service with out client TLS, and especially not without\nserver TLS.\n\nNOTE: The following all assumes you have x509 certificates in ascii format.\n\nPrepare the standard server side TLS bundle with a command similar to:\n\n```\n\n# Create the file containing the server or client certificates that either \n# will send.\ncat Root.crt CA.crt MyServerCert.{crt,key} > key-cert-ca-root-chain.pem\n\n# or\ncat Root.crt CA.crt MyClientCert.{crt,key} > key-cert-ca-root-chain.pem\n\n\n```\n\n--\n\nConfigure the server to service on SSL:\n\n1. Update both client and server configurations `serverUrl` to start with\n `https`\n2. Ensure the `tlsBundleFilePath` setting points to your pem bundle as\n prepared in the code block above.\n3. Restart both client and server services.\n\n## Mutual TLS\n\nMutual TLS or Client Certificate Authentication is when the client also\nsends certificates to the server, and the server verifies them.\n\nIn our case, our client also verifies that the server has provided a\nspecific trusted certificate.\n\n---\n\nFor Mutual TLS / Client Certificate Authentication, ensure you have a\ncertificate that has Client and Server capabilities.\n\n```\n# Run\nopenssl x509 -in mycert.crt -text | grep Web\n\n# Expect to see\n# TLS Web Server Authentication, TLS Web Client Authentication\n```\n\n---\n\nPrepare the certificates for mutual TLS, the same commands work for both\nsides, however, you put the servers certificates in the clients mutualTLS\nconfig and the clients certificates in the servers mutualTLS config.\n\n```\n# Create the file that contains the certificate chain of the trusted \ncertificates.\n\ncat Root.crt CA.crt > mtls/trusted-ca-chain.pem\n\n# Create the file containing the peer certificates to trust\ncat MyServerCert.{crt,key} > certs-of-peers.pem\n\n# or \ncat MyClientCert.{crt,key} > certs-of-peers.pem\n\n```\n\n---\n\nTo configure Mutual TLS, we will:\n\n* Tell the client to send it's own certificat and chain\n* Tell both the client and server what certificates to accept from the other\n\nOn the Server\n\n1. Set the `enableMutualTLS` to `true`\n2. Set the `mutualTLSTrustedCACertificateBundleFilePath` value to the path\n of a file containing the Clients root and certificate authority\n certificates.\n3. Set the `mutualTLSTrustedPeerCertificateBundleFilePath` to the path of a\n file containing the Clients public certificate.\n\nOn the Client\n\n1. Set the `tlsBundleFilePath` as per the last section.\n2. Set the `enableMutualTLS` to `true`\n3. Set the `mutualTLSTrustedCACertificateBundleFilePath` value to the path\n of a file containing the Servers root and certificate authority\n certificates.\n4. Set the `mutualTLSTrustedPeerCertificateBundleFilePath` to the path of a\n file containing the Servers public certificate.\n\n## Testing\n\nThe TCP over WebSocket service includes a comprehensive test suite that validates functionality across multiple scenarios. There are three different methods for running the unit tests:\n\n### 1. Docker Containers (Recommended for CI/Integration Testing)\n\nThe Docker approach provides the most isolated and reproducible testing environment.\n\n**Prerequisites:**\n```bash\n./docker/build.sh\ndocker-compose -f docker/docker-compose.yml up -d\n```\n\n**Run all tests:**\n```bash\ndocker-compose -f docker/docker-compose.yml --profile test up --build\n```\n\n**Run tests with verbose output:**\n```bash\ndocker-compose -f docker/docker-compose.yml --profile test run --rm tests /app/run_tests.sh --verbose\n```\n\n**Architecture:**\nThe Docker setup includes:\n- **Server** (`tcp-over-websocket-server`) - Routes traffic between clients\n- **Client 1** (`tcp-over-websocket-client1`) - First client with `clientId: 1`\n- **Client 2** (`tcp-over-websocket-client2`) - Second client with `clientId: 2`\n- **Tests** (`tcp-over-websocket-tests`) - Test runner with comprehensive test suites\n\n**Port Configuration:**\n- Server: 38080 (WebSocket), 38001-38002 (server-to-client tunnels)\n- Client 1: 38011-38012 (client-to-server tunnels)\n- Client 2: 38031-38032 (mapped to 38021-38022 internally)\n\n### 2. run_test_suite_locally.py (Recommended for Full Local Testing)\n\nThis method orchestrates all services and tests locally using subprocess management.\n\n**Prerequisites:**\n```bash\npip install -r requirements.txt\npip install -r tests/requirements.txt\n```\n\n**Run the complete test suite:**\n```bash\npython tests/run_test_suite_locally.py\n```\n\n**Features:**\n- Automatically starts and stops all three services (server, client1, client2)\n- Runs all test suites sequentially with proper cleanup\n- Captures and logs output from all services and tests\n- Provides comprehensive test reporting with pass/fail counts\n- Handles port conflicts and process cleanup\n- Saves detailed logs to `test-logs/` directory\n\n### 3. Running Services and Tests Individually (Best for Development)\n\nThis method gives you full control over each component, ideal for development and debugging specific issues.\n\n**Step 1: Start Services Manually**\n```bash\n# Terminal 1 - Start Server\npython tests/run_test_server_service.py\n\n# Terminal 2 - Start Client 1\npython tests/run_test_client1_service.py\n\n# Terminal 3 - Start Client 2\npython tests/run_test_client2_service.py\n```\n\n**Step 2: Run Individual Tests**\n```bash\n# Run specific test suite\npytest tests/test_suite_1_basic_echo.py -v\n\n# Run specific test\npytest tests/test_suite_1_basic_echo.py::TestBasicEcho::test_1_1_server_to_client_tun1_echo -v\n\n# Run with specific markers\npytest tests/ -m \"not slow\" -v # Skip slow tests\npytest tests/ -m \"failover\" -v # Run only failover tests\n```\n\n**Step 3: Development Workflow**\n```bash\n# Run all tests with detailed output\npytest tests/ -v --asyncio-mode=auto --tb=short\n\n# Run failed tests only\npytest tests/ --lf -v\n\n# Run with coverage\npytest tests/ --cov=tcp_over_websocket --cov-report=html\n```\n\n### Test Suites\n\nThe test suite covers comprehensive functionality validation:\n\n1. **Basic Echo Tests (Suite 1)** - Simple connectivity validation with both clients\n2. **Data Quality Tests (Suite 2)** - 100MB transfers with SHA-256 checksum validation \n3. **Ping Pong Tests (Suite 3)** - 1000 iterations with 10ms delays and 500-byte packets\n4. **Performance Tests (Suite 4)** - 5GB bidirectional transfers with throughput measurement\n5. **Concurrent Connection Tests (Suite 5)** - Multiple simultaneous connections and sequential cycles\n6. **Failover Impact Tests (Suite 6)** - Behavior validation during client transitions\n\n### Test Configuration\n\nTests use dedicated configurations in `test_config/`:\n- `websocket_server/config.json` - Server test configuration\n- `websocket_client_1/config.json` - Client 1 test configuration \n- `websocket_client_2/config.json` - Client 2 test configuration\n\n### Interpreting Results\n\nEach test provides:\n- **Pass/Fail status** with detailed error messages\n- **Performance metrics** (throughput, latency, connection rates)\n- **Data integrity validation** via SHA-256 checksums\n- **Connection state logging** for debugging failover scenarios\n- **Comprehensive logs** saved to `test-logs/` directory\n\n## Package Architecture\n\n### Core Classes\n\n#### **FileConfig** (`config/file_config.py`)\nMain configuration class providing typed access to all settings:\n- Manages `clientId` (1 or 2) for high availability setup\n- Determines if instance is server (`weAreServer` boolean)\n- Loads TCP tunnel listen and connect configurations\n- Integrates data exchange and logging configurations\n\n#### **ActiveRemoteController** (`controllers/active_remote_controller.py`)\nHigh availability manager that handles client switching:\n- Tracks which client (1 or 2) is currently active\n- Records tunnel connections to determine active client\n- Sends kill signals to inactive client connections\n- Manages client online/offline state tracking\n- Implements automatic failover when active client disconnects\n\n#### **TcpTunnelABC** (`tcp_tunnel/tcp_tunnel_abc.py`)\nAbstract base class that defines the core tunneling protocol:\n- Manages WebSocket \u2194 TCP data flow with packet sequencing\n- Handles connection lifecycle (made/lost/closed) events\n- Implements packet ordering and buffering logic\n- Provides control message handling (connection status)\n\n### Entry Points\n\n#### **run_tcp_over_websocket_service.py**\nMain service entry point:\n- Initializes Twisted reactor and WebSocket factories\n- Sets up server or client mode based on configuration\n- Creates and manages all TCP tunnels\n- Handles WebSocket connection establishment and monitoring\n\n#### **winsvc_tcp_over_websocket_service.py**\nWindows service wrapper:\n- Integrates with Windows Service Control Manager\n- Provides service installation, start, stop, and removal\n- Handles Windows service lifecycle events\n\n## Python 3.9 Compatibility\n\nThis project is designed for Python 3.9+ and uses modern Python features:\n\n- Type hints with `typing` module annotations\n- f-string formatting throughout\n- `pathlib.Path` for cross-platform file operations\n- Modern async/await patterns with Twisted's `inlineCallbacks`\n\n### Dependencies\n\nAll dependencies are verified compatible with Python 3.9:\n\n- `twisted[tls]==22.10.0` - Async networking and WebSocket support\n- `vortexpy==3.4.3` - Message routing and WebSocket abstraction \n- `reactivex==4.0.4` - Reactive observables for event handling\n- `json-cfg-rw==0.5.0` - JSON configuration file management\n- `txhttputil==1.2.8` - HTTP utilities for Twisted\n\n## Troubleshooting\n\n### Common Issues\n\n**Connection Refused**\n- Verify server is running and accessible\n- Check firewall settings and port availability\n- Validate `serverUrl` in client configuration\n\n**Failover Not Working** \n- Ensure both clients have different `clientId` values (1 and 2)\n- Check `standbySocketCloseDurationSecs` configuration\n- Verify both clients can connect to server\n\n**Certificate Errors**\n- Validate certificate file paths exist and are readable\n- Check certificate format (PEM) and content\n- Ensure certificate chains are complete\n\n**High Memory Usage**\n- Monitor connection counts and data buffer sizes\n- Check for connection leaks in TCP socket code\n- Consider adjusting packet sizes for large transfers\n\n### Test Troubleshooting\n\nIf tests fail:\n\n1. **Check service status** (for run_test_suite_locally.py):\n ```bash\n # Check if ports are in use\n netstat -tulpn | grep -E \"(38080|38001|38002|38011|38012|38021|38022)\"\n \n # Kill existing processes\n pkill -f run_test\n ```\n\n2. **Check Docker container status**:\n ```bash\n docker-compose -f docker/docker-compose.yml ps\n docker-compose -f docker/docker-compose.yml logs\n ```\n\n3. **View detailed logs**:\n ```bash\n # Local test logs\n ls -la test-logs/\n \n # Docker test logs \n docker-compose -f docker/docker-compose.yml logs tests\n ```\n\n4. **Run specific failing test**:\n ```bash\n pytest tests/test_suite_2_data_quality.py::TestDataQuality::test_2_1_100mb_server_to_client_tun1 -v -s\n ```\n\n### Performance Tuning\n\n**For High Throughput:**\n- Increase OS socket buffers\n- Use dedicated network interfaces\n- Monitor CPU usage during large transfers\n\n**For Many Connections:**\n- Adjust OS file descriptor limits\n- Monitor memory usage for connection tracking\n- Consider connection pooling in external TCP socket code\n\n## License\n\nMIT License - see LICENSE file for full text.\n",
"bugtrack_url": null,
"license": null,
"summary": "TCP over HTTPS Upgraded Websocket with Mutual TLS",
"version": "1.1.1",
"project_urls": {
"Download": "https://codeload.github.com/Synerty/tcp-over-websocket/zip/refs/heads/master",
"Homepage": "https://github.com/Synerty/tcp-over-websocket"
},
"split_keywords": [
"tcp",
" websocket",
" mutualtls",
" synerty",
" tunnel",
" proxy"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "b139030fb97008d833f779963632e0a96710649c6f14f916cbf0c8df69f06abf",
"md5": "f7117cf79e1f04061ca47f329b2cc621",
"sha256": "e967d6686d02e4f7bfb041fbfc8a4fa3233ff8cdacc1f33aa29a470f8c361302"
},
"downloads": -1,
"filename": "tcp_over_websocket-1.1.1.tar.gz",
"has_sig": false,
"md5_digest": "f7117cf79e1f04061ca47f329b2cc621",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 52119,
"upload_time": "2025-10-08T03:07:29",
"upload_time_iso_8601": "2025-10-08T03:07:29.811451Z",
"url": "https://files.pythonhosted.org/packages/b1/39/030fb97008d833f779963632e0a96710649c6f14f916cbf0c8df69f06abf/tcp_over_websocket-1.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-08 03:07:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Synerty",
"github_project": "tcp-over-websocket",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "alabaster",
"specs": [
[
"==",
"0.7.16"
]
]
},
{
"name": "annotated-types",
"specs": [
[
"==",
"0.7.0"
]
]
},
{
"name": "anthropic",
"specs": [
[
"==",
"0.69.0"
]
]
},
{
"name": "anyio",
"specs": [
[
"==",
"4.11.0"
]
]
},
{
"name": "attrs",
"specs": [
[
"==",
"25.3.0"
]
]
},
{
"name": "autobahn",
"specs": [
[
"==",
"23.6.2"
]
]
},
{
"name": "Automat",
"specs": [
[
"==",
"25.4.16"
]
]
},
{
"name": "babel",
"specs": [
[
"==",
"2.17.0"
]
]
},
{
"name": "backports.asyncio.runner",
"specs": [
[
"==",
"1.2.0"
]
]
},
{
"name": "certifi",
"specs": [
[
"==",
"2025.8.3"
]
]
},
{
"name": "cffi",
"specs": [
[
"==",
"2.0.0"
]
]
},
{
"name": "charset-normalizer",
"specs": [
[
"==",
"3.4.3"
]
]
},
{
"name": "constantly",
"specs": [
[
"==",
"23.10.4"
]
]
},
{
"name": "cryptography",
"specs": [
[
"==",
"46.0.2"
]
]
},
{
"name": "ddt",
"specs": [
[
"==",
"1.7.2"
]
]
},
{
"name": "distro",
"specs": [
[
"==",
"1.9.0"
]
]
},
{
"name": "docstring_parser",
"specs": [
[
"==",
"0.17.0"
]
]
},
{
"name": "docutils",
"specs": [
[
"==",
"0.21.2"
]
]
},
{
"name": "exceptiongroup",
"specs": [
[
"==",
"1.3.0"
]
]
},
{
"name": "filetype",
"specs": [
[
"==",
"1.2.0"
]
]
},
{
"name": "GeoAlchemy2",
"specs": [
[
"==",
"0.17.1"
]
]
},
{
"name": "greenlet",
"specs": [
[
"==",
"3.2.4"
]
]
},
{
"name": "h11",
"specs": [
[
"==",
"0.16.0"
]
]
},
{
"name": "httpcore",
"specs": [
[
"==",
"1.0.9"
]
]
},
{
"name": "httpx",
"specs": [
[
"==",
"0.28.1"
]
]
},
{
"name": "hyperlink",
"specs": [
[
"==",
"21.0.0"
]
]
},
{
"name": "idna",
"specs": [
[
"==",
"3.10"
]
]
},
{
"name": "imagesize",
"specs": [
[
"==",
"1.4.1"
]
]
},
{
"name": "importlib_metadata",
"specs": [
[
"==",
"8.7.0"
]
]
},
{
"name": "incremental",
"specs": [
[
"==",
"24.7.2"
]
]
},
{
"name": "iniconfig",
"specs": [
[
"==",
"2.1.0"
]
]
},
{
"name": "Jinja2",
"specs": [
[
"==",
"3.1.6"
]
]
},
{
"name": "jiter",
"specs": [
[
"==",
"0.11.0"
]
]
},
{
"name": "json-cfg-rw",
"specs": [
[
"==",
"0.5.0"
]
]
},
{
"name": "kwonly-args",
"specs": [
[
"==",
"1.0.10"
]
]
},
{
"name": "MarkupSafe",
"specs": [
[
"==",
"3.0.3"
]
]
},
{
"name": "packaging",
"specs": [
[
"==",
"25.0"
]
]
},
{
"name": "pem",
"specs": [
[
"==",
"23.1.0"
]
]
},
{
"name": "pluggy",
"specs": [
[
"==",
"1.6.0"
]
]
},
{
"name": "psutil",
"specs": [
[
"==",
"5.9.0"
]
]
},
{
"name": "pyasn1",
"specs": [
[
"==",
"0.6.1"
]
]
},
{
"name": "pyasn1_modules",
"specs": [
[
"==",
"0.4.2"
]
]
},
{
"name": "pycparser",
"specs": [
[
"==",
"2.23"
]
]
},
{
"name": "pydantic",
"specs": [
[
"==",
"2.11.9"
]
]
},
{
"name": "pydantic_core",
"specs": [
[
"==",
"2.33.2"
]
]
},
{
"name": "Pygments",
"specs": [
[
"==",
"2.19.2"
]
]
},
{
"name": "pyOpenSSL",
"specs": [
[
"==",
"25.3.0"
]
]
},
{
"name": "pytest",
"specs": [
[
"==",
"8.4.2"
]
]
},
{
"name": "pytest-asyncio",
"specs": [
[
"==",
"1.2.0"
]
]
},
{
"name": "pytmpdir",
"specs": [
[
"==",
"1.2.6"
]
]
},
{
"name": "pytz",
"specs": [
[
"==",
"2025.2"
]
]
},
{
"name": "reactivex",
"specs": [
[
"==",
"4.0.4"
]
]
},
{
"name": "requests",
"specs": [
[
"==",
"2.32.5"
]
]
},
{
"name": "Rx",
"specs": [
[
"==",
"1.6.3"
]
]
},
{
"name": "service-identity",
"specs": [
[
"==",
"24.2.0"
]
]
},
{
"name": "six",
"specs": [
[
"==",
"1.17.0"
]
]
},
{
"name": "sniffio",
"specs": [
[
"==",
"1.3.1"
]
]
},
{
"name": "snowballstemmer",
"specs": [
[
"==",
"3.0.1"
]
]
},
{
"name": "Sphinx",
"specs": [
[
"==",
"7.4.7"
]
]
},
{
"name": "sphinx-rtd-theme",
"specs": [
[
"==",
"3.0.2"
]
]
},
{
"name": "sphinxcontrib-applehelp",
"specs": [
[
"==",
"2.0.0"
]
]
},
{
"name": "sphinxcontrib-devhelp",
"specs": [
[
"==",
"2.0.0"
]
]
},
{
"name": "sphinxcontrib-htmlhelp",
"specs": [
[
"==",
"2.1.0"
]
]
},
{
"name": "sphinxcontrib-jquery",
"specs": [
[
"==",
"4.1"
]
]
},
{
"name": "sphinxcontrib-jsmath",
"specs": [
[
"==",
"1.0.1"
]
]
},
{
"name": "sphinxcontrib-qthelp",
"specs": [
[
"==",
"2.0.0"
]
]
},
{
"name": "sphinxcontrib-serializinghtml",
"specs": [
[
"==",
"2.0.0"
]
]
},
{
"name": "SQLAlchemy",
"specs": [
[
"==",
"2.0.43"
]
]
},
{
"name": "tomli",
"specs": [
[
"==",
"2.2.1"
]
]
},
{
"name": "tqdm",
"specs": [
[
"==",
"4.67.1"
]
]
},
{
"name": "treelib",
"specs": [
[
"==",
"1.8.0"
]
]
},
{
"name": "Twisted",
"specs": [
[
"==",
"22.10.0"
]
]
},
{
"name": "txaio",
"specs": [
[
"==",
"23.6.1"
]
]
},
{
"name": "txhttputil",
"specs": [
[
"==",
"1.2.8"
]
]
},
{
"name": "txwebsocket",
"specs": [
[
"==",
"1.1.1"
]
]
},
{
"name": "typing-inspection",
"specs": [
[
"==",
"0.4.2"
]
]
},
{
"name": "typing_extensions",
"specs": [
[
"==",
"4.15.0"
]
]
},
{
"name": "urllib3",
"specs": [
[
"==",
"2.5.0"
]
]
},
{
"name": "vcversioner",
"specs": [
[
"==",
"2.16.0.0"
]
]
},
{
"name": "vortexpy",
"specs": [
[
"==",
"3.4.3"
]
]
},
{
"name": "zipp",
"specs": [
[
"==",
"3.23.0"
]
]
},
{
"name": "zope.interface",
"specs": [
[
"==",
"8.0.1"
]
]
}
],
"lcname": "tcp-over-websocket"
}
