# Process Pilot
Process Pilot is a Python-based tool for managing and monitoring processes defined in a manifest file. It supports JSON and YAML formats for defining processes and their configurations.
## Features
- Load process configurations from JSON or YAML files.
- Manage process lifecycles with customizable hooks.
- Monitor process resource usage.
- Define shutdown strategies for processes.
- Define ready strategies to determine when launched processes are deemed to be "running".
## Installation
To install the dependencies, use [Poetry](https://python-poetry.org/):
```sh
poetry install
```
## Usage
You can use the `ProcessPilot` class directly in your Python code to manage processes defined in a YAML or JSON file.
### Process Control
You can restart specific processes by name:
```python
from process_pilot.pilot import ProcessPilot
from process_pilot.process import ProcessManifest
from pathlib import Path
# Load manifest and start ProcessPilot
manifest = ProcessManifest.from_json(Path("manifest.json"))
pilot = ProcessPilot(manifest)
pilot.start()
# Later, restart specific processes
try:
pilot.restart_processes(["api_server", "worker"])
except ValueError as e:
print(f"Error restarting processes: {e}")
```
### Example Usage
#### Using a JSON Manifest
```python
from pathlib import Path
from process_pilot.process import ProcessPilot, ProcessManifest
# Load the process manifest from a JSON file
manifest_path = Path("path/to/your/manifest.json")
manifest = ProcessManifest.from_json(manifest_path)
# Create a ProcessPilot instance with the loaded manifest
pilot = ProcessPilot(manifest)
# Start managing the processes
pilot.start()
```
#### Using a YAML Manifest
```python
from pathlib import Path
from process_pilot.process import ProcessPilot, ProcessManifest
# Load the process manifest from a YAML file
manifest_path = Path("path/to/your/manifest.yaml")
manifest = ProcessManifest.from_yaml(manifest_path)
# Create a ProcessPilot instance with the loaded manifest
pilot = ProcessPilot(manifest)
# Start managing the processes
pilot.start()
```
## Configuration
### Process Manifest
The process manifest defines the processes to be managed. It can be written in JSON or YAML format.
#### Parameters
- `name`: The name of the process. This should be unique within the manifest.
- `path`: The path to the executable or script to be run.
- `args`: A list of arguments to be passed to the process.
- `timeout`: The maximum time (in seconds) to wait for the process to stop.
- `shutdown_strategy`: The strategy to use when shutting down the process. Possible values are:
- `do_not_restart`: Do not restart the process after it stops.
- `restart`: Restart the process after it stops. This is the default.
- `shutdown_everything`: Stop all processes when this process stops.
- `ready_strategy`: The strategy to use to determine when the process is ready. Possible values are:
- `tcp`: The process is ready when it starts listening on a specified TCP port.
- `pipe`: The process is ready when it writes a specific signal to a named pipe.
- `file`: The process is ready when a specific file is created.
- `ready_timeout_sec`: The maximum time (in seconds) to wait for the process to be ready.
- `ready_params`: Additional parameters for the ready strategy. These vary based on the strategy:
- For `tcp`, specify the `port` to check.
- For `pipe`, specify the `path` to the named pipe.
- For `file`, specify the `path` to the file.
- `dependencies`: A list of other process names that must be started before this process can be started.
- `env`: A dictionary of environment variables to set for the process.
The following is an example of a JSON manifest:
```json
{
"processes": [
{
"name": "example",
"path": "sleep",
"args": ["5"],
"timeout": 3,
"shutdown_strategy": "do_not_restart",
"ready_strategy": "tcp",
"ready_timeout_sec": 10.0,
"ready_params": {
"port": 8080
},
"dependencies": ["another_process"],
"env": {
"ENV_VAR": "value"
}
}
]
}
```
The following is an example of a YAML manifest:
```yaml
processes:
- name: example
path: sleep
args: ["5"]
timeout: 1.0
shutdown_strategy: do_not_restart
ready_strategy: tcp
ready_timeout_sec: 10.0
ready_params:
port: 8080
dependencies:
- another_process
env:
ENV_VAR: value
```
## Plugin System
Process Pilot supports a plugin system that allows users to extend its functionality with custom lifecycle hooks, ready strategies, and process statistics handlers. Each created plugin can provide all three of the aforementioned groups, or just one. Each registration function links a name to a function or set of functions. The name provided must match what is in the manifest.
### Plugin Registration Scoping
Plugins in Process Pilot have one distinct registration scope:
1. **Process-Specific Hooks**: Process lifecycle hooks, ready strategies, and process statistic handlers are only registered for processes that explicitly request them in their manifest configuration.
### Creating a Plugin
To create a plugin, define a class that inherits from `Plugin` and implement the required methods:
```python
# import statements...
class FileReadyPlugin(Plugin):
def get_ready_strategies(self) -> dict[str, ReadyStrategyType]:
return {
"file": self._wait_file_ready,
}
def _wait_file_ready(self, process: "Process", ready_check_interval_secs: float) -> bool:
file_path = process.ready_params.get("path")
if not file_path:
msg = "Path not specified for file ready strategy"
raise RuntimeError(msg)
file_path = Path(file_path)
start_time = time.time()
while (time.time() - start_time) < process.ready_timeout_sec:
if file_path.exists():
return True
time.sleep(ready_check_interval_secs)
return False
```
When creating plugins that implement a ready strategy it is important to keep in mind that you should always be checking readiness relative to
the start time--and always comparing the difference to the timeout value that is specified in the manifest. The
simplest example of this can be seen in the `FileReadyPlugin` shown above:
```python
start_time = time.time()
while (time.time() - start_time) < process.ready_timeout_sec:
if file_path.exists():
return True
time.sleep(ready_check_interval_secs)
# Timeout
return False
```
Be careful not to use readiness checks that block the threads ability to check for a timeout condition.
### Plugin Registration
One way to use plugins with specific processes is to specify them in the manifest as shown below:
```json
{
"processes": [
{
"name": "example_process",
"path": "myapp",
"lifecycle_hooks": ["custom_lifecycle_hooks"],
"ready_strategy": "custom_strategy",
"ready_timeout_sec": 10.0
},
{
"name": "another_process",
"path": "otherapp"
}
]
}
```
In this example, you must have loaded a plugin that provides a lifecycle hook with the name "custom_lifecycle_hooks" and a plugin that implements a ready strategy named "custom_strategy"--they do not have to be in the same plugin, but they could be.
Plugins themselves are registered with process-pilot either in code directly:
```python
from pathlib import Path
from process_pilot.process import ProcessPilot, ProcessManifest
from custom_plugin import CustomPlugin
# Load manifest
manifest = ProcessManifest.from_json(Path("manifest.json"))
# Create pilot and register plugins
pilot = ProcessPilot(manifest)
pilot.register_plugins([CustomPlugin()])
# Start processes
pilot.start()
```
Or, by providing a plugin directory when the ProcessPilot object is constructed. See the API documentation for the ProcessPilot class, and the Plugin class for more details.
## Control Server Plugins
Control server plugins provide a way to add remote control capabilities to Process Pilot. A control server can be used to monitor and control processes at runtime through various interfaces (HTTP, TCP, Unix socket, etc.).
### Creating a Control Server Plugin
To create a control server plugin:
1. Create a class that implements your control server with at least these methods:
- `__init__(self, pilot: ProcessPilot)` - Constructor that receives the pilot instance
- `start(self)` - Called when Process Pilot starts
- `stop(self)` - Called when Process Pilot shuts down
2. Create a plugin class that inherits from `Plugin` and registers your control server:
```python
from process_pilot.plugin import Plugin, ControlServerType
from process_pilot.pilot import ProcessPilot
class MyControlServer:
def __init__(self, pilot: ProcessPilot):
self.pilot = pilot
def start(self):
# Start your control server
pass
def stop(self):
# Clean up resources
pass
class MyControlServerPlugin(Plugin):
def get_control_servers(self) -> dict[str, ControlServerType]:
return {"my_server": lambda pilot: MyControlServer(pilot)}
```
3. Register your plugin and enable it in the manifest:
```python
pilot = ProcessPilot(manifest)
pilot.register_plugins([MyControlServerPlugin()])
```
```yaml
manifest:
control_server: my_server # Enable your control server
processes:
# ... process definitions ...
```
### Control Server Lifecycle
The following diagram shows when the control server is started and stopped:
```mermaid
sequenceDiagram
participant User
participant ProcessPilot
participant ControlServer
participant Processes
User->>ProcessPilot: Create(manifest)
ProcessPilot->>ProcessPilot: Load plugins
ProcessPilot->>ControlServer: Create
User->>ProcessPilot: start()
ProcessPilot->>ControlServer: start()
ProcessPilot->>Processes: Start processes
Note over ControlServer,Processes: Control server running...
User->>ProcessPilot: shutdown()
ProcessPilot->>Processes: Stop processes
ProcessPilot->>ControlServer: stop()
ProcessPilot->>User: Return
```
### Built-in Control Server Features
The ProcessPilot instance passed to the control server provides methods to:
- Start/stop/restart processes
- Get process statistics
- Access the process manifest
- Query process status
## Process Lifecycle Hooks
The following diagram illustrates the process lifecycle and when various hook functions are called:
```mermaid
graph TD
A[Start Process Pilot] --> B[Initialize Processes]
B --> C[Execute PRE_START Hooks]
C --> D[Start Process]
D --> E[Execute POST_START Hooks]
E --> F[Monitor Process]
F -->|Process Running| F
F -->|Process Exits| G[Execute ON_SHUTDOWN Hooks]
G --> H{Shutdown Strategy}
H -->|restart| I[Restart Process]
I --> J[Execute ON_RESTART Hooks]
J --> F
H -->|do_not_restart| K[Stop Monitoring]
H -->|shutdown_everything| L[Stop All Processes]
```
Ready strategies tie in to this via:
```mermaid
graph TD
A[Start Process] --> B[Check Ready Strategy]
B -->|Strategy 1| C[Execute Strategy 1]
B -->|Strategy 2| D[Execute Strategy 2]
B -->|Strategy 3| E[Execute Strategy 3]
C --> F[Process Ready]
D --> F[Process Ready]
E --> F[Process Ready]
```
Statistic handlers tie into this via:
```mermaid
graph TD
A[Monitor Process] --> B[Collect Statistics]
B --> C[Execute Statistic Handlers]
C --> D[Update Statistics]
D --> A
```
## Ready Strategies
Process Pilot supports three different strategies to determine if a process is ready, but more strategies can be provided via plugins:
1. TCP Port Listening
2. Named Pipe Signal
3. File Presence
Each ready strategy is only relevant for determining when dependent processes should be started. That is, if a given process has no dependencies, then specifying a ready strategy isn't currently meaningful. The following diagrams illustrate how each strategy works:
### TCP Ready Strategy
```mermaid
sequenceDiagram
participant PP as Process Pilot
participant P as Process
participant TCP as TCP Port
PP->>P: Start Process
activate P
P->>TCP: Begin Listening
loop Until Ready or Timeout
PP->>TCP: Attempt Connection
alt Port is Listening
TCP-->>PP: Connection Success
PP->>PP: Process Ready
else Port not ready
TCP-->>PP: Connection Failed
Note over PP: Wait 0.1s
end
end
deactivate P
```
### Named Pipe Ready Strategy
```mermaid
sequenceDiagram
participant PP as Process Pilot
participant P as Process
participant Pipe as Named Pipe
PP->>Pipe: Create Pipe
PP->>P: Start Process
activate P
loop Until Ready or Timeout
PP->>Pipe: Read Pipe
alt Contains "ready"
Pipe-->>PP: "ready"
PP->>PP: Process Ready
else Not Ready
Pipe-->>PP: No Data/Error
Note over PP: Wait 0.1s
end
end
deactivate P
```
### File Ready Strategy
```mermaid
sequenceDiagram
participant PP as Process Pilot
participant P as Process
participant FS as File System
PP->>P: Start Process
activate P
loop Until Ready or Timeout
PP->>FS: Check File
alt File Exists
FS-->>PP: File Found
PP->>PP: Process Ready
else No File
FS-->>PP: No File
Note over PP: Wait 0.1s
end
end
deactivate P
```
Each strategy can be configured in the manifest:
```yaml
processes:
- name: example
path: myapp
ready_strategy: tcp # or "pipe" or "file"
ready_timeout_sec: 10.0
ready_params:
port: 8080 # for TCP
path: "/tmp/ready.txt" # for File
```
## Dependency Graph Visualization
Process Pilot includes a tool to visualize process dependencies using Graphviz. This helps you understand and validate the relationships between your processes.
### Prerequisites
The graph visualization requires Graphviz to be installed on your system:
```sh
# Ubuntu/Debian
apt-get install graphviz
# macOS
brew install graphviz
# Windows
choco install graphviz
```
### Generating Dependency Graphs
You can generate a dependency graph from your process manifest using the `graph.py` module:
```sh
process-graph manifest.json --format png --output-dir ./graphs
```
### Command Line Options
- manifest_path: Path to your JSON or YAML manifest file (required)
- --format: Output format (png, svg, or pdf) - defaults to png
- --output-dir: Directory to save the generated graph
- --detailed: Include detailed process information in tooltips
### Graph Features
- Process nodes with their names
- Directed edges showing dependencies
- Color-coding for ready strategies:
- Light blue: TCP ready strategy
- Light green: File ready strategy
- Light yellow: Pipe ready strategy
- Detailed tooltips (when using --detailed) showing:
- Process path
- Ready strategy
- Timeout values
> NOTE: Detailed output is only available when the output is SVG
### Example
Given this manifest:
```json
{
"processes": [
{
"name": "database",
"path": "postgresql",
"ready_strategy": "tcp",
"ready_params": {
"port": 5432
}
},
{
"name": "api",
"path": "api_server",
"dependencies": ["database"],
"ready_strategy": "file",
"ready_params": {
"path": "/tmp/api_ready"
}
},
{
"name": "worker",
"path": "worker_service",
"dependencies": ["api", "database"]
}
]
}
```
You could generate the graph via:
```sh
process-graph manifest.json --format svg
```
This will create a graph that will show:
- `database` node (light blue) with no dependencies
- `api` node (light green) depending on `database`
- `worker` node (white) depending
## Development
### Running Tests
To run the tests, use:
```sh
poetry run pytest
```
### Build the documentation
To build the documentation, run the following from the top level of the repository:
```sh
poetry run sphinx-build -b html docs docs/_build/html
```
### Linting and Formatting
To lint and format the code, use:
```sh
poetry run ruff check .
poetry run autopep8 --in-place --recursive .
```
## License
This project is licensed under the MIT License. See the [LICENSE](../LICENSE) file for details.
## Contributing
Contributions are welcome! Please open an issue or submit a pull request.
## Contact
For any inquiries, please contact James Armes at jamesleearmes@gmail.com.
Raw data
{
"_id": null,
"home_page": "https://github.com/engineerjames/process-pilot",
"name": "process-pilot",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.13,>=3.10",
"maintainer_email": null,
"keywords": "process, management, monitoring",
"author": "James Armes",
"author_email": "jamesleearmes@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/77/c8/ac06e41da1088840866f82ffd0aeea08217e78eb6802640368004102e347/process_pilot-0.3.5.tar.gz",
"platform": null,
"description": "# Process Pilot\n\nProcess Pilot is a Python-based tool for managing and monitoring processes defined in a manifest file. It supports JSON and YAML formats for defining processes and their configurations.\n\n## Features\n\n- Load process configurations from JSON or YAML files.\n- Manage process lifecycles with customizable hooks.\n- Monitor process resource usage.\n- Define shutdown strategies for processes.\n- Define ready strategies to determine when launched processes are deemed to be \"running\".\n\n## Installation\n\nTo install the dependencies, use [Poetry](https://python-poetry.org/):\n\n```sh\npoetry install\n```\n\n## Usage\n\nYou can use the `ProcessPilot` class directly in your Python code to manage processes defined in a YAML or JSON file.\n\n### Process Control\n\nYou can restart specific processes by name:\n\n```python\nfrom process_pilot.pilot import ProcessPilot\nfrom process_pilot.process import ProcessManifest\nfrom pathlib import Path\n\n# Load manifest and start ProcessPilot\nmanifest = ProcessManifest.from_json(Path(\"manifest.json\"))\npilot = ProcessPilot(manifest)\npilot.start()\n\n# Later, restart specific processes\ntry:\n pilot.restart_processes([\"api_server\", \"worker\"])\nexcept ValueError as e:\n print(f\"Error restarting processes: {e}\")\n```\n\n### Example Usage\n\n#### Using a JSON Manifest\n\n```python\nfrom pathlib import Path\nfrom process_pilot.process import ProcessPilot, ProcessManifest\n\n# Load the process manifest from a JSON file\nmanifest_path = Path(\"path/to/your/manifest.json\")\nmanifest = ProcessManifest.from_json(manifest_path)\n\n# Create a ProcessPilot instance with the loaded manifest\npilot = ProcessPilot(manifest)\n\n# Start managing the processes\npilot.start()\n```\n\n#### Using a YAML Manifest\n\n```python\nfrom pathlib import Path\nfrom process_pilot.process import ProcessPilot, ProcessManifest\n\n# Load the process manifest from a YAML file\nmanifest_path = Path(\"path/to/your/manifest.yaml\")\nmanifest = ProcessManifest.from_yaml(manifest_path)\n\n# Create a ProcessPilot instance with the loaded manifest\npilot = ProcessPilot(manifest)\n\n# Start managing the processes\npilot.start()\n```\n\n## Configuration\n\n### Process Manifest\n\nThe process manifest defines the processes to be managed. It can be written in JSON or YAML format.\n\n#### Parameters\n\n- `name`: The name of the process. This should be unique within the manifest.\n- `path`: The path to the executable or script to be run.\n- `args`: A list of arguments to be passed to the process.\n- `timeout`: The maximum time (in seconds) to wait for the process to stop.\n- `shutdown_strategy`: The strategy to use when shutting down the process. Possible values are:\n - `do_not_restart`: Do not restart the process after it stops.\n - `restart`: Restart the process after it stops. This is the default.\n - `shutdown_everything`: Stop all processes when this process stops.\n- `ready_strategy`: The strategy to use to determine when the process is ready. Possible values are:\n - `tcp`: The process is ready when it starts listening on a specified TCP port.\n - `pipe`: The process is ready when it writes a specific signal to a named pipe.\n - `file`: The process is ready when a specific file is created.\n- `ready_timeout_sec`: The maximum time (in seconds) to wait for the process to be ready.\n- `ready_params`: Additional parameters for the ready strategy. These vary based on the strategy:\n - For `tcp`, specify the `port` to check.\n - For `pipe`, specify the `path` to the named pipe.\n - For `file`, specify the `path` to the file.\n- `dependencies`: A list of other process names that must be started before this process can be started.\n- `env`: A dictionary of environment variables to set for the process.\n\nThe following is an example of a JSON manifest:\n\n```json\n{\n \"processes\": [\n {\n \"name\": \"example\",\n \"path\": \"sleep\",\n \"args\": [\"5\"],\n \"timeout\": 3,\n \"shutdown_strategy\": \"do_not_restart\",\n \"ready_strategy\": \"tcp\",\n \"ready_timeout_sec\": 10.0,\n \"ready_params\": {\n \"port\": 8080\n },\n \"dependencies\": [\"another_process\"],\n \"env\": {\n \"ENV_VAR\": \"value\"\n }\n }\n ]\n}\n```\n\nThe following is an example of a YAML manifest:\n\n```yaml\nprocesses:\n - name: example\n path: sleep\n args: [\"5\"]\n timeout: 1.0\n shutdown_strategy: do_not_restart\n ready_strategy: tcp\n ready_timeout_sec: 10.0\n ready_params:\n port: 8080\n dependencies:\n - another_process\n env:\n ENV_VAR: value\n```\n\n## Plugin System\n\nProcess Pilot supports a plugin system that allows users to extend its functionality with custom lifecycle hooks, ready strategies, and process statistics handlers. Each created plugin can provide all three of the aforementioned groups, or just one. Each registration function links a name to a function or set of functions. The name provided must match what is in the manifest.\n\n### Plugin Registration Scoping\n\nPlugins in Process Pilot have one distinct registration scope:\n\n1. **Process-Specific Hooks**: Process lifecycle hooks, ready strategies, and process statistic handlers are only registered for processes that explicitly request them in their manifest configuration.\n\n### Creating a Plugin\n\nTo create a plugin, define a class that inherits from `Plugin` and implement the required methods:\n\n```python\n# import statements...\n\nclass FileReadyPlugin(Plugin):\n def get_ready_strategies(self) -> dict[str, ReadyStrategyType]:\n return {\n \"file\": self._wait_file_ready,\n }\n\n def _wait_file_ready(self, process: \"Process\", ready_check_interval_secs: float) -> bool:\n file_path = process.ready_params.get(\"path\")\n\n if not file_path:\n msg = \"Path not specified for file ready strategy\"\n raise RuntimeError(msg)\n\n file_path = Path(file_path)\n\n start_time = time.time()\n while (time.time() - start_time) < process.ready_timeout_sec:\n if file_path.exists():\n return True\n time.sleep(ready_check_interval_secs)\n return False\n```\n\nWhen creating plugins that implement a ready strategy it is important to keep in mind that you should always be checking readiness relative to\nthe start time--and always comparing the difference to the timeout value that is specified in the manifest. The\nsimplest example of this can be seen in the `FileReadyPlugin` shown above:\n\n```python\nstart_time = time.time()\nwhile (time.time() - start_time) < process.ready_timeout_sec:\n if file_path.exists():\n return True\n time.sleep(ready_check_interval_secs)\n\n# Timeout\nreturn False\n```\n\nBe careful not to use readiness checks that block the threads ability to check for a timeout condition.\n\n### Plugin Registration\n\nOne way to use plugins with specific processes is to specify them in the manifest as shown below:\n\n```json\n{\n \"processes\": [\n {\n \"name\": \"example_process\",\n \"path\": \"myapp\",\n \"lifecycle_hooks\": [\"custom_lifecycle_hooks\"],\n \"ready_strategy\": \"custom_strategy\",\n \"ready_timeout_sec\": 10.0\n },\n {\n \"name\": \"another_process\",\n \"path\": \"otherapp\"\n }\n ]\n}\n```\n\nIn this example, you must have loaded a plugin that provides a lifecycle hook with the name \"custom_lifecycle_hooks\" and a plugin that implements a ready strategy named \"custom_strategy\"--they do not have to be in the same plugin, but they could be.\n\nPlugins themselves are registered with process-pilot either in code directly:\n\n```python\nfrom pathlib import Path\nfrom process_pilot.process import ProcessPilot, ProcessManifest\nfrom custom_plugin import CustomPlugin\n\n# Load manifest\nmanifest = ProcessManifest.from_json(Path(\"manifest.json\"))\n\n# Create pilot and register plugins\npilot = ProcessPilot(manifest)\npilot.register_plugins([CustomPlugin()])\n\n# Start processes\npilot.start()\n```\n\nOr, by providing a plugin directory when the ProcessPilot object is constructed. See the API documentation for the ProcessPilot class, and the Plugin class for more details.\n\n## Control Server Plugins\n\nControl server plugins provide a way to add remote control capabilities to Process Pilot. A control server can be used to monitor and control processes at runtime through various interfaces (HTTP, TCP, Unix socket, etc.).\n\n### Creating a Control Server Plugin\n\nTo create a control server plugin:\n\n1. Create a class that implements your control server with at least these methods:\n\n - `__init__(self, pilot: ProcessPilot)` - Constructor that receives the pilot instance\n - `start(self)` - Called when Process Pilot starts\n - `stop(self)` - Called when Process Pilot shuts down\n\n2. Create a plugin class that inherits from `Plugin` and registers your control server:\n\n```python\nfrom process_pilot.plugin import Plugin, ControlServerType\nfrom process_pilot.pilot import ProcessPilot\n\nclass MyControlServer:\n def __init__(self, pilot: ProcessPilot):\n self.pilot = pilot\n\n def start(self):\n # Start your control server\n pass\n\n def stop(self):\n # Clean up resources\n pass\n\nclass MyControlServerPlugin(Plugin):\n def get_control_servers(self) -> dict[str, ControlServerType]:\n return {\"my_server\": lambda pilot: MyControlServer(pilot)}\n```\n\n3. Register your plugin and enable it in the manifest:\n\n```python\npilot = ProcessPilot(manifest)\npilot.register_plugins([MyControlServerPlugin()])\n```\n\n```yaml\nmanifest:\n control_server: my_server # Enable your control server\n processes:\n # ... process definitions ...\n```\n\n### Control Server Lifecycle\n\nThe following diagram shows when the control server is started and stopped:\n\n```mermaid\nsequenceDiagram\n participant User\n participant ProcessPilot\n participant ControlServer\n participant Processes\n\n User->>ProcessPilot: Create(manifest)\n ProcessPilot->>ProcessPilot: Load plugins\n ProcessPilot->>ControlServer: Create\n\n User->>ProcessPilot: start()\n ProcessPilot->>ControlServer: start()\n ProcessPilot->>Processes: Start processes\n\n Note over ControlServer,Processes: Control server running...\n\n User->>ProcessPilot: shutdown()\n ProcessPilot->>Processes: Stop processes\n ProcessPilot->>ControlServer: stop()\n ProcessPilot->>User: Return\n```\n\n### Built-in Control Server Features\n\nThe ProcessPilot instance passed to the control server provides methods to:\n\n- Start/stop/restart processes\n- Get process statistics\n- Access the process manifest\n- Query process status\n\n## Process Lifecycle Hooks\n\nThe following diagram illustrates the process lifecycle and when various hook functions are called:\n\n```mermaid\ngraph TD\n A[Start Process Pilot] --> B[Initialize Processes]\n B --> C[Execute PRE_START Hooks]\n C --> D[Start Process]\n D --> E[Execute POST_START Hooks]\n E --> F[Monitor Process]\n F -->|Process Running| F\n F -->|Process Exits| G[Execute ON_SHUTDOWN Hooks]\n G --> H{Shutdown Strategy}\n H -->|restart| I[Restart Process]\n I --> J[Execute ON_RESTART Hooks]\n J --> F\n H -->|do_not_restart| K[Stop Monitoring]\n H -->|shutdown_everything| L[Stop All Processes]\n```\n\nReady strategies tie in to this via:\n\n```mermaid\ngraph TD\nA[Start Process] --> B[Check Ready Strategy]\nB -->|Strategy 1| C[Execute Strategy 1]\nB -->|Strategy 2| D[Execute Strategy 2]\nB -->|Strategy 3| E[Execute Strategy 3]\nC --> F[Process Ready]\nD --> F[Process Ready]\nE --> F[Process Ready]\n```\n\nStatistic handlers tie into this via:\n\n```mermaid\ngraph TD\n A[Monitor Process] --> B[Collect Statistics]\n B --> C[Execute Statistic Handlers]\n C --> D[Update Statistics]\n D --> A\n```\n\n## Ready Strategies\n\nProcess Pilot supports three different strategies to determine if a process is ready, but more strategies can be provided via plugins:\n\n1. TCP Port Listening\n2. Named Pipe Signal\n3. File Presence\n\nEach ready strategy is only relevant for determining when dependent processes should be started. That is, if a given process has no dependencies, then specifying a ready strategy isn't currently meaningful. The following diagrams illustrate how each strategy works:\n\n### TCP Ready Strategy\n\n```mermaid\nsequenceDiagram\n participant PP as Process Pilot\n participant P as Process\n participant TCP as TCP Port\n PP->>P: Start Process\n activate P\n P->>TCP: Begin Listening\n loop Until Ready or Timeout\n PP->>TCP: Attempt Connection\n alt Port is Listening\n TCP-->>PP: Connection Success\n PP->>PP: Process Ready\n else Port not ready\n TCP-->>PP: Connection Failed\n Note over PP: Wait 0.1s\n end\n end\n deactivate P\n```\n\n### Named Pipe Ready Strategy\n\n```mermaid\nsequenceDiagram\n participant PP as Process Pilot\n participant P as Process\n participant Pipe as Named Pipe\n PP->>Pipe: Create Pipe\n PP->>P: Start Process\n activate P\n loop Until Ready or Timeout\n PP->>Pipe: Read Pipe\n alt Contains \"ready\"\n Pipe-->>PP: \"ready\"\n PP->>PP: Process Ready\n else Not Ready\n Pipe-->>PP: No Data/Error\n Note over PP: Wait 0.1s\n end\n end\n deactivate P\n```\n\n### File Ready Strategy\n\n```mermaid\nsequenceDiagram\n participant PP as Process Pilot\n participant P as Process\n participant FS as File System\n PP->>P: Start Process\n activate P\n loop Until Ready or Timeout\n PP->>FS: Check File\n alt File Exists\n FS-->>PP: File Found\n PP->>PP: Process Ready\n else No File\n FS-->>PP: No File\n Note over PP: Wait 0.1s\n end\n end\n deactivate P\n```\n\nEach strategy can be configured in the manifest:\n\n```yaml\nprocesses:\n - name: example\n path: myapp\n ready_strategy: tcp # or \"pipe\" or \"file\"\n ready_timeout_sec: 10.0\n ready_params:\n port: 8080 # for TCP\n path: \"/tmp/ready.txt\" # for File\n```\n\n## Dependency Graph Visualization\n\nProcess Pilot includes a tool to visualize process dependencies using Graphviz. This helps you understand and validate the relationships between your processes.\n\n### Prerequisites\n\nThe graph visualization requires Graphviz to be installed on your system:\n\n```sh\n# Ubuntu/Debian\napt-get install graphviz\n\n# macOS\nbrew install graphviz\n\n# Windows\nchoco install graphviz\n```\n\n### Generating Dependency Graphs\n\nYou can generate a dependency graph from your process manifest using the `graph.py` module:\n\n```sh\nprocess-graph manifest.json --format png --output-dir ./graphs\n```\n\n### Command Line Options\n\n- manifest_path: Path to your JSON or YAML manifest file (required)\n- --format: Output format (png, svg, or pdf) - defaults to png\n- --output-dir: Directory to save the generated graph\n- --detailed: Include detailed process information in tooltips\n\n### Graph Features\n\n- Process nodes with their names\n- Directed edges showing dependencies\n- Color-coding for ready strategies:\n - Light blue: TCP ready strategy\n - Light green: File ready strategy\n - Light yellow: Pipe ready strategy\n- Detailed tooltips (when using --detailed) showing:\n - Process path\n - Ready strategy\n - Timeout values\n\n> NOTE: Detailed output is only available when the output is SVG\n\n### Example\n\nGiven this manifest:\n\n```json\n{\n \"processes\": [\n {\n \"name\": \"database\",\n \"path\": \"postgresql\",\n \"ready_strategy\": \"tcp\",\n \"ready_params\": {\n \"port\": 5432\n }\n },\n {\n \"name\": \"api\",\n \"path\": \"api_server\",\n \"dependencies\": [\"database\"],\n \"ready_strategy\": \"file\",\n \"ready_params\": {\n \"path\": \"/tmp/api_ready\"\n }\n },\n {\n \"name\": \"worker\",\n \"path\": \"worker_service\",\n \"dependencies\": [\"api\", \"database\"]\n }\n ]\n}\n```\n\nYou could generate the graph via:\n\n```sh\nprocess-graph manifest.json --format svg\n```\n\nThis will create a graph that will show:\n\n- `database` node (light blue) with no dependencies\n- `api` node (light green) depending on `database`\n- `worker` node (white) depending\n\n## Development\n\n### Running Tests\n\nTo run the tests, use:\n\n```sh\npoetry run pytest\n```\n\n### Build the documentation\n\nTo build the documentation, run the following from the top level of the repository:\n\n```sh\npoetry run sphinx-build -b html docs docs/_build/html\n```\n\n### Linting and Formatting\n\nTo lint and format the code, use:\n\n```sh\npoetry run ruff check .\npoetry run autopep8 --in-place --recursive .\n```\n\n## License\n\nThis project is licensed under the MIT License. See the [LICENSE](../LICENSE) file for details.\n\n## Contributing\n\nContributions are welcome! Please open an issue or submit a pull request.\n\n## Contact\n\nFor any inquiries, please contact James Armes at jamesleearmes@gmail.com.\n",
"bugtrack_url": null,
"license": null,
"summary": "Process manager for Python",
"version": "0.3.5",
"project_urls": {
"Documentation": "https://process-pilot.readthedocs.io/",
"Homepage": "https://github.com/engineerjames/process-pilot",
"Repository": "https://github.com/engineerjames/process-pilot"
},
"split_keywords": [
"process",
" management",
" monitoring"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "74a7cc27c627a9a3bf9e4bafa3153fbac44e88c2da0cc81e2016afd0c9509563",
"md5": "352ac28e2aaa1d4a1b568b7b39b0c585",
"sha256": "82112bb87d092e1930fe23d24536b1769bd66c5aee364a1c8f6b53506d77e20b"
},
"downloads": -1,
"filename": "process_pilot-0.3.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "352ac28e2aaa1d4a1b568b7b39b0c585",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.13,>=3.10",
"size": 23483,
"upload_time": "2025-01-13T16:33:01",
"upload_time_iso_8601": "2025-01-13T16:33:01.009582Z",
"url": "https://files.pythonhosted.org/packages/74/a7/cc27c627a9a3bf9e4bafa3153fbac44e88c2da0cc81e2016afd0c9509563/process_pilot-0.3.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "77c8ac06e41da1088840866f82ffd0aeea08217e78eb6802640368004102e347",
"md5": "204a130b0eaae0b4ecee9bfd6d6ffc84",
"sha256": "eb4e36273354e8316e88aa3323fd4202f71ea0018950bd9ca1184b692b96e96c"
},
"downloads": -1,
"filename": "process_pilot-0.3.5.tar.gz",
"has_sig": false,
"md5_digest": "204a130b0eaae0b4ecee9bfd6d6ffc84",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.13,>=3.10",
"size": 23610,
"upload_time": "2025-01-13T16:33:04",
"upload_time_iso_8601": "2025-01-13T16:33:04.914856Z",
"url": "https://files.pythonhosted.org/packages/77/c8/ac06e41da1088840866f82ffd0aeea08217e78eb6802640368004102e347/process_pilot-0.3.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-13 16:33:04",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "engineerjames",
"github_project": "process-pilot",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "sphinx",
"specs": []
},
{
"name": "sphinx-rtd-theme",
"specs": []
},
{
"name": "pylint",
"specs": []
},
{
"name": "pytest",
"specs": []
},
{
"name": "ruff",
"specs": []
},
{
"name": "mypy",
"specs": []
},
{
"name": "autopep8",
"specs": []
},
{
"name": "types-pyyaml",
"specs": []
},
{
"name": "pytest-mock",
"specs": []
},
{
"name": "types-psutil",
"specs": []
},
{
"name": "pydantic",
"specs": []
},
{
"name": "pyyaml",
"specs": []
},
{
"name": "psutil",
"specs": []
},
{
"name": "myst-parser",
"specs": []
},
{
"name": "sphinxcontrib-mermaid",
"specs": []
}
],
"lcname": "process-pilot"
}
JSON
