| Name | pysof JSON |
| Version |
0.1.3
JSON |
| download |
| home_page | None |
| Summary | Python wrapper for the Helios Software SOF (SQL on FHIR) toolkit. |
| upload_time | 2025-10-19 20:00:33 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | <3.12,>=3.11 |
| license | MIT |
| keywords |
sql on fhir
fhir
healthcare
sof
helios software
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# pysof
Python wrapper for the Helios SOF (SQL on FHIR) toolkit.
This package provides Python bindings for the Rust `helios-sof` library via PyO3 and maturin. Use `uv` to manage the environment and run builds.
> **Note**: This crate is excluded from the default workspace build via workspace default-members. When running `cargo build` from the repository root, `pysof` will not be built automatically. Build it explicitly when needed.
## Requirements
- Python 3.11
- uv (package and environment manager)
## Building the Crate
### Building with Cargo
This crate is excluded from the default workspace build to allow building the core Rust components without Python. To build it explicitly:
```bash
# Your current directory MUST be the pysof crate:
cd crates/pysof
# From the pysof folder
cargo build
# Or build with specific FHIR version features
cargo build -p pysof --features R4,R5
```
### Building with Maturin (Recommended)
For Python development, it's recommended to use `maturin` via `uv`:
```bash
# From repo root
cd crates/pysof
# Ensure Python 3.11 is available and create a venv
uv venv --python 3.11
# Install the project dev dependencies
uv sync --group dev
# Build and install the Rust extension into the venv
uv run maturin develop --release
# Build distributable artifacts
uv run maturin build --release -o dist # wheels
uv run maturin sdist -o dist # source distribution
# Sanity checks
uv run python -c "import pysof; print(pysof.__version__); print(pysof.get_status()); print(pysof.get_supported_fhir_versions())"
```
## Installation
### From PyPI (Recommended)
```bash
pip install pysof
```
### From Source
```bash
# Install from source (requires Rust toolchain)
pip install -e .
# Or build wheel locally
maturin build --release --out dist
pip install dist/*.whl
```
### From GitHub Releases
Download the appropriate wheel for your platform from the [releases page](https://github.com/HeliosSoftware/hfs/releases) and install:
```bash
pip install pysof-*.whl
```
### Supported Platforms
- **Linux**: x86_64 (glibc and musl)
- **Windows**: x86_64 (MSVC)
- **macOS**: AArch64 (x86_64 is not supported)
- **Python**: 3.11 only
For detailed information about wheel building and distribution, see [WHEEL_BUILDING.md](WHEEL_BUILDING.md).
## Testing
The project has separate test suites for Python and Rust components:
### Python Tests
Run the comprehensive Python test suite:
```bash
# Run all Python tests
uv run pytest python-tests/
# Run specific test files
uv run pytest python-tests/test_core_functions.py -v
uv run pytest python-tests/test_content_types.py -v
uv run pytest python-tests/test_import.py -v
# Run with coverage
uv run pytest python-tests/ --cov=pysof --cov-report=html
# Run tests with detailed output
uv run pytest python-tests/ -v --tb=short
```
Current Python test coverage:
- **58 total tests** across 5 test files
- Core API functions (19 tests)
- Content type support (14 tests)
- FHIR version support (16 tests)
- Package structure and imports (6 tests)
- Package metadata (3 tests)
### Rust Tests
Run the Rust unit and integration tests:
```bash
# Run all Rust tests
cargo test
# Run unit tests only
cargo test --test lib_tests
# Run integration tests only
cargo test --test integration
# Run with verbose output
cargo test -- --nocapture
```
Current Rust test coverage:
- **17 total tests** across 2 test files
- Unit tests: 14 tests covering core library functions
- Integration tests: 3 tests covering component interactions
## Usage
### Basic Example
```python
import pysof
import json
# Sample ViewDefinition for extracting patient data
view_definition = {
"resourceType": "ViewDefinition",
"id": "patient-demographics",
"name": "PatientDemographics",
"status": "active",
"resource": "Patient",
"select": [
{
"column": [
{"name": "id", "path": "id"},
{"name": "family_name", "path": "name.family"},
{"name": "given_name", "path": "name.given.first()"},
{"name": "gender", "path": "gender"},
{"name": "birth_date", "path": "birthDate"}
]
}
]
}
# Sample FHIR Bundle with patient data
bundle = {
"resourceType": "Bundle",
"type": "collection",
"entry": [
{
"resource": {
"resourceType": "Patient",
"id": "patient-1",
"name": [{"family": "Doe", "given": ["John"]}],
"gender": "male",
"birthDate": "1990-01-01"
}
},
{
"resource": {
"resourceType": "Patient",
"id": "patient-2",
"name": [{"family": "Smith", "given": ["Jane"]}],
"gender": "female",
"birthDate": "1985-05-15"
}
}
]
}
# Transform to different formats
csv_result = pysof.run_view_definition(view_definition, bundle, "csv")
json_result = pysof.run_view_definition(view_definition, bundle, "json")
ndjson_result = pysof.run_view_definition(view_definition, bundle, "ndjson")
parquet_result = pysof.run_view_definition(view_definition, bundle, "parquet")
print("CSV Output:")
print(csv_result.decode('utf-8'))
print("\nJSON Output:")
data = json.loads(json_result.decode('utf-8'))
print(json.dumps(data, indent=2))
print("\nParquet Output (binary):")
print(f"Parquet data: {len(parquet_result)} bytes")
```
### Advanced Usage with Options
```python
import pysof
from datetime import datetime
# Transform with pagination and filtering
result = pysof.run_view_definition_with_options(
view_definition,
bundle,
"json",
limit=10, # Limit to 10 results
page=1, # First page
since="2023-01-01T00:00:00Z", # Filter by modification date
fhir_version="R4" # Specify FHIR version
)
```
### Utility Functions
```python
import pysof
# Validate structures before processing
is_valid_view = pysof.validate_view_definition(view_definition)
is_valid_bundle = pysof.validate_bundle(bundle)
# Parse content types
format_str = pysof.parse_content_type("text/csv") # Returns "csv_with_header"
format_str = pysof.parse_content_type("application/json") # Returns "json"
# Check supported FHIR versions
versions = pysof.get_supported_fhir_versions() # Returns ["R4"] (or more based on build)
print(f"Supported FHIR versions: {versions}")
# Check package status
print(pysof.get_status()) # Shows current implementation status
print(f"Version: {pysof.get_version()}")
```
### Error Handling
```python
import pysof
try:
result = pysof.run_view_definition(view_definition, bundle, "json")
except pysof.InvalidViewDefinitionError as e:
print(f"ViewDefinition validation error: {e}")
except pysof.SerializationError as e:
print(f"JSON parsing error: {e}")
except pysof.UnsupportedContentTypeError as e:
print(f"Unsupported format: {e}")
except pysof.SofError as e:
print(f"General SOF error: {e}")
```
### Key Features
- **Performance**: Efficient processing of large FHIR Bundles using Rust
- **Parallel Processing**: Automatic multithreading of FHIR resources using rayon (5-7x speedup on multi-core systems)
- **GIL Release**: Python GIL is released during Rust execution for better performance
- **Multiple Formats**: Support for CSV, JSON, NDJSON, and Parquet outputs
- **FHIR Versions**: Support for R4, R4B, R5, and R6 (depending on build features)
### Available Options
The `run_view_definition_with_options()` function accepts the following parameters:
- **limit**: Limit the number of results returned
- **page**: Page number for pagination (1-based)
- **since**: Filter resources modified after this ISO8601 datetime
- **fhir_version**: FHIR version to use ("R4", "R4B", "R5", "R6")
### Supported Content Types
| Format | Description | Output |
|--------|-------------|---------|
| `csv` | CSV with headers | Comma-separated values with header row |
| `json` | JSON array | Array of objects, one per result row |
| `ndjson` | Newline-delimited JSON | One JSON object per line |
| `parquet` | Parquet format | Columnar binary format optimized for analytics |
### Supported FHIR Versions
- **R4** (default, always available)
- **R4B** (if compiled with R4B feature)
- **R5** (if compiled with R5 feature)
- **R6** (if compiled with R6 feature)
Use `pysof.get_supported_fhir_versions()` to check what's available in your build.
### Usage Notes
- The basic `run_view_definition()` function is suitable for simple use cases
- Use `run_view_definition_with_options()` for pagination, filtering, and other advanced features
- All heavy processing happens in Rust code; Python GIL is properly released for optimal performance
## Multithreading and Performance
### Automatic Parallel Processing
pysof automatically processes FHIR resources in parallel using rayon, providing significant performance improvements on multi-core systems:
- **5-7x speedup** for typical workloads on modern CPUs
- **Zero configuration required** - parallelization is always enabled
- **Python GIL released** during processing for true parallel execution
### Controlling Thread Count
By default, pysof uses all available CPU cores. You can control the thread count using the `RAYON_NUM_THREADS` environment variable:
```python
import os
import pysof
# Set thread count BEFORE first use (must be set before rayon initializes)
os.environ['RAYON_NUM_THREADS'] = '4'
# Now all operations will use 4 threads
result = pysof.run_view_definition(view_definition, bundle, "json")
```
Or from the command line:
```bash
# Linux/Mac
RAYON_NUM_THREADS=4 python my_script.py
# Windows (cmd)
set RAYON_NUM_THREADS=4
python my_script.py
# Windows (PowerShell)
$env:RAYON_NUM_THREADS=4
python my_script.py
```
**Important**: The environment variable must be set before rayon initializes its global thread pool (typically on first use of pysof). Once initialized, the thread count cannot be changed for that process.
### Performance Tips
- **Large datasets**: Use all available cores (default behavior)
- **Shared systems**: Limit threads to avoid resource contention (`RAYON_NUM_THREADS=4`)
- **Memory constrained**: Reduce thread count to lower memory usage
- **Benchmarking**: Test different thread counts to find optimal performance for your workload
## Configuring FHIR Version Support
By default, pysof is compiled with **R4 support only**. You can configure which FHIR versions are available by modifying the feature compilation settings.
### Change Default FHIR Version
To change from R4 to another version (e.g., R5):
1. **Edit `crates/pysof/Cargo.toml`**:
```toml
[features]
default = ["R5"] # Changed from ["R4"]
R4 = ["helios-sof/R4", "helios-fhir/R4"]
R4B = ["helios-sof/R4B", "helios-fhir/R4B"]
R5 = ["helios-sof/R5", "helios-fhir/R5"]
R6 = ["helios-sof/R6", "helios-fhir/R6"]
```
2. **Rebuild the extension**:
```bash
cd crates/pysof
uv run maturin develop --release
```
3. **Verify the change**:
```bash
uv run python -c "
import pysof
versions = pysof.get_supported_fhir_versions()
print('Supported FHIR versions:', versions)
"
```
This should now show `['R5']` instead of `['R4']`.
### Enable Multiple FHIR Versions
To support multiple FHIR versions simultaneously:
1. **Edit `crates/pysof/Cargo.toml`**:
```toml
[features]
default = ["R4", "R5"] # Enable both R4 and R5
# Or enable all versions:
# default = ["R4", "R4B", "R5", "R6"]
```
2. **Rebuild and verify**:
```bash
uv run maturin develop --release
uv run python -c "import pysof; print(pysof.get_supported_fhir_versions())"
```
This should show `['R4', 'R5']` (or all enabled versions).
3. **Use specific versions in code**:
```python
import pysof
# Use R4 explicitly
result_r4 = pysof.run_view_definition(view, bundle, "json", fhir_version="R4")
# Use R5 explicitly
result_r5 = pysof.run_view_definition(view, bundle, "json", fhir_version="R5")
```
### Build with Specific Features (Without Changing Default)
To temporarily build with different features without modifying `Cargo.toml`:
```bash
# Build with only R5
cargo build --features R5 --no-default-features
# Build with R4 and R6
cargo build --features R4,R6 --no-default-features
# With maturin
uv run --with maturin -- maturin develop --release --cargo-extra-args="--features R5 --no-default-features"
```
### Testing After Version Changes
After changing FHIR version support, run the test suite to ensure compatibility:
```bash
# Run all tests
uv run pytest
# Run FHIR version-specific tests
uv run pytest tests/test_fhir_versions.py -v
# Test with your new default version
uv run python -c "
import pysof
# Test with default version (should be your new default)
view = {'resourceType': 'ViewDefinition', 'id': 'test', 'name': 'Test', 'status': 'active', 'resource': 'Patient', 'select': [{'column': [{'name': 'id', 'path': 'id'}]}]}
bundle = {'resourceType': 'Bundle', 'type': 'collection', 'entry': [{'resource': {'resourceType': 'Patient', 'id': 'test'}}]}
result = pysof.run_view_definition(view, bundle, 'json')
print('Default version test successful:', len(result), 'bytes')
"
```
### Important Considerations
1. **Dependency Requirements**: Ensure the underlying `helios-sof` and `helios-fhir` crates support the features you want to enable
2. **Binary Size**: Enabling multiple FHIR versions increases the compiled binary size significantly
3. **Memory Usage**: Multiple versions require more memory at runtime
4. **Testing**: Always run your test suite after changing version support
5. **Documentation**: Update any project documentation that mentions specific FHIR versions
6. **CI/CD**: Update build scripts and CI workflows if they depend on specific versions
### Version Compatibility Matrix
| FHIR Version | Feature Flag | Status | Notes |
|--------------|--------------|---------|-------|
| R4 | `R4` | ✅ Stable | Default, always recommended |
| R4B | `R4B` | ✅ Stable | Minor updates to R4 |
| R5 | `R5` | ✅ Stable | Current latest stable version |
| R6 | `R6` | ⚠️ Preview | May have limited support |
### Common Configuration Examples
```toml
# Production: Single version for minimal size
default = ["R4"]
# Development: Multiple versions for testing
default = ["R4", "R5"]
# Bleeding edge: Latest versions only
default = ["R5", "R6"]
# Maximum compatibility: All versions (not recommended for production)
default = ["R4", "R4B", "R5", "R6"]
```
## Project layout
```
crates/pysof/
├─ pyproject.toml # PEP 621 metadata, Python >=3.11, uv-compatible
├─ README.md
├─ src/
│ ├─ pysof/
│ │ └─ __init__.py # Python package root
│ └─ lib.rs # Rust PyO3 bindings
├─ tests/ # Rust tests (17 tests)
│ ├─ lib_tests.rs # Unit tests for core library functions
│ ├─ integration.rs # Integration tests for component interactions
│ └─ integration/ # Organized integration test modules
│ ├─ mod.rs
│ ├─ content_types.rs
│ ├─ error_handling.rs
│ └─ fhir_versions.rs
├─ python-tests/ # Python test suite (58 tests)
│ ├─ __init__.py
│ ├─ test_core_functions.py
│ ├─ test_content_types.py
│ ├─ test_fhir_versions.py
│ ├─ test_import.py
│ └─ test_package_metadata.py
└─ Cargo.toml # Rust crate metadata
```
## License
This package inherits the license from the repository root.
Raw data
{
"_id": null,
"home_page": null,
"name": "pysof",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.12,>=3.11",
"maintainer_email": null,
"keywords": "SQL on FHIR, FHIR, healthcare, sof, Helios Software",
"author": null,
"author_email": "Helios Software <team@heliossoftware.com>",
"download_url": "https://files.pythonhosted.org/packages/4f/f9/8c223ad836c86ccba42b2c849981b3abcba9772f940f0d803750ab8c28f5/pysof-0.1.3.tar.gz",
"platform": null,
"description": "# pysof\r\n\r\nPython wrapper for the Helios SOF (SQL on FHIR) toolkit.\r\n\r\nThis package provides Python bindings for the Rust `helios-sof` library via PyO3 and maturin. Use `uv` to manage the environment and run builds.\r\n\r\n> **Note**: This crate is excluded from the default workspace build via workspace default-members. When running `cargo build` from the repository root, `pysof` will not be built automatically. Build it explicitly when needed.\r\n\r\n## Requirements\r\n\r\n- Python 3.11\r\n- uv (package and environment manager)\r\n\r\n## Building the Crate\r\n\r\n### Building with Cargo\r\n\r\nThis crate is excluded from the default workspace build to allow building the core Rust components without Python. To build it explicitly:\r\n\r\n```bash\r\n# Your current directory MUST be the pysof crate:\r\ncd crates/pysof\r\n\r\n# From the pysof folder\r\ncargo build\r\n\r\n# Or build with specific FHIR version features\r\ncargo build -p pysof --features R4,R5\r\n```\r\n\r\n### Building with Maturin (Recommended)\r\n\r\nFor Python development, it's recommended to use `maturin` via `uv`:\r\n\r\n```bash\r\n# From repo root\r\ncd crates/pysof\r\n\r\n# Ensure Python 3.11 is available and create a venv\r\nuv venv --python 3.11\r\n\r\n# Install the project dev dependencies\r\nuv sync --group dev\r\n\r\n# Build and install the Rust extension into the venv\r\nuv run maturin develop --release\r\n\r\n# Build distributable artifacts\r\nuv run maturin build --release -o dist # wheels\r\nuv run maturin sdist -o dist # source distribution\r\n\r\n# Sanity checks\r\nuv run python -c \"import pysof; print(pysof.__version__); print(pysof.get_status()); print(pysof.get_supported_fhir_versions())\"\r\n```\r\n\r\n## Installation\r\n\r\n### From PyPI (Recommended)\r\n\r\n```bash\r\npip install pysof\r\n```\r\n\r\n### From Source\r\n\r\n```bash\r\n# Install from source (requires Rust toolchain)\r\npip install -e .\r\n\r\n# Or build wheel locally\r\nmaturin build --release --out dist\r\npip install dist/*.whl\r\n```\r\n\r\n### From GitHub Releases\r\n\r\nDownload the appropriate wheel for your platform from the [releases page](https://github.com/HeliosSoftware/hfs/releases) and install:\r\n\r\n```bash\r\npip install pysof-*.whl\r\n```\r\n\r\n### Supported Platforms\r\n\r\n- **Linux**: x86_64 (glibc and musl)\r\n- **Windows**: x86_64 (MSVC)\r\n- **macOS**: AArch64 (x86_64 is not supported) \r\n- **Python**: 3.11 only\r\n\r\nFor detailed information about wheel building and distribution, see [WHEEL_BUILDING.md](WHEEL_BUILDING.md).\r\n\r\n## Testing\r\n\r\nThe project has separate test suites for Python and Rust components:\r\n\r\n### Python Tests\r\n\r\nRun the comprehensive Python test suite:\r\n\r\n```bash\r\n# Run all Python tests\r\nuv run pytest python-tests/\r\n\r\n# Run specific test files\r\nuv run pytest python-tests/test_core_functions.py -v\r\nuv run pytest python-tests/test_content_types.py -v\r\nuv run pytest python-tests/test_import.py -v\r\n\r\n# Run with coverage\r\nuv run pytest python-tests/ --cov=pysof --cov-report=html\r\n\r\n# Run tests with detailed output\r\nuv run pytest python-tests/ -v --tb=short\r\n```\r\n\r\nCurrent Python test coverage:\r\n- **58 total tests** across 5 test files\r\n- Core API functions (19 tests)\r\n- Content type support (14 tests) \r\n- FHIR version support (16 tests)\r\n- Package structure and imports (6 tests)\r\n- Package metadata (3 tests)\r\n\r\n### Rust Tests\r\n\r\nRun the Rust unit and integration tests:\r\n\r\n```bash\r\n# Run all Rust tests\r\ncargo test\r\n\r\n# Run unit tests only\r\ncargo test --test lib_tests\r\n\r\n# Run integration tests only\r\ncargo test --test integration\r\n\r\n# Run with verbose output\r\ncargo test -- --nocapture\r\n```\r\n\r\nCurrent Rust test coverage:\r\n- **17 total tests** across 2 test files\r\n- Unit tests: 14 tests covering core library functions\r\n- Integration tests: 3 tests covering component interactions\r\n\r\n## Usage\r\n\r\n### Basic Example\r\n\r\n```python\r\nimport pysof\r\nimport json\r\n\r\n# Sample ViewDefinition for extracting patient data\r\nview_definition = {\r\n \"resourceType\": \"ViewDefinition\",\r\n \"id\": \"patient-demographics\",\r\n \"name\": \"PatientDemographics\", \r\n \"status\": \"active\",\r\n \"resource\": \"Patient\",\r\n \"select\": [\r\n {\r\n \"column\": [\r\n {\"name\": \"id\", \"path\": \"id\"},\r\n {\"name\": \"family_name\", \"path\": \"name.family\"},\r\n {\"name\": \"given_name\", \"path\": \"name.given.first()\"},\r\n {\"name\": \"gender\", \"path\": \"gender\"},\r\n {\"name\": \"birth_date\", \"path\": \"birthDate\"}\r\n ]\r\n }\r\n ]\r\n}\r\n\r\n# Sample FHIR Bundle with patient data\r\nbundle = {\r\n \"resourceType\": \"Bundle\",\r\n \"type\": \"collection\", \r\n \"entry\": [\r\n {\r\n \"resource\": {\r\n \"resourceType\": \"Patient\",\r\n \"id\": \"patient-1\",\r\n \"name\": [{\"family\": \"Doe\", \"given\": [\"John\"]}],\r\n \"gender\": \"male\",\r\n \"birthDate\": \"1990-01-01\"\r\n }\r\n },\r\n {\r\n \"resource\": {\r\n \"resourceType\": \"Patient\", \r\n \"id\": \"patient-2\",\r\n \"name\": [{\"family\": \"Smith\", \"given\": [\"Jane\"]}],\r\n \"gender\": \"female\",\r\n \"birthDate\": \"1985-05-15\"\r\n }\r\n }\r\n ]\r\n}\r\n\r\n# Transform to different formats\r\ncsv_result = pysof.run_view_definition(view_definition, bundle, \"csv\")\r\njson_result = pysof.run_view_definition(view_definition, bundle, \"json\")\r\nndjson_result = pysof.run_view_definition(view_definition, bundle, \"ndjson\")\r\nparquet_result = pysof.run_view_definition(view_definition, bundle, \"parquet\")\r\n\r\nprint(\"CSV Output:\")\r\nprint(csv_result.decode('utf-8'))\r\n\r\nprint(\"\\nJSON Output:\")\r\ndata = json.loads(json_result.decode('utf-8'))\r\nprint(json.dumps(data, indent=2))\r\n\r\nprint(\"\\nParquet Output (binary):\")\r\nprint(f\"Parquet data: {len(parquet_result)} bytes\")\r\n```\r\n\r\n### Advanced Usage with Options\r\n\r\n```python\r\nimport pysof\r\nfrom datetime import datetime\r\n\r\n# Transform with pagination and filtering\r\nresult = pysof.run_view_definition_with_options(\r\n view_definition,\r\n bundle, \r\n \"json\",\r\n limit=10, # Limit to 10 results\r\n page=1, # First page\r\n since=\"2023-01-01T00:00:00Z\", # Filter by modification date\r\n fhir_version=\"R4\" # Specify FHIR version\r\n)\r\n```\r\n\r\n### Utility Functions\r\n\r\n```python\r\nimport pysof\r\n\r\n# Validate structures before processing\r\nis_valid_view = pysof.validate_view_definition(view_definition)\r\nis_valid_bundle = pysof.validate_bundle(bundle)\r\n\r\n# Parse content types\r\nformat_str = pysof.parse_content_type(\"text/csv\") # Returns \"csv_with_header\"\r\nformat_str = pysof.parse_content_type(\"application/json\") # Returns \"json\"\r\n\r\n# Check supported FHIR versions\r\nversions = pysof.get_supported_fhir_versions() # Returns [\"R4\"] (or more based on build)\r\nprint(f\"Supported FHIR versions: {versions}\")\r\n\r\n# Check package status\r\nprint(pysof.get_status()) # Shows current implementation status\r\nprint(f\"Version: {pysof.get_version()}\")\r\n```\r\n\r\n### Error Handling\r\n\r\n```python\r\nimport pysof\r\n\r\ntry:\r\n result = pysof.run_view_definition(view_definition, bundle, \"json\")\r\nexcept pysof.InvalidViewDefinitionError as e:\r\n print(f\"ViewDefinition validation error: {e}\")\r\nexcept pysof.SerializationError as e:\r\n print(f\"JSON parsing error: {e}\")\r\nexcept pysof.UnsupportedContentTypeError as e:\r\n print(f\"Unsupported format: {e}\")\r\nexcept pysof.SofError as e:\r\n print(f\"General SOF error: {e}\")\r\n```\r\n\r\n### Key Features\r\n\r\n- **Performance**: Efficient processing of large FHIR Bundles using Rust\r\n- **Parallel Processing**: Automatic multithreading of FHIR resources using rayon (5-7x speedup on multi-core systems)\r\n- **GIL Release**: Python GIL is released during Rust execution for better performance\r\n- **Multiple Formats**: Support for CSV, JSON, NDJSON, and Parquet outputs\r\n- **FHIR Versions**: Support for R4, R4B, R5, and R6 (depending on build features)\r\n\r\n### Available Options\r\n\r\nThe `run_view_definition_with_options()` function accepts the following parameters:\r\n\r\n- **limit**: Limit the number of results returned\r\n- **page**: Page number for pagination (1-based)\r\n- **since**: Filter resources modified after this ISO8601 datetime\r\n- **fhir_version**: FHIR version to use (\"R4\", \"R4B\", \"R5\", \"R6\")\r\n\r\n### Supported Content Types\r\n\r\n| Format | Description | Output |\r\n|--------|-------------|---------|\r\n| `csv` | CSV with headers | Comma-separated values with header row |\r\n| `json` | JSON array | Array of objects, one per result row |\r\n| `ndjson` | Newline-delimited JSON | One JSON object per line |\r\n| `parquet` | Parquet format | Columnar binary format optimized for analytics |\r\n\r\n### Supported FHIR Versions\r\n\r\n- **R4** (default, always available)\r\n- **R4B** (if compiled with R4B feature)\r\n- **R5** (if compiled with R5 feature) \r\n- **R6** (if compiled with R6 feature)\r\n\r\nUse `pysof.get_supported_fhir_versions()` to check what's available in your build.\r\n\r\n### Usage Notes\r\n\r\n- The basic `run_view_definition()` function is suitable for simple use cases\r\n- Use `run_view_definition_with_options()` for pagination, filtering, and other advanced features\r\n- All heavy processing happens in Rust code; Python GIL is properly released for optimal performance\r\n\r\n## Multithreading and Performance\r\n\r\n### Automatic Parallel Processing\r\n\r\npysof automatically processes FHIR resources in parallel using rayon, providing significant performance improvements on multi-core systems:\r\n\r\n- **5-7x speedup** for typical workloads on modern CPUs\r\n- **Zero configuration required** - parallelization is always enabled\r\n- **Python GIL released** during processing for true parallel execution\r\n\r\n### Controlling Thread Count\r\n\r\nBy default, pysof uses all available CPU cores. You can control the thread count using the `RAYON_NUM_THREADS` environment variable:\r\n\r\n```python\r\nimport os\r\nimport pysof\r\n\r\n# Set thread count BEFORE first use (must be set before rayon initializes)\r\nos.environ['RAYON_NUM_THREADS'] = '4'\r\n\r\n# Now all operations will use 4 threads\r\nresult = pysof.run_view_definition(view_definition, bundle, \"json\")\r\n```\r\n\r\nOr from the command line:\r\n\r\n```bash\r\n# Linux/Mac\r\nRAYON_NUM_THREADS=4 python my_script.py\r\n\r\n# Windows (cmd)\r\nset RAYON_NUM_THREADS=4\r\npython my_script.py\r\n\r\n# Windows (PowerShell)\r\n$env:RAYON_NUM_THREADS=4\r\npython my_script.py\r\n```\r\n\r\n**Important**: The environment variable must be set before rayon initializes its global thread pool (typically on first use of pysof). Once initialized, the thread count cannot be changed for that process.\r\n\r\n### Performance Tips\r\n\r\n- **Large datasets**: Use all available cores (default behavior)\r\n- **Shared systems**: Limit threads to avoid resource contention (`RAYON_NUM_THREADS=4`)\r\n- **Memory constrained**: Reduce thread count to lower memory usage\r\n- **Benchmarking**: Test different thread counts to find optimal performance for your workload\r\n\r\n## Configuring FHIR Version Support\r\n\r\nBy default, pysof is compiled with **R4 support only**. You can configure which FHIR versions are available by modifying the feature compilation settings.\r\n\r\n### Change Default FHIR Version\r\n\r\nTo change from R4 to another version (e.g., R5):\r\n\r\n1. **Edit `crates/pysof/Cargo.toml`**:\r\n ```toml\r\n [features]\r\n default = [\"R5\"] # Changed from [\"R4\"]\r\n R4 = [\"helios-sof/R4\", \"helios-fhir/R4\"]\r\n R4B = [\"helios-sof/R4B\", \"helios-fhir/R4B\"]\r\n R5 = [\"helios-sof/R5\", \"helios-fhir/R5\"]\r\n R6 = [\"helios-sof/R6\", \"helios-fhir/R6\"]\r\n ```\r\n\r\n2. **Rebuild the extension**:\r\n ```bash\r\n cd crates/pysof\r\n uv run maturin develop --release\r\n ```\r\n\r\n3. **Verify the change**:\r\n ```bash\r\n uv run python -c \"\r\n import pysof\r\n versions = pysof.get_supported_fhir_versions()\r\n print('Supported FHIR versions:', versions)\r\n \"\r\n ```\r\n This should now show `['R5']` instead of `['R4']`.\r\n\r\n### Enable Multiple FHIR Versions\r\n\r\nTo support multiple FHIR versions simultaneously:\r\n\r\n1. **Edit `crates/pysof/Cargo.toml`**:\r\n ```toml\r\n [features]\r\n default = [\"R4\", \"R5\"] # Enable both R4 and R5\r\n # Or enable all versions:\r\n # default = [\"R4\", \"R4B\", \"R5\", \"R6\"]\r\n ```\r\n\r\n2. **Rebuild and verify**:\r\n ```bash\r\n uv run maturin develop --release\r\n uv run python -c \"import pysof; print(pysof.get_supported_fhir_versions())\"\r\n ```\r\n This should show `['R4', 'R5']` (or all enabled versions).\r\n\r\n3. **Use specific versions in code**:\r\n ```python\r\n import pysof\r\n \r\n # Use R4 explicitly\r\n result_r4 = pysof.run_view_definition(view, bundle, \"json\", fhir_version=\"R4\")\r\n \r\n # Use R5 explicitly \r\n result_r5 = pysof.run_view_definition(view, bundle, \"json\", fhir_version=\"R5\")\r\n ```\r\n\r\n### Build with Specific Features (Without Changing Default)\r\n\r\nTo temporarily build with different features without modifying `Cargo.toml`:\r\n\r\n```bash\r\n# Build with only R5\r\ncargo build --features R5 --no-default-features\r\n\r\n# Build with R4 and R6\r\ncargo build --features R4,R6 --no-default-features\r\n\r\n# With maturin\r\nuv run --with maturin -- maturin develop --release --cargo-extra-args=\"--features R5 --no-default-features\"\r\n```\r\n\r\n### Testing After Version Changes\r\n\r\nAfter changing FHIR version support, run the test suite to ensure compatibility:\r\n\r\n```bash\r\n# Run all tests\r\nuv run pytest\r\n\r\n# Run FHIR version-specific tests\r\nuv run pytest tests/test_fhir_versions.py -v\r\n\r\n# Test with your new default version\r\nuv run python -c \"\r\nimport pysof\r\n\r\n# Test with default version (should be your new default)\r\nview = {'resourceType': 'ViewDefinition', 'id': 'test', 'name': 'Test', 'status': 'active', 'resource': 'Patient', 'select': [{'column': [{'name': 'id', 'path': 'id'}]}]}\r\nbundle = {'resourceType': 'Bundle', 'type': 'collection', 'entry': [{'resource': {'resourceType': 'Patient', 'id': 'test'}}]}\r\n\r\nresult = pysof.run_view_definition(view, bundle, 'json')\r\nprint('Default version test successful:', len(result), 'bytes')\r\n\"\r\n```\r\n\r\n### Important Considerations\r\n\r\n1. **Dependency Requirements**: Ensure the underlying `helios-sof` and `helios-fhir` crates support the features you want to enable\r\n2. **Binary Size**: Enabling multiple FHIR versions increases the compiled binary size significantly\r\n3. **Memory Usage**: Multiple versions require more memory at runtime\r\n4. **Testing**: Always run your test suite after changing version support\r\n5. **Documentation**: Update any project documentation that mentions specific FHIR versions\r\n6. **CI/CD**: Update build scripts and CI workflows if they depend on specific versions\r\n\r\n### Version Compatibility Matrix\r\n\r\n| FHIR Version | Feature Flag | Status | Notes |\r\n|--------------|--------------|---------|-------|\r\n| R4 | `R4` | \u2705 Stable | Default, always recommended |\r\n| R4B | `R4B` | \u2705 Stable | Minor updates to R4 |\r\n| R5 | `R5` | \u2705 Stable | Current latest stable version |\r\n| R6 | `R6` | \u26a0\ufe0f Preview | May have limited support |\r\n\r\n### Common Configuration Examples\r\n\r\n```toml\r\n# Production: Single version for minimal size\r\ndefault = [\"R4\"]\r\n\r\n# Development: Multiple versions for testing\r\ndefault = [\"R4\", \"R5\"]\r\n\r\n# Bleeding edge: Latest versions only\r\ndefault = [\"R5\", \"R6\"]\r\n\r\n# Maximum compatibility: All versions (not recommended for production)\r\ndefault = [\"R4\", \"R4B\", \"R5\", \"R6\"]\r\n```\r\n\r\n## Project layout\r\n\r\n```\r\ncrates/pysof/\r\n\u251c\u2500 pyproject.toml # PEP 621 metadata, Python >=3.11, uv-compatible\r\n\u251c\u2500 README.md\r\n\u251c\u2500 src/\r\n\u2502 \u251c\u2500 pysof/\r\n\u2502 \u2502 \u2514\u2500 __init__.py # Python package root\r\n\u2502 \u2514\u2500 lib.rs # Rust PyO3 bindings\r\n\u251c\u2500 tests/ # Rust tests (17 tests)\r\n\u2502 \u251c\u2500 lib_tests.rs # Unit tests for core library functions\r\n\u2502 \u251c\u2500 integration.rs # Integration tests for component interactions\r\n\u2502 \u2514\u2500 integration/ # Organized integration test modules\r\n\u2502 \u251c\u2500 mod.rs\r\n\u2502 \u251c\u2500 content_types.rs\r\n\u2502 \u251c\u2500 error_handling.rs\r\n\u2502 \u2514\u2500 fhir_versions.rs\r\n\u251c\u2500 python-tests/ # Python test suite (58 tests)\r\n\u2502 \u251c\u2500 __init__.py\r\n\u2502 \u251c\u2500 test_core_functions.py\r\n\u2502 \u251c\u2500 test_content_types.py\r\n\u2502 \u251c\u2500 test_fhir_versions.py\r\n\u2502 \u251c\u2500 test_import.py\r\n\u2502 \u2514\u2500 test_package_metadata.py\r\n\u2514\u2500 Cargo.toml # Rust crate metadata\r\n```\r\n\r\n## License\r\n\r\nThis package inherits the license from the repository root.\r\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Python wrapper for the Helios Software SOF (SQL on FHIR) toolkit.",
"version": "0.1.3",
"project_urls": null,
"split_keywords": [
"sql on fhir",
" fhir",
" healthcare",
" sof",
" helios software"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "bfab376c6ebc6c5166984faa8ab2d3bcc4c5b5d47121d9cff4e83fa56bdb51b1",
"md5": "01b81977d7abac440f6c734e24fd09ca",
"sha256": "4ba9a67dd2133d4b78cef464cbe136f78c6bbb19c63ca3819ef4a33089c8be22"
},
"downloads": -1,
"filename": "pysof-0.1.3-cp311-cp311-macosx_11_0_arm64.whl",
"has_sig": false,
"md5_digest": "01b81977d7abac440f6c734e24fd09ca",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": "<3.12,>=3.11",
"size": 19747283,
"upload_time": "2025-10-19T20:00:19",
"upload_time_iso_8601": "2025-10-19T20:00:19.762763Z",
"url": "https://files.pythonhosted.org/packages/bf/ab/376c6ebc6c5166984faa8ab2d3bcc4c5b5d47121d9cff4e83fa56bdb51b1/pysof-0.1.3-cp311-cp311-macosx_11_0_arm64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "71b340f20e0dde86957c630a47d48fae7da0153c0c11d33f084e546a076b19fa",
"md5": "87448a7a49fac27186dcb11331739a75",
"sha256": "01a0c874e65e004389db88b3580514bc64cc8597b48a1249665eceae8d8606b8"
},
"downloads": -1,
"filename": "pysof-0.1.3-cp311-cp311-manylinux_2_34_x86_64.whl",
"has_sig": false,
"md5_digest": "87448a7a49fac27186dcb11331739a75",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": "<3.12,>=3.11",
"size": 26304616,
"upload_time": "2025-10-19T20:00:26",
"upload_time_iso_8601": "2025-10-19T20:00:26.590692Z",
"url": "https://files.pythonhosted.org/packages/71/b3/40f20e0dde86957c630a47d48fae7da0153c0c11d33f084e546a076b19fa/pysof-0.1.3-cp311-cp311-manylinux_2_34_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "29c46e097e305dce6d0d48321aa91087645426bed77cbdcfef5ee6c8776a40c4",
"md5": "370e475c54a0604ae5f96d844cf2ab8f",
"sha256": "1f5aaed509c51d15d8536088a92072f1b6106091c294b6fb2e5926c18896b8f1"
},
"downloads": -1,
"filename": "pysof-0.1.3-cp311-cp311-win_amd64.whl",
"has_sig": false,
"md5_digest": "370e475c54a0604ae5f96d844cf2ab8f",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": "<3.12,>=3.11",
"size": 23016662,
"upload_time": "2025-10-19T20:00:30",
"upload_time_iso_8601": "2025-10-19T20:00:30.855011Z",
"url": "https://files.pythonhosted.org/packages/29/c4/6e097e305dce6d0d48321aa91087645426bed77cbdcfef5ee6c8776a40c4/pysof-0.1.3-cp311-cp311-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "4ff98c223ad836c86ccba42b2c849981b3abcba9772f940f0d803750ab8c28f5",
"md5": "5baf8c8c86350952e8764cf3976a74e9",
"sha256": "2f685f061e49151e6d5a9bd953f4ecf530dece5cc3d1e85533418df544a54298"
},
"downloads": -1,
"filename": "pysof-0.1.3.tar.gz",
"has_sig": false,
"md5_digest": "5baf8c8c86350952e8764cf3976a74e9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.12,>=3.11",
"size": 4469434,
"upload_time": "2025-10-19T20:00:33",
"upload_time_iso_8601": "2025-10-19T20:00:33.430329Z",
"url": "https://files.pythonhosted.org/packages/4f/f9/8c223ad836c86ccba42b2c849981b3abcba9772f940f0d803750ab8c28f5/pysof-0.1.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-19 20:00:33",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "pysof"
}
