## Usage
### Installation
```bash
pip install clio-api-model-generator
```
### Generating Models
To execute the generator and produce dataclass models, run:
```python
from clio_api_model_generator import clio_manage as manage_model_generator
manage_model_generator.generate_models(output_dir=models_dir, overwrite=False)
```
This script will initialize the models directory, download the OpenAPI spec, and generate all necessary dataclasses.
---
# Model Generator Generation Flow
**This document explains the setup process for handling the `models` directory, running generator scripts, and ensuring the correct paths are set dynamically.**
## Overview
The project handles the `models` directory in the following way:
1. **Checks for the existence of the `models` folder in the parent directory first (preferred).**
2. **If not found in the parent, it checks the project root directory.**
3. **If the folder exists and is not empty, it is backed up with a unique name (e.g., `models.backup`, `models.backup1`, etc.).**
4. **If no models folder is found, a new one is created in the preferred location (parent or project root).**
5. **Static files from the `static/` directory are copied into the models folder.**
6. **Environment variables are set to ensure the generator scripts use the correct folder paths.**
7. **Generator scripts are imported after paths are set.**
## Setup Process
## Execution Flow Summary
1. **Check parent directory for `models` folder first.**
2. **Backup existing folder if not empty and create a new one.**
3. **Copy static files into the models folder.**
4. **Set environment variables with the correct paths.**
5. **Import generator scripts that rely on the updated paths.**
## Expected Outputs
### Case 1: Models folder found in the parent directory and backed up
```
Existing non-empty models directory found at: /home/user/models
Renamed "/home/user/models" to "/home/user/models.backup"
Created new "/home/user/models" directory.
Copied all files from "static/" to "/home/user/models".
Set environment variables for model paths: /home/user/models
```
### Case 2: No models folder found, creating a new one
```
No existing "models" directory found in parent or current directory.
Created new "/home/user/project/models" directory.
Copied all files from "static/" to "/home/user/project/models".
Set environment variables for model paths: /home/user/project/models
```
---
## Notes
- Ensure the `static/` directory exists and contains the required files before running the script.
- The parent directory is prioritized when determining the location of the `models` folder.
- Environment variables are crucial to dynamically set paths without modifying scripts manually.
---
# Dataclass Generator Scripts
This document provides an overview of the dataclass generator scripts, explaining their order of execution, flow, and parsing/extraction logic for each generator.
## Overview
The scripts process an OpenAPI specification to generate Python dataclasses for:
1. **Schemas** – Representing API response objects.
2. **Fields** – Handling OpenAPI field mappings.
3. **Query Parameters** – Handling API query parameters.
4. **Request Bodies** – Handling request payloads.
5. **Endpoints** – Registering API endpoints with generated models.
---
## Order of Execution
The generation process is orchestrated by `generate_models.py`, which follows this sequence:
1. **Initialization (folder setup and OpenAPI spec download)**
- Ensures the `models` directory is ready.
- Downloads the OpenAPI spec if not present.
2. **Schema Generation (`components.py`)**
- Generates dataclasses from OpenAPI schemas.
3. **Field Extraction (`fields.py`)**
- Extracts field definitions for models based on schema dependencies.
4. **Query Parameter Handling (`query.py`)**
- Creates query dataclasses based on API query parameters.
5. **Request Body Handling (`request_body.py`)**
- Processes and generates dataclasses for API request bodies.
6. **Endpoint Registry Creation (`endpoints.py`)**
- Registers generated models to the API endpoint paths.
---
## Parsing and Extraction Logic
### 1. **Schema Generator (`components.py`)**
**Purpose:**
Processes OpenAPI `components.schemas` to generate Python dataclasses.
**Key Logic:**
- Reads the OpenAPI spec from `SPEC_FILE_PATH`.
- Iterates over schemas containing `_base` in the name.
- Recursively processes properties:
- Required fields are listed first.
- Optional fields are annotated with `Optional`.
- Handles nested objects by generating sub-dataclasses.
**Example Output:**
```python
@dataclass
class UserBase:
id: int
name: str
email: Optional[str] = None
```
---
## Fields Generation (`fields.py`)
### **Purpose**
The `fields.py` script generates dataclasses that represent OpenAPI schema field definitions, including handling dependencies and nested resources.
### **Key Steps in Fields Generation**
1. **Reading OpenAPI Spec:**
- Reads schema definitions from `SPEC_FILE_PATH`.
- Focuses on schemas without underscores (filters `_base` schemas).
2. **Processing Base Fields:**
- Base schema references are extracted from `allOf` definitions.
- Fields are copied from the base schema into the generated dataclass.
- Field names are sanitized to be valid Python identifiers.
3. **Handling Nested Resources:**
- Nested resources are identified within the schema.
- If a nested reference is found, the corresponding nested resource dataclass is created and included in the parent class.
- Nested fields are marked as `Optional` and are postponed if dependencies exist.
4. **Topological Sorting:**
- Ensures that dataclasses are generated in the correct dependency order.
- Handles cyclic dependencies by raising errors.
5. **Mapping to Endpoints:**
- Generates a mapping of fields to API responses using list and show schemas.
- Outputs a `endpoint_field_mapping.json` file for later use in endpoint definitions.
---
### **Example Workflow in `fields.py`**
#### Input OpenAPI Schema Example
```json
"Account": {
"allOf": [
{ "$ref": "#/components/schemas/BaseEntity" },
{
"properties": {
"name": { "type": "string" },
"contacts": {
"$ref": "#/components/schemas/Contact"
}
}
}
]
},
"Contact": {
"properties": {
"email": { "type": "string" },
"phone": { "type": "string" }
}
}
```
#### Output Generated Dataclass
```python
from dataclasses import dataclass
from typing import Optional
from models.schemas import *
@dataclass
class Account_Fields:
id: Optional[int] = None
created_at: Optional[str] = None
name: Optional[str] = None
contacts: Optional[Contact_Fields] = None
```
---
### **Handling Base Fields**
1. **Base Schema Reference Processing:**
- Extracted from the first `allOf` item.
- Fields from the referenced schema (e.g., `BaseEntity`) are added.
2. **Base Fields Example:**
```python
@dataclass
class BaseEntity:
id: Optional[int] = None
created_at: Optional[str] = None
```
---
### **Handling Nested Resources**
1. **Detecting Nested Fields:**
- Any field containing a reference (`$ref`) to another schema is treated as a nested resource.
- These are appended with `_Fields` and added to the parent dataclass.
2. **Nested Field Example:**
```python
@dataclass
class Contact_Fields:
email: Optional[str] = None
phone: Optional[str] = None
```
3. **Embedding in Parent:**
```python
@dataclass
class Account_Fields:
name: Optional[str] = None
contacts: Optional[Contact_Fields] = None
```
---
### **Mapping to Endpoints**
The generator script processes schemas with `_list` and `_show` suffixes to establish relationships between fields and endpoint responses.
**Example Mapping in `endpoint_field_mapping.json`:**
```json
{
"Account_list": "Account_Fields",
"User_show": "User_Fields"
}
```
**Logic:**
- The mapping associates the API response schemas to the generated field dataclasses.
- Helps the endpoint generator (`endpoints.py`) link responses to correct models.
---
### 3. **Query Generator (`query.py`)**
**Purpose:**
Generates dataclasses for API query parameters.
**Key Logic:**
- Reads query parameters from API operations.
- Transforms query parameter names to valid Python identifiers.
- Maps OpenAPI types to Python types.
- Writes required and optional fields separately.
**Example Output:**
```python
@dataclass
class AccountQuery:
account_id: int
status: Optional[str] = None
```
---
### 4. **Request Body Generator (`request_body.py`)**
**Purpose:**
Processes request body parameters for API operations.
**Key Logic:**
- Extracts the request body schema from `application/json` content.
- Handles nested object properties.
- Generates dataclasses with required and optional fields.
**Example Output:**
```python
@dataclass
class CreateAccountRequestBody:
name: str
email: str
age: Optional[int] = None
```
---
### 5. **Endpoints Generator (`endpoints.py`)**
**Purpose:**
Creates an `Endpoints` registry to store endpoint mappings.
**Key Logic:**
- Iterates through API paths and methods.
- Stores metadata such as path, method, query, and request body models.
- Handles special download cases based on HTTP 303 responses.
**Example Output:**
```python
class Endpoints:
registry = {
'get_user': {
'path': '/users/{user_id}',
'method': 'GET',
'query_model': UserQuery,
'request_body_model': None
}
}
```
---
## Directory Structure
```
project-root/
│-- models/
│ ├── query.py
│ ├── request_body.py
│ ├── endpoints.py
│ ├── fields.py
│ ├── schemas.py
│-- static/
│ ├── __init__.py
│ ├── models_registry.py
│-- generate_models.py
│-- openapi.json
```
**Static files get copied to models directory automatically before generating the model dataclasses**
---
## Conclusion
This setup automates the creation of Python dataclasses from an OpenAPI spec, providing a structured way to handle API interactions and ensuring maintainability through a clear generation flow.
Raw data
{
"_id": null,
"home_page": null,
"name": "clio-api-model-generator",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": null,
"author": null,
"author_email": "Unigrated Partners <dev@unigratedpartners.com>",
"download_url": "https://files.pythonhosted.org/packages/fe/86/4161367020de03289238687863eea3c21cf1cd538fa524ec2a07b803c204/clio_api_model_generator-0.2.1.tar.gz",
"platform": null,
"description": "\n## Usage\n\n### Installation\n```bash\npip install clio-api-model-generator\n```\n### Generating Models\nTo execute the generator and produce dataclass models, run:\n\n```python\nfrom clio_api_model_generator import clio_manage as manage_model_generator\n\nmanage_model_generator.generate_models(output_dir=models_dir, overwrite=False)\n```\n\nThis script will initialize the models directory, download the OpenAPI spec, and generate all necessary dataclasses.\n\n---\n\n# Model Generator Generation Flow\n\n**This document explains the setup process for handling the `models` directory, running generator scripts, and ensuring the correct paths are set dynamically.**\n\n## Overview\n\nThe project handles the `models` directory in the following way:\n\n1. **Checks for the existence of the `models` folder in the parent directory first (preferred).**\n2. **If not found in the parent, it checks the project root directory.**\n3. **If the folder exists and is not empty, it is backed up with a unique name (e.g., `models.backup`, `models.backup1`, etc.).**\n4. **If no models folder is found, a new one is created in the preferred location (parent or project root).**\n5. **Static files from the `static/` directory are copied into the models folder.**\n6. **Environment variables are set to ensure the generator scripts use the correct folder paths.**\n7. **Generator scripts are imported after paths are set.**\n\n## Setup Process\n\n## Execution Flow Summary\n\n1. **Check parent directory for `models` folder first.** \n2. **Backup existing folder if not empty and create a new one.** \n3. **Copy static files into the models folder.** \n4. **Set environment variables with the correct paths.** \n5. **Import generator scripts that rely on the updated paths.** \n\n## Expected Outputs\n\n### Case 1: Models folder found in the parent directory and backed up\n```\nExisting non-empty models directory found at: /home/user/models\nRenamed \"/home/user/models\" to \"/home/user/models.backup\"\nCreated new \"/home/user/models\" directory.\nCopied all files from \"static/\" to \"/home/user/models\".\nSet environment variables for model paths: /home/user/models\n```\n\n### Case 2: No models folder found, creating a new one\n```\nNo existing \"models\" directory found in parent or current directory.\nCreated new \"/home/user/project/models\" directory.\nCopied all files from \"static/\" to \"/home/user/project/models\".\nSet environment variables for model paths: /home/user/project/models\n```\n\n---\n\n## Notes\n\n- Ensure the `static/` directory exists and contains the required files before running the script.\n- The parent directory is prioritized when determining the location of the `models` folder.\n- Environment variables are crucial to dynamically set paths without modifying scripts manually.\n\n---\n\n# Dataclass Generator Scripts\n\nThis document provides an overview of the dataclass generator scripts, explaining their order of execution, flow, and parsing/extraction logic for each generator.\n\n## Overview\n\nThe scripts process an OpenAPI specification to generate Python dataclasses for:\n\n1. **Schemas** \u2013 Representing API response objects.\n2. **Fields** \u2013 Handling OpenAPI field mappings.\n3. **Query Parameters** \u2013 Handling API query parameters.\n4. **Request Bodies** \u2013 Handling request payloads.\n5. **Endpoints** \u2013 Registering API endpoints with generated models.\n\n---\n\n## Order of Execution\n\nThe generation process is orchestrated by `generate_models.py`, which follows this sequence:\n\n1. **Initialization (folder setup and OpenAPI spec download)**\n - Ensures the `models` directory is ready.\n - Downloads the OpenAPI spec if not present.\n\n2. **Schema Generation (`components.py`)**\n - Generates dataclasses from OpenAPI schemas.\n\n3. **Field Extraction (`fields.py`)**\n - Extracts field definitions for models based on schema dependencies.\n\n4. **Query Parameter Handling (`query.py`)**\n - Creates query dataclasses based on API query parameters.\n\n5. **Request Body Handling (`request_body.py`)**\n - Processes and generates dataclasses for API request bodies.\n\n6. **Endpoint Registry Creation (`endpoints.py`)**\n - Registers generated models to the API endpoint paths.\n\n---\n\n## Parsing and Extraction Logic\n\n### 1. **Schema Generator (`components.py`)**\n\n**Purpose:** \nProcesses OpenAPI `components.schemas` to generate Python dataclasses.\n\n**Key Logic:** \n- Reads the OpenAPI spec from `SPEC_FILE_PATH`.\n- Iterates over schemas containing `_base` in the name.\n- Recursively processes properties:\n - Required fields are listed first.\n - Optional fields are annotated with `Optional`.\n- Handles nested objects by generating sub-dataclasses.\n\n**Example Output:**\n\n```python\n@dataclass\nclass UserBase:\n id: int\n name: str\n email: Optional[str] = None\n```\n\n---\n\n## Fields Generation (`fields.py`)\n\n### **Purpose**\n\nThe `fields.py` script generates dataclasses that represent OpenAPI schema field definitions, including handling dependencies and nested resources.\n\n### **Key Steps in Fields Generation**\n\n1. **Reading OpenAPI Spec:** \n - Reads schema definitions from `SPEC_FILE_PATH`.\n - Focuses on schemas without underscores (filters `_base` schemas).\n\n2. **Processing Base Fields:** \n - Base schema references are extracted from `allOf` definitions.\n - Fields are copied from the base schema into the generated dataclass.\n - Field names are sanitized to be valid Python identifiers.\n\n3. **Handling Nested Resources:** \n - Nested resources are identified within the schema.\n - If a nested reference is found, the corresponding nested resource dataclass is created and included in the parent class.\n - Nested fields are marked as `Optional` and are postponed if dependencies exist.\n\n4. **Topological Sorting:** \n - Ensures that dataclasses are generated in the correct dependency order.\n - Handles cyclic dependencies by raising errors.\n\n5. **Mapping to Endpoints:** \n - Generates a mapping of fields to API responses using list and show schemas.\n - Outputs a `endpoint_field_mapping.json` file for later use in endpoint definitions.\n\n---\n\n### **Example Workflow in `fields.py`**\n\n#### Input OpenAPI Schema Example\n\n```json\n\"Account\": {\n \"allOf\": [\n { \"$ref\": \"#/components/schemas/BaseEntity\" },\n {\n \"properties\": {\n \"name\": { \"type\": \"string\" },\n \"contacts\": {\n \"$ref\": \"#/components/schemas/Contact\"\n }\n }\n }\n ]\n},\n\"Contact\": {\n \"properties\": {\n \"email\": { \"type\": \"string\" },\n \"phone\": { \"type\": \"string\" }\n }\n}\n```\n\n#### Output Generated Dataclass\n\n```python\nfrom dataclasses import dataclass\nfrom typing import Optional\nfrom models.schemas import *\n\n@dataclass\nclass Account_Fields:\n id: Optional[int] = None\n created_at: Optional[str] = None\n name: Optional[str] = None\n contacts: Optional[Contact_Fields] = None\n```\n\n---\n\n### **Handling Base Fields**\n\n1. **Base Schema Reference Processing:** \n - Extracted from the first `allOf` item.\n - Fields from the referenced schema (e.g., `BaseEntity`) are added.\n\n2. **Base Fields Example:** \n\n ```python\n @dataclass\n class BaseEntity:\n id: Optional[int] = None\n created_at: Optional[str] = None\n ```\n\n---\n\n### **Handling Nested Resources**\n\n1. **Detecting Nested Fields:** \n - Any field containing a reference (`$ref`) to another schema is treated as a nested resource.\n - These are appended with `_Fields` and added to the parent dataclass.\n\n2. **Nested Field Example:**\n\n ```python\n @dataclass\n class Contact_Fields:\n email: Optional[str] = None\n phone: Optional[str] = None\n ```\n\n3. **Embedding in Parent:**\n\n ```python\n @dataclass\n class Account_Fields:\n name: Optional[str] = None\n contacts: Optional[Contact_Fields] = None\n ```\n\n---\n\n### **Mapping to Endpoints**\n\nThe generator script processes schemas with `_list` and `_show` suffixes to establish relationships between fields and endpoint responses.\n\n**Example Mapping in `endpoint_field_mapping.json`:**\n\n```json\n{\n \"Account_list\": \"Account_Fields\",\n \"User_show\": \"User_Fields\"\n}\n```\n\n**Logic:** \n- The mapping associates the API response schemas to the generated field dataclasses.\n- Helps the endpoint generator (`endpoints.py`) link responses to correct models.\n\n---\n\n### 3. **Query Generator (`query.py`)**\n\n**Purpose:** \nGenerates dataclasses for API query parameters.\n\n**Key Logic:** \n- Reads query parameters from API operations.\n- Transforms query parameter names to valid Python identifiers.\n- Maps OpenAPI types to Python types.\n- Writes required and optional fields separately.\n\n**Example Output:**\n\n```python\n@dataclass\nclass AccountQuery:\n account_id: int\n status: Optional[str] = None\n```\n\n---\n\n### 4. **Request Body Generator (`request_body.py`)**\n\n**Purpose:** \nProcesses request body parameters for API operations.\n\n**Key Logic:** \n- Extracts the request body schema from `application/json` content.\n- Handles nested object properties.\n- Generates dataclasses with required and optional fields.\n\n**Example Output:**\n\n```python\n@dataclass\nclass CreateAccountRequestBody:\n name: str\n email: str\n age: Optional[int] = None\n```\n\n---\n\n### 5. **Endpoints Generator (`endpoints.py`)**\n\n**Purpose:** \nCreates an `Endpoints` registry to store endpoint mappings.\n\n**Key Logic:** \n- Iterates through API paths and methods.\n- Stores metadata such as path, method, query, and request body models.\n- Handles special download cases based on HTTP 303 responses.\n\n**Example Output:**\n\n```python\nclass Endpoints:\n registry = {\n 'get_user': {\n 'path': '/users/{user_id}',\n 'method': 'GET',\n 'query_model': UserQuery,\n 'request_body_model': None\n }\n }\n```\n\n---\n\n## Directory Structure\n\n```\nproject-root/\n\u2502-- models/\n\u2502 \u251c\u2500\u2500 query.py\n\u2502 \u251c\u2500\u2500 request_body.py\n\u2502 \u251c\u2500\u2500 endpoints.py\n\u2502 \u251c\u2500\u2500 fields.py\n\u2502 \u251c\u2500\u2500 schemas.py\n\u2502-- static/\n\u2502 \u251c\u2500\u2500 __init__.py\n\u2502 \u251c\u2500\u2500 models_registry.py\n\u2502-- generate_models.py\n\u2502-- openapi.json\n```\n**Static files get copied to models directory automatically before generating the model dataclasses**\n---\n\n## Conclusion\n\nThis setup automates the creation of Python dataclasses from an OpenAPI spec, providing a structured way to handle API interactions and ensuring maintainability through a clear generation flow.\n\n",
"bugtrack_url": null,
"license": null,
"summary": "A utility to generate Clio API models from OpenAPI specifications for use with our Clio API Python Client.",
"version": "0.2.1",
"project_urls": {
"Clio API Python Client": "https://github.com/unigrated-solutions/clio-api-python-client",
"Homepage": "https://github.com/unigrated-solutions/clio-api-model-generator"
},
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "418dea8768ca04672871b2d9eb6b45bf5a363bc8c727d13953e2a6bd370bfc92",
"md5": "c2be250f7b066526fd84680027e66d93",
"sha256": "94bcbdcf5de92c6d071c4bf821def29af631c52beecd98bc583d206bd92f1e9f"
},
"downloads": -1,
"filename": "clio_api_model_generator-0.2.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c2be250f7b066526fd84680027e66d93",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 23271,
"upload_time": "2025-08-12T23:34:04",
"upload_time_iso_8601": "2025-08-12T23:34:04.044912Z",
"url": "https://files.pythonhosted.org/packages/41/8d/ea8768ca04672871b2d9eb6b45bf5a363bc8c727d13953e2a6bd370bfc92/clio_api_model_generator-0.2.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "fe864161367020de03289238687863eea3c21cf1cd538fa524ec2a07b803c204",
"md5": "187ff207ab9d626ba5f702dc9eb2e16e",
"sha256": "27808ba64ccb4c1f5ad4c5ac5d3740818d31fb77eece8e486dc3d9f995abd022"
},
"downloads": -1,
"filename": "clio_api_model_generator-0.2.1.tar.gz",
"has_sig": false,
"md5_digest": "187ff207ab9d626ba5f702dc9eb2e16e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 21182,
"upload_time": "2025-08-12T23:34:05",
"upload_time_iso_8601": "2025-08-12T23:34:05.213461Z",
"url": "https://files.pythonhosted.org/packages/fe/86/4161367020de03289238687863eea3c21cf1cd538fa524ec2a07b803c204/clio_api_model_generator-0.2.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-12 23:34:05",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "unigrated-solutions",
"github_project": "clio-api-python-client",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "certifi",
"specs": []
},
{
"name": "charset-normalizer",
"specs": []
},
{
"name": "et_xmlfile",
"specs": []
},
{
"name": "idna",
"specs": []
},
{
"name": "mypy-extensions",
"specs": []
},
{
"name": "numpy",
"specs": [
[
"<",
"2.0"
]
]
},
{
"name": "openpyxl",
"specs": []
},
{
"name": "pandas",
"specs": []
},
{
"name": "python-dateutil",
"specs": []
},
{
"name": "pytz",
"specs": []
},
{
"name": "requests",
"specs": []
},
{
"name": "six",
"specs": []
},
{
"name": "typing-inspect",
"specs": []
},
{
"name": "typing_extensions",
"specs": []
},
{
"name": "tzdata",
"specs": []
},
{
"name": "urllib3",
"specs": []
},
{
"name": "aiohttp",
"specs": []
},
{
"name": "pyyaml",
"specs": []
},
{
"name": "clio-api-model-generator",
"specs": [
[
"==",
"0.2.0"
]
]
}
],
"lcname": "clio-api-model-generator"
}
JSON
