| Name | csvalchemy JSON |
| Version |
0.1.0
JSON |
| download |
| home_page | None |
| Summary | Read, validate, and write CSV files using Pydantic models with dydactic. |
| upload_time | 2025-10-29 18:48:38 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.10 |
| license | MIT License |
| keywords |
csv
validation
orm
model
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# csvalchemy
A Python package for reading and writing CSV files using Pydantic models.
## Overview
csvalchemy provides a clean interface for validating CSV data against Pydantic models,
handling errors gracefully, and writing validated results back to CSV files. It integrates
with [dydactic](https://github.com/eddiethedean/dydactic) for robust validation of data records.
## Features
- **CSV Reading**: Read CSV files and validate each row against Pydantic models
- **Error Handling**: Continue processing even when individual rows fail validation
- **Type Safety**: Full type hints and validation using Pydantic
- **CSV Writing**: Write validated results back to CSV files
- **Integration**: Built on dydactic for reliable validation
## Dependencies
- **Python**: 3.10 or higher
- **pydantic**: >=2.9.2 (Data validation using Python type annotations)
- **dydactic**: >=0.2.0 (Validation engine - requires Python 3.10+)
- **python-dateutil**: >=2.8.0 (DateTime parsing)
## Installation
```bash
pip install csvalchemy
```
## Quick Start
```python
from pydantic import BaseModel
from csvalchemy import read
from io import StringIO
# Define your model
class Person(BaseModel):
name: str
age: int
email: str | None = None
# Sample CSV content
csv_content = """name,age,email
Alice,30,alice@example.com
Bob,25,bob@example.com
Charlie,35,charlie@example.com
"""
# Read and validate CSV
with StringIO(csv_content) as f:
for result in read(f, Person):
if result.error:
print(f"Validation error: {result.error}")
else:
print(f"Valid person: {result.result.name}, age {result.result.age}")
```
**Output:**
```
Valid person: Alice, age 30
Valid person: Bob, age 25
Valid person: Charlie, age 35
```
## Examples
### Error Handling
csvalchemy continues processing even when individual rows fail validation:
```python
from pydantic import BaseModel
from csvalchemy import read
from io import StringIO
class Person(BaseModel):
name: str
age: int
email: str | None = None
# CSV with some invalid rows
csv_content = """name,age,email
Alice,30,alice@example.com
Bob,not_a_number,bob@example.com
Charlie,35,charlie@example.com
Diana,not_a_number,diana@example.com
"""
with StringIO(csv_content) as f:
valid_count = 0
error_count = 0
for result in read(f, Person):
if result.error:
error_count += 1
print(f"Error on row {error_count}: {result.error}")
else:
valid_count += 1
print(f"Valid: {result.result.name}")
print(f"\nSummary: {valid_count} valid, {error_count} errors")
```
**Output:**
```
Valid: Alice
Error on row 1: 1 validation error for Person
age
Input should be a valid integer, unable to parse string as an integer [type=int_parsing, input_value='not_a_number', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/int_parsing
Valid: Charlie
Error on row 2: 1 validation error for Person
age
Input should be a valid integer, unable to parse string as an integer [type=int_parsing, input_value='not_a_number', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/int_parsing
Summary: 2 valid, 2 errors
```
### Writing Validated CSV
Write only validated results back to CSV:
```python
from pydantic import BaseModel
from csvalchemy import read
from io import StringIO
class Product(BaseModel):
id: int
name: str
price: float
in_stock: bool
# Input CSV
input_csv = """id,name,price,in_stock
1,Widget,19.99,True
2,Gadget,29.99,False
3,Invalid,not_a_number,True
4,Thing,39.99,True
"""
# Read and validate
input_file = StringIO(input_csv)
validator = read(input_file, Product)
# Write validated results to new CSV
output_file = StringIO()
# Recreate validator since iterator was consumed
input_file2 = StringIO(input_csv)
validator2 = read(input_file2, Product)
writer = validator2.csv_writer(output_file)
# Consume writer to trigger CSV writing
for result in writer:
if result.error:
print(f"Skipped invalid row: {result.error}")
else:
print(f"Wrote: {result.result.name}")
# Show output CSV
output_file.seek(0)
print("\n=== Output CSV ===")
print(output_file.read())
```
**Output:**
```
Wrote: Widget
Wrote: Gadget
Skipped invalid row: 1 validation error for Product
price
Input should be a valid number, unable to parse string as a number [type=float_parsing, input_value='not_a_number', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/float_parsing
Wrote: Thing
=== Output CSV ===
id,name,price,in_stock
1,Widget,19.99,True
2,Gadget,29.99,False
4,Thing,39.99,True
```
### Using Validator Directly
Validate data not from CSV files:
```python
from pydantic import BaseModel
from csvalchemy import Validator
import dydactic.options
class Person(BaseModel):
name: str
age: int
email: str | None = None
# Data not from CSV
records = [
{"name": "Alice", "age": "30", "email": "alice@example.com"},
{"name": "Bob", "age": "not_a_number", "email": "bob@example.com"},
{"name": "Charlie", "age": "35"},
]
# Standard validation
print("=== Using Validator directly ===")
validator = Validator(iter(records), Person)
for result in validator:
if result.error:
print(f"Error: {result.error}")
else:
print(f"Valid: {result.result.name}, age {result.result.age}")
# Skip invalid records
print("\n=== Using SKIP error option ===")
validator_skip = Validator(
iter(records),
Person,
error_option=dydactic.options.ErrorOption.SKIP
)
valid_results = list(validator_skip)
print(f"Got {len(valid_results)} valid results (invalid ones skipped)")
```
**Output:**
```
=== Using Validator directly ===
Valid: Alice, age 30
Error: 1 validation error for Person
age
Input should be a valid integer, unable to parse string as an integer [type=int_parsing, input_value='not_a_number', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/int_parsing
Valid: Charlie, age 35
=== Using SKIP error option ===
Got 2 valid results (invalid ones skipped)
```
## Integration with dydactic
csvalchemy uses [dydactic](https://github.com/eddiethedean/dydactic) as its core validation
engine. The `Validator` and `ValidatorIterator` classes wrap `dydactic.validate()` to provide
a consistent API for CSV data validation.
### How it works
1. **CSV Reading**: `read()` creates a `CSVReaderValidator` that reads CSV rows using Python's `csv.DictReader`
2. **Validation**: Each row is validated using `dydactic.validate()`, which handles Pydantic model validation
3. **Error Handling**: Validation errors are captured without stopping the iteration
4. **Result Mapping**: dydactic's result objects are mapped to csvalchemy's `Result` type for consistent API
### Benefits
- Leverages dydactic's robust validation handling
- Independent validation of each record (errors don't stop processing)
- Type-safe error handling with clear error messages
- Compatible with dydactic's validation strategies
- Configurable error handling (RETURN, RAISE, or SKIP)
- Support for strict validation and attribute-based validation
### Configuration Options
The `Validator` class supports dydactic's configuration options:
- **error_option**: Control how validation errors are handled:
- `RETURN` (default): Errors are returned in `Result.error`
- `RAISE`: Exceptions are raised immediately on validation errors
- `SKIP`: Records with errors are skipped entirely
- **strict**: Enable strict Pydantic validation
- **from_attributes**: Validate from object attributes
Example:
```python
from pydantic import BaseModel
from csvalchemy import Validator
import dydactic.options
class Person(BaseModel):
name: str
age: int
records = [
{"name": "Alice", "age": "30"},
{"name": "Bob", "age": "invalid"},
{"name": "Charlie", "age": "35"},
]
# Default: RETURN errors
validator_return = Validator(iter(records), Person)
results_return = list(validator_return)
print(f"RETURN mode: {len(results_return)} results (including errors)")
# SKIP invalid records
validator_skip = Validator(
iter(records),
Person,
error_option=dydactic.options.ErrorOption.SKIP
)
results_skip = list(validator_skip)
print(f"SKIP mode: {len(results_skip)} results (errors skipped)")
```
**Output:**
```
RETURN mode: 3 results (including errors)
SKIP mode: 2 results (errors skipped)
```
## Architecture Notes
### Casting and Validation
csvalchemy provides two approaches to validation:
1. **Full Validation (Recommended)**: Use `Validator` or `read()` which leverage dydactic's complete validation pipeline including dydactic's casting functionality. This is the primary and recommended approach for CSV validation.
2. **Standalone Casting**: The `cast.py` module provides casting utilities similar to `dydactic.cast`. This module is kept for:
- Standalone use cases that don't require full dydactic validation
- Direct class instantiation without Pydantic models
- Testing scenarios
Note: The main validation flow uses dydactic's casting internally, so `cast.py` is not used in the primary validation pipeline.
## Requirements
- Python 3.10+ (required by dydactic)
- See `pyproject.toml` for complete dependency list
Raw data
{
"_id": null,
"home_page": null,
"name": "csvalchemy",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "csv, validation, orm, model",
"author": null,
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/b3/37/53a2bd641738771dc7902f4b2d4e506f1ad13b27cf8d9f375e62891496b1/csvalchemy-0.1.0.tar.gz",
"platform": null,
"description": "# csvalchemy\n\nA Python package for reading and writing CSV files using Pydantic models.\n\n## Overview\n\ncsvalchemy provides a clean interface for validating CSV data against Pydantic models,\nhandling errors gracefully, and writing validated results back to CSV files. It integrates\nwith [dydactic](https://github.com/eddiethedean/dydactic) for robust validation of data records.\n\n## Features\n\n- **CSV Reading**: Read CSV files and validate each row against Pydantic models\n- **Error Handling**: Continue processing even when individual rows fail validation\n- **Type Safety**: Full type hints and validation using Pydantic\n- **CSV Writing**: Write validated results back to CSV files\n- **Integration**: Built on dydactic for reliable validation\n\n## Dependencies\n\n- **Python**: 3.10 or higher\n- **pydantic**: >=2.9.2 (Data validation using Python type annotations)\n- **dydactic**: >=0.2.0 (Validation engine - requires Python 3.10+)\n- **python-dateutil**: >=2.8.0 (DateTime parsing)\n\n## Installation\n\n```bash\npip install csvalchemy\n```\n\n## Quick Start\n\n```python\nfrom pydantic import BaseModel\nfrom csvalchemy import read\nfrom io import StringIO\n\n# Define your model\nclass Person(BaseModel):\n name: str\n age: int\n email: str | None = None\n\n# Sample CSV content\ncsv_content = \"\"\"name,age,email\nAlice,30,alice@example.com\nBob,25,bob@example.com\nCharlie,35,charlie@example.com\n\"\"\"\n\n# Read and validate CSV\nwith StringIO(csv_content) as f:\n for result in read(f, Person):\n if result.error:\n print(f\"Validation error: {result.error}\")\n else:\n print(f\"Valid person: {result.result.name}, age {result.result.age}\")\n```\n\n**Output:**\n```\nValid person: Alice, age 30\nValid person: Bob, age 25\nValid person: Charlie, age 35\n```\n\n## Examples\n\n### Error Handling\n\ncsvalchemy continues processing even when individual rows fail validation:\n\n```python\nfrom pydantic import BaseModel\nfrom csvalchemy import read\nfrom io import StringIO\n\nclass Person(BaseModel):\n name: str\n age: int\n email: str | None = None\n\n# CSV with some invalid rows\ncsv_content = \"\"\"name,age,email\nAlice,30,alice@example.com\nBob,not_a_number,bob@example.com\nCharlie,35,charlie@example.com\nDiana,not_a_number,diana@example.com\n\"\"\"\n\nwith StringIO(csv_content) as f:\n valid_count = 0\n error_count = 0\n \n for result in read(f, Person):\n if result.error:\n error_count += 1\n print(f\"Error on row {error_count}: {result.error}\")\n else:\n valid_count += 1\n print(f\"Valid: {result.result.name}\")\n \n print(f\"\\nSummary: {valid_count} valid, {error_count} errors\")\n```\n\n**Output:**\n```\nValid: Alice\nError on row 1: 1 validation error for Person\nage\n Input should be a valid integer, unable to parse string as an integer [type=int_parsing, input_value='not_a_number', input_type=str]\n For further information visit https://errors.pydantic.dev/2.12/v/int_parsing\nValid: Charlie\nError on row 2: 1 validation error for Person\nage\n Input should be a valid integer, unable to parse string as an integer [type=int_parsing, input_value='not_a_number', input_type=str]\n For further information visit https://errors.pydantic.dev/2.12/v/int_parsing\n\nSummary: 2 valid, 2 errors\n```\n\n### Writing Validated CSV\n\nWrite only validated results back to CSV:\n\n```python\nfrom pydantic import BaseModel\nfrom csvalchemy import read\nfrom io import StringIO\n\nclass Product(BaseModel):\n id: int\n name: str\n price: float\n in_stock: bool\n\n# Input CSV\ninput_csv = \"\"\"id,name,price,in_stock\n1,Widget,19.99,True\n2,Gadget,29.99,False\n3,Invalid,not_a_number,True\n4,Thing,39.99,True\n\"\"\"\n\n# Read and validate\ninput_file = StringIO(input_csv)\nvalidator = read(input_file, Product)\n\n# Write validated results to new CSV\noutput_file = StringIO()\n\n# Recreate validator since iterator was consumed\ninput_file2 = StringIO(input_csv)\nvalidator2 = read(input_file2, Product)\nwriter = validator2.csv_writer(output_file)\n\n# Consume writer to trigger CSV writing\nfor result in writer:\n if result.error:\n print(f\"Skipped invalid row: {result.error}\")\n else:\n print(f\"Wrote: {result.result.name}\")\n\n# Show output CSV\noutput_file.seek(0)\nprint(\"\\n=== Output CSV ===\")\nprint(output_file.read())\n```\n\n**Output:**\n```\nWrote: Widget\nWrote: Gadget\nSkipped invalid row: 1 validation error for Product\nprice\n Input should be a valid number, unable to parse string as a number [type=float_parsing, input_value='not_a_number', input_type=str]\n For further information visit https://errors.pydantic.dev/2.12/v/float_parsing\nWrote: Thing\n\n=== Output CSV ===\nid,name,price,in_stock\n1,Widget,19.99,True\n2,Gadget,29.99,False\n4,Thing,39.99,True\n```\n\n### Using Validator Directly\n\nValidate data not from CSV files:\n\n```python\nfrom pydantic import BaseModel\nfrom csvalchemy import Validator\nimport dydactic.options\n\nclass Person(BaseModel):\n name: str\n age: int\n email: str | None = None\n\n# Data not from CSV\nrecords = [\n {\"name\": \"Alice\", \"age\": \"30\", \"email\": \"alice@example.com\"},\n {\"name\": \"Bob\", \"age\": \"not_a_number\", \"email\": \"bob@example.com\"},\n {\"name\": \"Charlie\", \"age\": \"35\"},\n]\n\n# Standard validation\nprint(\"=== Using Validator directly ===\")\nvalidator = Validator(iter(records), Person)\n\nfor result in validator:\n if result.error:\n print(f\"Error: {result.error}\")\n else:\n print(f\"Valid: {result.result.name}, age {result.result.age}\")\n\n# Skip invalid records\nprint(\"\\n=== Using SKIP error option ===\")\nvalidator_skip = Validator(\n iter(records),\n Person,\n error_option=dydactic.options.ErrorOption.SKIP\n)\n\nvalid_results = list(validator_skip)\nprint(f\"Got {len(valid_results)} valid results (invalid ones skipped)\")\n```\n\n**Output:**\n```\n=== Using Validator directly ===\nValid: Alice, age 30\nError: 1 validation error for Person\nage\n Input should be a valid integer, unable to parse string as an integer [type=int_parsing, input_value='not_a_number', input_type=str]\n For further information visit https://errors.pydantic.dev/2.12/v/int_parsing\nValid: Charlie, age 35\n\n=== Using SKIP error option ===\nGot 2 valid results (invalid ones skipped)\n```\n\n## Integration with dydactic\n\ncsvalchemy uses [dydactic](https://github.com/eddiethedean/dydactic) as its core validation\nengine. The `Validator` and `ValidatorIterator` classes wrap `dydactic.validate()` to provide\na consistent API for CSV data validation.\n\n### How it works\n\n1. **CSV Reading**: `read()` creates a `CSVReaderValidator` that reads CSV rows using Python's `csv.DictReader`\n2. **Validation**: Each row is validated using `dydactic.validate()`, which handles Pydantic model validation\n3. **Error Handling**: Validation errors are captured without stopping the iteration\n4. **Result Mapping**: dydactic's result objects are mapped to csvalchemy's `Result` type for consistent API\n\n### Benefits\n\n- Leverages dydactic's robust validation handling\n- Independent validation of each record (errors don't stop processing)\n- Type-safe error handling with clear error messages\n- Compatible with dydactic's validation strategies\n- Configurable error handling (RETURN, RAISE, or SKIP)\n- Support for strict validation and attribute-based validation\n\n### Configuration Options\n\nThe `Validator` class supports dydactic's configuration options:\n\n- **error_option**: Control how validation errors are handled:\n - `RETURN` (default): Errors are returned in `Result.error`\n - `RAISE`: Exceptions are raised immediately on validation errors\n - `SKIP`: Records with errors are skipped entirely\n- **strict**: Enable strict Pydantic validation\n- **from_attributes**: Validate from object attributes\n\nExample:\n\n```python\nfrom pydantic import BaseModel\nfrom csvalchemy import Validator\nimport dydactic.options\n\nclass Person(BaseModel):\n name: str\n age: int\n\nrecords = [\n {\"name\": \"Alice\", \"age\": \"30\"},\n {\"name\": \"Bob\", \"age\": \"invalid\"},\n {\"name\": \"Charlie\", \"age\": \"35\"},\n]\n\n# Default: RETURN errors\nvalidator_return = Validator(iter(records), Person)\nresults_return = list(validator_return)\nprint(f\"RETURN mode: {len(results_return)} results (including errors)\")\n\n# SKIP invalid records\nvalidator_skip = Validator(\n iter(records),\n Person,\n error_option=dydactic.options.ErrorOption.SKIP\n)\nresults_skip = list(validator_skip)\nprint(f\"SKIP mode: {len(results_skip)} results (errors skipped)\")\n```\n\n**Output:**\n```\nRETURN mode: 3 results (including errors)\nSKIP mode: 2 results (errors skipped)\n```\n\n## Architecture Notes\n\n### Casting and Validation\n\ncsvalchemy provides two approaches to validation:\n\n1. **Full Validation (Recommended)**: Use `Validator` or `read()` which leverage dydactic's complete validation pipeline including dydactic's casting functionality. This is the primary and recommended approach for CSV validation.\n\n2. **Standalone Casting**: The `cast.py` module provides casting utilities similar to `dydactic.cast`. This module is kept for:\n - Standalone use cases that don't require full dydactic validation\n - Direct class instantiation without Pydantic models\n - Testing scenarios\n\nNote: The main validation flow uses dydactic's casting internally, so `cast.py` is not used in the primary validation pipeline.\n\n## Requirements\n\n- Python 3.10+ (required by dydactic)\n- See `pyproject.toml` for complete dependency list\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "Read, validate, and write CSV files using Pydantic models with dydactic.",
"version": "0.1.0",
"project_urls": null,
"split_keywords": [
"csv",
" validation",
" orm",
" model"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "155b6292e3b1e8046c2a5aa0fdca7d2eb0939cbbb073510aa5a4dac58e0a4243",
"md5": "c88d43ef48e4b32311920a43e6c594e0",
"sha256": "705330849911918d2734a9051a1cac416cdd8cfd81aac4865dd1acd182a5ba9e"
},
"downloads": -1,
"filename": "csvalchemy-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c88d43ef48e4b32311920a43e6c594e0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 23451,
"upload_time": "2025-10-29T18:48:37",
"upload_time_iso_8601": "2025-10-29T18:48:37.510654Z",
"url": "https://files.pythonhosted.org/packages/15/5b/6292e3b1e8046c2a5aa0fdca7d2eb0939cbbb073510aa5a4dac58e0a4243/csvalchemy-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "b33753a2bd641738771dc7902f4b2d4e506f1ad13b27cf8d9f375e62891496b1",
"md5": "a3c09dbec986f654b051f8aaa80df5fd",
"sha256": "26661d7122a643dc3001284c5a9e67d5a8fcc38ed9bc1a7b6a7f3c564aa5c87f"
},
"downloads": -1,
"filename": "csvalchemy-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "a3c09dbec986f654b051f8aaa80df5fd",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 18256,
"upload_time": "2025-10-29T18:48:38",
"upload_time_iso_8601": "2025-10-29T18:48:38.530785Z",
"url": "https://files.pythonhosted.org/packages/b3/37/53a2bd641738771dc7902f4b2d4e506f1ad13b27cf8d9f375e62891496b1/csvalchemy-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-29 18:48:38",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "csvalchemy"
}
