# BatchWizard
BatchWizard is a powerful CLI tool for managing OpenAI batch processing jobs with ease. It provides functionalities to upload files, create batch jobs, check their status, and download the results. The tool uses asynchronous processing to efficiently handle multiple jobs concurrently.
## Table of Contents
- [Installation](#installation)
- [Usage](#usage)
- [Configuration](#configuration)
- [Commands](#commands)
- [Features](#features)
- [Contributing](#contributing)
- [License](#license)
## Installation
You can install BatchWizard using `pipx` for an isolated environment or directly via `pip`.
### Using pipx (recommended)
```bash
pipx install batchwizard
```
### Using pip
```bash
pip install batchwizard
```
Ensure you have `pipx` or `pip` installed on your system. For `pipx`, you can follow the installation instructions [here](https://pipx.pypa.io/stable/installation/).
## Usage
BatchWizard provides a command-line interface (CLI) for managing batch jobs. Here are some example commands:
### Process Batch Jobs
To process input files or directories:
```bash
batchwizard process <input_paths>... [--output-directory OUTPUT_DIR] [--max-concurrent-jobs NUM] [--check-interval SECONDS]
```
You can provide multiple input paths, which can be individual JSONL files or directories containing JSONL files.
#### Example with Sample Input
Let's say you have a file named `batchinput.jsonl` with the following content:
```jsonl
{"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-4o-mini", "messages": [{"role": "system", "content": "You are a helpful assistant."},{"role": "user", "content": "Hello world!"}],"max_tokens": 1000}}
{"custom_id": "request-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-4o-mini", "messages": [{"role": "system", "content": "You are an unhelpful assistant."},{"role": "user", "content": "Hello world!"}],"max_tokens": 1000}}
```
To process this file using BatchWizard:
1. First, ensure your OpenAI API key is set:
```bash
batchwizard configure --set-key YOUR_API_KEY
```
2. Then, run the process command:
```bash
batchwizard process /path/to/batchinput.jsonl --output-directory /path/to/output
```
This command will:
- Upload the `batchinput.jsonl` file to OpenAI
- Create a batch job
- Monitor the job status
- Download the results to the specified output directory when complete
You can also process multiple files or directories:
```bash
batchwizard process /path/to/file1.jsonl /path/to/directory_with_jsonl_files /path/to/file2.jsonl
```
### List Recent Jobs
To list recent batch jobs:
```bash
batchwizard list-jobs [--limit NUM] [--all]
```
### Cancel a Job
To cancel a specific batch job:
```bash
batchwizard cancel <job_id>
```
### Download Job Results
To download results for a completed batch job:
```bash
batchwizard download <job_id> [--output-file FILE_PATH]
```
## Configuration
### Setting up the OpenAI API Key
To set the OpenAI API key:
```bash
batchwizard configure --set-key YOUR_API_KEY
```
### Show Current Configuration
To show the current configuration:
```bash
batchwizard configure --show
```
### Reset Configuration
To reset the configuration to default values:
```bash
batchwizard configure --reset
```
## Commands
BatchWizard supports the following commands:
- `process`: Process batch jobs from input files or directories.
- `configure`: Manage BatchWizard configuration.
- `list-jobs`: List recent batch jobs.
- `cancel`: Cancel a specific batch job.
- `download`: Download results for a completed batch job.
For detailed information on each command, use the `--help` option:
```bash
batchwizard <command> --help
```
## Features
- **Flexible Input**: Process individual JSONL files or entire directories containing JSONL files.
- **Asynchronous Processing**: Efficiently handle multiple batch jobs concurrently.
- **Rich UI**: Display progress and job status using a rich, interactive interface.
- **Flexible Configuration**: Easily manage API keys and other settings.
- **Job Management**: List, cancel, and download results for batch jobs.
- **Error Handling**: Robust error handling and informative error messages.
## Contributing
We welcome contributions to BatchWizard! To contribute, follow these steps:
1. Fork the repository.
2. Create a new branch: `git checkout -b feature/your-feature-name`.
3. Make your changes and commit them: `git commit -m 'Add some feature'`.
4. Push to the branch: `git push origin feature/your-feature-name`.
5. Open a pull request.
### Running Tests
To run tests, use `pytest`:
```bash
pytest --cov=batchwizard tests/
```
Ensure your code passes all tests and meets the coding standards before opening a pull request.
## License
This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
## Contact
For any questions or feedback, feel free to open an issue on the [GitHub repository](https://github.com/cmakafui/batchwizard).
Raw data
{
"_id": null,
"home_page": "https://github.com/cmakafui/batchwizard",
"name": "batchwizard",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.9",
"maintainer_email": null,
"keywords": "openai, batch, cli, async",
"author": "Carl Kugblenu",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/06/d6/d79bc14bc78027b5377926e33fa0826efacff76332c173c0b74c3faa8a68/batchwizard-0.2.0.tar.gz",
"platform": null,
"description": "# BatchWizard\n\nBatchWizard is a powerful CLI tool for managing OpenAI batch processing jobs with ease. It provides functionalities to upload files, create batch jobs, check their status, and download the results. The tool uses asynchronous processing to efficiently handle multiple jobs concurrently.\n\n## Table of Contents\n\n- [Installation](#installation)\n- [Usage](#usage)\n- [Configuration](#configuration)\n- [Commands](#commands)\n- [Features](#features)\n- [Contributing](#contributing)\n- [License](#license)\n\n## Installation\n\nYou can install BatchWizard using `pipx` for an isolated environment or directly via `pip`.\n\n### Using pipx (recommended)\n\n```bash\npipx install batchwizard\n```\n\n### Using pip\n\n```bash\npip install batchwizard\n```\n\nEnsure you have `pipx` or `pip` installed on your system. For `pipx`, you can follow the installation instructions [here](https://pipx.pypa.io/stable/installation/).\n\n## Usage\n\nBatchWizard provides a command-line interface (CLI) for managing batch jobs. Here are some example commands:\n\n### Process Batch Jobs\n\nTo process input files or directories:\n\n```bash\nbatchwizard process <input_paths>... [--output-directory OUTPUT_DIR] [--max-concurrent-jobs NUM] [--check-interval SECONDS]\n```\n\nYou can provide multiple input paths, which can be individual JSONL files or directories containing JSONL files.\n\n#### Example with Sample Input\n\nLet's say you have a file named `batchinput.jsonl` with the following content:\n\n```jsonl\n{\"custom_id\": \"request-1\", \"method\": \"POST\", \"url\": \"/v1/chat/completions\", \"body\": {\"model\": \"gpt-4o-mini\", \"messages\": [{\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},{\"role\": \"user\", \"content\": \"Hello world!\"}],\"max_tokens\": 1000}}\n{\"custom_id\": \"request-2\", \"method\": \"POST\", \"url\": \"/v1/chat/completions\", \"body\": {\"model\": \"gpt-4o-mini\", \"messages\": [{\"role\": \"system\", \"content\": \"You are an unhelpful assistant.\"},{\"role\": \"user\", \"content\": \"Hello world!\"}],\"max_tokens\": 1000}}\n```\n\nTo process this file using BatchWizard:\n\n1. First, ensure your OpenAI API key is set:\n ```bash\n batchwizard configure --set-key YOUR_API_KEY\n ```\n2. Then, run the process command:\n ```bash\n batchwizard process /path/to/batchinput.jsonl --output-directory /path/to/output\n ```\n This command will:\n - Upload the `batchinput.jsonl` file to OpenAI\n - Create a batch job\n - Monitor the job status\n - Download the results to the specified output directory when complete\n\nYou can also process multiple files or directories:\n\n```bash\nbatchwizard process /path/to/file1.jsonl /path/to/directory_with_jsonl_files /path/to/file2.jsonl\n```\n\n### List Recent Jobs\n\nTo list recent batch jobs:\n\n```bash\nbatchwizard list-jobs [--limit NUM] [--all]\n```\n\n### Cancel a Job\n\nTo cancel a specific batch job:\n\n```bash\nbatchwizard cancel <job_id>\n```\n\n### Download Job Results\n\nTo download results for a completed batch job:\n\n```bash\nbatchwizard download <job_id> [--output-file FILE_PATH]\n```\n\n## Configuration\n\n### Setting up the OpenAI API Key\n\nTo set the OpenAI API key:\n\n```bash\nbatchwizard configure --set-key YOUR_API_KEY\n```\n\n### Show Current Configuration\n\nTo show the current configuration:\n\n```bash\nbatchwizard configure --show\n```\n\n### Reset Configuration\n\nTo reset the configuration to default values:\n\n```bash\nbatchwizard configure --reset\n```\n\n## Commands\n\nBatchWizard supports the following commands:\n\n- `process`: Process batch jobs from input files or directories.\n- `configure`: Manage BatchWizard configuration.\n- `list-jobs`: List recent batch jobs.\n- `cancel`: Cancel a specific batch job.\n- `download`: Download results for a completed batch job.\n\nFor detailed information on each command, use the `--help` option:\n\n```bash\nbatchwizard <command> --help\n```\n\n## Features\n\n- **Flexible Input**: Process individual JSONL files or entire directories containing JSONL files.\n- **Asynchronous Processing**: Efficiently handle multiple batch jobs concurrently.\n- **Rich UI**: Display progress and job status using a rich, interactive interface.\n- **Flexible Configuration**: Easily manage API keys and other settings.\n- **Job Management**: List, cancel, and download results for batch jobs.\n- **Error Handling**: Robust error handling and informative error messages.\n\n## Contributing\n\nWe welcome contributions to BatchWizard! To contribute, follow these steps:\n\n1. Fork the repository.\n2. Create a new branch: `git checkout -b feature/your-feature-name`.\n3. Make your changes and commit them: `git commit -m 'Add some feature'`.\n4. Push to the branch: `git push origin feature/your-feature-name`.\n5. Open a pull request.\n\n### Running Tests\n\nTo run tests, use `pytest`:\n\n```bash\npytest --cov=batchwizard tests/\n```\n\nEnsure your code passes all tests and meets the coding standards before opening a pull request.\n\n## License\n\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.\n\n## Contact\n\nFor any questions or feedback, feel free to open an issue on the [GitHub repository](https://github.com/cmakafui/batchwizard).\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "BatchWizard: Manage OpenAI batch processing jobs with ease",
"version": "0.2.0",
"project_urls": {
"Homepage": "https://github.com/cmakafui/batchwizard",
"Repository": "https://github.com/cmakafui/batchwizard"
},
"split_keywords": [
"openai",
" batch",
" cli",
" async"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c09ddc95caf4549f8440887840ed1d475e860999a75202f42b0c09993cde1bc7",
"md5": "0f0398b51652ed670aaefa7843629813",
"sha256": "ca357c15b04240d1b1d9e657db47f552c6adc14abe2f25172be874e733596c9f"
},
"downloads": -1,
"filename": "batchwizard-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "0f0398b51652ed670aaefa7843629813",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.9",
"size": 12399,
"upload_time": "2024-08-25T18:12:11",
"upload_time_iso_8601": "2024-08-25T18:12:11.675267Z",
"url": "https://files.pythonhosted.org/packages/c0/9d/dc95caf4549f8440887840ed1d475e860999a75202f42b0c09993cde1bc7/batchwizard-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "06d6d79bc14bc78027b5377926e33fa0826efacff76332c173c0b74c3faa8a68",
"md5": "36747c102ffefa49d95ea29930d943e9",
"sha256": "90d98e9c8b08614326645131507d303c5ac35017fa577189782be613dee3f58b"
},
"downloads": -1,
"filename": "batchwizard-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "36747c102ffefa49d95ea29930d943e9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.9",
"size": 11477,
"upload_time": "2024-08-25T18:12:13",
"upload_time_iso_8601": "2024-08-25T18:12:13.132957Z",
"url": "https://files.pythonhosted.org/packages/06/d6/d79bc14bc78027b5377926e33fa0826efacff76332c173c0b74c3faa8a68/batchwizard-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-25 18:12:13",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cmakafui",
"github_project": "batchwizard",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "batchwizard"
}
JSON
