# Medical LLM Fine-tuning with RAG System
A comprehensive PyPI package for fine-tuning large language models on medical literature with entity relationship extraction capabilities, featuring Qwen3-4B-Thinking model integration and advanced RAG-Anything multimodal document processing.
## 📦 Quick Start
```bash
# Install the package
pip install medllm-finetune-rag[all]
# Manual Unsloth installation (for 2x faster training)
pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
# Quick training
medllm train medical_data.json
# Or use Python API
python -c "from medllm import quick_train; quick_train('medical_data.json')"
```
## 🚀 Features
### 🤖 Advanced Model Support
- **Qwen3-4B-Thinking Integration**: State-of-the-art reasoning model with thinking capabilities
- **Unsloth Acceleration**: 2x faster training with reduced VRAM usage
- **Thinking Mode**: Support for reasoning-based inference with `<think></think>` tags
- **Multiple Training Formats**: Instruction fine-tuning and NER training formats
### 📄 RAG-Anything Integration
- **Multimodal Document Processing**: PDF, DOCX, images, tables, equations
- **Advanced Parsers**: MinerU and Docling for robust document extraction
- **LightRAG Knowledge Graph**: Graph-based retrieval for enhanced context
- **Multiple Search Modes**: Naive, local, global, and hybrid retrieval strategies
### 🏥 Medical Specialization
- **Medical Entity Extraction**: Specialized for Bacteria, Disease, Evidence entities
- **Relationship Extraction**: Complex medical relationship understanding
- **Domain-Specific Processing**: Optimized for medical literature analysis
- **Evaluation Metrics**: Built-in medical NER evaluation tools
### ⚡ Platform Optimization
- **Apple Silicon Support**: Optimized for Mac M2/M3 with MPS backend
- **CUDA Acceleration**: Full GPU acceleration for NVIDIA cards
- **CPU Fallback**: Reliable CPU-only training option
- **Universal Configuration**: Single config system for all platforms
## 🏗️ Project Structure
```
medllm-finetune-rag/
├── core/ # Core training modules
│ ├── medical_llm_trainer.py # Main training class with Unsloth integration
│ ├── data_processing.py # Data preprocessing and format conversion
│ ├── medical_rag_system.py # RAG-Anything system implementation
│ ├── evaluation_metrics.py # Model evaluation tools
│ └── huggingface_uploader.py # HuggingFace model upload utility
├── config/ # Configuration files
│ ├── config_mac_m2.yaml # Mac M2/M3 optimized configuration
│ ├── config_cuda.yaml # CUDA GPU configuration
│ └── config_cpu.yaml # CPU-only configuration
├── scripts/ # Utility scripts
│ ├── setup_environment.py # Environment setup and verification
│ ├── run_training_pipeline.py # Legacy training pipeline runner
│ ├── quick_start.sh # Quick setup script
│ └── setup_qwen3.sh # Qwen3 specific environment setup
├── docs/ # Documentation
│ ├── UNIVERSAL_TRAINER_GUIDE.md # Universal trainer documentation
│ ├── README_QWEN3.md # Qwen3 integration guide
│ └── README_MAC_M2.md # Mac M2 setup guide
├── examples/ # Example scripts and demos
│ ├── medical_rag_demo.py # Comprehensive RAG-Anything demo
│ ├── quick_rag_test.py # Quick RAG functionality test
│ ├── qwen3_example.py # Qwen3 model example
│ └── run_mac_m2.py # Mac M2 optimized runner
├── universal_trainer.py # Universal training script
├── run.py # Simple command wrapper
├── requirements.txt # Python dependencies
├── .env # Environment variables (API keys)
└── README.md # This file
```
## 🔧 Installation
### Prerequisites
- Python 3.9-3.13
- PyTorch with appropriate backend (CUDA/MPS/CPU)
- Git and GitHub CLI (optional, for repository management)
- OpenAI API key (optional, for RAG-Anything functionality)
### Quick Setup
1. **Clone the repository**:
```bash
git clone <repository-url>
cd medllm-finetune-rag
```
2. **Automated setup** (recommended):
```bash
chmod +x scripts/setup_qwen3.sh
./scripts/setup_qwen3.sh
```
This will automatically:
- Create and activate a virtual environment
- Install all required dependencies including RAG-Anything
- Set up platform-specific optimizations (Mac M2/CUDA/CPU)
- Verify the installation
3. **Configure API keys** (optional, for RAG functionality):
```bash
# Create .env file with your API keys
echo "OPENAI_API_KEY=your_openai_api_key_here" > .env
echo "HF_TOKEN=your_huggingface_token_here" >> .env
```
4. **Quick test**:
```bash
# Test basic functionality
python examples/quick_rag_test.py
# Test model inference
python run.py test
```
### Manual Installation
```bash
# Core dependencies
pip install torch torchvision torchaudio
pip install transformers>=4.36.0
pip install datasets>=2.14.0
pip install peft>=0.6.0
pip install bitsandbytes>=0.41.0
# Unsloth for efficient training (if compatible)
pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
pip install unsloth_zoo
pip install trl>=0.7.0
# Additional utilities
pip install wandb
pip install scikit-learn
pip install numpy pandas
pip install tqdm
pip install PyYAML
```
## 🚀 Quick Start
### 1. Universal Trainer (Recommended)
The universal trainer provides a unified interface for all training operations:
```bash
# Quick inference test
python run.py test
# Training with different configurations
python run.py train # Mac M2 optimized
python run.py cuda # CUDA GPU training
python run.py cpu # CPU-only training
# Training with HuggingFace upload
python run.py train-upload
# Interactive mode
python run.py interactive
# Full pipeline with upload
python run.py full-upload
```
### 2. RAG-Anything Demo
```bash
# Test RAG functionality
python examples/quick_rag_test.py
# Run comprehensive RAG demo
python examples/medical_rag_demo.py
```
### 3. Configuration-Based Training
```bash
# Use specific configuration files
python universal_trainer.py --config config/config_mac_m2.yaml --mode train
python universal_trainer.py --config config/config_cuda.yaml --mode inference
python universal_trainer.py --config config/config_cpu.yaml --mode eval
```
### 4. Medical Training Pipeline
For research and development with full control over each stage:
```bash
# Complete medical pipeline (all stages)
python scripts/run_training_pipeline.py --stage all
# Individual stages
python scripts/run_training_pipeline.py --stage 1 # Data preprocessing
python scripts/run_training_pipeline.py --stage 2 # Model training
python scripts/run_training_pipeline.py --stage 3 # RAG system building
python scripts/run_training_pipeline.py --stage 4 # Evaluation
python scripts/run_training_pipeline.py --stage 5 # Demo inference
# Custom parameters
python scripts/run_training_pipeline.py --stage 2 --model Qwen/Qwen3-4B-Thinking-2507 --epochs 5
```
### 5. Direct Training API
```python
from core.medical_llm_trainer import MedicalLLMTrainer
# Initialize trainer with Qwen3-4B-Thinking
trainer = MedicalLLMTrainer(
model_name="Qwen/Qwen3-4B-Thinking-2507",
use_unsloth=True,
use_qlora=True,
max_seq_length=2048
)
# Train on your data
trainer.train("path/to/your/training_data.json")
# Inference with thinking mode
result = trainer.inference(
"Hepatitis C virus causes chronic liver infection.",
enable_thinking=True
)
print(result)
```
## 🛠️ Training Tools Comparison
This project provides multiple ways to train and use medical LLMs, each optimized for different use cases:
### 📋 Tools Overview
| **Tool** | **Purpose** | **Best For** | **Complexity** |
|----------|-------------|--------------|----------------|
| `scripts/run.py` | Simple command wrapper | Quick tasks, daily use | ⭐ Low |
| `scripts/run_training_pipeline.py` | Complete medical pipeline | Research, development | ⭐⭐⭐ High |
| `universal_trainer.py` | Configuration-driven trainer | Production, customization | ⭐⭐ Medium |
| Direct API | Python integration | Custom applications | ⭐⭐ Medium |
### 🚀 run.py - Quick & Easy Commands
**Perfect for**: Daily use, quick testing, simple training
```bash
# Quick commands - just works!
python run.py test # Quick inference test
python run.py train # Full training (Mac M2 optimized)
python run.py train-upload # Train + upload to HuggingFace
python run.py cuda # CUDA optimized inference
python run.py interactive # Interactive chat mode
python run.py full-upload # Complete pipeline + upload
```
**How it works**: Simple wrapper that translates commands to `universal_trainer.py` calls
- 113 lines of code
- No dependencies beyond subprocess
- Instant gratification
### 🏥 run_training_pipeline.py - Medical Research Pipeline
**Perfect for**: Medical NLP research, stage-by-stage development, detailed analysis
```bash
# Stage-based execution with full control
python scripts/run_training_pipeline.py --stage all # Complete pipeline
python scripts/run_training_pipeline.py --stage 1 # Data preprocessing only
python scripts/run_training_pipeline.py --stage 2 --epochs 10 # Custom training
python scripts/run_training_pipeline.py --stage 3 # RAG system building
python scripts/run_training_pipeline.py --stage 4 # Evaluation with metrics
python scripts/run_training_pipeline.py --stage 5 # Demo inference
# Advanced options
python scripts/run_training_pipeline.py --stage 2 \
--model Qwen/Qwen3-4B-Thinking-2507 \
--device cuda \
--batch_size 4 \
--lr 3e-4
```
**Pipeline Stages**:
1. **Data Preprocessing**: Load, clean, augment medical data
2. **Model Training**: Fine-tune Qwen3-4B-Thinking for medical tasks
3. **RAG System Building**: Create RAG-Anything enhanced retrieval system
4. **Evaluation**: Comprehensive metrics and performance analysis
5. **Demo Inference**: Multi-mode RAG inference demonstration
**Features**:
- 673 lines of comprehensive functionality
- Async RAG-Anything integration
- Detailed error handling and recovery
- Medical-specific evaluation metrics
- Stage-by-stage execution control
### ⚙️ universal_trainer.py - Configuration-Driven
**Perfect for**: Production deployments, custom configurations, reproducible experiments
```bash
# Configuration-based training
python universal_trainer.py --config config/config_mac_m2.yaml --mode train
python universal_trainer.py --config config/config_cuda.yaml --mode inference
python universal_trainer.py --config config/config_cpu.yaml --mode eval
# Custom configurations
python universal_trainer.py --config my_custom_config.yaml --mode full --upload-to-hf
```
**Features**:
- YAML-based configuration
- Platform optimization (Mac M2/CUDA/CPU)
- HuggingFace integration
- RAG-Anything support
### 🎯 When to Use Which Tool?
| **Scenario** | **Recommended Tool** | **Command Example** |
|--------------|---------------------|---------------------|
| **Quick test** | `run.py` | `python run.py test` |
| **Daily training** | `run.py` | `python run.py train` |
| **Research experiment** | `run_training_pipeline.py` | `python scripts/run_training_pipeline.py --stage all` |
| **Custom evaluation** | `run_training_pipeline.py` | `python scripts/run_training_pipeline.py --stage 4` |
| **RAG development** | `run_training_pipeline.py` | `python scripts/run_training_pipeline.py --stage 3` |
| **Production deployment** | `universal_trainer.py` | `python universal_trainer.py --config prod_config.yaml` |
| **Custom integration** | Direct API | See Python examples below |
### 2. Data Processing
```python
from core.data_processing import MedicalDataProcessor
# Process raw medical data
processor = MedicalDataProcessor("raw_data.json")
processor.load_data()
processor.save_processed_data("processed_data/")
```
### 3. Using the Mock Trainer (for development)
```python
from examples.english_stable_solution import MockMedicalTrainer
# Use when network issues prevent model download
trainer = MockMedicalTrainer()
result = trainer.inference(
"Streptococcus pneumoniae causes pneumonia.",
enable_thinking=True
)
```
## 📊 Model Configuration
### Supported Models
- **Qwen3-4B-Thinking-2507** (Recommended): Advanced reasoning capabilities
- **Qwen2.5-7B-Instruct**: General instruction following
- **Custom models**: Compatible with HuggingFace transformers
### Training Parameters
```yaml
model:
name: "Qwen/Qwen3-4B-Thinking-2507"
use_unsloth: true
use_qlora: true
torch_dtype: "bfloat16"
max_seq_length: 2048
training:
batch_size: 4
gradient_accumulation_steps: 4
learning_rate: 2.0e-4
num_train_epochs: 3
warmup_steps: 10
save_steps: 100
```
## 🧠 Entity Types and Relations
### Entity Types
- **Bacteria**: Bacteria, viruses, and other pathogens
- **Disease**: Diseases, symptoms, and pathological conditions
- **Evidence**: Research evidence, conclusions, and findings
### Relationship Types
- **is_a**: Hierarchical relationship
- **biomarker_for**: Biomarker relationship
- **correlated_with**: Correlation relationship
- **has_relationship**: General relationship
## 🍎 Mac M2/M3 Support
Special optimizations for Apple Silicon:
- **MPS Backend**: Automatic detection and usage of Metal Performance Shaders
- **Virtual Environment**: Automatic setup to avoid system conflicts
- **Fallback Mechanisms**: Graceful degradation when Unsloth is incompatible
- **Memory Optimization**: Efficient memory usage for limited RAM
See [README_MAC_M2.md](docs/README_MAC_M2.md) for detailed setup instructions.
## 🔄 Thinking Mode
The Qwen3-4B-Thinking model supports reasoning mode with `<think></think>` tags:
```python
# Enable thinking mode for complex reasoning
result = trainer.inference(
"Analyze the relationship between H. pylori and gastric cancer.",
enable_thinking=True
)
# Output includes reasoning process:
# <think>
# I need to analyze the relationship between H. pylori and gastric cancer...
# </think>
#
# Final analysis in JSON format...
```
## 📈 Evaluation
Built-in evaluation metrics:
- **Entity Recognition**: Precision, Recall, F1-score
- **Relation Extraction**: Accuracy and relationship-specific metrics
- **Medical Accuracy**: Domain-specific evaluation criteria
```python
from core.evaluation_metrics import MedicalEvaluator
evaluator = MedicalEvaluator()
results = evaluator.evaluate_model(trainer, test_data)
print(f"F1 Score: {results['f1_score']:.4f}")
```
## 🛠️ Troubleshooting
### Common Issues
1. **Network Download Errors**:
- Use the mock trainer for development: `examples/english_stable_solution.py`
- Try different networks or download during off-peak hours
- Use `examples/download_step_by_step.py` for manual downloads
2. **Mac M2 Compatibility**:
- Use virtual environment to avoid system conflicts
- Fallback to standard transformers if Unsloth fails
- Check `docs/README_MAC_M2.md` for specific solutions
3. **Memory Issues**:
- Reduce batch size and max sequence length
- Enable gradient checkpointing
- Use QLoRA for memory-efficient fine-tuning
### Environment Issues
```bash
# Reset virtual environment
rm -rf venv
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
```
## 📚 Quick Reference
### 🚀 Most Common Commands
```bash
# For everyday use (recommended)
python run.py test # Quick test
python run.py train # Train model
python run.py interactive # Chat mode
# For research and development
python scripts/run_training_pipeline.py --stage all # Full pipeline
python scripts/run_training_pipeline.py --stage 4 # Evaluation only
# For RAG functionality
python examples/quick_rag_test.py # Test RAG
python examples/medical_rag_demo.py # Full RAG demo
# For setup and configuration
python scripts/setup_environment.py # Auto setup
python scripts/setup_environment.py --mode install # Install deps only
```
### 🔧 File Structure Quick Guide
```
medllm-finetune-rag/
├── scripts/run.py # 👈 Start here - simple commands
├── scripts/run_training_pipeline.py # 👈 Research pipeline
├── universal_trainer.py # Configuration-driven trainer
├── examples/ # Demo scripts and examples
├── config/ # Platform-specific configs
└── core/ # Core modules (advanced use)
```
### 💡 Troubleshooting Quick Fixes
```bash
# Dependencies missing?
python scripts/setup_environment.py --mode install
# Import errors?
pip install -r requirements.txt
# Mac M2 issues?
python scripts/setup_qwen3.sh
# Want to start fresh?
rm -rf venv && python -m venv venv && source venv/bin/activate
pip install -r requirements.txt
```
## 🤝 Contributing
1. Fork the repository
2. Create a feature branch: `git checkout -b feature-name`
3. Commit changes: `git commit -am 'Add feature'`
4. Push to branch: `git push origin feature-name`
5. Submit a Pull Request
## 📄 License
This project is licensed under the MIT License - see the LICENSE file for details.
## 🙏 Acknowledgments
- [Unsloth](https://github.com/unslothai/unsloth) for efficient LLM training
- [Qwen Team](https://github.com/QwenLM/Qwen) for the Qwen3-4B-Thinking model
- [HuggingFace](https://huggingface.co/) for the transformers ecosystem
- Medical research community for domain expertise
## 📞 Support
- 📧 Issues: Use GitHub Issues for bug reports and feature requests
- 📚 Documentation: Check the `docs/` directory for detailed guides
- 💬 Discussions: Use GitHub Discussions for questions and community support
---
**Note**: This project is designed for research and educational purposes. Ensure compliance with relevant medical data regulations and ethical guidelines when working with medical literature and patient data.
Raw data
{
"_id": null,
"home_page": "https://github.com/chenxingqiang/medllm-finetune-rag",
"name": "medllm-finetune-rag",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "Xingqiang Chen <joy66777@gmail.com>",
"keywords": "medical, llm, fine-tuning, rag, qwen, healthcare, nlp, ai",
"author": "Xingqiang Chen",
"author_email": "Xingqiang Chen <joy66777@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/56/77/94bb9c39bb15acb6ea856b781028b8a6013a005d03b3d63b555a6738d138/medllm_finetune_rag-0.1.2.tar.gz",
"platform": null,
"description": "# Medical LLM Fine-tuning with RAG System\n\nA comprehensive PyPI package for fine-tuning large language models on medical literature with entity relationship extraction capabilities, featuring Qwen3-4B-Thinking model integration and advanced RAG-Anything multimodal document processing.\n\n## \ud83d\udce6 Quick Start\n\n```bash\n# Install the package\npip install medllm-finetune-rag[all]\n\n# Manual Unsloth installation (for 2x faster training)\npip install \"unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git\"\n\n# Quick training\nmedllm train medical_data.json\n\n# Or use Python API\npython -c \"from medllm import quick_train; quick_train('medical_data.json')\"\n```\n\n## \ud83d\ude80 Features\n\n### \ud83e\udd16 Advanced Model Support\n- **Qwen3-4B-Thinking Integration**: State-of-the-art reasoning model with thinking capabilities\n- **Unsloth Acceleration**: 2x faster training with reduced VRAM usage\n- **Thinking Mode**: Support for reasoning-based inference with `<think></think>` tags\n- **Multiple Training Formats**: Instruction fine-tuning and NER training formats\n\n### \ud83d\udcc4 RAG-Anything Integration\n- **Multimodal Document Processing**: PDF, DOCX, images, tables, equations\n- **Advanced Parsers**: MinerU and Docling for robust document extraction\n- **LightRAG Knowledge Graph**: Graph-based retrieval for enhanced context\n- **Multiple Search Modes**: Naive, local, global, and hybrid retrieval strategies\n\n### \ud83c\udfe5 Medical Specialization\n- **Medical Entity Extraction**: Specialized for Bacteria, Disease, Evidence entities\n- **Relationship Extraction**: Complex medical relationship understanding\n- **Domain-Specific Processing**: Optimized for medical literature analysis\n- **Evaluation Metrics**: Built-in medical NER evaluation tools\n\n### \u26a1 Platform Optimization\n- **Apple Silicon Support**: Optimized for Mac M2/M3 with MPS backend\n- **CUDA Acceleration**: Full GPU acceleration for NVIDIA cards\n- **CPU Fallback**: Reliable CPU-only training option\n- **Universal Configuration**: Single config system for all platforms\n\n## \ud83c\udfd7\ufe0f Project Structure\n\n```\nmedllm-finetune-rag/\n\u251c\u2500\u2500 core/ # Core training modules\n\u2502 \u251c\u2500\u2500 medical_llm_trainer.py # Main training class with Unsloth integration\n\u2502 \u251c\u2500\u2500 data_processing.py # Data preprocessing and format conversion\n\u2502 \u251c\u2500\u2500 medical_rag_system.py # RAG-Anything system implementation\n\u2502 \u251c\u2500\u2500 evaluation_metrics.py # Model evaluation tools\n\u2502 \u2514\u2500\u2500 huggingface_uploader.py # HuggingFace model upload utility\n\u251c\u2500\u2500 config/ # Configuration files\n\u2502 \u251c\u2500\u2500 config_mac_m2.yaml # Mac M2/M3 optimized configuration\n\u2502 \u251c\u2500\u2500 config_cuda.yaml # CUDA GPU configuration\n\u2502 \u2514\u2500\u2500 config_cpu.yaml # CPU-only configuration\n\u251c\u2500\u2500 scripts/ # Utility scripts\n\u2502 \u251c\u2500\u2500 setup_environment.py # Environment setup and verification\n\u2502 \u251c\u2500\u2500 run_training_pipeline.py # Legacy training pipeline runner\n\u2502 \u251c\u2500\u2500 quick_start.sh # Quick setup script\n\u2502 \u2514\u2500\u2500 setup_qwen3.sh # Qwen3 specific environment setup\n\u251c\u2500\u2500 docs/ # Documentation\n\u2502 \u251c\u2500\u2500 UNIVERSAL_TRAINER_GUIDE.md # Universal trainer documentation\n\u2502 \u251c\u2500\u2500 README_QWEN3.md # Qwen3 integration guide\n\u2502 \u2514\u2500\u2500 README_MAC_M2.md # Mac M2 setup guide\n\u251c\u2500\u2500 examples/ # Example scripts and demos\n\u2502 \u251c\u2500\u2500 medical_rag_demo.py # Comprehensive RAG-Anything demo\n\u2502 \u251c\u2500\u2500 quick_rag_test.py # Quick RAG functionality test\n\u2502 \u251c\u2500\u2500 qwen3_example.py # Qwen3 model example\n\u2502 \u2514\u2500\u2500 run_mac_m2.py # Mac M2 optimized runner\n\u251c\u2500\u2500 universal_trainer.py # Universal training script\n\u251c\u2500\u2500 run.py # Simple command wrapper\n\u251c\u2500\u2500 requirements.txt # Python dependencies\n\u251c\u2500\u2500 .env # Environment variables (API keys)\n\u2514\u2500\u2500 README.md # This file\n```\n\n## \ud83d\udd27 Installation\n\n### Prerequisites\n\n- Python 3.9-3.13\n- PyTorch with appropriate backend (CUDA/MPS/CPU)\n- Git and GitHub CLI (optional, for repository management)\n- OpenAI API key (optional, for RAG-Anything functionality)\n\n### Quick Setup\n\n1. **Clone the repository**:\n ```bash\n git clone <repository-url>\ncd medllm-finetune-rag\n ```\n\n2. **Automated setup** (recommended):\n ```bash\n chmod +x scripts/setup_qwen3.sh\n ./scripts/setup_qwen3.sh\n ```\n \n This will automatically:\n - Create and activate a virtual environment\n - Install all required dependencies including RAG-Anything\n - Set up platform-specific optimizations (Mac M2/CUDA/CPU)\n - Verify the installation\n\n3. **Configure API keys** (optional, for RAG functionality):\n ```bash\n # Create .env file with your API keys\n echo \"OPENAI_API_KEY=your_openai_api_key_here\" > .env\n echo \"HF_TOKEN=your_huggingface_token_here\" >> .env\n ```\n\n4. **Quick test**:\n ```bash\n # Test basic functionality\n python examples/quick_rag_test.py\n \n # Test model inference\n python run.py test\n ```\n\n### Manual Installation\n\n```bash\n# Core dependencies\npip install torch torchvision torchaudio\npip install transformers>=4.36.0\npip install datasets>=2.14.0\npip install peft>=0.6.0\npip install bitsandbytes>=0.41.0\n\n# Unsloth for efficient training (if compatible)\npip install \"unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git\"\npip install unsloth_zoo\npip install trl>=0.7.0\n\n# Additional utilities\npip install wandb\npip install scikit-learn\npip install numpy pandas\npip install tqdm\npip install PyYAML\n```\n\n## \ud83d\ude80 Quick Start\n\n### 1. Universal Trainer (Recommended)\n\nThe universal trainer provides a unified interface for all training operations:\n\n```bash\n# Quick inference test\npython run.py test\n\n# Training with different configurations\npython run.py train # Mac M2 optimized\npython run.py cuda # CUDA GPU training\npython run.py cpu # CPU-only training\n\n# Training with HuggingFace upload\npython run.py train-upload\n\n# Interactive mode\npython run.py interactive\n\n# Full pipeline with upload\npython run.py full-upload\n```\n\n### 2. RAG-Anything Demo\n\n```bash\n# Test RAG functionality\npython examples/quick_rag_test.py\n\n# Run comprehensive RAG demo\npython examples/medical_rag_demo.py\n```\n\n### 3. Configuration-Based Training\n\n```bash\n# Use specific configuration files\npython universal_trainer.py --config config/config_mac_m2.yaml --mode train\npython universal_trainer.py --config config/config_cuda.yaml --mode inference\npython universal_trainer.py --config config/config_cpu.yaml --mode eval\n```\n\n### 4. Medical Training Pipeline\n\nFor research and development with full control over each stage:\n\n```bash\n# Complete medical pipeline (all stages)\npython scripts/run_training_pipeline.py --stage all\n\n# Individual stages\npython scripts/run_training_pipeline.py --stage 1 # Data preprocessing\npython scripts/run_training_pipeline.py --stage 2 # Model training\npython scripts/run_training_pipeline.py --stage 3 # RAG system building\npython scripts/run_training_pipeline.py --stage 4 # Evaluation\npython scripts/run_training_pipeline.py --stage 5 # Demo inference\n\n# Custom parameters\npython scripts/run_training_pipeline.py --stage 2 --model Qwen/Qwen3-4B-Thinking-2507 --epochs 5\n```\n\n### 5. Direct Training API\n\n```python\nfrom core.medical_llm_trainer import MedicalLLMTrainer\n\n# Initialize trainer with Qwen3-4B-Thinking\ntrainer = MedicalLLMTrainer(\n model_name=\"Qwen/Qwen3-4B-Thinking-2507\",\n use_unsloth=True,\n use_qlora=True,\n max_seq_length=2048\n)\n\n# Train on your data\ntrainer.train(\"path/to/your/training_data.json\")\n\n# Inference with thinking mode\nresult = trainer.inference(\n \"Hepatitis C virus causes chronic liver infection.\",\n enable_thinking=True\n)\nprint(result)\n```\n\n## \ud83d\udee0\ufe0f Training Tools Comparison\n\nThis project provides multiple ways to train and use medical LLMs, each optimized for different use cases:\n\n### \ud83d\udccb Tools Overview\n\n| **Tool** | **Purpose** | **Best For** | **Complexity** |\n|----------|-------------|--------------|----------------|\n| `scripts/run.py` | Simple command wrapper | Quick tasks, daily use | \u2b50 Low |\n| `scripts/run_training_pipeline.py` | Complete medical pipeline | Research, development | \u2b50\u2b50\u2b50 High |\n| `universal_trainer.py` | Configuration-driven trainer | Production, customization | \u2b50\u2b50 Medium |\n| Direct API | Python integration | Custom applications | \u2b50\u2b50 Medium |\n\n### \ud83d\ude80 run.py - Quick & Easy Commands\n\n**Perfect for**: Daily use, quick testing, simple training\n\n```bash\n# Quick commands - just works!\npython run.py test # Quick inference test\npython run.py train # Full training (Mac M2 optimized)\npython run.py train-upload # Train + upload to HuggingFace\npython run.py cuda # CUDA optimized inference\npython run.py interactive # Interactive chat mode\npython run.py full-upload # Complete pipeline + upload\n```\n\n**How it works**: Simple wrapper that translates commands to `universal_trainer.py` calls\n- 113 lines of code\n- No dependencies beyond subprocess\n- Instant gratification\n\n### \ud83c\udfe5 run_training_pipeline.py - Medical Research Pipeline\n\n**Perfect for**: Medical NLP research, stage-by-stage development, detailed analysis\n\n```bash\n# Stage-based execution with full control\npython scripts/run_training_pipeline.py --stage all # Complete pipeline\npython scripts/run_training_pipeline.py --stage 1 # Data preprocessing only\npython scripts/run_training_pipeline.py --stage 2 --epochs 10 # Custom training\npython scripts/run_training_pipeline.py --stage 3 # RAG system building\npython scripts/run_training_pipeline.py --stage 4 # Evaluation with metrics\npython scripts/run_training_pipeline.py --stage 5 # Demo inference\n\n# Advanced options\npython scripts/run_training_pipeline.py --stage 2 \\\n --model Qwen/Qwen3-4B-Thinking-2507 \\\n --device cuda \\\n --batch_size 4 \\\n --lr 3e-4\n```\n\n**Pipeline Stages**:\n1. **Data Preprocessing**: Load, clean, augment medical data\n2. **Model Training**: Fine-tune Qwen3-4B-Thinking for medical tasks\n3. **RAG System Building**: Create RAG-Anything enhanced retrieval system\n4. **Evaluation**: Comprehensive metrics and performance analysis\n5. **Demo Inference**: Multi-mode RAG inference demonstration\n\n**Features**:\n- 673 lines of comprehensive functionality\n- Async RAG-Anything integration\n- Detailed error handling and recovery\n- Medical-specific evaluation metrics\n- Stage-by-stage execution control\n\n### \u2699\ufe0f universal_trainer.py - Configuration-Driven\n\n**Perfect for**: Production deployments, custom configurations, reproducible experiments\n\n```bash\n# Configuration-based training\npython universal_trainer.py --config config/config_mac_m2.yaml --mode train\npython universal_trainer.py --config config/config_cuda.yaml --mode inference\npython universal_trainer.py --config config/config_cpu.yaml --mode eval\n\n# Custom configurations\npython universal_trainer.py --config my_custom_config.yaml --mode full --upload-to-hf\n```\n\n**Features**:\n- YAML-based configuration\n- Platform optimization (Mac M2/CUDA/CPU)\n- HuggingFace integration\n- RAG-Anything support\n\n### \ud83c\udfaf When to Use Which Tool?\n\n| **Scenario** | **Recommended Tool** | **Command Example** |\n|--------------|---------------------|---------------------|\n| **Quick test** | `run.py` | `python run.py test` |\n| **Daily training** | `run.py` | `python run.py train` |\n| **Research experiment** | `run_training_pipeline.py` | `python scripts/run_training_pipeline.py --stage all` |\n| **Custom evaluation** | `run_training_pipeline.py` | `python scripts/run_training_pipeline.py --stage 4` |\n| **RAG development** | `run_training_pipeline.py` | `python scripts/run_training_pipeline.py --stage 3` |\n| **Production deployment** | `universal_trainer.py` | `python universal_trainer.py --config prod_config.yaml` |\n| **Custom integration** | Direct API | See Python examples below |\n\n### 2. Data Processing\n\n```python\nfrom core.data_processing import MedicalDataProcessor\n\n# Process raw medical data\nprocessor = MedicalDataProcessor(\"raw_data.json\")\nprocessor.load_data()\nprocessor.save_processed_data(\"processed_data/\")\n```\n\n### 3. Using the Mock Trainer (for development)\n\n```python\nfrom examples.english_stable_solution import MockMedicalTrainer\n\n# Use when network issues prevent model download\ntrainer = MockMedicalTrainer()\nresult = trainer.inference(\n \"Streptococcus pneumoniae causes pneumonia.\",\n enable_thinking=True\n)\n```\n\n## \ud83d\udcca Model Configuration\n\n### Supported Models\n\n- **Qwen3-4B-Thinking-2507** (Recommended): Advanced reasoning capabilities\n- **Qwen2.5-7B-Instruct**: General instruction following\n- **Custom models**: Compatible with HuggingFace transformers\n\n### Training Parameters\n\n```yaml\nmodel:\n name: \"Qwen/Qwen3-4B-Thinking-2507\"\n use_unsloth: true\n use_qlora: true\n torch_dtype: \"bfloat16\"\n max_seq_length: 2048\n\ntraining:\n batch_size: 4\n gradient_accumulation_steps: 4\n learning_rate: 2.0e-4\n num_train_epochs: 3\n warmup_steps: 10\n save_steps: 100\n```\n\n## \ud83e\udde0 Entity Types and Relations\n\n### Entity Types\n- **Bacteria**: Bacteria, viruses, and other pathogens\n- **Disease**: Diseases, symptoms, and pathological conditions\n- **Evidence**: Research evidence, conclusions, and findings\n\n### Relationship Types\n- **is_a**: Hierarchical relationship\n- **biomarker_for**: Biomarker relationship\n- **correlated_with**: Correlation relationship\n- **has_relationship**: General relationship\n\n## \ud83c\udf4e Mac M2/M3 Support\n\nSpecial optimizations for Apple Silicon:\n\n- **MPS Backend**: Automatic detection and usage of Metal Performance Shaders\n- **Virtual Environment**: Automatic setup to avoid system conflicts\n- **Fallback Mechanisms**: Graceful degradation when Unsloth is incompatible\n- **Memory Optimization**: Efficient memory usage for limited RAM\n\nSee [README_MAC_M2.md](docs/README_MAC_M2.md) for detailed setup instructions.\n\n## \ud83d\udd04 Thinking Mode\n\nThe Qwen3-4B-Thinking model supports reasoning mode with `<think></think>` tags:\n\n```python\n# Enable thinking mode for complex reasoning\nresult = trainer.inference(\n \"Analyze the relationship between H. pylori and gastric cancer.\",\n enable_thinking=True\n)\n\n# Output includes reasoning process:\n# <think>\n# I need to analyze the relationship between H. pylori and gastric cancer...\n# </think>\n# \n# Final analysis in JSON format...\n```\n\n## \ud83d\udcc8 Evaluation\n\nBuilt-in evaluation metrics:\n\n- **Entity Recognition**: Precision, Recall, F1-score\n- **Relation Extraction**: Accuracy and relationship-specific metrics\n- **Medical Accuracy**: Domain-specific evaluation criteria\n\n```python\nfrom core.evaluation_metrics import MedicalEvaluator\n\nevaluator = MedicalEvaluator()\nresults = evaluator.evaluate_model(trainer, test_data)\nprint(f\"F1 Score: {results['f1_score']:.4f}\")\n```\n\n## \ud83d\udee0\ufe0f Troubleshooting\n\n### Common Issues\n\n1. **Network Download Errors**:\n - Use the mock trainer for development: `examples/english_stable_solution.py`\n - Try different networks or download during off-peak hours\n - Use `examples/download_step_by_step.py` for manual downloads\n\n2. **Mac M2 Compatibility**:\n - Use virtual environment to avoid system conflicts\n - Fallback to standard transformers if Unsloth fails\n - Check `docs/README_MAC_M2.md` for specific solutions\n\n3. **Memory Issues**:\n - Reduce batch size and max sequence length\n - Enable gradient checkpointing\n - Use QLoRA for memory-efficient fine-tuning\n\n### Environment Issues\n\n```bash\n# Reset virtual environment\nrm -rf venv\npython -m venv venv\nsource venv/bin/activate\npip install -r requirements.txt\n```\n\n## \ud83d\udcda Quick Reference\n\n### \ud83d\ude80 Most Common Commands\n\n```bash\n# For everyday use (recommended)\npython run.py test # Quick test\npython run.py train # Train model\npython run.py interactive # Chat mode\n\n# For research and development\npython scripts/run_training_pipeline.py --stage all # Full pipeline\npython scripts/run_training_pipeline.py --stage 4 # Evaluation only\n\n# For RAG functionality\npython examples/quick_rag_test.py # Test RAG\npython examples/medical_rag_demo.py # Full RAG demo\n\n# For setup and configuration\npython scripts/setup_environment.py # Auto setup\npython scripts/setup_environment.py --mode install # Install deps only\n```\n\n### \ud83d\udd27 File Structure Quick Guide\n\n```\nmedllm-finetune-rag/\n\u251c\u2500\u2500 scripts/run.py # \ud83d\udc48 Start here - simple commands\n\u251c\u2500\u2500 scripts/run_training_pipeline.py # \ud83d\udc48 Research pipeline\n\u251c\u2500\u2500 universal_trainer.py # Configuration-driven trainer\n\u251c\u2500\u2500 examples/ # Demo scripts and examples\n\u251c\u2500\u2500 config/ # Platform-specific configs\n\u2514\u2500\u2500 core/ # Core modules (advanced use)\n```\n\n### \ud83d\udca1 Troubleshooting Quick Fixes\n\n```bash\n# Dependencies missing?\npython scripts/setup_environment.py --mode install\n\n# Import errors?\npip install -r requirements.txt\n\n# Mac M2 issues?\npython scripts/setup_qwen3.sh\n\n# Want to start fresh?\nrm -rf venv && python -m venv venv && source venv/bin/activate\npip install -r requirements.txt\n```\n\n## \ud83e\udd1d Contributing\n\n1. Fork the repository\n2. Create a feature branch: `git checkout -b feature-name`\n3. Commit changes: `git commit -am 'Add feature'`\n4. Push to branch: `git push origin feature-name`\n5. Submit a Pull Request\n\n## \ud83d\udcc4 License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.\n\n## \ud83d\ude4f Acknowledgments\n\n- [Unsloth](https://github.com/unslothai/unsloth) for efficient LLM training\n- [Qwen Team](https://github.com/QwenLM/Qwen) for the Qwen3-4B-Thinking model\n- [HuggingFace](https://huggingface.co/) for the transformers ecosystem\n- Medical research community for domain expertise\n\n## \ud83d\udcde Support\n\n- \ud83d\udce7 Issues: Use GitHub Issues for bug reports and feature requests\n- \ud83d\udcda Documentation: Check the `docs/` directory for detailed guides\n- \ud83d\udcac Discussions: Use GitHub Discussions for questions and community support\n\n---\n\n**Note**: This project is designed for research and educational purposes. Ensure compliance with relevant medical data regulations and ethical guidelines when working with medical literature and patient data.\n",
"bugtrack_url": null,
"license": null,
"summary": "A comprehensive toolkit for fine-tuning medical large language models with RAG capabilities",
"version": "0.1.2",
"project_urls": {
"Bug Reports": "https://github.com/chenxingqiang/medllm-finetune-rag/issues",
"Documentation": "https://github.com/chenxingqiang/medllm-finetune-rag/blob/main/README.md",
"Homepage": "https://github.com/chenxingqiang/medllm-finetune-rag",
"Repository": "https://github.com/chenxingqiang/medllm-finetune-rag"
},
"split_keywords": [
"medical",
" llm",
" fine-tuning",
" rag",
" qwen",
" healthcare",
" nlp",
" ai"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "05f9c9cf8c3ba4a9777f3f05dae6e912500e75180194ed4eb9df6dfe1662baff",
"md5": "ed8dab5a1e3c4a5420fe31cd00480a8c",
"sha256": "e0da0b0f80d13ac2bba34104542e4719da408651125158ca53ab6f971a50867b"
},
"downloads": -1,
"filename": "medllm_finetune_rag-0.1.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ed8dab5a1e3c4a5420fe31cd00480a8c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 98353,
"upload_time": "2025-08-19T01:46:10",
"upload_time_iso_8601": "2025-08-19T01:46:10.406058Z",
"url": "https://files.pythonhosted.org/packages/05/f9/c9cf8c3ba4a9777f3f05dae6e912500e75180194ed4eb9df6dfe1662baff/medllm_finetune_rag-0.1.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "567794bb9c39bb15acb6ea856b781028b8a6013a005d03b3d63b555a6738d138",
"md5": "1178584b9dfcc29019c31f49f8c553cb",
"sha256": "6bb1baed65ebff27d69bfb611257a0600cc088377cf28f17d2323b24dd7bd9b6"
},
"downloads": -1,
"filename": "medllm_finetune_rag-0.1.2.tar.gz",
"has_sig": false,
"md5_digest": "1178584b9dfcc29019c31f49f8c553cb",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 110280,
"upload_time": "2025-08-19T01:46:12",
"upload_time_iso_8601": "2025-08-19T01:46:12.325639Z",
"url": "https://files.pythonhosted.org/packages/56/77/94bb9c39bb15acb6ea856b781028b8a6013a005d03b3d63b555a6738d138/medllm_finetune_rag-0.1.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-19 01:46:12",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "chenxingqiang",
"github_project": "medllm-finetune-rag",
"github_not_found": true,
"lcname": "medllm-finetune-rag"
}
JSON
