# YtQGen: YouTube to Questionnaire Generator
YtQGen is a Python library and CLI that turns educational YouTube videos into clean, printable questionnaires. It downloads audio, transcribes speech, summarizes content, and generates balanced questions, then exports a PDF.
## Highlights
- Multiple AI providers: OpenAI, Mistral, and Groq for both LLM tasks and transcription
- No external ffmpeg required: audio handled via PyAV and yt-dlp
- Robust on long inputs: automatic audio chunking and summary chunking
- Configurable outputs: control question count, type mix, difficulty, and style
- Plain-text safe for PDFs: prompts enforce ASCII-safe formatting for reliable PDF rendering
- CLI and Python API, with YAML config and .env support
## Requirements
- Python 3.10 or newer
- API key for at least one provider
- OpenAI: https://platform.openai.com/api-keys
- Mistral: https://console.mistral.ai/api-keys/
- Groq: https://console.groq.com/keys
## Installation
Install from PyPI using extras for your provider:
```bash
# Core package only
pip install ytqgen
# With a specific provider
pip install "ytqgen[openai]"
pip install "ytqgen[mistral]"
pip install "ytqgen[groq]"
# With all providers
pip install "ytqgen[all]"
# For development and tests
pip install -e ".[dev]"
```
The package entry point installs a command named `ytqgen`.
## Quickstart
### CLI
```bash
# Minimal: provide a YouTube URL and choose an LLM provider
ytqgen process --input "https://www.youtube.com/watch?v=VIDEO_ID" --provider openai --output quiz.pdf
# With a config file that includes keys and defaults
ytqgen process --config examples/config.yml --input "https://www.youtube.com/watch?v=VIDEO_ID"
# Override generation settings at the CLI
ytqgen process \
--input "https://www.youtube.com/watch?v=VIDEO_ID" \
--num-questions 20 \
--mcq-percentage 40 \
--subjective-percentage 60
```
CLI precedence: CLI arguments override YAML config, which overrides .env, which overrides existing environment variables.
Notes
- The CLI currently accepts only YouTube URLs for `--input`.
- YouTube downloads are best run on a local machine. Some cloud or hosted environments may block downloads.
### Python API
```python
import os
from ytqgen import process_stream
# Configure providers and keys via environment
os.environ["LLM_PROVIDER"] = "openai" # or "mistral" or "groq"
os.environ["TRANSCRIPTION_PROVIDER"] = "openai" # or "mistral" or "groq"
os.environ["OPENAI_API_KEY"] = "your-openai-key"
pdf_path = process_stream("https://www.youtube.com/watch?v=VIDEO_ID", output_pdf="questionnaire.pdf")
print(pdf_path)
```
## Configuration
You can configure YtQGen with a YAML file, environment variables, or CLI arguments. A sample is provided in `examples/config_template.yml`.
Supported keys in YAML
- provider: LLM provider for summarization and question generation. One of openai, mistral, groq
- transcription_provider: provider for audio transcription. One of openai, mistral, groq
- keys: API keys per provider
- openai: string
- mistral: string
- groq: string
- models: optional override of model names per provider
- openai: for example gpt-4o-mini, gpt-4o, gpt-4, gpt-3.5-turbo
- mistral: for example mistral-small-latest, mistral-medium-latest, mistral-large-latest
- groq: for example llama-3.1-70b-versatile, llama-3.3-70b-versatile, mixtral-8x7b-32768
- generation: questionnaire settings
- num_questions: integer greater than 0
- question_types: percentages that sum to 100
- mcq_percentage
- subjective_percentage
- difficulty: percentages that sum to 100
- easy_percentage
- medium_percentage
- hard_percentage
- style: percentages that sum to 100 (applied to subjective)
- scenario_percentage
- theoretical_percentage
Example
```yaml
input: https://www.youtube.com/watch?v=dQw4w9WgXcQ
output: my_questionnaire.pdf
provider: openai
transcription_provider: openai
keys:
openai: YOUR_OPENAI_API_KEY
models:
openai: gpt-4o-mini
generation:
num_questions: 10
question_types:
mcq_percentage: 20
subjective_percentage: 80
difficulty:
easy_percentage: 30
medium_percentage: 50
hard_percentage: 20
style:
scenario_percentage: 40
theoretical_percentage: 60
```
Environment variables
- LLM_PROVIDER, TRANSCRIPTION_PROVIDER
- OPENAI_API_KEY, MISTRAL_API_KEY, GROQ_API_KEY
- YTQGEN_OPENAI_MODEL, YTQGEN_MISTRAL_MODEL, YTQGEN_GROQ_MODEL
- YTQGEN_NUM_QUESTIONS, YTQGEN_MCQ_PERCENTAGE, YTQGEN_SUBJECTIVE_PERCENTAGE
- YTQGEN_EASY_PERCENTAGE, YTQGEN_MEDIUM_PERCENTAGE, YTQGEN_HARD_PERCENTAGE
- YTQGEN_SCENARIO_PERCENTAGE, YTQGEN_THEORETICAL_PERCENTAGE
## Providers and models
Transcription
- OpenAI: Whisper model whisper-1
- Groq: Whisper model whisper-large-v3
- Mistral: Voxtral endpoint voxtral-mini-latest
LLM summarization and question generation
- OpenAI: defaults to gpt-4o-mini unless overridden
- Mistral: defaults to mistral-small-latest unless overridden
- Groq: set with YTQGEN_GROQ_MODEL. Common choices include llama-3.1-70b-versatile, openai/gpt-oss-120b, and openai/gpt-oss-20b
Set explicit model names in your YAML or environment when in doubt.
If you are lost regarding models, check out the respective documentations.
- [OpenAI Documentation](https://api.openai.com/v1/models)
- [Groq Documentation](https://console.groq.com/docs/models)
- [Mistral Documentation](https://docs.mistral.ai/getting-started/models)
## How it works
Pipeline steps
1. Download audio with yt-dlp and convert to MP3 with PyAV
2. Transcribe audio with the selected provider. Long audio is automatically split into 10 minute chunks
3. Summarize the transcription using the selected LLM. Long summaries are created via chunked summarization and a final synthesis pass
4. Generate a balanced questionnaire from the summary using configurable distributions
5. Render a plain text PDF with PyMuPDF for broad compatibility
Key modules
- `ytqgen.audio_processor`: audio download and conversion with PyAV
- `ytqgen.transcription`: transcription with OpenAI, Groq, or Mistral, with chunking
- `ytqgen.summarizer`: chunked summarization using provider LLMs
- `ytqgen.questionnaire_generator`: dynamic prompts and distribution controls
- `ytqgen.pdf_generator`: plain text PDF writer using PyMuPDF
- `ytqgen.cli`: CLI implementation with YAML config and validation
- `ytqgen.pipeline`: orchestrates the end to end flow
## Examples
- `examples/config_template.yml`: copy to your project and fill in keys
- `examples/example_youtube.py`: simple programmatic usage with a YouTube URL
Note: A sample `examples/example_local_file.py` exists in the repository but the current pipeline expects a YouTube URL. Local file processing is not yet supported by the default pipeline.
## Troubleshooting
Common issues
- Missing API key: set the corresponding environment variable or place it in YAML under `keys`
- Invalid input: the CLI accepts only YouTube URLs
- yt-dlp blocked: some hosted environments block downloads. Run locally or provide a different network
- Audio conversion errors: ensure the `av` package is installed and up to date
- PDF output looks odd: the generator enforces ASCII in prompts, but if your model returns formatted text, reduce creativity or re-run
Update tools
```bash
pip install --upgrade yt-dlp av PyMuPDF
```
## Development and testing
Install dev dependencies and run the test suite
```bash
pip install -e ".[dev]"
pytest tests/ -v
```
Useful targets
- Run import checks: `python test_imports.py`
- Try a fast configuration: `ytqgen process --input "https://youtube.com/watch?v=VIDEO_ID" --config test_config.yml --output test_output.pdf`
## License
MIT License. See `LICENSE` for details.
## Contributing
Contributions are welcome. Please open an issue or a pull request with a clear description and steps to reproduce for bugs.
## Security and privacy
Downloaded audio is stored in the system temporary directory and removed after processing. API keys are read from environment variables or YAML at runtime and are not persisted by the library.
## Advanced usage (library)
YtQGen can be used as both a CLI and a Python library. The following examples show how to use individual components programmatically.
### Transcription (audio to text)
```python
from ytqgen.transcription import transcribe_audio
import os
# Configure transcription provider
os.environ['TRANSCRIPTION_PROVIDER'] = 'openai' # or 'groq', 'mistral'
os.environ['OPENAI_API_KEY'] = 'your-key'
# Transcribe audio file (automatically handles chunking for long files)
transcript = transcribe_audio(
audio_file_path='lecture.mp3',
chunk_duration_minutes=10 # Optional: chunk size for long audio
)
print(f"Transcript: {transcript[:200]}...")
```
Features
- Supports OpenAI Whisper, Groq Whisper, Mistral Voxtral
- Automatic chunking for audio longer than 10 minutes
- Returns full transcription as text
### Summarization (long text to summary)
```python
from ytqgen.summarizer import summarize_text
import os
# Configure LLM provider
os.environ['LLM_PROVIDER'] = 'groq'
os.environ['GROQ_API_KEY'] = 'your-key'
# Summarize long transcription
summary = summarize_text(
transcription=transcript,
max_tokens=2048 # Optional: control summary length
)
print(f"Summary: {summary}")
```
Features
- Automatic chunking for very long texts (>1000 words)
- Hierarchical summarization (chunk, combine, final)
- Token-based length control
### Question generation
```python
from ytqgen.questionnaire_generator import generate_questionnaire
import os
# Configure LLM provider (if not already set)
os.environ['LLM_PROVIDER'] = 'openai'
os.environ['OPENAI_API_KEY'] = 'your-key'
# Configure question distribution (optional)
os.environ['YTQGEN_NUM_QUESTIONS'] = '20'
os.environ['YTQGEN_MCQ_PERCENTAGE'] = '70'
os.environ['YTQGEN_SUBJECTIVE_PERCENTAGE'] = '30'
# Generate questionnaire from summary
questionnaire = generate_questionnaire(summary=summary)
print(questionnaire)
```
Features
- MCQ (multiple choice) and subjective questions
- Configurable question counts and difficulty distribution
- Scenario-based and theoretical questions
### PDF generation
```python
from ytqgen.pdf_generator import save_text_as_pdf
# Save questionnaire as PDF
save_text_as_pdf(
text=questionnaire,
filename='my_quiz.pdf'
)
print("PDF saved successfully.")
```
### LLM client (direct access)
```python
from ytqgen.model_loader import get_llm_client
import os
os.environ['LLM_PROVIDER'] = 'mistral'
os.environ['MISTRAL_API_KEY'] = 'your-key'
# Get initialized LLM client
client, model_name, provider = get_llm_client()
print(f"Using {provider} with model {model_name}")
# Use client directly for custom tasks
if provider == 'mistral':
response = client.chat.complete(
model=model_name,
messages=[{"role": "user", "content": "Explain AI"}]
)
else: # OpenAI or Groq
response = client.chat.completions.create(
model=model_name,
messages=[{"role": "user", "content": "Explain AI"}]
)
```
## Complete custom workflow example
```python
import os
from ytqgen.transcription import transcribe_audio
from ytqgen.summarizer import summarize_text
from ytqgen.questionnaire_generator import generate_questionnaire
from ytqgen.pdf_generator import save_text_as_pdf
# 1. Configure environment
os.environ['LLM_PROVIDER'] = 'groq'
os.environ['GROQ_API_KEY'] = 'gsk_...'
os.environ['TRANSCRIPTION_PROVIDER'] = 'groq'
os.environ['YTQGEN_NUM_QUESTIONS'] = '15'
# 2. Transcribe
print("Step 1: Transcribing...")
transcript = transcribe_audio('lecture.mp3')
# 3. Summarize
print("Step 2: Summarizing...")
summary = summarize_text(transcript, max_tokens=2048)
# 4. Generate questions
print("Step 3: Generating questions...")
questionnaire = generate_questionnaire(summary)
# 5. Save as PDF
print("Step 4: Creating PDF...")
save_text_as_pdf(questionnaire, 'custom_quiz.pdf')
print("Complete. Your quiz is ready.")
```
## Configuration via environment variables
All configuration can be done via environment variables:
```python
import os
# Provider selection
os.environ['LLM_PROVIDER'] = 'groq' # openai, mistral, groq
os.environ['TRANSCRIPTION_PROVIDER'] = 'groq'
# API keys
os.environ['OPENAI_API_KEY'] = 'sk-...'
os.environ['MISTRAL_API_KEY'] = 'mistral-...'
os.environ['GROQ_API_KEY'] = 'gsk_...'
# Model selection (optional)
os.environ['YTQGEN_OPENAI_MODEL'] = 'gpt-4o'
os.environ['YTQGEN_MISTRAL_MODEL'] = 'mistral-large-latest'
os.environ['YTQGEN_GROQ_MODEL'] = 'llama-3.3-70b-versatile'
# Question generation settings
os.environ['YTQGEN_NUM_QUESTIONS'] = '20'
os.environ['YTQGEN_MCQ_PERCENTAGE'] = '70'
os.environ['YTQGEN_SUBJECTIVE_PERCENTAGE'] = '30'
os.environ['YTQGEN_EASY_PERCENTAGE'] = '30'
os.environ['YTQGEN_MEDIUM_PERCENTAGE'] = '50'
os.environ['YTQGEN_HARD_PERCENTAGE'] = '20'
# Summary settings
os.environ['YTQGEN_SUMMARY_MAX_LENGTH'] = '500' # words
os.environ['YTQGEN_SUMMARY_MIN_LENGTH'] = '100' # words
```
## Error handling
```python
from ytqgen.transcription import transcribe_audio
try:
transcript = transcribe_audio('lecture.mp3')
except ValueError as e:
print(f"Configuration error: {e}")
# Handle missing API keys, invalid providers, etc.
except Exception as e:
print(f"Transcription failed: {e}")
# Handle API errors, file errors, etc.
```
Common errors
- ValueError: Missing API keys, invalid provider names
- FileNotFoundError: Audio file does not exist
- API errors: rate limits, invalid keys, network issues
## Use cases
### Batch processing multiple files
```python
import os
from ytqgen import process_stream
os.environ['LLM_PROVIDER'] = 'groq'
os.environ['GROQ_API_KEY'] = 'your-key'
os.environ['TRANSCRIPTION_PROVIDER'] = 'groq'
files = ['lecture1.mp4', 'lecture2.mp4', 'lecture3.mp4']
for i, file in enumerate(files, 1):
print(f"\nProcessing {i}/{len(files)}: {file}")
output = f'quiz_{i}.pdf'
try:
process_stream(file, output)
print(f"Saved to {output}")
except Exception as e:
print(f"Failed: {e}")
```
### Custom question distribution
```python
import os
from ytqgen.questionnaire_generator import generate_questionnaire
# More MCQs, fewer subjective
os.environ['YTQGEN_NUM_QUESTIONS'] = '25'
os.environ['YTQGEN_MCQ_PERCENTAGE'] = '80'
os.environ['YTQGEN_SUBJECTIVE_PERCENTAGE'] = '20'
# Harder questions
os.environ['YTQGEN_EASY_PERCENTAGE'] = '20'
os.environ['YTQGEN_MEDIUM_PERCENTAGE'] = '40'
os.environ['YTQGEN_HARD_PERCENTAGE'] = '40'
questionnaire = generate_questionnaire(summary)
```
### Multi-provider workflow
```python
import os
from ytqgen.transcription import transcribe_audio
from ytqgen.summarizer import summarize_text
from ytqgen.questionnaire_generator import generate_questionnaire
# Use Groq for transcription
os.environ['TRANSCRIPTION_PROVIDER'] = 'groq'
os.environ['GROQ_API_KEY'] = 'gsk_...'
transcript = transcribe_audio('lecture.mp3')
# Use OpenAI for summarization and questions
os.environ['LLM_PROVIDER'] = 'openai'
os.environ['OPENAI_API_KEY'] = 'sk-...'
os.environ['YTQGEN_OPENAI_MODEL'] = 'gpt-4o'
summary = summarize_text(transcript)
questionnaire = generate_questionnaire(summary)
```
## API reference
### `process_stream(video_url, output_pdf)`
Full pipeline to process video/audio and generate a questionnaire.
Parameters
- `video_url` (str): Path to local file or YouTube URL
- `output_pdf` (str): Output PDF filename
Returns
- `str`: Path to generated PDF, or `None` if error
Environment variables used
- `LLM_PROVIDER`, `TRANSCRIPTION_PROVIDER`
- API keys for selected providers
- All `YTQGEN_*` configuration variables
### `transcribe_audio(audio_file_path, chunk_duration_minutes=10)`
Transcribe audio to text using API services.
Parameters
- `audio_file_path` (str): Path to audio/video file
- `chunk_duration_minutes` (int): Chunk size for long audio (default: 10)
Returns
- `str`: Transcribed text
Environment variables used
- `TRANSCRIPTION_PROVIDER` (default: 'openai')
- `OPENAI_API_KEY`, `GROQ_API_KEY`, or `MISTRAL_API_KEY`
### `summarize_text(transcription, max_tokens=MAX_TOKENS)`
Summarize long transcription text.
Parameters
- `transcription` (str): Input text to summarize
- `max_tokens` (int): Maximum tokens for summary (default: 2048)
Returns
- `str`: Summarized text
Environment variables used
- `LLM_PROVIDER`
- API key for selected provider
- `YTQGEN_SUMMARY_MAX_LENGTH`, `YTQGEN_SUMMARY_MIN_LENGTH`
### `generate_questionnaire(summary)`
Generate a structured questionnaire from a summary.
Parameters
- `summary` (str): Summarized content
Returns
- `str`: Formatted questionnaire with questions and answers
Environment variables used
- `LLM_PROVIDER`
- API key for selected provider
- `YTQGEN_NUM_QUESTIONS`
- `YTQGEN_MCQ_PERCENTAGE`, `YTQGEN_SUBJECTIVE_PERCENTAGE`
- `YTQGEN_EASY_PERCENTAGE`, `YTQGEN_MEDIUM_PERCENTAGE`, `YTQGEN_HARD_PERCENTAGE`
### `save_text_as_pdf(text, filename)`
Save text content as a formatted PDF.
Parameters
- `text` (str): Content to save
- `filename` (str): Output PDF path
Returns
- `None`
### `get_llm_client()`
Get an initialized LLM client for the selected provider.
Returns
- `tuple`: (client, model_name, provider)
- `client`: Initialized API client object
- `model_name` (str): Model being used
- `provider` (str): Provider name ('openai', 'mistral', 'groq')
Environment variables used
- `LLM_PROVIDER`
- API key for selected provider
- `YTQGEN_OPENAI_MODEL`, `YTQGEN_MISTRAL_MODEL`, `YTQGEN_GROQ_MODEL`
## When to use library vs CLI
Use CLI when
- Quick one-off processing
- Simple workflows
- YAML configuration preferred
- Running from terminal or scripts
Use Python library when
- Custom processing pipelines
- Integration with existing Python applications
- Batch processing multiple files
- Need fine-grained control over each step
- Building applications on top of YtQGen
- Programmatic configuration required
Questions or issues?
Open an issue on [GitHub](https://github.com/JaiAnshSB26/YtQGen_Public/issues) or contact us at [team@jbac.dev](mailto:team@jbac.dev).
Raw data
{
"_id": null,
"home_page": null,
"name": "ytqgen",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "youtube, education, questionnaire, transcription, summarization, quiz, learning",
"author": "YtQGen Contributors",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/63/f2/7082318a10a7f2a080c9031aca11892baf5893fd7cfed550b8d42fc771a6/ytqgen-0.1.2.tar.gz",
"platform": null,
"description": "# YtQGen: YouTube to Questionnaire Generator\r\n\r\nYtQGen is a Python library and CLI that turns educational YouTube videos into clean, printable questionnaires. It downloads audio, transcribes speech, summarizes content, and generates balanced questions, then exports a PDF.\r\n\r\n## Highlights\r\n\r\n- Multiple AI providers: OpenAI, Mistral, and Groq for both LLM tasks and transcription\r\n- No external ffmpeg required: audio handled via PyAV and yt-dlp\r\n- Robust on long inputs: automatic audio chunking and summary chunking\r\n- Configurable outputs: control question count, type mix, difficulty, and style\r\n- Plain-text safe for PDFs: prompts enforce ASCII-safe formatting for reliable PDF rendering\r\n- CLI and Python API, with YAML config and .env support\r\n\r\n## Requirements\r\n\r\n- Python 3.10 or newer\r\n- API key for at least one provider\r\n - OpenAI: https://platform.openai.com/api-keys\r\n - Mistral: https://console.mistral.ai/api-keys/\r\n - Groq: https://console.groq.com/keys\r\n\r\n## Installation\r\n\r\nInstall from PyPI using extras for your provider:\r\n\r\n```bash\r\n# Core package only\r\npip install ytqgen\r\n\r\n# With a specific provider\r\npip install \"ytqgen[openai]\"\r\npip install \"ytqgen[mistral]\"\r\npip install \"ytqgen[groq]\"\r\n\r\n# With all providers\r\npip install \"ytqgen[all]\"\r\n\r\n# For development and tests\r\npip install -e \".[dev]\"\r\n```\r\n\r\nThe package entry point installs a command named `ytqgen`.\r\n\r\n## Quickstart\r\n\r\n### CLI\r\n\r\n```bash\r\n# Minimal: provide a YouTube URL and choose an LLM provider\r\nytqgen process --input \"https://www.youtube.com/watch?v=VIDEO_ID\" --provider openai --output quiz.pdf\r\n\r\n# With a config file that includes keys and defaults\r\nytqgen process --config examples/config.yml --input \"https://www.youtube.com/watch?v=VIDEO_ID\"\r\n\r\n# Override generation settings at the CLI\r\nytqgen process \\\r\n --input \"https://www.youtube.com/watch?v=VIDEO_ID\" \\\r\n --num-questions 20 \\\r\n --mcq-percentage 40 \\\r\n --subjective-percentage 60\r\n```\r\n\r\nCLI precedence: CLI arguments override YAML config, which overrides .env, which overrides existing environment variables.\r\n\r\nNotes\r\n- The CLI currently accepts only YouTube URLs for `--input`.\r\n- YouTube downloads are best run on a local machine. Some cloud or hosted environments may block downloads.\r\n\r\n### Python API\r\n\r\n```python\r\nimport os\r\nfrom ytqgen import process_stream\r\n\r\n# Configure providers and keys via environment\r\nos.environ[\"LLM_PROVIDER\"] = \"openai\" # or \"mistral\" or \"groq\"\r\nos.environ[\"TRANSCRIPTION_PROVIDER\"] = \"openai\" # or \"mistral\" or \"groq\"\r\nos.environ[\"OPENAI_API_KEY\"] = \"your-openai-key\"\r\n\r\npdf_path = process_stream(\"https://www.youtube.com/watch?v=VIDEO_ID\", output_pdf=\"questionnaire.pdf\")\r\nprint(pdf_path)\r\n```\r\n\r\n## Configuration\r\n\r\nYou can configure YtQGen with a YAML file, environment variables, or CLI arguments. A sample is provided in `examples/config_template.yml`.\r\n\r\nSupported keys in YAML\r\n\r\n- provider: LLM provider for summarization and question generation. One of openai, mistral, groq\r\n- transcription_provider: provider for audio transcription. One of openai, mistral, groq\r\n- keys: API keys per provider\r\n - openai: string\r\n - mistral: string\r\n - groq: string\r\n- models: optional override of model names per provider\r\n - openai: for example gpt-4o-mini, gpt-4o, gpt-4, gpt-3.5-turbo\r\n - mistral: for example mistral-small-latest, mistral-medium-latest, mistral-large-latest\r\n - groq: for example llama-3.1-70b-versatile, llama-3.3-70b-versatile, mixtral-8x7b-32768\r\n- generation: questionnaire settings\r\n - num_questions: integer greater than 0\r\n - question_types: percentages that sum to 100\r\n - mcq_percentage\r\n - subjective_percentage\r\n - difficulty: percentages that sum to 100\r\n - easy_percentage\r\n - medium_percentage\r\n - hard_percentage\r\n - style: percentages that sum to 100 (applied to subjective)\r\n - scenario_percentage\r\n - theoretical_percentage\r\n\r\nExample\r\n\r\n```yaml\r\ninput: https://www.youtube.com/watch?v=dQw4w9WgXcQ\r\noutput: my_questionnaire.pdf\r\n\r\nprovider: openai\r\ntranscription_provider: openai\r\n\r\nkeys:\r\n openai: YOUR_OPENAI_API_KEY\r\n\r\nmodels:\r\n openai: gpt-4o-mini\r\n\r\ngeneration:\r\n num_questions: 10\r\n question_types:\r\n mcq_percentage: 20\r\n subjective_percentage: 80\r\n difficulty:\r\n easy_percentage: 30\r\n medium_percentage: 50\r\n hard_percentage: 20\r\n style:\r\n scenario_percentage: 40\r\n theoretical_percentage: 60\r\n```\r\n\r\nEnvironment variables\r\n\r\n- LLM_PROVIDER, TRANSCRIPTION_PROVIDER\r\n- OPENAI_API_KEY, MISTRAL_API_KEY, GROQ_API_KEY\r\n- YTQGEN_OPENAI_MODEL, YTQGEN_MISTRAL_MODEL, YTQGEN_GROQ_MODEL\r\n- YTQGEN_NUM_QUESTIONS, YTQGEN_MCQ_PERCENTAGE, YTQGEN_SUBJECTIVE_PERCENTAGE\r\n- YTQGEN_EASY_PERCENTAGE, YTQGEN_MEDIUM_PERCENTAGE, YTQGEN_HARD_PERCENTAGE\r\n- YTQGEN_SCENARIO_PERCENTAGE, YTQGEN_THEORETICAL_PERCENTAGE\r\n\r\n## Providers and models\r\n\r\nTranscription\r\n- OpenAI: Whisper model whisper-1\r\n- Groq: Whisper model whisper-large-v3\r\n- Mistral: Voxtral endpoint voxtral-mini-latest\r\n\r\nLLM summarization and question generation\r\n- OpenAI: defaults to gpt-4o-mini unless overridden\r\n- Mistral: defaults to mistral-small-latest unless overridden\r\n- Groq: set with YTQGEN_GROQ_MODEL. Common choices include llama-3.1-70b-versatile, openai/gpt-oss-120b, and openai/gpt-oss-20b\r\n\r\nSet explicit model names in your YAML or environment when in doubt.\r\n\r\nIf you are lost regarding models, check out the respective documentations.\r\n- [OpenAI Documentation](https://api.openai.com/v1/models)\r\n- [Groq Documentation](https://console.groq.com/docs/models)\r\n- [Mistral Documentation](https://docs.mistral.ai/getting-started/models)\r\n\r\n## How it works\r\n\r\nPipeline steps\r\n1. Download audio with yt-dlp and convert to MP3 with PyAV\r\n2. Transcribe audio with the selected provider. Long audio is automatically split into 10 minute chunks\r\n3. Summarize the transcription using the selected LLM. Long summaries are created via chunked summarization and a final synthesis pass\r\n4. Generate a balanced questionnaire from the summary using configurable distributions\r\n5. Render a plain text PDF with PyMuPDF for broad compatibility\r\n\r\nKey modules\r\n- `ytqgen.audio_processor`: audio download and conversion with PyAV\r\n- `ytqgen.transcription`: transcription with OpenAI, Groq, or Mistral, with chunking\r\n- `ytqgen.summarizer`: chunked summarization using provider LLMs\r\n- `ytqgen.questionnaire_generator`: dynamic prompts and distribution controls\r\n- `ytqgen.pdf_generator`: plain text PDF writer using PyMuPDF\r\n- `ytqgen.cli`: CLI implementation with YAML config and validation\r\n- `ytqgen.pipeline`: orchestrates the end to end flow\r\n\r\n## Examples\r\n\r\n- `examples/config_template.yml`: copy to your project and fill in keys\r\n- `examples/example_youtube.py`: simple programmatic usage with a YouTube URL\r\n\r\nNote: A sample `examples/example_local_file.py` exists in the repository but the current pipeline expects a YouTube URL. Local file processing is not yet supported by the default pipeline.\r\n\r\n## Troubleshooting\r\n\r\nCommon issues\r\n\r\n- Missing API key: set the corresponding environment variable or place it in YAML under `keys`\r\n- Invalid input: the CLI accepts only YouTube URLs\r\n- yt-dlp blocked: some hosted environments block downloads. Run locally or provide a different network\r\n- Audio conversion errors: ensure the `av` package is installed and up to date\r\n- PDF output looks odd: the generator enforces ASCII in prompts, but if your model returns formatted text, reduce creativity or re-run\r\n\r\nUpdate tools\r\n\r\n```bash\r\npip install --upgrade yt-dlp av PyMuPDF\r\n```\r\n\r\n## Development and testing\r\n\r\nInstall dev dependencies and run the test suite\r\n\r\n```bash\r\npip install -e \".[dev]\"\r\npytest tests/ -v\r\n```\r\n\r\nUseful targets\r\n- Run import checks: `python test_imports.py`\r\n- Try a fast configuration: `ytqgen process --input \"https://youtube.com/watch?v=VIDEO_ID\" --config test_config.yml --output test_output.pdf`\r\n\r\n## License\r\n\r\nMIT License. See `LICENSE` for details.\r\n\r\n## Contributing\r\n\r\nContributions are welcome. Please open an issue or a pull request with a clear description and steps to reproduce for bugs.\r\n\r\n## Security and privacy\r\n\r\nDownloaded audio is stored in the system temporary directory and removed after processing. API keys are read from environment variables or YAML at runtime and are not persisted by the library.\r\n\r\n\r\n## Advanced usage (library)\r\n\r\nYtQGen can be used as both a CLI and a Python library. The following examples show how to use individual components programmatically.\r\n\r\n### Transcription (audio to text)\r\n\r\n```python\r\nfrom ytqgen.transcription import transcribe_audio\r\nimport os\r\n\r\n# Configure transcription provider\r\nos.environ['TRANSCRIPTION_PROVIDER'] = 'openai' # or 'groq', 'mistral'\r\nos.environ['OPENAI_API_KEY'] = 'your-key'\r\n\r\n# Transcribe audio file (automatically handles chunking for long files)\r\ntranscript = transcribe_audio(\r\n audio_file_path='lecture.mp3',\r\n chunk_duration_minutes=10 # Optional: chunk size for long audio\r\n)\r\n\r\nprint(f\"Transcript: {transcript[:200]}...\")\r\n```\r\n\r\nFeatures\r\n- Supports OpenAI Whisper, Groq Whisper, Mistral Voxtral\r\n- Automatic chunking for audio longer than 10 minutes\r\n- Returns full transcription as text\r\n\r\n### Summarization (long text to summary)\r\n\r\n```python\r\nfrom ytqgen.summarizer import summarize_text\r\nimport os\r\n\r\n# Configure LLM provider\r\nos.environ['LLM_PROVIDER'] = 'groq'\r\nos.environ['GROQ_API_KEY'] = 'your-key'\r\n\r\n# Summarize long transcription\r\nsummary = summarize_text(\r\n transcription=transcript,\r\n max_tokens=2048 # Optional: control summary length\r\n)\r\n\r\nprint(f\"Summary: {summary}\")\r\n```\r\n\r\nFeatures\r\n- Automatic chunking for very long texts (>1000 words)\r\n- Hierarchical summarization (chunk, combine, final)\r\n- Token-based length control\r\n\r\n### Question generation\r\n\r\n```python\r\nfrom ytqgen.questionnaire_generator import generate_questionnaire\r\nimport os\r\n\r\n# Configure LLM provider (if not already set)\r\nos.environ['LLM_PROVIDER'] = 'openai'\r\nos.environ['OPENAI_API_KEY'] = 'your-key'\r\n\r\n# Configure question distribution (optional)\r\nos.environ['YTQGEN_NUM_QUESTIONS'] = '20'\r\nos.environ['YTQGEN_MCQ_PERCENTAGE'] = '70'\r\nos.environ['YTQGEN_SUBJECTIVE_PERCENTAGE'] = '30'\r\n\r\n# Generate questionnaire from summary\r\nquestionnaire = generate_questionnaire(summary=summary)\r\n\r\nprint(questionnaire)\r\n```\r\n\r\nFeatures\r\n- MCQ (multiple choice) and subjective questions\r\n- Configurable question counts and difficulty distribution\r\n- Scenario-based and theoretical questions\r\n\r\n### PDF generation\r\n\r\n```python\r\nfrom ytqgen.pdf_generator import save_text_as_pdf\r\n\r\n# Save questionnaire as PDF\r\nsave_text_as_pdf(\r\n text=questionnaire,\r\n filename='my_quiz.pdf'\r\n)\r\n\r\nprint(\"PDF saved successfully.\")\r\n```\r\n\r\n### LLM client (direct access)\r\n\r\n```python\r\nfrom ytqgen.model_loader import get_llm_client\r\nimport os\r\n\r\nos.environ['LLM_PROVIDER'] = 'mistral'\r\nos.environ['MISTRAL_API_KEY'] = 'your-key'\r\n\r\n# Get initialized LLM client\r\nclient, model_name, provider = get_llm_client()\r\n\r\nprint(f\"Using {provider} with model {model_name}\")\r\n\r\n# Use client directly for custom tasks\r\nif provider == 'mistral':\r\n response = client.chat.complete(\r\n model=model_name,\r\n messages=[{\"role\": \"user\", \"content\": \"Explain AI\"}]\r\n )\r\nelse: # OpenAI or Groq\r\n response = client.chat.completions.create(\r\n model=model_name,\r\n messages=[{\"role\": \"user\", \"content\": \"Explain AI\"}]\r\n )\r\n```\r\n\r\n## Complete custom workflow example\r\n\r\n```python\r\nimport os\r\nfrom ytqgen.transcription import transcribe_audio\r\nfrom ytqgen.summarizer import summarize_text\r\nfrom ytqgen.questionnaire_generator import generate_questionnaire\r\nfrom ytqgen.pdf_generator import save_text_as_pdf\r\n\r\n# 1. Configure environment\r\nos.environ['LLM_PROVIDER'] = 'groq'\r\nos.environ['GROQ_API_KEY'] = 'gsk_...'\r\nos.environ['TRANSCRIPTION_PROVIDER'] = 'groq'\r\nos.environ['YTQGEN_NUM_QUESTIONS'] = '15'\r\n\r\n# 2. Transcribe\r\nprint(\"Step 1: Transcribing...\")\r\ntranscript = transcribe_audio('lecture.mp3')\r\n\r\n# 3. Summarize\r\nprint(\"Step 2: Summarizing...\")\r\nsummary = summarize_text(transcript, max_tokens=2048)\r\n\r\n# 4. Generate questions\r\nprint(\"Step 3: Generating questions...\")\r\nquestionnaire = generate_questionnaire(summary)\r\n\r\n# 5. Save as PDF\r\nprint(\"Step 4: Creating PDF...\")\r\nsave_text_as_pdf(questionnaire, 'custom_quiz.pdf')\r\n\r\nprint(\"Complete. Your quiz is ready.\")\r\n```\r\n\r\n## Configuration via environment variables\r\n\r\nAll configuration can be done via environment variables:\r\n\r\n```python\r\nimport os\r\n\r\n# Provider selection\r\nos.environ['LLM_PROVIDER'] = 'groq' # openai, mistral, groq\r\nos.environ['TRANSCRIPTION_PROVIDER'] = 'groq'\r\n\r\n# API keys\r\nos.environ['OPENAI_API_KEY'] = 'sk-...'\r\nos.environ['MISTRAL_API_KEY'] = 'mistral-...'\r\nos.environ['GROQ_API_KEY'] = 'gsk_...'\r\n\r\n# Model selection (optional)\r\nos.environ['YTQGEN_OPENAI_MODEL'] = 'gpt-4o'\r\nos.environ['YTQGEN_MISTRAL_MODEL'] = 'mistral-large-latest'\r\nos.environ['YTQGEN_GROQ_MODEL'] = 'llama-3.3-70b-versatile'\r\n\r\n# Question generation settings\r\nos.environ['YTQGEN_NUM_QUESTIONS'] = '20'\r\nos.environ['YTQGEN_MCQ_PERCENTAGE'] = '70'\r\nos.environ['YTQGEN_SUBJECTIVE_PERCENTAGE'] = '30'\r\nos.environ['YTQGEN_EASY_PERCENTAGE'] = '30'\r\nos.environ['YTQGEN_MEDIUM_PERCENTAGE'] = '50'\r\nos.environ['YTQGEN_HARD_PERCENTAGE'] = '20'\r\n\r\n# Summary settings\r\nos.environ['YTQGEN_SUMMARY_MAX_LENGTH'] = '500' # words\r\nos.environ['YTQGEN_SUMMARY_MIN_LENGTH'] = '100' # words\r\n```\r\n\r\n## Error handling\r\n\r\n```python\r\nfrom ytqgen.transcription import transcribe_audio\r\n\r\ntry:\r\n transcript = transcribe_audio('lecture.mp3')\r\nexcept ValueError as e:\r\n print(f\"Configuration error: {e}\")\r\n # Handle missing API keys, invalid providers, etc.\r\nexcept Exception as e:\r\n print(f\"Transcription failed: {e}\")\r\n # Handle API errors, file errors, etc.\r\n```\r\n\r\nCommon errors\r\n- ValueError: Missing API keys, invalid provider names\r\n- FileNotFoundError: Audio file does not exist\r\n- API errors: rate limits, invalid keys, network issues\r\n\r\n## Use cases\r\n\r\n### Batch processing multiple files\r\n\r\n```python\r\nimport os\r\nfrom ytqgen import process_stream\r\n\r\nos.environ['LLM_PROVIDER'] = 'groq'\r\nos.environ['GROQ_API_KEY'] = 'your-key'\r\nos.environ['TRANSCRIPTION_PROVIDER'] = 'groq'\r\n\r\nfiles = ['lecture1.mp4', 'lecture2.mp4', 'lecture3.mp4']\r\n\r\nfor i, file in enumerate(files, 1):\r\n print(f\"\\nProcessing {i}/{len(files)}: {file}\")\r\n output = f'quiz_{i}.pdf'\r\n try:\r\n process_stream(file, output)\r\n print(f\"Saved to {output}\")\r\n except Exception as e:\r\n print(f\"Failed: {e}\")\r\n```\r\n\r\n### Custom question distribution\r\n\r\n```python\r\nimport os\r\nfrom ytqgen.questionnaire_generator import generate_questionnaire\r\n\r\n# More MCQs, fewer subjective\r\nos.environ['YTQGEN_NUM_QUESTIONS'] = '25'\r\nos.environ['YTQGEN_MCQ_PERCENTAGE'] = '80'\r\nos.environ['YTQGEN_SUBJECTIVE_PERCENTAGE'] = '20'\r\n\r\n# Harder questions\r\nos.environ['YTQGEN_EASY_PERCENTAGE'] = '20'\r\nos.environ['YTQGEN_MEDIUM_PERCENTAGE'] = '40'\r\nos.environ['YTQGEN_HARD_PERCENTAGE'] = '40'\r\n\r\nquestionnaire = generate_questionnaire(summary)\r\n```\r\n\r\n### Multi-provider workflow\r\n\r\n```python\r\nimport os\r\nfrom ytqgen.transcription import transcribe_audio\r\nfrom ytqgen.summarizer import summarize_text\r\nfrom ytqgen.questionnaire_generator import generate_questionnaire\r\n\r\n# Use Groq for transcription\r\nos.environ['TRANSCRIPTION_PROVIDER'] = 'groq'\r\nos.environ['GROQ_API_KEY'] = 'gsk_...'\r\n\r\ntranscript = transcribe_audio('lecture.mp3')\r\n\r\n# Use OpenAI for summarization and questions\r\nos.environ['LLM_PROVIDER'] = 'openai'\r\nos.environ['OPENAI_API_KEY'] = 'sk-...'\r\nos.environ['YTQGEN_OPENAI_MODEL'] = 'gpt-4o'\r\n\r\nsummary = summarize_text(transcript)\r\nquestionnaire = generate_questionnaire(summary)\r\n```\r\n\r\n## API reference\r\n\r\n### `process_stream(video_url, output_pdf)`\r\n\r\nFull pipeline to process video/audio and generate a questionnaire.\r\n\r\nParameters\r\n- `video_url` (str): Path to local file or YouTube URL\r\n- `output_pdf` (str): Output PDF filename\r\n\r\nReturns\r\n- `str`: Path to generated PDF, or `None` if error\r\n\r\nEnvironment variables used\r\n- `LLM_PROVIDER`, `TRANSCRIPTION_PROVIDER`\r\n- API keys for selected providers\r\n- All `YTQGEN_*` configuration variables\r\n\r\n### `transcribe_audio(audio_file_path, chunk_duration_minutes=10)`\r\n\r\nTranscribe audio to text using API services.\r\n\r\nParameters\r\n- `audio_file_path` (str): Path to audio/video file\r\n- `chunk_duration_minutes` (int): Chunk size for long audio (default: 10)\r\n\r\nReturns\r\n- `str`: Transcribed text\r\n\r\nEnvironment variables used\r\n- `TRANSCRIPTION_PROVIDER` (default: 'openai')\r\n- `OPENAI_API_KEY`, `GROQ_API_KEY`, or `MISTRAL_API_KEY`\r\n\r\n### `summarize_text(transcription, max_tokens=MAX_TOKENS)`\r\n\r\nSummarize long transcription text.\r\n\r\nParameters\r\n- `transcription` (str): Input text to summarize\r\n- `max_tokens` (int): Maximum tokens for summary (default: 2048)\r\n\r\nReturns\r\n- `str`: Summarized text\r\n\r\nEnvironment variables used\r\n- `LLM_PROVIDER`\r\n- API key for selected provider\r\n- `YTQGEN_SUMMARY_MAX_LENGTH`, `YTQGEN_SUMMARY_MIN_LENGTH`\r\n\r\n### `generate_questionnaire(summary)`\r\n\r\nGenerate a structured questionnaire from a summary.\r\n\r\nParameters\r\n- `summary` (str): Summarized content\r\n\r\nReturns\r\n- `str`: Formatted questionnaire with questions and answers\r\n\r\nEnvironment variables used\r\n- `LLM_PROVIDER`\r\n- API key for selected provider\r\n- `YTQGEN_NUM_QUESTIONS`\r\n- `YTQGEN_MCQ_PERCENTAGE`, `YTQGEN_SUBJECTIVE_PERCENTAGE`\r\n- `YTQGEN_EASY_PERCENTAGE`, `YTQGEN_MEDIUM_PERCENTAGE`, `YTQGEN_HARD_PERCENTAGE`\r\n\r\n### `save_text_as_pdf(text, filename)`\r\n\r\nSave text content as a formatted PDF.\r\n\r\nParameters\r\n- `text` (str): Content to save\r\n- `filename` (str): Output PDF path\r\n\r\nReturns\r\n- `None`\r\n\r\n### `get_llm_client()`\r\n\r\nGet an initialized LLM client for the selected provider.\r\n\r\nReturns\r\n- `tuple`: (client, model_name, provider)\r\n - `client`: Initialized API client object\r\n - `model_name` (str): Model being used\r\n - `provider` (str): Provider name ('openai', 'mistral', 'groq')\r\n\r\nEnvironment variables used\r\n- `LLM_PROVIDER`\r\n- API key for selected provider\r\n- `YTQGEN_OPENAI_MODEL`, `YTQGEN_MISTRAL_MODEL`, `YTQGEN_GROQ_MODEL`\r\n\r\n## When to use library vs CLI\r\n\r\nUse CLI when\r\n- Quick one-off processing\r\n- Simple workflows\r\n- YAML configuration preferred\r\n- Running from terminal or scripts\r\n\r\nUse Python library when\r\n- Custom processing pipelines\r\n- Integration with existing Python applications\r\n- Batch processing multiple files\r\n- Need fine-grained control over each step\r\n- Building applications on top of YtQGen\r\n- Programmatic configuration required\r\n\r\nQuestions or issues?\r\n\r\nOpen an issue on [GitHub](https://github.com/JaiAnshSB26/YtQGen_Public/issues) or contact us at [team@jbac.dev](mailto:team@jbac.dev).\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Educational video to questionnaire generator - transcribe, summarize, and generate interactive quizzes from video content",
"version": "0.1.2",
"project_urls": {
"Bug Tracker": "https://github.com/AnubhavChoudhery/YtQGen/issues",
"Documentation": "https://github.com/AnubhavChoudhery/YtQGen#readme",
"Homepage": "https://github.com/AnubhavChoudhery/YtQGen",
"Repository": "https://github.com/AnubhavChoudhery/YtQGen"
},
"split_keywords": [
"youtube",
" education",
" questionnaire",
" transcription",
" summarization",
" quiz",
" learning"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "04c744c4e0aa9586a40cb5241f648ebcbd83ba587473bd01182ae57b63755e29",
"md5": "be75465592b2786d76de780037fb7eaa",
"sha256": "c9b9fe29fc2708faabb0ca1328f07d2ee876c4698c3e5ddbb86a65c4dca498ea"
},
"downloads": -1,
"filename": "ytqgen-0.1.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "be75465592b2786d76de780037fb7eaa",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 28404,
"upload_time": "2025-10-26T03:03:41",
"upload_time_iso_8601": "2025-10-26T03:03:41.370205Z",
"url": "https://files.pythonhosted.org/packages/04/c7/44c4e0aa9586a40cb5241f648ebcbd83ba587473bd01182ae57b63755e29/ytqgen-0.1.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "63f27082318a10a7f2a080c9031aca11892baf5893fd7cfed550b8d42fc771a6",
"md5": "8ccf193f55dc71d5729169082d029ad6",
"sha256": "1f2ecc0926bb50d92bd0028c535aec91547956c9d011d4a8330e1213e42973d4"
},
"downloads": -1,
"filename": "ytqgen-0.1.2.tar.gz",
"has_sig": false,
"md5_digest": "8ccf193f55dc71d5729169082d029ad6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 40957,
"upload_time": "2025-10-26T03:03:42",
"upload_time_iso_8601": "2025-10-26T03:03:42.689310Z",
"url": "https://files.pythonhosted.org/packages/63/f2/7082318a10a7f2a080c9031aca11892baf5893fd7cfed550b8d42fc771a6/ytqgen-0.1.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-26 03:03:42",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "AnubhavChoudhery",
"github_project": "YtQGen",
"github_not_found": true,
"lcname": "ytqgen"
}
JSON
