# Code Context Compiler
Code Context Compiler is a powerful CLI tool that scans a code project and creates a single file containing the entire project code with file-wise separation. The tool also masks sensitive information in the code to enhance security and provides various customization options.
## Features
- Scans an entire project directory
- Processes all file types by default
- Compiles all code files into a single output file
- Masks sensitive information such as passwords, API keys, and tokens
- Respects `.gitignore` patterns and custom ignore patterns
- Supports custom configuration via YAML files
- Asynchronous file processing for improved performance
- Progress indicator for large projects
- Multiple output formats: text, JSON, and YAML
- Option to only process files tracked by Git
- Customizable file extension filtering (optional)
- Customizable masking patterns
- AI-friendly output format, ideal for feeding project context to language models
## Use Cases
### Feeding Project Context to Language Models (LLMs)
Code Context Compiler is designed to be AI-friendly, making it an excellent tool for preparing entire project contexts for language models. Some key benefits include:
1. **Comprehensive Context**: By compiling the entire project into a single file, you provide LLMs with a complete view of your codebase, enabling more accurate and context-aware responses.
2. **Structured Output**: The file-wise separation in the output allows LLMs to understand the project structure and relationships between different files.
3. **Sensitive Information Protection**: With the built-in masking feature, you can safely share your project context with AI models without exposing sensitive data.
4. **Customizable Content**: Use the configuration options to include only the files and information relevant to your AI-related tasks.
5. **Multiple Output Formats**: Choose between text, JSON, or YAML output to best suit your LLM integration needs.
By using Code Context Compiler to prepare your project data, you can enhance the effectiveness of AI-powered code analysis, documentation generation, code review assistance, and other AI-driven development tools.
## LLM-Friendly Output
Code Context Compiler generates output that is specifically designed to be easily understood by Large Language Models (LLMs). Each compiled output includes:
1. A comprehensive prompt at the beginning of the file, explaining:
- The structure of the document
- How to interpret file markers
- The presence and meaning of masked sensitive information
- Guidelines for analyzing the code
2. Clear file demarcation using "File: " prefixes before each file's content.
3. Consistent formatting across all files in the project.
This structure allows LLMs to easily parse and understand the entire project context, making it ideal for tasks such as:
- Code analysis and review
- Documentation generation
- Answering questions about the project structure and functionality
- Identifying patterns and potential improvements across the codebase
When using the JSON or YAML output formats, the LLM prompt is included as a separate field, making it even easier for automated systems to leverage this information.
## Installation
## Using Pip
You can install Code Context Compiler directly from PyPI:
```
pip install code_context_compiler
```
## Usage
After installation, you can use the tool directly from the command line:
```
code_context_compiler [OPTIONS] PROJECT_PATH OUTPUT_FILE
```
...
## Using Git clone
To install Code Context Compiler, you need Python 3.8 or later and Poetry. Follow these steps:
1. Clone the repository:
```
git clone https://github.com/yourusername/code_context_compiler.git
cd code_context_compiler
```
2. Install dependencies using Poetry:
```
poetry install
```
## Usage
To use Code Context Compiler, run the following command:
```
poetry run code_context_compiler [OPTIONS] PROJECT_PATH OUTPUT_FILE
```
Arguments:
- `PROJECT_PATH`: Path to the project to scan
- `OUTPUT_FILE`: Path to the output file
Options:
- `--config-file PATH`: Path to the configuration file
- `--output-format [text|json|yaml]`: Output format (default: text)
- `--help`: Show this message and exit
Example:
```
poetry run code_context_compiler /path/to/your/project /path/to/output/file.txt --config-file config.yaml --output-format json
```
## Configuration
You can customize the behavior of Code Context Compiler by creating a YAML configuration file. Here's an example configuration:
```yaml
ignore_patterns:
- "*.log"
- "*.tmp"
- "poetry.lock"
- "package-lock.json"
file_extensions:
- ".py"
- ".js"
- ".java"
mask_patterns:
- 'password\s*=\s*["\'].*?["\']'
- 'api[_-]?key\s*=\s*["\'].*?["\']'
use_git: true
```
- `ignore_patterns`: List of file patterns to ignore. By default, common lock files (like poetry.lock, package-lock.json, yarn.lock, etc.) are ignored.
- `file_extensions`: List of file extensions to process (if empty, all files are processed)
- `mask_patterns`: List of regex patterns to mask sensitive information
- `use_git`: Boolean to only process Git-tracked files
Note: If `file_extensions` is not specified or is an empty list, the tool will process all file types.
The tool comes with sensible defaults, including ignoring common lock files. You can override or extend these defaults in your configuration file.
## Development
To set up the development environment:
1. Ensure you have Python 3.8+ and Poetry installed.
2. Clone the repository and navigate to the project directory.
3. Install dependencies:
```
poetry install
```
4. Run tests:
```
poetry run pytest
```
5. Run tests with coverage:
```
poetry run pytest --cov=code_context_compiler
```
## Contributing
Contributions to Code Context Compiler are welcome! Please feel free to submit a Pull Request.
1. Fork the repository
2. Create your feature branch (`git checkout -b feature/AmazingFeature`)
3. Commit your changes (`git commit -m 'Add some AmazingFeature'`)
4. Push to the branch (`git push origin feature/AmazingFeature`)
5. Open a Pull Request
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
Raw data
{
"_id": null,
"home_page": "https://github.com/chetanchoudhary/code_context_compiler",
"name": "code_context_compiler",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8",
"maintainer_email": null,
"keywords": "cli, code, compiler, context, llm, ai",
"author": "Chetan Choudhary",
"author_email": "chetan.choudhary.office@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/ee/72/88806f8edbabc17adba62ef1228889e3757fe1dc60a0bff04b746c8904e2/code_context_compiler-0.1.4.tar.gz",
"platform": null,
"description": "# Code Context Compiler\n\nCode Context Compiler is a powerful CLI tool that scans a code project and creates a single file containing the entire project code with file-wise separation. The tool also masks sensitive information in the code to enhance security and provides various customization options.\n\n## Features\n\n- Scans an entire project directory\n- Processes all file types by default\n- Compiles all code files into a single output file\n- Masks sensitive information such as passwords, API keys, and tokens\n- Respects `.gitignore` patterns and custom ignore patterns\n- Supports custom configuration via YAML files\n- Asynchronous file processing for improved performance\n- Progress indicator for large projects\n- Multiple output formats: text, JSON, and YAML\n- Option to only process files tracked by Git\n- Customizable file extension filtering (optional)\n- Customizable masking patterns\n- AI-friendly output format, ideal for feeding project context to language models\n\n## Use Cases\n\n### Feeding Project Context to Language Models (LLMs)\n\nCode Context Compiler is designed to be AI-friendly, making it an excellent tool for preparing entire project contexts for language models. Some key benefits include:\n\n1. **Comprehensive Context**: By compiling the entire project into a single file, you provide LLMs with a complete view of your codebase, enabling more accurate and context-aware responses.\n\n2. **Structured Output**: The file-wise separation in the output allows LLMs to understand the project structure and relationships between different files.\n\n3. **Sensitive Information Protection**: With the built-in masking feature, you can safely share your project context with AI models without exposing sensitive data.\n\n4. **Customizable Content**: Use the configuration options to include only the files and information relevant to your AI-related tasks.\n\n5. **Multiple Output Formats**: Choose between text, JSON, or YAML output to best suit your LLM integration needs.\n\nBy using Code Context Compiler to prepare your project data, you can enhance the effectiveness of AI-powered code analysis, documentation generation, code review assistance, and other AI-driven development tools.\n\n## LLM-Friendly Output\n\nCode Context Compiler generates output that is specifically designed to be easily understood by Large Language Models (LLMs). Each compiled output includes:\n\n1. A comprehensive prompt at the beginning of the file, explaining:\n - The structure of the document\n - How to interpret file markers\n - The presence and meaning of masked sensitive information\n - Guidelines for analyzing the code\n\n2. Clear file demarcation using \"File: \" prefixes before each file's content.\n\n3. Consistent formatting across all files in the project.\n\nThis structure allows LLMs to easily parse and understand the entire project context, making it ideal for tasks such as:\n\n- Code analysis and review\n- Documentation generation\n- Answering questions about the project structure and functionality\n- Identifying patterns and potential improvements across the codebase\n\nWhen using the JSON or YAML output formats, the LLM prompt is included as a separate field, making it even easier for automated systems to leverage this information.\n\n\n\n## Installation\n\n## Using Pip\n\nYou can install Code Context Compiler directly from PyPI:\n\n```\npip install code_context_compiler\n```\n\n## Usage\n\nAfter installation, you can use the tool directly from the command line:\n\n```\ncode_context_compiler [OPTIONS] PROJECT_PATH OUTPUT_FILE\n```\n\n...\n\n## Using Git clone\n\nTo install Code Context Compiler, you need Python 3.8 or later and Poetry. Follow these steps:\n\n1. Clone the repository:\n ```\n git clone https://github.com/yourusername/code_context_compiler.git\n cd code_context_compiler\n ```\n\n2. Install dependencies using Poetry:\n ```\n poetry install\n ```\n\n## Usage\n\nTo use Code Context Compiler, run the following command:\n\n```\npoetry run code_context_compiler [OPTIONS] PROJECT_PATH OUTPUT_FILE\n```\n\nArguments:\n- `PROJECT_PATH`: Path to the project to scan\n- `OUTPUT_FILE`: Path to the output file\n\nOptions:\n- `--config-file PATH`: Path to the configuration file\n- `--output-format [text|json|yaml]`: Output format (default: text)\n- `--help`: Show this message and exit\n\nExample:\n```\npoetry run code_context_compiler /path/to/your/project /path/to/output/file.txt --config-file config.yaml --output-format json\n```\n\n## Configuration\n\nYou can customize the behavior of Code Context Compiler by creating a YAML configuration file. Here's an example configuration:\n\n```yaml\nignore_patterns:\n - \"*.log\"\n - \"*.tmp\"\n - \"poetry.lock\"\n - \"package-lock.json\"\nfile_extensions:\n - \".py\"\n - \".js\"\n - \".java\"\nmask_patterns:\n - 'password\\s*=\\s*[\"\\'].*?[\"\\']'\n - 'api[_-]?key\\s*=\\s*[\"\\'].*?[\"\\']'\nuse_git: true\n```\n\n- `ignore_patterns`: List of file patterns to ignore. By default, common lock files (like poetry.lock, package-lock.json, yarn.lock, etc.) are ignored.\n- `file_extensions`: List of file extensions to process (if empty, all files are processed)\n- `mask_patterns`: List of regex patterns to mask sensitive information\n- `use_git`: Boolean to only process Git-tracked files\n\nNote: If `file_extensions` is not specified or is an empty list, the tool will process all file types.\n\nThe tool comes with sensible defaults, including ignoring common lock files. You can override or extend these defaults in your configuration file.\n\n## Development\n\nTo set up the development environment:\n\n1. Ensure you have Python 3.8+ and Poetry installed.\n2. Clone the repository and navigate to the project directory.\n3. Install dependencies:\n ```\n poetry install\n ```\n4. Run tests:\n ```\n poetry run pytest\n ```\n5. Run tests with coverage:\n ```\n poetry run pytest --cov=code_context_compiler\n ```\n\n## Contributing\n\nContributions to Code Context Compiler are welcome! Please feel free to submit a Pull Request.\n\n1. Fork the repository\n2. Create your feature branch (`git checkout -b feature/AmazingFeature`)\n3. Commit your changes (`git commit -m 'Add some AmazingFeature'`)\n4. Push to the branch (`git push origin feature/AmazingFeature`)\n5. Open a Pull Request\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.",
"bugtrack_url": null,
"license": "MIT",
"summary": "A CLI tool to scan a code project and create a single file with the entire project code, ideal for feeding context to LLMs.",
"version": "0.1.4",
"project_urls": {
"Homepage": "https://github.com/chetanchoudhary/code_context_compiler",
"Repository": "https://github.com/chetanchoudhary/code_context_compiler"
},
"split_keywords": [
"cli",
" code",
" compiler",
" context",
" llm",
" ai"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "4c514a0b12ae37b30572acc2191bdb7a723054b644622041c957048c73f86e18",
"md5": "69becd65574770773204c19d7c3cc153",
"sha256": "95112cb1b7efbd89356557c5302ca552eef7512c48ae202f8ccfef497a977ec5"
},
"downloads": -1,
"filename": "code_context_compiler-0.1.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "69becd65574770773204c19d7c3cc153",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8",
"size": 10963,
"upload_time": "2024-07-25T11:18:19",
"upload_time_iso_8601": "2024-07-25T11:18:19.002381Z",
"url": "https://files.pythonhosted.org/packages/4c/51/4a0b12ae37b30572acc2191bdb7a723054b644622041c957048c73f86e18/code_context_compiler-0.1.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ee7288806f8edbabc17adba62ef1228889e3757fe1dc60a0bff04b746c8904e2",
"md5": "4a5b6ae63377d4f2ece38d2788ba9a5b",
"sha256": "9c0b02b2478ca5452209f402c7ae5378fd033ccf5b26ad9aebde113e7c21dc40"
},
"downloads": -1,
"filename": "code_context_compiler-0.1.4.tar.gz",
"has_sig": false,
"md5_digest": "4a5b6ae63377d4f2ece38d2788ba9a5b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8",
"size": 11018,
"upload_time": "2024-07-25T11:18:20",
"upload_time_iso_8601": "2024-07-25T11:18:20.838091Z",
"url": "https://files.pythonhosted.org/packages/ee/72/88806f8edbabc17adba62ef1228889e3757fe1dc60a0bff04b746c8904e2/code_context_compiler-0.1.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-25 11:18:20",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "chetanchoudhary",
"github_project": "code_context_compiler",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "code_context_compiler"
}
JSON
