| Name | ctxctx JSON |
| Version |
0.3.10
JSON |
| download |
| home_page | None |
| Summary | Intelligently select, format, and present relevant project files and directory structure as context for LLMs. |
| upload_time | 2025-09-06 15:23:30 |
| maintainer | None |
| docs_url | None |
| author | Toomes Gud |
| requires_python | >=3.10 |
| license | MIT |
| keywords |
llm
context
cli
code
directory
files
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# LLM Context Builder: Focused Project Context for Large Language Models
A powerful Python package designed to intelligently select, format, and present relevant project files and directory structure as context for Large Language Models (LLMs). Avoid token limits, reduce noise, and get more accurate, actionable responses from your AI assistant.
---
### 🌟 Why Use This?
Working with LLMs for code-related tasks is incredible, but they often struggle with:
1. **Token Limits:** Sending an entire codebase is impossible and wasteful.
2. **Information Overload:** Even if possible, too much irrelevant code confuses the model.
3. **Lack of Structure:** Raw file dumps lack the directory context a human developer would have.
**LLM Context Builder** solves these problems by allowing you to:
* **Precisely select** the files, folders, or specific line ranges you want to include.
* **Automatically ignore** irrelevant files (like `node_modules` or build artifacts) using powerful ignore rules (including `.gitignore`).
* **Provide a clear project overview** with an automatically generated directory tree.
* **Format output** in easily parsable Markdown or JSON.
This results in **more focused, relevant, and accurate responses** from your LLM, helping you code faster and more effectively.
---
### 🚀 Getting Started
1. **Installation:**
Install `ctxctx` directly from PyPI:
```bash
pip install ctxctx
```
Alternatively, if cloning the repository for development:
```bash
git clone https://github.com/gkegke/ctxctx.git
cd llm-context-builder
# Install with poetry (recommended for development)
poetry install --with dev
```
2. **Basic Usage:**
Once installed, you can use the `ctxctx` command directly from your terminal in any project directory:
```bash
ctxctx
```
This will generate `prompt_input_files.md` and `prompt_input_files.json` containing only the directory tree of your project, up to a default depth of 3.
---
### 📖 Table of Contents
- [LLM Context Builder: Focused Project Context for Large Language Models](#llm-context-builder-focused-project-context-for-large-language-models)
- [🌟 Why Use This?](#-why-use-this)
- [🚀 Getting Started](#-getting-started)
- [📖 Table of Contents](#-table-of-contents)
- [✨ Key Features \& Usage Examples](#-key-features--usage-examples)
- [1. Basic Usage: Include Directory Tree](#1-basic-usage-include-directory-tree)
- [2. Including Specific Files \& Folders](#2-including-specific-files--folders)
- [3. Ignoring Files \& Folders](#3-ignoring-files--folders)
- [4. Force Including Files \& Folders (Override Ignores)](#4-force-including-files--folders-override-ignores)
- [5. Targeting Specific Line Ranges](#5-targeting-specific-line-ranges)
- [6. Using Glob Patterns for Flexible Selection](#6-using-glob-patterns-for-flexible-selection)
- [7. Passing Arguments from a File](#7-passing-arguments-from-a-file)
- [8. Pre-defined Context Profiles](#8-pre-defined-context-profiles)
- [9. Output Formats (Markdown \& JSON)](#9-output-formats-markdown--json)
- [10. Dry Run Mode](#10-dry-run-mode)
- [11. Discovering Files with --list-files](#11-discovering-files-with---list-files)
- [⚙️ Configuration](#️-configuration)
- [🪵 Logging Configuration](#-logging-configuration)
- [Console Output](#console-output)
- [File-based Logging](#file-based-logging)
- [Benefits for CI/CD](#benefits-for-cicd)
- [🤝 Contributing](#-contributing)
- [📄 License](#-license)
---
### ✨ Key Features & Usage Examples
The tool outputs its results into `prompt_input_files.md` (Markdown) and `prompt_input_files.json` (JSON) by default, based on the `OUTPUT_FORMATS` configuration.
#### 1. Basic Usage: Include Directory Tree
The simplest way to get context is just to include your project's directory structure. This gives the LLM a high-level overview of your project's layout, which is often very helpful.
```bash
ctxctx
```
This will generate `prompt_input_files.md` and `prompt_input_files.json` containing only the directory tree of your project, up to a default depth of 3.
#### 2. Including Specific Files & Folders
The most common use case is to provide the content of a few specific files or all files within a specific folder.
* **Include a single file:**
```bash
ctxctx src/main.py
```
* **Include multiple files:**
```bash
ctxctx src/utils.js README.md
```
* **Include all files within a folder (recursively, up to `SEARCH_MAX_DEPTH`):**
```bash
ctxctx config/
```
* **Combine files and folders:**
```bash
ctxctx tests/backend/ src/data_models.py
```
#### 3. Ignoring Files & Folders
Crucial for large projects! The tool uses a robust ignore system to ensure you don't send irrelevant or sensitive files to the LLM.
* **Automatic Gitignore:** By default, the script respects your project's `.gitignore` file.
* **Built-in Ignores:** Common build artifacts, temporary files, and environment directories (`node_modules`, `__pycache__`, `.venv`, `.git`, `.DS_Store`, etc.) are ignored automatically.
* **Additional Ignore Files:** The script also looks for other common ignore files like `.dockerignore`, `.npmignore`, and `.eslintignore` defined in `ADDITIONAL_IGNORE_FILENAMES`.
* **ctxctx-Specific Ignores:** The tool automatically ignores its own configuration and output files (like `.ctxctx.yaml`, the `.ctxctx_cache/` directory, `prompt_input_files.md`, and `prompt_input_files.json`) by default. These are embedded directly in the tool's core configuration's `EXPLICIT_IGNORE_NAMES`. You can always add more explicit ignore patterns to your `.ctxctx.yaml` file if needed.
#### 4. Force Including Files & Folders (Override Ignores)
Sometimes, you want to include a file or folder that is normally ignored by `ctxctx`'s default rules, `.gitignore`, or your custom ignore rules in `.ctxctx.yaml`. The "force include" feature allows you to explicitly override these ignore rules for specific paths.
* **Syntax:** Prefix the file or folder path (or glob pattern) with `force:`.
* Example: `ctxctx 'force:path/to/file.js'`
* **How it works:** When `ctxctx` encounters a query starting with `force:`, it marks that path as "force included". During processing, if a file matches *any* ignore rule, `ctxctx` first checks if it's explicitly force-included. If it is, the file will be included in the context, regardless of other ignore patterns.
* **Important Nuance for Simple Filenames:**
When using `force:` followed by a *simple filename* (i.e., no directory separators like `/` or `\\`, and no glob wildcards like `*` or `?`), `ctxctx` will **only look for that file directly in the project's root directory. If you want to force-include a simple filename that resides in a subdirectory, you must specify its full path or a glob pattern that includes its path (e.g., `force:src/config.py`).** This prevents unintended inclusion of identically named files deep within subdirectories, which is particularly useful for project-level files like `LICENSE` or `.gitignore`.
* **Example: `force:.gitignore`**
If you have `/project/.gitignore` and `/project/frontend/.gitignore`, `ctxctx 'force:.gitignore'` will **only** include `/project/.gitignore`. If you wanted the one in `frontend/`, you would need to specify its path: `ctxctx 'force:frontend/.gitignore'`.
* **Example: `force:README.md`**
If you have `README.md` at the root and `docs/README.md`, `ctxctx 'force:README.md'` will **only** include the root `README.md`.
* This specific root-only behavior **does not apply** if your `force:` query includes directory separators (e.g., `force:src/config.py`) or glob patterns (e.g., `force:*.log`). In those cases, the search remains recursive up to `SEARCH_MAX_DEPTH`, and force-include simply overrides ignore rules wherever the pattern matches.
* **Examples:**
* **Force include a specific log file:**
```bash
ctxctx 'force:debug.log'
```
(Even if `*.log` is in your `.gitignore` or `debug.log` is in your `.ctxctx.yaml` `EXPLICIT_IGNORE_NAMES`, it will be included. If `debug.log` only exists in a subdirectory, this command will *not* find it, due to the nuance described above.)
* **Force include a file inside an ignored directory:**
```bash
ctxctx 'force:node_modules/my_custom_module/index.js'
```
(Normally `node_modules` is ignored, but this specific file will be included. This is a path-specific query, so it searches deeply.)
* **Force include all build artifacts (using a glob):**
```bash
ctxctx 'force:build/**/*.js'
```
(If your `build` directory is typically ignored, this will include all JavaScript files within it. This is a glob query, so it searches deeply.)
* **Combine force-include with line ranges:**
```bash
ctxctx 'force:temp/sensitive_data.py:10,20'
```
(Includes only lines 10-20 from `sensitive_data.py`, even if `temp/` is ignored. This is a path-specific query, so it searches deeply.)
This feature provides granular control, ensuring that critical files are always part of your LLM context, even if they would otherwise be filtered out.
#### 5. Targeting Specific Line Ranges
For very precise context, you can include one or more specific ranges of lines from a file. This is perfect for focusing on a few key sections of code for debugging or refactoring. The output will clearly mark the included line ranges and indicate where content has been omitted.
* **Syntax:** `filepath:start1,end1:start2,end2...` (lines are 1-indexed and inclusive).
* **Example (Single Range):**
```bash
ctxctx 'src/api/user_routes.py:100,150'
```
This will include lines 100 through 150 from `src/api/user_routes.py`.
* **Example (Multiple Ranges):**
```bash
ctxctx 'src/data_processor.py:20,45:200,215'
```
This will include lines 20-45 and 200-215 from the same file, with a comment indicating the omitted lines in between.
#### 6. Using Glob Patterns for Flexible Selection
Glob patterns provide a powerful way to select multiple files based on wildcards.
* **Syntax:** Standard Unix-style glob patterns (e.g., `*.py`, `src/**/*.js`). Remember to quote patterns to prevent shell expansion.
* **Example:**
* **All Python files:**
```bash
ctxctx '*.py'
```
* **All JavaScript or TypeScript files within `src/` and its subdirectories:**
```bash
ctxctx 'src/**/*.{js,ts}' # (Note: Shell might expand {js,ts}, quote carefully or run in a compatible shell)
# Safer alternative for cross-platform (multiple arguments):
ctxctx 'src/**/*.js' 'src/**/*.ts'
```
* **All Markdown files in the root or `docs/` folder:**
```bash
ctxctx '*.md' 'docs/*.md'
```
#### 7. Passing Arguments from a File
For very long or complex `ctxctx` commands, or for commands you use frequently, you can store your queries and flags in a text file and pass that file to `ctxctx`. This helps keep your terminal commands clean and makes them easily repeatable.
* **Syntax:** `ctxctx @filename`
* **How it works:** `ctxctx` will read each line from the specified file as if it were a separate command-line argument. Lines starting with `#` are treated as comments and ignored.
* **Example `my_queries.txt`:**
```
# This is a comment, it will be ignored
src/main.py
tests/unit/test_config.py:10,25:50,60
'*.md'
docs/api/
--profile backend_dev
```
* **Usage:**
```bash
ctxctx @my_queries.txt
```
This command would be equivalent to running:
```bash
ctxctx src/main.py 'tests/unit/test_config.py:10,25:50,60' '*.md' docs/api/ --profile backend_dev
```
#### 8. Pre-defined Context Profiles
For common tasks, you can define **profiles** within your main `.ctxctx.yaml` file to create reusable context definitions. Profiles now use a powerful funnel-based system:
1. **`include`**: A list of glob patterns that defines the initial set of files.
2. **`queries`**: (Optional) Ad-hoc queries (like specific files, line ranges, or `force:` includes) are added to the set.
3. **`exclude`**: (Optional) A list of glob patterns that removes files from the final set.
This allows you to build broad contexts with `include` and then precisely refine them with `exclude` and `queries`.
1. **Define Profiles in your `.ctxctx.yaml` file** in your project's root directory:
```yaml
# .ctxctx.yaml
ROOT: .
OUTPUT_FILE_BASE_NAME: prompt_input_files
OUTPUT_FORMATS:
- md
- json
TREE_MAX_DEPTH: 3
# ... other global settings ...
profiles: # Profiles are now a top-level key in .ctxctx.yaml
backend_api:
description: "Core backend API files, excluding tests and configs."
include:
- 'src/server/**/*.py' # All python files under src/server
exclude:
- 'src/server/tests/**' # Exclude the test subdirectory
- 'src/server/config.py' # Exclude the config file
queries:
- 'requirements.txt' # Explicitly add the requirements file
frontend_component:
description: "Focus on a specific component, its styles, and tests."
include:
- 'src/components/UserProfile/**' # All files for the component
exclude:
- 'src/components/UserProfile/**/*.snap' # Exclude snapshots
# Override a global config setting just for this profile
tree_max_depth: 4
refactor_task:
description: "A specific refactoring task with precise line numbers."
queries:
# Use 'queries' when you only need a few specific files/ranges
- 'src/data/processor.py:10,45:100,120'
- 'src/utils/helpers.py:1,30'
- 'tests/test_processor.py'
```
2. **Use a profile:**
```bash
ctxctx --profile backend_api
ctxctx --profile refactor_task
```
You can also combine profiles with additional ad-hoc queries from the command line:
```bash
ctxctx --profile frontend_component 'public/index.html'
```
#### 9. Output Formats (Markdown & JSON)
The tool generates two output files by default (`prompt_input_files.md` and `prompt_input_files.json`) to give you flexibility depending on what your LLM prefers or how you want to review the context.
* **Markdown (`.md`):** Human-readable, includes directory tree, and uses Markdown code blocks with syntax highlighting hints for file contents. Great for reviewing the context yourself before sending it, or for models that prefer structured text.
* **JSON (`.json`):** Machine-readable structured data. Contains the directory tree as a string and an array of file objects, each with path, content, and any line/function details. Ideal for programmatic use or models that perform better with structured JSON input.
You can configure which formats are generated in your `.ctxctx.yaml` file (or via a profile).
#### 10. Dry Run Mode
Test your queries and configurations without writing any files. The full output will be printed directly to your console.
```bash
ctxctx --dry-run 'src/config.py' '*.md'
```
#### 11. Discovering Files with --list-files
When you need to select from a large number of files, the `--list-files` command simplifies the process. It prints a clean, sorted list of all files that `ctxctx` can see after applying all ignore rules (from `.gitignore`, `.ctxctx.yaml` `EXPLICIT_IGNORE_NAMES`, etc.).
Its primary use is to generate an **argument file** that you can edit and pass back to `ctxctx`.
* **How it works:**
1. Generate a list of all potential files. The command directs logs to `stderr`, so you can safely redirect the output.
```bash
ctxctx --list-files > my_context.txt
```
2. Open `my_context.txt` in your editor. Comment out (`#`) or delete the lines for files you wish to exclude.
3. Feed the curated list back into `ctxctx` using the `@` prefix.
```bash
ctxctx @my_context.txt
```
* **Combining with Profiles:** You can also use it with profiles to see exactly what files a profile includes, providing a great starting point for a more specific context.
```bash
ctxctx --profile backend_api --list-files > backend_files.txt
# Now edit backend_files.txt and run:
ctxctx @backend_files.txt
```
---
### ⚙️ Configuration
The tool's behavior can be customized by creating a `.ctxctx.yaml` file in your project's root directory. Any values defined in this file will override the tool's defaults. Additionally, profiles can override these global settings.
Key configurable options include:
* `ROOT`: The base directory for your project (defaults to `.` - current directory).
* `OUTPUT_FILE_BASE_NAME`: Base name for output files (e.g., `prompt_input_files`).
* `OUTPUT_FORMATS`: List of desired output formats (`markdown`, `json`).
* `TREE_MAX_DEPTH`: Maximum recursion depth for the directory tree view.
* `TREE_EXCLUDE_EMPTY_DIRS`: If `true`, empty directories (after applying ignore rules) will not be included in the tree.
* `SEARCH_MAX_DEPTH`: Maximum recursion depth for file content search.
* `MAX_MATCHES_PER_QUERY`: Max number of files a single query can return before an error is raised (prevents accidental large inclusions).
* `EXPLICIT_IGNORE_NAMES`: A set of exact file/folder names or relative paths to always ignore. **This now includes the tool's internal output and cache files by default.** You can add your own custom ignore names here.
* `SUBSTRING_IGNORE_PATTERNS`: A list of substrings that, if found anywhere in a file's relative path, will cause it to be ignored.
* `ADDITIONAL_IGNORE_FILENAMES`: List of other ignore files (e.g., `.dockerignore`) to load in addition to `.gitignore`.
* `DEFAULT_CONFIG_FILENAME`: The name of the main configuration file (defaults to `.ctxctx.yaml`).
* `USE_GITIGNORE`: Boolean to enable/disable `.gitignore` integration.
* `GITIGNORE_PATH`: Relative path to your main `.gitignore` file.
* `USE_CACHE`: Boolean to enable/disable caching of the project's file list.
---
### 🪵 Logging Configuration
`ctxctx` provides flexible logging options to help you debug issues, monitor execution, and capture detailed output, especially useful in automated environments like CI/CD.
#### Console Output
By default, `ctxctx` logs informational messages to your console (`stdout`).
* **Enable Debug Mode:** Use the `--debug` flag to increase the verbosity of console output, showing detailed debugging information.
```bash
ctxctx --debug src/main.py
```
#### File-based Logging
For persistent logs or detailed analysis, you can direct all logging output to a file.
* **Log to a File:** Use the `--log-file <path>` argument to write all logs (including DEBUG level) to the specified file. This is highly recommended for CI/CD pipelines or when running `ctxctx` in non-interactive scripts, as it ensures all details are captured without relying on console output.
```bash
# Log all output to ctxctx.log at DEBUG level
ctxctx src/cli.py --log-file ctxctx.log
# Combine with other arguments
ctxctx 'src/**/*.py' --profile backend_dev --log-file debug_output.txt
```
#### Benefits for CI/CD
The `--log-file` argument is invaluable for Continuous Integration/Continuous Deployment (CI/CD) pipelines:
* **Persistent Records:** Capture full execution logs for every build, even if the pipeline fails, allowing for post-mortem analysis.
* **Detailed Debugging:** Provide engineers with comprehensive information for troubleshooting build issues or unexpected `ctxctx` behavior within automated workflows.
* **Clean Console:** Avoids flooding the CI/CD console output with verbose details, keeping the primary build logs focused.
* **Auditing:** Maintain an auditable trail of what context was generated for specific code changes.
---
### 🤝 Contributing
Contributions are welcome! If you have ideas for new features, improvements, or bug fixes, please open an issue or submit a pull request.
Areas for future improvement include:
* **Git Integration:** Automatically include files based on Git status (e.g., staged, modified).
* **Code-aware Extraction:** Use AST (Abstract Syntax Tree) parsing to extract specific functions, classes, or methods from code files.
* **Advanced Ignore Logic:** More robust `.gitignore` parsing, including support for negation patterns (`!`).
* **Interactive Mode:** A CLI mode for interactively selecting files and folders to include in the context.
Performance improvements being considered IF real world usage calls for it (increases codebase complexity):
* **Content Change Detection for Cache:** Currently, the cache only tracks file *list* changes based on config/ignore file mtimes. Adding content hashing could make cache invalidation more precise.
* **Parallel File Content Reading**
### 📄 License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
Raw data
{
"_id": null,
"home_page": null,
"name": "ctxctx",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "llm, context, cli, code, directory, files",
"author": "Toomes Gud",
"author_email": "toomesgud@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/cb/20/9ce5427cb9b8864346aeb3ee9434c33ac2e14d1a1f4ce5201f815976da02/ctxctx-0.3.10.tar.gz",
"platform": null,
"description": "# LLM Context Builder: Focused Project Context for Large Language Models\n\n\n\n\nA powerful Python package designed to intelligently select, format, and present relevant project files and directory structure as context for Large Language Models (LLMs). Avoid token limits, reduce noise, and get more accurate, actionable responses from your AI assistant.\n\n---\n\n### \ud83c\udf1f Why Use This?\n\nWorking with LLMs for code-related tasks is incredible, but they often struggle with:\n\n1. **Token Limits:** Sending an entire codebase is impossible and wasteful.\n2. **Information Overload:** Even if possible, too much irrelevant code confuses the model.\n3. **Lack of Structure:** Raw file dumps lack the directory context a human developer would have.\n\n**LLM Context Builder** solves these problems by allowing you to:\n* **Precisely select** the files, folders, or specific line ranges you want to include.\n* **Automatically ignore** irrelevant files (like `node_modules` or build artifacts) using powerful ignore rules (including `.gitignore`).\n* **Provide a clear project overview** with an automatically generated directory tree.\n* **Format output** in easily parsable Markdown or JSON.\n\nThis results in **more focused, relevant, and accurate responses** from your LLM, helping you code faster and more effectively.\n\n---\n\n### \ud83d\ude80 Getting Started\n\n1. **Installation:**\n Install `ctxctx` directly from PyPI:\n ```bash\n pip install ctxctx\n ```\n Alternatively, if cloning the repository for development:\n ```bash\n git clone https://github.com/gkegke/ctxctx.git\n cd llm-context-builder\n # Install with poetry (recommended for development)\n poetry install --with dev\n ```\n\n2. **Basic Usage:**\n Once installed, you can use the `ctxctx` command directly from your terminal in any project directory:\n ```bash\n ctxctx\n ```\n This will generate `prompt_input_files.md` and `prompt_input_files.json` containing only the directory tree of your project, up to a default depth of 3.\n\n---\n\n### \ud83d\udcd6 Table of Contents\n\n- [LLM Context Builder: Focused Project Context for Large Language Models](#llm-context-builder-focused-project-context-for-large-language-models)\n - [\ud83c\udf1f Why Use This?](#-why-use-this)\n - [\ud83d\ude80 Getting Started](#-getting-started)\n - [\ud83d\udcd6 Table of Contents](#-table-of-contents)\n - [\u2728 Key Features \\& Usage Examples](#-key-features--usage-examples)\n - [1. Basic Usage: Include Directory Tree](#1-basic-usage-include-directory-tree)\n - [2. Including Specific Files \\& Folders](#2-including-specific-files--folders)\n - [3. Ignoring Files \\& Folders](#3-ignoring-files--folders)\n - [4. Force Including Files \\& Folders (Override Ignores)](#4-force-including-files--folders-override-ignores)\n - [5. Targeting Specific Line Ranges](#5-targeting-specific-line-ranges)\n - [6. Using Glob Patterns for Flexible Selection](#6-using-glob-patterns-for-flexible-selection)\n - [7. Passing Arguments from a File](#7-passing-arguments-from-a-file)\n - [8. Pre-defined Context Profiles](#8-pre-defined-context-profiles)\n - [9. Output Formats (Markdown \\& JSON)](#9-output-formats-markdown--json)\n - [10. Dry Run Mode](#10-dry-run-mode)\n - [11. Discovering Files with --list-files](#11-discovering-files-with---list-files)\n - [\u2699\ufe0f Configuration](#\ufe0f-configuration)\n - [\ud83e\udeb5 Logging Configuration](#-logging-configuration)\n - [Console Output](#console-output)\n - [File-based Logging](#file-based-logging)\n - [Benefits for CI/CD](#benefits-for-cicd)\n - [\ud83e\udd1d Contributing](#-contributing)\n - [\ud83d\udcc4 License](#-license)\n\n---\n\n### \u2728 Key Features & Usage Examples\n\nThe tool outputs its results into `prompt_input_files.md` (Markdown) and `prompt_input_files.json` (JSON) by default, based on the `OUTPUT_FORMATS` configuration.\n\n#### 1. Basic Usage: Include Directory Tree\n\nThe simplest way to get context is just to include your project's directory structure. This gives the LLM a high-level overview of your project's layout, which is often very helpful.\n\n```bash\nctxctx\n```\nThis will generate `prompt_input_files.md` and `prompt_input_files.json` containing only the directory tree of your project, up to a default depth of 3.\n\n#### 2. Including Specific Files & Folders\n\nThe most common use case is to provide the content of a few specific files or all files within a specific folder.\n\n* **Include a single file:**\n ```bash\n ctxctx src/main.py\n ```\n* **Include multiple files:**\n ```bash\n ctxctx src/utils.js README.md\n ```\n* **Include all files within a folder (recursively, up to `SEARCH_MAX_DEPTH`):**\n ```bash\n ctxctx config/\n ```\n* **Combine files and folders:**\n ```bash\n ctxctx tests/backend/ src/data_models.py\n ```\n\n#### 3. Ignoring Files & Folders\n\nCrucial for large projects! The tool uses a robust ignore system to ensure you don't send irrelevant or sensitive files to the LLM.\n\n* **Automatic Gitignore:** By default, the script respects your project's `.gitignore` file.\n* **Built-in Ignores:** Common build artifacts, temporary files, and environment directories (`node_modules`, `__pycache__`, `.venv`, `.git`, `.DS_Store`, etc.) are ignored automatically.\n* **Additional Ignore Files:** The script also looks for other common ignore files like `.dockerignore`, `.npmignore`, and `.eslintignore` defined in `ADDITIONAL_IGNORE_FILENAMES`.\n* **ctxctx-Specific Ignores:** The tool automatically ignores its own configuration and output files (like `.ctxctx.yaml`, the `.ctxctx_cache/` directory, `prompt_input_files.md`, and `prompt_input_files.json`) by default. These are embedded directly in the tool's core configuration's `EXPLICIT_IGNORE_NAMES`. You can always add more explicit ignore patterns to your `.ctxctx.yaml` file if needed.\n\n#### 4. Force Including Files & Folders (Override Ignores)\n\nSometimes, you want to include a file or folder that is normally ignored by `ctxctx`'s default rules, `.gitignore`, or your custom ignore rules in `.ctxctx.yaml`. The \"force include\" feature allows you to explicitly override these ignore rules for specific paths.\n\n* **Syntax:** Prefix the file or folder path (or glob pattern) with `force:`.\n * Example: `ctxctx 'force:path/to/file.js'`\n* **How it works:** When `ctxctx` encounters a query starting with `force:`, it marks that path as \"force included\". During processing, if a file matches *any* ignore rule, `ctxctx` first checks if it's explicitly force-included. If it is, the file will be included in the context, regardless of other ignore patterns.\n\n* **Important Nuance for Simple Filenames:**\n When using `force:` followed by a *simple filename* (i.e., no directory separators like `/` or `\\\\`, and no glob wildcards like `*` or `?`), `ctxctx` will **only look for that file directly in the project's root directory. If you want to force-include a simple filename that resides in a subdirectory, you must specify its full path or a glob pattern that includes its path (e.g., `force:src/config.py`).** This prevents unintended inclusion of identically named files deep within subdirectories, which is particularly useful for project-level files like `LICENSE` or `.gitignore`.\n\n * **Example: `force:.gitignore`**\n If you have `/project/.gitignore` and `/project/frontend/.gitignore`, `ctxctx 'force:.gitignore'` will **only** include `/project/.gitignore`. If you wanted the one in `frontend/`, you would need to specify its path: `ctxctx 'force:frontend/.gitignore'`.\n\n * **Example: `force:README.md`**\n If you have `README.md` at the root and `docs/README.md`, `ctxctx 'force:README.md'` will **only** include the root `README.md`.\n\n * This specific root-only behavior **does not apply** if your `force:` query includes directory separators (e.g., `force:src/config.py`) or glob patterns (e.g., `force:*.log`). In those cases, the search remains recursive up to `SEARCH_MAX_DEPTH`, and force-include simply overrides ignore rules wherever the pattern matches.\n\n* **Examples:**\n * **Force include a specific log file:**\n ```bash\n ctxctx 'force:debug.log'\n ```\n (Even if `*.log` is in your `.gitignore` or `debug.log` is in your `.ctxctx.yaml` `EXPLICIT_IGNORE_NAMES`, it will be included. If `debug.log` only exists in a subdirectory, this command will *not* find it, due to the nuance described above.)\n\n * **Force include a file inside an ignored directory:**\n ```bash\n ctxctx 'force:node_modules/my_custom_module/index.js'\n ```\n (Normally `node_modules` is ignored, but this specific file will be included. This is a path-specific query, so it searches deeply.)\n\n * **Force include all build artifacts (using a glob):**\n ```bash\n ctxctx 'force:build/**/*.js'\n ```\n (If your `build` directory is typically ignored, this will include all JavaScript files within it. This is a glob query, so it searches deeply.)\n\n * **Combine force-include with line ranges:**\n ```bash\n ctxctx 'force:temp/sensitive_data.py:10,20'\n ```\n (Includes only lines 10-20 from `sensitive_data.py`, even if `temp/` is ignored. This is a path-specific query, so it searches deeply.)\n\nThis feature provides granular control, ensuring that critical files are always part of your LLM context, even if they would otherwise be filtered out.\n\n#### 5. Targeting Specific Line Ranges\n\nFor very precise context, you can include one or more specific ranges of lines from a file. This is perfect for focusing on a few key sections of code for debugging or refactoring. The output will clearly mark the included line ranges and indicate where content has been omitted.\n\n* **Syntax:** `filepath:start1,end1:start2,end2...` (lines are 1-indexed and inclusive).\n* **Example (Single Range):**\n ```bash\n ctxctx 'src/api/user_routes.py:100,150'\n ```\n This will include lines 100 through 150 from `src/api/user_routes.py`.\n\n* **Example (Multiple Ranges):**\n ```bash\n ctxctx 'src/data_processor.py:20,45:200,215'\n ```\n This will include lines 20-45 and 200-215 from the same file, with a comment indicating the omitted lines in between.\n\n#### 6. Using Glob Patterns for Flexible Selection\n\nGlob patterns provide a powerful way to select multiple files based on wildcards.\n\n* **Syntax:** Standard Unix-style glob patterns (e.g., `*.py`, `src/**/*.js`). Remember to quote patterns to prevent shell expansion.\n* **Example:**\n * **All Python files:**\n ```bash\n ctxctx '*.py'\n ```\n * **All JavaScript or TypeScript files within `src/` and its subdirectories:**\n ```bash\n ctxctx 'src/**/*.{js,ts}' # (Note: Shell might expand {js,ts}, quote carefully or run in a compatible shell)\n # Safer alternative for cross-platform (multiple arguments):\n ctxctx 'src/**/*.js' 'src/**/*.ts'\n ```\n * **All Markdown files in the root or `docs/` folder:**\n ```bash\n ctxctx '*.md' 'docs/*.md'\n ```\n\n#### 7. Passing Arguments from a File\n\nFor very long or complex `ctxctx` commands, or for commands you use frequently, you can store your queries and flags in a text file and pass that file to `ctxctx`. This helps keep your terminal commands clean and makes them easily repeatable.\n\n* **Syntax:** `ctxctx @filename`\n* **How it works:** `ctxctx` will read each line from the specified file as if it were a separate command-line argument. Lines starting with `#` are treated as comments and ignored.\n\n* **Example `my_queries.txt`:**\n ```\n # This is a comment, it will be ignored\n src/main.py\n tests/unit/test_config.py:10,25:50,60\n '*.md'\n docs/api/\n --profile backend_dev\n ```\n* **Usage:**\n ```bash\n ctxctx @my_queries.txt\n ```\n This command would be equivalent to running:\n ```bash\n ctxctx src/main.py 'tests/unit/test_config.py:10,25:50,60' '*.md' docs/api/ --profile backend_dev\n ```\n\n#### 8. Pre-defined Context Profiles\n\nFor common tasks, you can define **profiles** within your main `.ctxctx.yaml` file to create reusable context definitions. Profiles now use a powerful funnel-based system:\n\n1. **`include`**: A list of glob patterns that defines the initial set of files.\n2. **`queries`**: (Optional) Ad-hoc queries (like specific files, line ranges, or `force:` includes) are added to the set.\n3. **`exclude`**: (Optional) A list of glob patterns that removes files from the final set.\n\nThis allows you to build broad contexts with `include` and then precisely refine them with `exclude` and `queries`.\n\n1. **Define Profiles in your `.ctxctx.yaml` file** in your project's root directory:\n ```yaml\n # .ctxctx.yaml\n ROOT: .\n OUTPUT_FILE_BASE_NAME: prompt_input_files\n OUTPUT_FORMATS:\n - md\n - json\n TREE_MAX_DEPTH: 3\n # ... other global settings ...\n\n profiles: # Profiles are now a top-level key in .ctxctx.yaml\n backend_api:\n description: \"Core backend API files, excluding tests and configs.\"\n include:\n - 'src/server/**/*.py' # All python files under src/server\n exclude:\n - 'src/server/tests/**' # Exclude the test subdirectory\n - 'src/server/config.py' # Exclude the config file\n queries:\n - 'requirements.txt' # Explicitly add the requirements file\n\n frontend_component:\n description: \"Focus on a specific component, its styles, and tests.\"\n include:\n - 'src/components/UserProfile/**' # All files for the component\n exclude:\n - 'src/components/UserProfile/**/*.snap' # Exclude snapshots\n # Override a global config setting just for this profile\n tree_max_depth: 4\n\n refactor_task:\n description: \"A specific refactoring task with precise line numbers.\"\n queries:\n # Use 'queries' when you only need a few specific files/ranges\n - 'src/data/processor.py:10,45:100,120'\n - 'src/utils/helpers.py:1,30'\n - 'tests/test_processor.py'\n ```\n\n2. **Use a profile:**\n ```bash\n ctxctx --profile backend_api\n ctxctx --profile refactor_task\n ```\n You can also combine profiles with additional ad-hoc queries from the command line:\n ```bash\n ctxctx --profile frontend_component 'public/index.html'\n ```\n\n#### 9. Output Formats (Markdown & JSON)\n\nThe tool generates two output files by default (`prompt_input_files.md` and `prompt_input_files.json`) to give you flexibility depending on what your LLM prefers or how you want to review the context.\n\n* **Markdown (`.md`):** Human-readable, includes directory tree, and uses Markdown code blocks with syntax highlighting hints for file contents. Great for reviewing the context yourself before sending it, or for models that prefer structured text.\n* **JSON (`.json`):** Machine-readable structured data. Contains the directory tree as a string and an array of file objects, each with path, content, and any line/function details. Ideal for programmatic use or models that perform better with structured JSON input.\n\nYou can configure which formats are generated in your `.ctxctx.yaml` file (or via a profile).\n\n#### 10. Dry Run Mode\n\nTest your queries and configurations without writing any files. The full output will be printed directly to your console.\n\n```bash\nctxctx --dry-run 'src/config.py' '*.md'\n```\n\n#### 11. Discovering Files with --list-files\n\nWhen you need to select from a large number of files, the `--list-files` command simplifies the process. It prints a clean, sorted list of all files that `ctxctx` can see after applying all ignore rules (from `.gitignore`, `.ctxctx.yaml` `EXPLICIT_IGNORE_NAMES`, etc.).\n\nIts primary use is to generate an **argument file** that you can edit and pass back to `ctxctx`.\n\n* **How it works:**\n 1. Generate a list of all potential files. The command directs logs to `stderr`, so you can safely redirect the output.\n ```bash\n ctxctx --list-files > my_context.txt\n ```\n 2. Open `my_context.txt` in your editor. Comment out (`#`) or delete the lines for files you wish to exclude.\n 3. Feed the curated list back into `ctxctx` using the `@` prefix.\n ```bash\n ctxctx @my_context.txt\n ```\n\n* **Combining with Profiles:** You can also use it with profiles to see exactly what files a profile includes, providing a great starting point for a more specific context.\n ```bash\n ctxctx --profile backend_api --list-files > backend_files.txt\n # Now edit backend_files.txt and run:\n ctxctx @backend_files.txt\n ```\n\n---\n\n### \u2699\ufe0f Configuration\n\nThe tool's behavior can be customized by creating a `.ctxctx.yaml` file in your project's root directory. Any values defined in this file will override the tool's defaults. Additionally, profiles can override these global settings.\n\nKey configurable options include:\n\n* `ROOT`: The base directory for your project (defaults to `.` - current directory).\n* `OUTPUT_FILE_BASE_NAME`: Base name for output files (e.g., `prompt_input_files`).\n* `OUTPUT_FORMATS`: List of desired output formats (`markdown`, `json`).\n* `TREE_MAX_DEPTH`: Maximum recursion depth for the directory tree view.\n* `TREE_EXCLUDE_EMPTY_DIRS`: If `true`, empty directories (after applying ignore rules) will not be included in the tree.\n* `SEARCH_MAX_DEPTH`: Maximum recursion depth for file content search.\n* `MAX_MATCHES_PER_QUERY`: Max number of files a single query can return before an error is raised (prevents accidental large inclusions).\n* `EXPLICIT_IGNORE_NAMES`: A set of exact file/folder names or relative paths to always ignore. **This now includes the tool's internal output and cache files by default.** You can add your own custom ignore names here.\n* `SUBSTRING_IGNORE_PATTERNS`: A list of substrings that, if found anywhere in a file's relative path, will cause it to be ignored.\n* `ADDITIONAL_IGNORE_FILENAMES`: List of other ignore files (e.g., `.dockerignore`) to load in addition to `.gitignore`.\n* `DEFAULT_CONFIG_FILENAME`: The name of the main configuration file (defaults to `.ctxctx.yaml`).\n* `USE_GITIGNORE`: Boolean to enable/disable `.gitignore` integration.\n* `GITIGNORE_PATH`: Relative path to your main `.gitignore` file.\n* `USE_CACHE`: Boolean to enable/disable caching of the project's file list.\n\n---\n\n### \ud83e\udeb5 Logging Configuration\n\n`ctxctx` provides flexible logging options to help you debug issues, monitor execution, and capture detailed output, especially useful in automated environments like CI/CD.\n\n#### Console Output\n\nBy default, `ctxctx` logs informational messages to your console (`stdout`).\n\n* **Enable Debug Mode:** Use the `--debug` flag to increase the verbosity of console output, showing detailed debugging information.\n\n ```bash\n ctxctx --debug src/main.py\n ```\n\n#### File-based Logging\n\nFor persistent logs or detailed analysis, you can direct all logging output to a file.\n\n* **Log to a File:** Use the `--log-file <path>` argument to write all logs (including DEBUG level) to the specified file. This is highly recommended for CI/CD pipelines or when running `ctxctx` in non-interactive scripts, as it ensures all details are captured without relying on console output.\n\n ```bash\n # Log all output to ctxctx.log at DEBUG level\n ctxctx src/cli.py --log-file ctxctx.log\n\n # Combine with other arguments\n ctxctx 'src/**/*.py' --profile backend_dev --log-file debug_output.txt\n ```\n\n#### Benefits for CI/CD\n\nThe `--log-file` argument is invaluable for Continuous Integration/Continuous Deployment (CI/CD) pipelines:\n\n* **Persistent Records:** Capture full execution logs for every build, even if the pipeline fails, allowing for post-mortem analysis.\n* **Detailed Debugging:** Provide engineers with comprehensive information for troubleshooting build issues or unexpected `ctxctx` behavior within automated workflows.\n* **Clean Console:** Avoids flooding the CI/CD console output with verbose details, keeping the primary build logs focused.\n* **Auditing:** Maintain an auditable trail of what context was generated for specific code changes.\n\n---\n\n### \ud83e\udd1d Contributing\n\nContributions are welcome! If you have ideas for new features, improvements, or bug fixes, please open an issue or submit a pull request.\n\nAreas for future improvement include:\n* **Git Integration:** Automatically include files based on Git status (e.g., staged, modified).\n* **Code-aware Extraction:** Use AST (Abstract Syntax Tree) parsing to extract specific functions, classes, or methods from code files.\n* **Advanced Ignore Logic:** More robust `.gitignore` parsing, including support for negation patterns (`!`).\n* **Interactive Mode:** A CLI mode for interactively selecting files and folders to include in the context.\n\n\nPerformance improvements being considered IF real world usage calls for it (increases codebase complexity):\n* **Content Change Detection for Cache:** Currently, the cache only tracks file *list* changes based on config/ignore file mtimes. Adding content hashing could make cache invalidation more precise.\n* **Parallel File Content Reading**\n\n### \ud83d\udcc4 License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Intelligently select, format, and present relevant project files and directory structure as context for LLMs.",
"version": "0.3.10",
"project_urls": null,
"split_keywords": [
"llm",
" context",
" cli",
" code",
" directory",
" files"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2328405e7f78c9b8783a90bab85b8755e1b3df2223e582ea5b01be9f939aa72b",
"md5": "4be5d01750b951059e0d3f67e7ce8d62",
"sha256": "521c097ea210631aa7d8a736055208c27b8e0c653c228efe7d56555868f0a66b"
},
"downloads": -1,
"filename": "ctxctx-0.3.10-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4be5d01750b951059e0d3f67e7ce8d62",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 34332,
"upload_time": "2025-09-06T15:23:29",
"upload_time_iso_8601": "2025-09-06T15:23:29.521889Z",
"url": "https://files.pythonhosted.org/packages/23/28/405e7f78c9b8783a90bab85b8755e1b3df2223e582ea5b01be9f939aa72b/ctxctx-0.3.10-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "cb209ce5427cb9b8864346aeb3ee9434c33ac2e14d1a1f4ce5201f815976da02",
"md5": "dfe37f9d06eb86a042a7cea8561c9f21",
"sha256": "a385568c34825fb7c74038b31a481abba8fe3e0d0229bee4ba38c25949099670"
},
"downloads": -1,
"filename": "ctxctx-0.3.10.tar.gz",
"has_sig": false,
"md5_digest": "dfe37f9d06eb86a042a7cea8561c9f21",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 35574,
"upload_time": "2025-09-06T15:23:30",
"upload_time_iso_8601": "2025-09-06T15:23:30.940938Z",
"url": "https://files.pythonhosted.org/packages/cb/20/9ce5427cb9b8864346aeb3ee9434c33ac2e14d1a1f4ce5201f815976da02/ctxctx-0.3.10.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-09-06 15:23:30",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "ctxctx"
}
