| Name | git-crossref JSON |
| Version |
0.1.1
JSON |
| download |
| home_page | None |
| Summary | A Git plugin for cross-referencing and syncing specific files from multiple repositories. |
| upload_time | 2025-08-18 08:14:30 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.11 |
| license | None |
| keywords |
git
sync
files
repository
cli
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# git-crossref
A Git-based vendoring tool that lets you selectively sync specific files and directories from multiple remote repositories into your project. Think of it as a lightweight alternative to Git submodules, giving you fine-grained control over what gets included and when it gets updated.
## Why Use git-crossref?
Perfect for scenarios where you need to:
- Vendor specific files: pull only the files you need, not entire repositories
- Keep dependencies current: easily update to specific commits, tags, or branches
- Multiple sources: combine files from different repositories into your project
- Fast synchronization: hash-based change detection means only modified files are updated
- Team coordination: version-controlled configuration ensures everyone syncs the same files
- Safe updates: local change detection prevents accidental overwrites
- File transformations: apply text transformations during sync (e.g., import path updates)
## vs. Other Tools
| Tool | git-crossref | Git Submodules | npm/pip packages |
|------|-------------|----------------|------------------|
| **Granularity** | Individual files/dirs | Entire repositories | Entire packages |
| **Updates** | On-demand, per file | Manual submodule update | Version-based |
| **Transformations** | ✅ Built-in | ❌ Manual | ❌ Post-install scripts |
| **Multi-source** | ✅ Multiple remotes | ✅ Multiple submodules | ✅ Multiple registries |
| **Local changes** | ✅ Protected | ⚠️ Conflicts | ❌ Overwritten |
| **Performance** | ⚡ Hash-based | 🐌 Full checkout | ⚡ Registry-based |
Perfect for when you need more control than submodules but don't want full package overhead.
## Installation
You can install git-crossref using pip:
pip install git-crossref
Alternatively, clone the repository and install manually:
git clone https://github.com/yourusername/git-crossref.git
cd git-crossref
pip install .
## Quick Start
1. Initialize a configuration file:
```bash
git-crossref init
```
2. Edit `.gitcrossref` to configure your remotes and files
3. Sync files:
```bash
git-crossref sync
```
## Configuration
The tool uses a YAML configuration file located at `.gitcrossref` in your repository root. Here's an example:
```yaml
remotes:
upstream:
url: "https://github.com/example/source-repo.git"
base_path: "src/library"
version: "main" # All files from this remote default to 'main'
another-source:
url: "https://github.com/example/another-repo.git"
base_path: "scripts"
version: "v1.2.3" # All files from this remote default to 'v1.2.3'
files:
upstream:
# Single file sync
- source: "utils.py"
destination: "libs/utils.py"
# No hash provided, so it uses 'main' from upstream
# File with specific commit
- source: "config.yaml"
destination: "config/defaults.yaml"
hash: "abc123" # Overrides 'main' with a specific commit
# Directory tree sync
- source: "templates/"
destination: "project-templates/"
# Syncs entire templates directory
another-source:
# Script file
- source: "deploy.sh"
destination: "scripts/deploy.sh"
# No hash provided, so it uses 'v1.2.3' from another-source
```
### Configuration Options
**Remotes:**
- `url`: Git URL of the remote repository
- `base_path`: Optional base path within the repository
- `version`: Default branch/tag/commit for files from this remote
**Files:**
- `source`: Path to file or directory in the remote repository (relative to `base_path`)
- `destination`: Local path where content should be copied
- `hash`: Optional specific commit hash (overrides `version`)
- `ignore_changes`: If true, overwrites local files without checking for changes
**Git Object Types:**
- **Blob (file)**: `source: "utils.py"` - single file and/or glob patterns
- **Tree (directory)**: `source: "templates/"` - entire directory (note trailing slash)
## Commands
### Sync files
```bash
# Sync all configured files
git-crossref sync
# Sync only files from a specific remote
git-crossref sync --remote upstream
# Force sync (override local changes)
git-crossref sync --force
# Sync specific files by pattern
git-crossref sync libs/
```
### Check status
```bash
# Check status of all files
git-crossref status
# Check specific files
git-crossref check libs/utils.py
# Verbose output with hash information
git-crossref status --verbose
```
### Configuration management
```bash
# Initialize new configuration
git-crossref init
# Validate configuration
git-crossref validate
# Clean up cached repositories
git-crossref clean
```
## How it Works
git-crossref provides efficient, reliable vendoring through a Git-native approach:
1. **Smart caching**: bare clone remote repositories once and reuses them for all operations
2. **Precise targeting**: Extracts only the specific files/directories you need
3. **Hash-based detection**: Compares Git blob hashes to detect changes reliably
4. **Safe operations**: Checks for local modifications before overwriting files
This approach is both fast and reliable compared to timestamp-based synchronization tools.
## Use Cases
### Common Scenarios
**📁 Shared Configuration Files**
```yaml
# Sync common CI/build configs across projects
remotes:
shared-configs:
url: "https://github.com/company/shared-configs.git"
version: "main"
files:
shared-configs:
- source: ".github/workflows/"
destination: ".github/workflows/"
- source: "Makefile"
destination: "Makefile"
```
**🔧 Utility Libraries**
```yaml
# Cherry-pick utility functions from larger codebases
remotes:
utils:
url: "https://github.com/company/utils.git"
base_path: "src"
files:
utils:
- source: "database/helpers.py"
destination: "lib/db_helpers.py"
- source: "validation/*.py"
destination: "lib/validators/"
```
**📦 Dependency Vendoring**
```yaml
# Vendor specific versions of dependencies
remotes:
vendor:
url: "https://github.com/external/library.git"
version: "v2.1.0" # Pin to specific version
files:
vendor:
- source: "src/core/"
destination: "vendor/library/"
ignore_changes: true # Don't warn about local modifications
```
## Technical Details
### Implementation
For those interested in the technical implementation:
- **Bare repository caching**: Remote repositories are cloned to `.git/crossref-cache/` as bare repositories (Git objects only, no working directory)
- **Hash-based comparison**: Uses `git hash-object` to compare local and remote file blob hashes for reliable change detection
- **Git status integration**: Leverages `git status` to detect uncommitted local changes before sync operations
### Architecture
The tool is built around several key components:
- **Configuration layer**: YAML-based configuration with JSON schema validation
- **Git operations**: Efficient Git object manipulation using GitPython
- **Sync engines**: Specialized classes for different content types (files, directories, patterns)
- **Status tracking**: Rich status enumeration with success/failure categorization
- **Error handling**: Comprehensive exception system with actionable error messages
## Contributing
Contributions are welcome! Please see [README.developers.md](README.developers.md) for detailed development setup, code quality guidelines, and contribution workflow.
## License
This project is licensed under the MIT License - see the LICENSE file for details.
Raw data
{
"_id": null,
"home_page": null,
"name": "git-crossref",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.11",
"maintainer_email": "Albert Esteve <aesteve@redhat.com>",
"keywords": "git, sync, files, repository, cli",
"author": null,
"author_email": "Albert Esteve <aesteve@redhat.com>",
"download_url": "https://files.pythonhosted.org/packages/e0/1d/6eaf30919f16cb61cdf1c83cc723dd3071653eb67f058667f3746975006d/git_crossref-0.1.1.tar.gz",
"platform": null,
"description": "# git-crossref\n\nA Git-based vendoring tool that lets you selectively sync specific files and directories from multiple remote repositories into your project. Think of it as a lightweight alternative to Git submodules, giving you fine-grained control over what gets included and when it gets updated.\n\n## Why Use git-crossref?\n\nPerfect for scenarios where you need to:\n\n- Vendor specific files: pull only the files you need, not entire repositories\n- Keep dependencies current: easily update to specific commits, tags, or branches\n- Multiple sources: combine files from different repositories into your project\n- Fast synchronization: hash-based change detection means only modified files are updated\n- Team coordination: version-controlled configuration ensures everyone syncs the same files\n- Safe updates: local change detection prevents accidental overwrites\n- File transformations: apply text transformations during sync (e.g., import path updates)\n\n## vs. Other Tools\n\n| Tool | git-crossref | Git Submodules | npm/pip packages |\n|------|-------------|----------------|------------------|\n| **Granularity** | Individual files/dirs | Entire repositories | Entire packages |\n| **Updates** | On-demand, per file | Manual submodule update | Version-based |\n| **Transformations** | \u2705 Built-in | \u274c Manual | \u274c Post-install scripts |\n| **Multi-source** | \u2705 Multiple remotes | \u2705 Multiple submodules | \u2705 Multiple registries |\n| **Local changes** | \u2705 Protected | \u26a0\ufe0f Conflicts | \u274c Overwritten |\n| **Performance** | \u26a1 Hash-based | \ud83d\udc0c Full checkout | \u26a1 Registry-based |\n\nPerfect for when you need more control than submodules but don't want full package overhead.\n\n## Installation\n\nYou can install git-crossref using pip:\n\n pip install git-crossref\n\nAlternatively, clone the repository and install manually:\n\n git clone https://github.com/yourusername/git-crossref.git\n cd git-crossref\n pip install .\n\n## Quick Start\n\n1. Initialize a configuration file:\n ```bash\n git-crossref init\n ```\n\n2. Edit `.gitcrossref` to configure your remotes and files\n\n3. Sync files:\n ```bash\n git-crossref sync\n ```\n\n## Configuration\n\nThe tool uses a YAML configuration file located at `.gitcrossref` in your repository root. Here's an example:\n\n```yaml\nremotes:\n upstream:\n url: \"https://github.com/example/source-repo.git\"\n base_path: \"src/library\"\n version: \"main\" # All files from this remote default to 'main'\n\n another-source:\n url: \"https://github.com/example/another-repo.git\"\n base_path: \"scripts\"\n version: \"v1.2.3\" # All files from this remote default to 'v1.2.3'\n\nfiles:\n upstream:\n # Single file sync\n - source: \"utils.py\"\n destination: \"libs/utils.py\"\n # No hash provided, so it uses 'main' from upstream\n\n # File with specific commit\n - source: \"config.yaml\"\n destination: \"config/defaults.yaml\"\n hash: \"abc123\" # Overrides 'main' with a specific commit\n\n # Directory tree sync\n - source: \"templates/\"\n destination: \"project-templates/\"\n # Syncs entire templates directory\n\n another-source:\n # Script file\n - source: \"deploy.sh\"\n destination: \"scripts/deploy.sh\"\n # No hash provided, so it uses 'v1.2.3' from another-source\n```\n\n### Configuration Options\n\n**Remotes:**\n\n- `url`: Git URL of the remote repository\n- `base_path`: Optional base path within the repository \n- `version`: Default branch/tag/commit for files from this remote\n\n**Files:**\n\n- `source`: Path to file or directory in the remote repository (relative to `base_path`)\n- `destination`: Local path where content should be copied\n- `hash`: Optional specific commit hash (overrides `version`)\n- `ignore_changes`: If true, overwrites local files without checking for changes\n\n**Git Object Types:**\n\n- **Blob (file)**: `source: \"utils.py\"` - single file and/or glob patterns\n- **Tree (directory)**: `source: \"templates/\"` - entire directory (note trailing slash)\n\n## Commands\n\n### Sync files\n```bash\n# Sync all configured files\ngit-crossref sync\n\n# Sync only files from a specific remote\ngit-crossref sync --remote upstream\n\n# Force sync (override local changes)\ngit-crossref sync --force\n\n# Sync specific files by pattern\ngit-crossref sync libs/\n```\n\n### Check status\n```bash\n# Check status of all files\ngit-crossref status\n\n# Check specific files\ngit-crossref check libs/utils.py\n\n# Verbose output with hash information\ngit-crossref status --verbose\n```\n\n### Configuration management\n```bash\n# Initialize new configuration\ngit-crossref init\n\n# Validate configuration\ngit-crossref validate\n\n# Clean up cached repositories\ngit-crossref clean\n```\n\n## How it Works\n\ngit-crossref provides efficient, reliable vendoring through a Git-native approach:\n\n1. **Smart caching**: bare clone remote repositories once and reuses them for all operations\n2. **Precise targeting**: Extracts only the specific files/directories you need\n3. **Hash-based detection**: Compares Git blob hashes to detect changes reliably\n4. **Safe operations**: Checks for local modifications before overwriting files\n\nThis approach is both fast and reliable compared to timestamp-based synchronization tools.\n\n## Use Cases\n\n### Common Scenarios\n\n**\ud83d\udcc1 Shared Configuration Files**\n```yaml\n# Sync common CI/build configs across projects\nremotes:\n shared-configs:\n url: \"https://github.com/company/shared-configs.git\"\n version: \"main\"\nfiles:\n shared-configs:\n - source: \".github/workflows/\"\n destination: \".github/workflows/\"\n - source: \"Makefile\"\n destination: \"Makefile\"\n```\n\n**\ud83d\udd27 Utility Libraries** \n```yaml\n# Cherry-pick utility functions from larger codebases\nremotes:\n utils:\n url: \"https://github.com/company/utils.git\"\n base_path: \"src\"\nfiles:\n utils:\n - source: \"database/helpers.py\"\n destination: \"lib/db_helpers.py\"\n - source: \"validation/*.py\"\n destination: \"lib/validators/\"\n```\n\n**\ud83d\udce6 Dependency Vendoring**\n```yaml\n# Vendor specific versions of dependencies\nremotes:\n vendor:\n url: \"https://github.com/external/library.git\"\n version: \"v2.1.0\" # Pin to specific version\nfiles:\n vendor:\n - source: \"src/core/\"\n destination: \"vendor/library/\"\n ignore_changes: true # Don't warn about local modifications\n```\n\n## Technical Details\n\n### Implementation\n\nFor those interested in the technical implementation:\n\n- **Bare repository caching**: Remote repositories are cloned to `.git/crossref-cache/` as bare repositories (Git objects only, no working directory)\n- **Hash-based comparison**: Uses `git hash-object` to compare local and remote file blob hashes for reliable change detection\n- **Git status integration**: Leverages `git status` to detect uncommitted local changes before sync operations\n\n### Architecture\n\nThe tool is built around several key components:\n\n- **Configuration layer**: YAML-based configuration with JSON schema validation\n- **Git operations**: Efficient Git object manipulation using GitPython\n- **Sync engines**: Specialized classes for different content types (files, directories, patterns)\n- **Status tracking**: Rich status enumeration with success/failure categorization\n- **Error handling**: Comprehensive exception system with actionable error messages\n\n## Contributing\n\nContributions are welcome! Please see [README.developers.md](README.developers.md) for detailed development setup, code quality guidelines, and contribution workflow.\n\n## License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.\n\n",
"bugtrack_url": null,
"license": null,
"summary": "A Git plugin for cross-referencing and syncing specific files from multiple repositories.",
"version": "0.1.1",
"project_urls": {
"Homepage": "https://github.com/aesteve-rh/git-crossref",
"Issues": "https://github.com/aesteve-rh/git-crossref/issues",
"Repository": "https://github.com/aesteve-rh/git-crossref"
},
"split_keywords": [
"git",
" sync",
" files",
" repository",
" cli"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "e01d6eaf30919f16cb61cdf1c83cc723dd3071653eb67f058667f3746975006d",
"md5": "a71d7d015fd28836c443a1a70413a05d",
"sha256": "7667050f945a47484f4c1e90de969233cb2ba2edc396d8be83b0c2bbca7b33b6"
},
"downloads": -1,
"filename": "git_crossref-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "a71d7d015fd28836c443a1a70413a05d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11",
"size": 56091,
"upload_time": "2025-08-18T08:14:30",
"upload_time_iso_8601": "2025-08-18T08:14:30.063642Z",
"url": "https://files.pythonhosted.org/packages/e0/1d/6eaf30919f16cb61cdf1c83cc723dd3071653eb67f058667f3746975006d/git_crossref-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-18 08:14:30",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "aesteve-rh",
"github_project": "git-crossref",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"tox": true,
"lcname": "git-crossref"
}
