| Name | gil JSON |
| Version |
1.25.0.0
JSON |
| download |
| home_page | |
| Summary | Gil (git links) tool |
| upload_time | 2023-01-08 11:37:06 |
| maintainer | |
| docs_url | None |
| author | Ivan Shynkarenka |
| requires_python | >=3.7 |
| license | MIT License Copyright (c) 2018-2023 Ivan Shynkarenka Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
git
git-addons
devops-tools
subtree
submodule
dependency-links
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Gil (git links) tool
[](https://github.com/chronoxor/gil/actions/workflows/build-linux-clang.yml)
[](https://github.com/chronoxor/gil/actions/workflows/build-linux-gcc.yml)
<br/>
[](https://github.com/chronoxor/gil/actions/workflows/build-macos.yml)
<br/>
[](https://github.com/chronoxor/gil/actions/workflows/build-windows-cygwin.yml)
[](https://github.com/chronoxor/gil/actions/workflows/build-windows-msys2.yml)
[](https://github.com/chronoxor/gil/actions/workflows/build-windows-mingw.yml)
[](https://github.com/chronoxor/gil/actions/workflows/build-windows-vs.yml)
Gil is a git links tool to manage complex git repositories dependencies with
cycles and cross references.
This tool provides a solution to the [git recursive submodules dependency problem](https://github.com/chronoxor/gil#recursive-submodules-problem).
# Contents
* [Requirements](#requirements)
* [Setup](#setup)
* [Sample](#sample)
* [Usage](#usage)
* [Motivation: git submodules vs git links](#motivation-git-submodules-vs-git-links)
* [Git submodules management](#git-submodules-management)
* [Recursive submodules problem](#recursive-submodules-problem)
* [Motivation: git subtrees vs git links](#motivation-git-subtrees-vs-git-links)
* [Git subtrees management](#git-subtrees-management)
* [Subtrees manual management problem](#subtrees-manual-management-problem)
# Requirements
* Linux
* MacOS
* Windows
* [git](https://git-scm.com)
* [python3](https://www.python.org)
# Setup
```shell
sudo pip3 install --prefix /usr/local gil
```
# Sample
Consider we have the following git repository dependency graph:
Where:
* Blue nodes are third-party components
* Green nodes are components that we develop
* Edges show components dependency with a branch
### Root .gitlinks file
In order to describe the sample model with git links we have to define root
.gitlinks file in your sample repository with a following content:
```shell
# Projects
CppBenchmark CppBenchmark https://github.com/chronoxor/CppBenchmark.git master
CppCommon CppCommon https://github.com/chronoxor/CppCommon.git master
CppLogging CppLogging https://github.com/chronoxor/CppLogging.git master
# Modules
Catch2 modules/Catch2 https://github.com/catchorg/Catch2.git master
cpp-optparse modules/cpp-optparse https://github.com/weisslj/cpp-optparse.git master
fmt modules/fmt https://github.com/fmtlib/fmt.git master
HdrHistogram modules/HdrHistogram https://github.com/HdrHistogram/HdrHistogram_c.git master
zlib modules/zlib https://github.com/madler/zlib.git master
# Scripts
build scripts/build https://github.com/chronoxor/CppBuildScripts.git master
cmake scripts/cmake https://github.com/chronoxor/CppCMakeScripts.git master
```
Each line describe git link in the following format:
1. Unique name of the repository
2. Relative path of the repository (started from the path of .gitlinks file)
3. Git repository which will be used in `git clone` command
4. Repository branch to checkout
5. (Optional) Relative path in the base repository
6. (Optional) Relative path in the target repository
Empty line or line started with `#` are not parsed (treated as comment).
### CppBenchmark .gitlinks file
CppBenchmark .gitlinks file should be committed into the CppBenchmark project
and have the following content:
```shell
# Modules
Catch2 modules/Catch2 https://github.com/catchorg/Catch2.git master
cpp-optparse modules/cpp-optparse https://github.com/weisslj/cpp-optparse.git master
HdrHistogram modules/HdrHistogram https://github.com/HdrHistogram/HdrHistogram_c.git master
zlib modules/zlib https://github.com/madler/zlib.git master
# Scripts
build build https://github.com/chronoxor/CppBuildScripts.git master
cmake cmake https://github.com/chronoxor/CppCMakeScripts.git master
```
### CppCommon .gitlinks file
CppCommon .gitlinks file should be committed into the CppCommon project and
have the following content:
```shell
# Modules
Catch2 modules/Catch2 https://github.com/catchorg/Catch2.git master
CppBenchmark modules/CppBenchmark https://github.com/chronoxor/CppBenchmark.git master
fmt modules/fmt https://github.com/fmtlib/fmt.git master
# Scripts
build build https://github.com/chronoxor/CppBuildScripts.git master
cmake cmake https://github.com/chronoxor/CppCMakeScripts.git master
```
### CppLogging .gitlinks file
CppLogging .gitlinks file should be committed into the CppLogging project and
have the following content:
```shell
# Modules
Catch2 modules/Catch2 https://github.com/catchorg/Catch2.git master
cpp-optparse modules/cpp-optparse https://github.com/weisslj/cpp-optparse.git master
CppBenchmark modules/CppBenchmark https://github.com/chronoxor/CppBenchmark.git master
CppCommon modules/CppCommon https://github.com/chronoxor/CppCommon.git master
zlib modules/zlib https://github.com/madler/zlib.git master
# Scripts
build build https://github.com/chronoxor/CppBuildScripts.git master
cmake cmake https://github.com/chronoxor/CppCMakeScripts.git master
```
### Update the repository
Finally you have to update your root sample repository:
```shell
# Clone and link all git links dependencies from .gitlinks file
gil clone
gil link
# The same result with a single command
gil update
```
As the result you'll clone all required projects and link them to each other
in a proper way:
```shell
Working path: ~/gil/sample
...
Updating git links: ~/gil/sample/.gitlinks
Updating git links: ~/gil/sample/CppBenchmark/.gitlinks
Update git link: ~/gil/sample/modules/Catch2 -> ~/gil/sample/CppBenchmark/modules/Catch2
Update git link: ~/gil/sample/modules/cpp-optparse -> ~/gil/sample/CppBenchmark/modules/cpp-optparse
Update git link: ~/gil/sample/modules/HdrHistogram -> ~/gil/sample/CppBenchmark/modules/HdrHistogram
Update git link: ~/gil/sample/modules/zlib -> ~/gil/sample/CppBenchmark/modules/zlib
Update git link: ~/gil/sample/scripts/build -> ~/gil/sample/CppBenchmark/build
Update git link: ~/gil/sample/scripts/cmake -> ~/gil/sample/CppBenchmark/cmake
Updating git links: ~/gil/sample/CppCommon/.gitlinks
Update git link: ~/gil/sample/modules/Catch2 -> ~/gil/sample/CppCommon/modules/Catch2
Update git link: ~/gil/sample/CppBenchmark -> ~/gil/sample/CppCommon/modules/CppBenchmark
Update git link: ~/gil/sample/modules/fmt -> ~/gil/sample/CppCommon/modules/fmt
Update git link: ~/gil/sample/scripts/build -> ~/gil/sample/CppCommon/build
Update git link: ~/gil/sample/scripts/cmake -> ~/gil/sample/CppCommon/cmake
Updating git links: ~/gil/sample/CppLogging/.gitlinks
Update git link: ~/gil/sample/modules/Catch2 -> ~/gil/sample/CppLogging/modules/Catch2
Update git link: ~/gil/sample/modules/cpp-optparse -> ~/gil/sample/CppLogging/modules/cpp-optparse
Update git link: ~/gil/sample/CppBenchmark -> ~/gil/sample/CppLogging/modules/CppBenchmark
Update git link: ~/gil/sample/CppCommon -> ~/gil/sample/CppLogging/modules/CppCommon
Update git link: ~/gil/sample/modules/zlib -> ~/gil/sample/CppLogging/modules/zlib
Update git link: ~/gil/sample/scripts/build -> ~/gil/sample/CppLogging/build
Update git link: ~/gil/sample/scripts/cmake -> ~/gil/sample/CppLogging/cmake
```
All repositories will be checkout to required branches.
If content of any .gitlinks changes it is required to run `gil update` command
to re-update git links - new repositories will be cloned and linked properly!
### Commit, push, pull
You can work and change any files in your repositories and perform all git
operations for any repository to commit and contribute your changes.
If you want to commit all changes in some repository with all changes in
child linked repositories you can do it with a single command:
```shell
gil commit -a -m "Some big update"
```
This command will visit the current and all child repositories and perform
the corresponding `git commit` command with provided arguments.
Please note that gil command visit only child repositories that are described
in .gitlinks file no parent dependencies will be processed. For this reason
if you want to commit, pull or push all repositories the corresponding command
should be run in the root directory where root .gitlinks is placed.
Pull, push commands works in a similar way:
```shell
gil pull
gil push
```
### Build
Please investigate and follow links in the sample repository in order to
understand the logic how gil (git links) tool manages dependencies.
Finally you can build sample projects with provided build scripts:
* `~/gil/sample/CppBenchmark/build`
* `~/gil/sample/CppCommon/build`
* `~/gil/sample/CppLogging/build`
Additionally each module can be cloned and successfully build without root
repository. In this case local .gitlinks file will be used to resolve all
dependencies!
# Usage
Gil (git links) tool supports the following commands:
```
usage: gil command arguments
Supported commands:
help - show this help
version - show version
context - show git links context
clone [args] - clone git repositories
link - link git repositories
update - update git repositories (clone & link)
pull [args] - pull git repositories
push [args] - push git repositories
commit [args] - commit git repositories
status [args] - show status of git repositories
```
* `gil context` - command will show the current git link context of the current directory;
* `gil clone [args]` - clone all repositories that are missed in the current context;
* `gil link` - link all repositories that are missed in the current context;
* `gil update` - clone and link in a single operation;
* `gil pull [args]` - pull all repositories in the current directory;
* `gil push [args]` - push all repositories in the current directory;
* `gil commit [args]` - commit all repositories in the current directory;
* `gil status [args]` - show status of all repositories in the current directory;
# Motivation: git submodules vs git links
Git submodules allow to get the same idea of repositories dependency, but with
a serious limitation - git submodules degrade to tree-like recursive structure
with repositories duplication.
## Git submodules management
Git submodules provides several operations to manage repositories dependency.
### Add submodule dependency
```shell
# Add git submodule of 'master' branch into the relative directory 'modules/zlib'
git submodule add -b master https://github.com/madler/zlib.git modules/zlib
```
### Initialize submodules
```shell
# Update all submodules in the current repository
git submodule update --init --recursive --remote
```
### Refresh submodules from remote repositories
```shell
git pull
git submodule update --init --recursive --remote
git pull --recurse-submodules
git submodule foreach "(git checkout master; git pull)"
```
### Synchronize submodules with remote repositories
```shell
git pull
git submodule update --init --recursive --remote
git pull --recurse-submodules
git submodule foreach "(git checkout master; git pull)"
git add --all
git commit -m "Submodule Sync"
git push
```
### Remove submodule dependency
```shell
# Remove git submodule from the relative directory 'modules/zlib'
git submodule deinit -f -- modules/zlib
rm -rf .git/modules/modules/zlib
git rm -f modules/zlib
```
## Recursive submodules problem
Git submodules representation of the described sample repository is the
following:
```shell
~/gil/sample/CppBenchmark
~/gil/sample/CppBenchmark/build
~/gil/sample/CppBenchmark/cmake
~/gil/sample/CppBenchmark/modules/Catch2
~/gil/sample/CppBenchmark/modules/cpp-optparse
~/gil/sample/CppBenchmark/modules/HdrHistogram
~/gil/sample/CppBenchmark/modules/zlib
~/gil/sample/CppCommon
~/gil/sample/CppCommon/build - DUPLICATE!
~/gil/sample/CppCommon/cmake - DUPLICATE!
~/gil/sample/CppCommon/modules/Catch2 - DUPLICATE!
~/gil/sample/CppCommon/modules/CppBenchmark - DUPLICATE!
~/gil/sample/CppCommon/modules/CppBenchmark/build - DUPLICATE!
~/gil/sample/CppCommon/modules/CppBenchmark/cmake - DUPLICATE!
~/gil/sample/CppCommon/modules/CppBenchmark/modules/Catch2 - DUPLICATE!
~/gil/sample/CppCommon/modules/CppBenchmark/modules/cpp-optparse - DUPLICATE!
~/gil/sample/CppCommon/modules/CppBenchmark/modules/HdrHistogram - DUPLICATE!
~/gil/sample/CppCommon/modules/CppBenchmark/modules/zlib - DUPLICATE!
~/gil/sample/CppCommon/modules/fmt
~/gil/sample/CppLogging
~/gil/sample/CppLogging/build - DUPLICATE!
~/gil/sample/CppLogging/cmake - DUPLICATE!
~/gil/sample/CppLogging/modules/Catch2 - DUPLICATE!
~/gil/sample/CppLogging/modules/cpp-optparse - DUPLICATE!
~/gil/sample/CppLogging/modules/CppBenchmark - DUPLICATE!
~/gil/sample/CppLogging/modules/CppBenchmark/build - DUPLICATE!
~/gil/sample/CppLogging/modules/CppBenchmark/cmake - DUPLICATE!
~/gil/sample/CppLogging/modules/CppBenchmark/modules/Catch2 - DUPLICATE!
~/gil/sample/CppLogging/modules/CppBenchmark/modules/cpp-optparse - DUPLICATE!
~/gil/sample/CppLogging/modules/CppBenchmark/modules/HdrHistogram - DUPLICATE!
~/gil/sample/CppLogging/modules/CppBenchmark/modules/zlib - DUPLICATE!
~/gil/sample/CppLogging/modules/CppCommon - DUPLICATE!
~/gil/sample/CppLogging/modules/CppCommon/build - DUPLICATE!
~/gil/sample/CppLogging/modules/CppCommon/cmake - DUPLICATE!
~/gil/sample/CppLogging/modules/CppCommon/modules/Catch2 - DUPLICATE!
~/gil/sample/CppLogging/modules/CppCommon/modules/CppBenchmark - DUPLICATE!
~/gil/sample/CppLogging/modules/CppCommon/modules/CppBenchmark/build - DUPLICATE!
~/gil/sample/CppLogging/modules/CppCommon/modules/CppBenchmark/cmake - DUPLICATE!
~/gil/sample/CppLogging/modules/CppCommon/modules/CppBenchmark/modules/Catch2 - DUPLICATE!
~/gil/sample/CppLogging/modules/CppCommon/modules/CppBenchmark/modules/cpp-optparse - DUPLICATE!
~/gil/sample/CppLogging/modules/CppCommon/modules/CppBenchmark/modules/HdrHistogram - DUPLICATE!
~/gil/sample/CppLogging/modules/CppCommon/modules/CppBenchmark/modules/zlib - DUPLICATE!
~/gil/sample/CppLogging/modules/zlib - DUPLICATE!
```
Only 10 of 41 repositories are unique. Other duplicates each other multiple
times. Moreover some nested repositories included recursively! CppLogging
project has the third level of nesting (`CppLogging -> CppCommon -> CppBenchmark`).
Each new level heavily increases the rate of duplication. As the result all
submodule management operations becomes slow and requires multiple executions
of synchronize script to pass changes from top level to the bottom. The count
of synchronizations in the worst case equals to recursive levels count.
All this issues are solved with gil (git links) tool which allows to link all
required repositories together avoiding duplication. The tool also provides
operations to manage changes in all repositories with a [simple easy to use
commands](https://github.com/chronoxor/gil#usage).
# Motivation: git subtrees vs git links
Git subtrees is an easy to use alternative to git submodules. It allows to nest
one repository inside another as a sub-directory.
From the one side git subtrees are very lightweight with simple workflow. They
don't add any metadata files like git submodule does (i.e., .gitmodule) and
sub-project's code is available right after the clone of the super project is
done. Contents of the module can be modified without having a separate
repository copy of the dependency somewhere else.
However they have several drawbacks. Contributing code back upstream for the
sub-projects is slightly more complicated. And developer should be very careful
of not mixing super and sub-project code in commits.
Git subtrees requires precise manual management in case use have complicated
projects structure and lots of upstream commits to dependent projects.
## Git subtrees management
Git subtrees provides several operations to manage repositories dependency.
### Add subtree dependency
```shell
# Add git subtree of 'master' branch into the relative directory 'modules/zlib'
git subtree add -P modules/zlib https://github.com/madler/zlib.git master --squash
```
### Update subtree from the upstream remote repository
```shell
git subtree pull -P modules/zlib https://github.com/madler/zlib.git master --squash
```
### Contribute changes from subtree back to the upstream remote repository
```shell
git subtree push -P modules/zlib https://github.com/madler/zlib.git master
```
### Remove subtree dependency
```shell
# Remove git subtree from the relative directory 'modules/zlib'
git rm -f modules/zlib
```
## Subtrees manual management problem
Initially git subtrees lack of the same [repository duplicate problem](#recursive-submodules-problem)
which git submodules have. Therefore all repositories with multiple dependencies
will be duplicated and require manual and careful management.
For instance `CppBenchmark` repository presented four times in our [sample](#sample):
```shell
~/gil/sample/CppBenchmark
~/gil/sample/CppCommon/modules/CppBenchmark - DUPLICATE!
~/gil/sample/CppLogging/modules/CppBenchmark - DUPLICATE!
~/gil/sample/CppLogging/modules/CppCommon/modules/CppBenchmark - DUPLICATE!
```
The first problem if you updated `CppBenchmark` and want to get the fix in all
dependent repositories you have to update them manually with the set of
`git subtree pull` commands:
```shell
# Commit and push changes in CppBenchmark repository
cd CppBenchmark
git commit -a -m "Bugfix in CppBenchmark"
git push
cd ..
# Update CppCommon repository
cd CppCommon
git subtree pull --prefix modules/CppBenchmark https://github.com/chronoxor/test-CppBenchmark.git main --squash
git push
cd ..
# Update CppLogging repository
cd CppLogging
git subtree pull --prefix modules/CppBenchmark https://github.com/chronoxor/test-CppBenchmark.git main --squash
git subtree pull --prefix modules/CppCommon https://github.com/chronoxor/test-CppCommon.git main --squash
git push
cd ..
```
The second problem if you found a bug in `CppBenchmark` while debugging
`CppLogging` and want to fix it in place, push the changes and update all
repositories properly. In this case you need to make the following steps:
```shell
# Commit and push changes in CppBenchmark but from CppLogging repository
cd CppLogging
git commit -a -m "Bugfix in CppBenchmark (found in CppLogging repository)"
git push
git subtree push --prefix modules/CppBenchmark https://github.com/chronoxor/test-CppBenchmark.git main --squash
cd ..
# Update CppBenchmark repository
cd CppBenchmark
git pull
cd ..
# Update CppCommon repository
cd CppCommon
git subtree pull --prefix modules/CppBenchmark https://github.com/chronoxor/test-CppBenchmark.git main --squash
git push
cd ..
# Update CppLogging repository
cd CppLogging
git subtree pull --prefix modules/CppBenchmark https://github.com/chronoxor/test-CppBenchmark.git main --squash
git subtree pull --prefix modules/CppCommon https://github.com/chronoxor/test-CppCommon.git main --squash
git push
cd ..
```
As the result too much manual work should be done to make all repositories up
to date. Definitely all commands should be stored into scripts with care. And
this job requires some efforts and testing because it is very easy to make a
mistake. Git subtrees are good and easy to use for small repositories without
complicated hierarchy. But working with huge complicated repositories using
git subtrees is a big challenge as git subtrees do not suppot recursing
pull/push/update commands.
Raw data
{
"_id": null,
"home_page": "",
"name": "gil",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "git,git-addons,devops-tools,subtree,submodule,dependency-links",
"author": "Ivan Shynkarenka",
"author_email": "chronoxor@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/13/c2/7899e192386b79d9d7c93b45d274733e444065b0eef616c119ea77360a19/gil-1.25.0.0.tar.gz",
"platform": "Linux",
"description": "# Gil (git links) tool\n\n[](https://github.com/chronoxor/gil/actions/workflows/build-linux-clang.yml)\n[](https://github.com/chronoxor/gil/actions/workflows/build-linux-gcc.yml)\n<br/>\n[](https://github.com/chronoxor/gil/actions/workflows/build-macos.yml)\n<br/>\n[](https://github.com/chronoxor/gil/actions/workflows/build-windows-cygwin.yml)\n[](https://github.com/chronoxor/gil/actions/workflows/build-windows-msys2.yml)\n[](https://github.com/chronoxor/gil/actions/workflows/build-windows-mingw.yml)\n[](https://github.com/chronoxor/gil/actions/workflows/build-windows-vs.yml)\n\nGil is a git links tool to manage complex git repositories dependencies with\ncycles and cross references.\n\nThis tool provides a solution to the [git recursive submodules dependency problem](https://github.com/chronoxor/gil#recursive-submodules-problem).\n\n# Contents\n * [Requirements](#requirements)\n * [Setup](#setup)\n * [Sample](#sample)\n * [Usage](#usage)\n * [Motivation: git submodules vs git links](#motivation-git-submodules-vs-git-links)\n * [Git submodules management](#git-submodules-management)\n * [Recursive submodules problem](#recursive-submodules-problem)\n * [Motivation: git subtrees vs git links](#motivation-git-subtrees-vs-git-links)\n * [Git subtrees management](#git-subtrees-management)\n * [Subtrees manual management problem](#subtrees-manual-management-problem)\n\n# Requirements\n* Linux\n* MacOS\n* Windows\n* [git](https://git-scm.com)\n* [python3](https://www.python.org)\n\n# Setup\n\n```shell\nsudo pip3 install --prefix /usr/local gil\n```\n\n# Sample\nConsider we have the following git repository dependency graph:\n\n\n\nWhere:\n* Blue nodes are third-party components\n* Green nodes are components that we develop\n* Edges show components dependency with a branch\n\n### Root .gitlinks file\nIn order to describe the sample model with git links we have to define root\n.gitlinks file in your sample repository with a following content:\n```shell\n# Projects\nCppBenchmark CppBenchmark https://github.com/chronoxor/CppBenchmark.git master\nCppCommon CppCommon https://github.com/chronoxor/CppCommon.git master\nCppLogging CppLogging https://github.com/chronoxor/CppLogging.git master\n\n# Modules\nCatch2 modules/Catch2 https://github.com/catchorg/Catch2.git master\ncpp-optparse modules/cpp-optparse https://github.com/weisslj/cpp-optparse.git master\nfmt modules/fmt https://github.com/fmtlib/fmt.git master\nHdrHistogram modules/HdrHistogram https://github.com/HdrHistogram/HdrHistogram_c.git master\nzlib modules/zlib https://github.com/madler/zlib.git master\n\n# Scripts\nbuild scripts/build https://github.com/chronoxor/CppBuildScripts.git master\ncmake scripts/cmake https://github.com/chronoxor/CppCMakeScripts.git master\n```\n\nEach line describe git link in the following format:\n1. Unique name of the repository\n2. Relative path of the repository (started from the path of .gitlinks file)\n3. Git repository which will be used in `git clone` command\n4. Repository branch to checkout\n5. (Optional) Relative path in the base repository\n6. (Optional) Relative path in the target repository\n\nEmpty line or line started with `#` are not parsed (treated as comment).\n\n### CppBenchmark .gitlinks file\nCppBenchmark .gitlinks file should be committed into the CppBenchmark project\nand have the following content:\n```shell\n# Modules\nCatch2 modules/Catch2 https://github.com/catchorg/Catch2.git master\ncpp-optparse modules/cpp-optparse https://github.com/weisslj/cpp-optparse.git master\nHdrHistogram modules/HdrHistogram https://github.com/HdrHistogram/HdrHistogram_c.git master\nzlib modules/zlib https://github.com/madler/zlib.git master\n\n# Scripts\nbuild build https://github.com/chronoxor/CppBuildScripts.git master\ncmake cmake https://github.com/chronoxor/CppCMakeScripts.git master\n```\n\n### CppCommon .gitlinks file\nCppCommon .gitlinks file should be committed into the CppCommon project and\nhave the following content:\n```shell\n# Modules\nCatch2 modules/Catch2 https://github.com/catchorg/Catch2.git master\nCppBenchmark modules/CppBenchmark https://github.com/chronoxor/CppBenchmark.git master\nfmt modules/fmt https://github.com/fmtlib/fmt.git master\n\n# Scripts\nbuild build https://github.com/chronoxor/CppBuildScripts.git master\ncmake cmake https://github.com/chronoxor/CppCMakeScripts.git master\n```\n\n### CppLogging .gitlinks file\nCppLogging .gitlinks file should be committed into the CppLogging project and\nhave the following content:\n```shell\n# Modules\nCatch2 modules/Catch2 https://github.com/catchorg/Catch2.git master\ncpp-optparse modules/cpp-optparse https://github.com/weisslj/cpp-optparse.git master\nCppBenchmark modules/CppBenchmark https://github.com/chronoxor/CppBenchmark.git master\nCppCommon modules/CppCommon https://github.com/chronoxor/CppCommon.git master\nzlib modules/zlib https://github.com/madler/zlib.git master\n\n# Scripts\nbuild build https://github.com/chronoxor/CppBuildScripts.git master\ncmake cmake https://github.com/chronoxor/CppCMakeScripts.git master\n```\n\n### Update the repository\nFinally you have to update your root sample repository:\n```shell\n# Clone and link all git links dependencies from .gitlinks file\ngil clone\ngil link\n\n# The same result with a single command\ngil update\n```\n\nAs the result you'll clone all required projects and link them to each other\nin a proper way:\n```shell\nWorking path: ~/gil/sample\n...\nUpdating git links: ~/gil/sample/.gitlinks\nUpdating git links: ~/gil/sample/CppBenchmark/.gitlinks\nUpdate git link: ~/gil/sample/modules/Catch2 -> ~/gil/sample/CppBenchmark/modules/Catch2\nUpdate git link: ~/gil/sample/modules/cpp-optparse -> ~/gil/sample/CppBenchmark/modules/cpp-optparse\nUpdate git link: ~/gil/sample/modules/HdrHistogram -> ~/gil/sample/CppBenchmark/modules/HdrHistogram\nUpdate git link: ~/gil/sample/modules/zlib -> ~/gil/sample/CppBenchmark/modules/zlib\nUpdate git link: ~/gil/sample/scripts/build -> ~/gil/sample/CppBenchmark/build\nUpdate git link: ~/gil/sample/scripts/cmake -> ~/gil/sample/CppBenchmark/cmake\nUpdating git links: ~/gil/sample/CppCommon/.gitlinks\nUpdate git link: ~/gil/sample/modules/Catch2 -> ~/gil/sample/CppCommon/modules/Catch2\nUpdate git link: ~/gil/sample/CppBenchmark -> ~/gil/sample/CppCommon/modules/CppBenchmark\nUpdate git link: ~/gil/sample/modules/fmt -> ~/gil/sample/CppCommon/modules/fmt\nUpdate git link: ~/gil/sample/scripts/build -> ~/gil/sample/CppCommon/build\nUpdate git link: ~/gil/sample/scripts/cmake -> ~/gil/sample/CppCommon/cmake\nUpdating git links: ~/gil/sample/CppLogging/.gitlinks\nUpdate git link: ~/gil/sample/modules/Catch2 -> ~/gil/sample/CppLogging/modules/Catch2\nUpdate git link: ~/gil/sample/modules/cpp-optparse -> ~/gil/sample/CppLogging/modules/cpp-optparse\nUpdate git link: ~/gil/sample/CppBenchmark -> ~/gil/sample/CppLogging/modules/CppBenchmark\nUpdate git link: ~/gil/sample/CppCommon -> ~/gil/sample/CppLogging/modules/CppCommon\nUpdate git link: ~/gil/sample/modules/zlib -> ~/gil/sample/CppLogging/modules/zlib\nUpdate git link: ~/gil/sample/scripts/build -> ~/gil/sample/CppLogging/build\nUpdate git link: ~/gil/sample/scripts/cmake -> ~/gil/sample/CppLogging/cmake\n```\n\nAll repositories will be checkout to required branches.\n\nIf content of any .gitlinks changes it is required to run `gil update` command\nto re-update git links - new repositories will be cloned and linked properly!\n\n### Commit, push, pull\nYou can work and change any files in your repositories and perform all git\noperations for any repository to commit and contribute your changes.\n\nIf you want to commit all changes in some repository with all changes in\nchild linked repositories you can do it with a single command:\n```shell\ngil commit -a -m \"Some big update\"\n```\n\nThis command will visit the current and all child repositories and perform\nthe corresponding `git commit` command with provided arguments.\n\nPlease note that gil command visit only child repositories that are described\nin .gitlinks file no parent dependencies will be processed. For this reason\nif you want to commit, pull or push all repositories the corresponding command\nshould be run in the root directory where root .gitlinks is placed.\n\nPull, push commands works in a similar way:\n```shell\ngil pull\ngil push\n```\n\n### Build\nPlease investigate and follow links in the sample repository in order to\nunderstand the logic how gil (git links) tool manages dependencies.\n\nFinally you can build sample projects with provided build scripts:\n* `~/gil/sample/CppBenchmark/build`\n* `~/gil/sample/CppCommon/build`\n* `~/gil/sample/CppLogging/build`\n\nAdditionally each module can be cloned and successfully build without root\nrepository. In this case local .gitlinks file will be used to resolve all\ndependencies!\n\n# Usage\nGil (git links) tool supports the following commands:\n```\nusage: gil command arguments\nSupported commands:\n help - show this help\n version - show version\n context - show git links context\n clone [args] - clone git repositories\n link - link git repositories\n update - update git repositories (clone & link)\n pull [args] - pull git repositories\n push [args] - push git repositories\n commit [args] - commit git repositories\n status [args] - show status of git repositories\n```\n\n* `gil context` - command will show the current git link context of the current directory;\n* `gil clone [args]` - clone all repositories that are missed in the current context;\n* `gil link` - link all repositories that are missed in the current context;\n* `gil update` - clone and link in a single operation;\n* `gil pull [args]` - pull all repositories in the current directory;\n* `gil push [args]` - push all repositories in the current directory;\n* `gil commit [args]` - commit all repositories in the current directory;\n* `gil status [args]` - show status of all repositories in the current directory;\n\n# Motivation: git submodules vs git links\nGit submodules allow to get the same idea of repositories dependency, but with\na serious limitation - git submodules degrade to tree-like recursive structure\nwith repositories duplication.\n\n## Git submodules management\nGit submodules provides several operations to manage repositories dependency.\n\n### Add submodule dependency\n```shell\n# Add git submodule of 'master' branch into the relative directory 'modules/zlib'\ngit submodule add -b master https://github.com/madler/zlib.git modules/zlib\n```\n\n### Initialize submodules\n```shell\n# Update all submodules in the current repository\ngit submodule update --init --recursive --remote\n```\n\n### Refresh submodules from remote repositories\n```shell\ngit pull\ngit submodule update --init --recursive --remote\ngit pull --recurse-submodules\ngit submodule foreach \"(git checkout master; git pull)\"\n```\n\n### Synchronize submodules with remote repositories\n```shell\ngit pull\ngit submodule update --init --recursive --remote\ngit pull --recurse-submodules\ngit submodule foreach \"(git checkout master; git pull)\"\ngit add --all\ngit commit -m \"Submodule Sync\"\ngit push\n```\n\n### Remove submodule dependency\n```shell\n# Remove git submodule from the relative directory 'modules/zlib'\ngit submodule deinit -f -- modules/zlib\nrm -rf .git/modules/modules/zlib\ngit rm -f modules/zlib\n```\n\n## Recursive submodules problem\nGit submodules representation of the described sample repository is the\nfollowing:\n```shell\n~/gil/sample/CppBenchmark\n~/gil/sample/CppBenchmark/build\n~/gil/sample/CppBenchmark/cmake\n~/gil/sample/CppBenchmark/modules/Catch2\n~/gil/sample/CppBenchmark/modules/cpp-optparse\n~/gil/sample/CppBenchmark/modules/HdrHistogram\n~/gil/sample/CppBenchmark/modules/zlib\n~/gil/sample/CppCommon\n~/gil/sample/CppCommon/build - DUPLICATE!\n~/gil/sample/CppCommon/cmake - DUPLICATE!\n~/gil/sample/CppCommon/modules/Catch2 - DUPLICATE!\n~/gil/sample/CppCommon/modules/CppBenchmark - DUPLICATE!\n~/gil/sample/CppCommon/modules/CppBenchmark/build - DUPLICATE!\n~/gil/sample/CppCommon/modules/CppBenchmark/cmake - DUPLICATE!\n~/gil/sample/CppCommon/modules/CppBenchmark/modules/Catch2 - DUPLICATE!\n~/gil/sample/CppCommon/modules/CppBenchmark/modules/cpp-optparse - DUPLICATE!\n~/gil/sample/CppCommon/modules/CppBenchmark/modules/HdrHistogram - DUPLICATE!\n~/gil/sample/CppCommon/modules/CppBenchmark/modules/zlib - DUPLICATE!\n~/gil/sample/CppCommon/modules/fmt\n~/gil/sample/CppLogging\n~/gil/sample/CppLogging/build - DUPLICATE!\n~/gil/sample/CppLogging/cmake - DUPLICATE!\n~/gil/sample/CppLogging/modules/Catch2 - DUPLICATE!\n~/gil/sample/CppLogging/modules/cpp-optparse - DUPLICATE!\n~/gil/sample/CppLogging/modules/CppBenchmark - DUPLICATE!\n~/gil/sample/CppLogging/modules/CppBenchmark/build - DUPLICATE!\n~/gil/sample/CppLogging/modules/CppBenchmark/cmake - DUPLICATE!\n~/gil/sample/CppLogging/modules/CppBenchmark/modules/Catch2 - DUPLICATE!\n~/gil/sample/CppLogging/modules/CppBenchmark/modules/cpp-optparse - DUPLICATE!\n~/gil/sample/CppLogging/modules/CppBenchmark/modules/HdrHistogram - DUPLICATE!\n~/gil/sample/CppLogging/modules/CppBenchmark/modules/zlib - DUPLICATE!\n~/gil/sample/CppLogging/modules/CppCommon - DUPLICATE!\n~/gil/sample/CppLogging/modules/CppCommon/build - DUPLICATE!\n~/gil/sample/CppLogging/modules/CppCommon/cmake - DUPLICATE!\n~/gil/sample/CppLogging/modules/CppCommon/modules/Catch2 - DUPLICATE!\n~/gil/sample/CppLogging/modules/CppCommon/modules/CppBenchmark - DUPLICATE!\n~/gil/sample/CppLogging/modules/CppCommon/modules/CppBenchmark/build - DUPLICATE!\n~/gil/sample/CppLogging/modules/CppCommon/modules/CppBenchmark/cmake - DUPLICATE!\n~/gil/sample/CppLogging/modules/CppCommon/modules/CppBenchmark/modules/Catch2 - DUPLICATE!\n~/gil/sample/CppLogging/modules/CppCommon/modules/CppBenchmark/modules/cpp-optparse - DUPLICATE!\n~/gil/sample/CppLogging/modules/CppCommon/modules/CppBenchmark/modules/HdrHistogram - DUPLICATE!\n~/gil/sample/CppLogging/modules/CppCommon/modules/CppBenchmark/modules/zlib - DUPLICATE!\n~/gil/sample/CppLogging/modules/zlib - DUPLICATE!\n```\n\nOnly 10 of 41 repositories are unique. Other duplicates each other multiple\ntimes. Moreover some nested repositories included recursively! CppLogging\nproject has the third level of nesting (`CppLogging -> CppCommon -> CppBenchmark`).\n\nEach new level heavily increases the rate of duplication. As the result all\nsubmodule management operations becomes slow and requires multiple executions\nof synchronize script to pass changes from top level to the bottom. The count\nof synchronizations in the worst case equals to recursive levels count.\n\nAll this issues are solved with gil (git links) tool which allows to link all\nrequired repositories together avoiding duplication. The tool also provides\noperations to manage changes in all repositories with a [simple easy to use\ncommands](https://github.com/chronoxor/gil#usage).\n\n# Motivation: git subtrees vs git links\nGit subtrees is an easy to use alternative to git submodules. It allows to nest\none repository inside another as a sub-directory.\n\nFrom the one side git subtrees are very lightweight with simple workflow. They\ndon't add any metadata files like git submodule does (i.e., .gitmodule) and\nsub-project's code is available right after the clone of the super project is\ndone. Contents of the module can be modified without having a separate\nrepository copy of the dependency somewhere else.\n\nHowever they have several drawbacks. Contributing code back upstream for the\nsub-projects is slightly more complicated. And developer should be very careful\nof not mixing super and sub-project code in commits.\n\nGit subtrees requires precise manual management in case use have complicated\nprojects structure and lots of upstream commits to dependent projects.\n\n## Git subtrees management\nGit subtrees provides several operations to manage repositories dependency.\n\n### Add subtree dependency\n```shell\n# Add git subtree of 'master' branch into the relative directory 'modules/zlib'\ngit subtree add -P modules/zlib https://github.com/madler/zlib.git master --squash\n```\n\n### Update subtree from the upstream remote repository\n```shell\ngit subtree pull -P modules/zlib https://github.com/madler/zlib.git master --squash\n```\n\n### Contribute changes from subtree back to the upstream remote repository\n```shell\ngit subtree push -P modules/zlib https://github.com/madler/zlib.git master\n```\n\n### Remove subtree dependency\n```shell\n# Remove git subtree from the relative directory 'modules/zlib'\ngit rm -f modules/zlib\n```\n\n## Subtrees manual management problem\nInitially git subtrees lack of the same [repository duplicate problem](#recursive-submodules-problem)\nwhich git submodules have. Therefore all repositories with multiple dependencies\nwill be duplicated and require manual and careful management.\n\nFor instance `CppBenchmark` repository presented four times in our [sample](#sample):\n```shell\n~/gil/sample/CppBenchmark\n~/gil/sample/CppCommon/modules/CppBenchmark - DUPLICATE!\n~/gil/sample/CppLogging/modules/CppBenchmark - DUPLICATE!\n~/gil/sample/CppLogging/modules/CppCommon/modules/CppBenchmark - DUPLICATE!\n```\n\nThe first problem if you updated `CppBenchmark` and want to get the fix in all\ndependent repositories you have to update them manually with the set of\n`git subtree pull` commands:\n```shell\n# Commit and push changes in CppBenchmark repository\ncd CppBenchmark\ngit commit -a -m \"Bugfix in CppBenchmark\"\ngit push\ncd ..\n\n# Update CppCommon repository\ncd CppCommon\ngit subtree pull --prefix modules/CppBenchmark https://github.com/chronoxor/test-CppBenchmark.git main --squash\ngit push\ncd ..\n\n# Update CppLogging repository\ncd CppLogging\ngit subtree pull --prefix modules/CppBenchmark https://github.com/chronoxor/test-CppBenchmark.git main --squash\ngit subtree pull --prefix modules/CppCommon https://github.com/chronoxor/test-CppCommon.git main --squash\ngit push\ncd ..\n```\n\nThe second problem if you found a bug in `CppBenchmark` while debugging\n`CppLogging` and want to fix it in place, push the changes and update all\nrepositories properly. In this case you need to make the following steps:\n```shell\n# Commit and push changes in CppBenchmark but from CppLogging repository\ncd CppLogging\ngit commit -a -m \"Bugfix in CppBenchmark (found in CppLogging repository)\"\ngit push\ngit subtree push --prefix modules/CppBenchmark https://github.com/chronoxor/test-CppBenchmark.git main --squash\ncd ..\n\n# Update CppBenchmark repository\ncd CppBenchmark\ngit pull\ncd ..\n\n# Update CppCommon repository\ncd CppCommon\ngit subtree pull --prefix modules/CppBenchmark https://github.com/chronoxor/test-CppBenchmark.git main --squash\ngit push\ncd ..\n\n# Update CppLogging repository\ncd CppLogging\ngit subtree pull --prefix modules/CppBenchmark https://github.com/chronoxor/test-CppBenchmark.git main --squash\ngit subtree pull --prefix modules/CppCommon https://github.com/chronoxor/test-CppCommon.git main --squash\ngit push\ncd ..\n```\n\nAs the result too much manual work should be done to make all repositories up\nto date. Definitely all commands should be stored into scripts with care. And\nthis job requires some efforts and testing because it is very easy to make a\nmistake. Git subtrees are good and easy to use for small repositories without\ncomplicated hierarchy. But working with huge complicated repositories using\ngit subtrees is a big challenge as git subtrees do not suppot recursing\npull/push/update commands.\n",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2018-2023 Ivan Shynkarenka Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ",
"summary": "Gil (git links) tool",
"version": "1.25.0.0",
"split_keywords": [
"git",
"git-addons",
"devops-tools",
"subtree",
"submodule",
"dependency-links"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "abacf2bfa3732bb09b0436cedc06e56f74aae268a16fa2c66f9ae6f7038a6f6c",
"md5": "9ad68f4f7a445ebfc856f7c0d49c0b81",
"sha256": "a1d599cc6ec003a01b4801b5e0f4ebd10eacfb5fb216fb8ddae3feecbaab48b7"
},
"downloads": -1,
"filename": "gil-1.25.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9ad68f4f7a445ebfc856f7c0d49c0b81",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 11188,
"upload_time": "2023-01-08T11:37:03",
"upload_time_iso_8601": "2023-01-08T11:37:03.136000Z",
"url": "https://files.pythonhosted.org/packages/ab/ac/f2bfa3732bb09b0436cedc06e56f74aae268a16fa2c66f9ae6f7038a6f6c/gil-1.25.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "13c27899e192386b79d9d7c93b45d274733e444065b0eef616c119ea77360a19",
"md5": "bba85a0dcc5d9e0a0d2770e9079711df",
"sha256": "3227d199034bb0e54b5e6792257adba44f0f259d8e909aea83970290d7601d64"
},
"downloads": -1,
"filename": "gil-1.25.0.0.tar.gz",
"has_sig": false,
"md5_digest": "bba85a0dcc5d9e0a0d2770e9079711df",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 11528,
"upload_time": "2023-01-08T11:37:06",
"upload_time_iso_8601": "2023-01-08T11:37:06.740913Z",
"url": "https://files.pythonhosted.org/packages/13/c2/7899e192386b79d9d7c93b45d274733e444065b0eef616c119ea77360a19/gil-1.25.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-01-08 11:37:06",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "gil"
}
