The good and the bad of submodules, using Wind River's Pulsar Linux as a convenient example.
Stuff to add:
git config submodule.<what>.url PRIVATE_URL
git submodule [--quiet] add [<options>] [--] <repository> [<path>] git submodule [--quiet] status [--cached] [--recursive] [--] [<path>...] git submodule [--quiet] init [--] [<path>...] git submodule [--quiet] deinit [-f|--force] (--all|[--] <path>...) git submodule [--quiet] update [<options>] [--] [<path>...] git submodule [--quiet] summary [<options>] [--] [<path>...] git submodule [--quiet] foreach [--recursive] <command> git submodule [--quiet] sync [--recursive] [--] [<path>...] git submodule [--quiet] absorbgitdirs [--] [<path>...]
Example:
$ git clone -b pulsar-x -o wr-core \ https://github.com/WindRiver-OpenSourceLabs/wr-core $ cd wr-core $ ls -F docs/ install_templates/ scripts/ init-intel-x86-env* layers/ todo/ init-intel-x86-secure-env* overc-installer/ init-raspberrypi-env* README.TXT $
At this point, the submodules are not checked out yet, so this is not taking up a lot of space:
$ du -ks . 1228 $
and there's nothing under the layers/
subdirectories:
$ tree -F layers layers ├── bitbake/ ├── external-binaries/ ├── fsl-ls10xx/ ├── intel-iot-refkit/ ├── meta-browser/ ├── meta-cloud-services/ ├── meta-dpdk/ ├── meta-flatpak/ ├── meta-gateway/ ├── meta-intel/ ├── meta-iot-cloud/ ├── meta-java/ ├── meta-openembedded/ ├── meta-overc/ ├── meta-raspberrypi/ ├── meta-secure-core/ ├── meta-security/ ├── meta-up-board/ ├── meta-virtualization/ ├── oe-core/ ├── oe-meta-go/ ├── qemuarm/ ├── qemuarma9/ ├── vendor-proprietary/ ├── wrlabs-integration/ └── xilinx-zynq/ 26 directories, 0 files $
The repository comes with a versioned .gitmodules
file, identifying where to find the submodules (relative or absolute URLs) and where to clone them locally:
$ cat .gitmodules [submodule "bitbake"] path = layers/bitbake url = ../bitbake branch = master [submodule "oe-core"] path = layers/oe-core url = ../oe-core branch = master [submodule "meta-intel"] path = layers/meta-intel url = ../meta-intel branch = master ... snip ... [submodule "meta-raspberrypi"] path = layers/meta-raspberrypi url = git://git.yoctoproject.org/meta-raspberrypi branch = master ... snip ...
So far, there's nothing in the repository's config
file about submodules:
$ cat .git/config [core] repositoryformatversion = 0 filemode = true bare = false logallrefupdates = true [remote "wr-core"] url = https://github.com/WindRiver-OpenSourceLabs/wr-core fetch = +refs/heads/*:refs/remotes/wr-core/* [branch "pulsar-x"] remote = wr-core merge = refs/heads/pulsar-x $
Initialize the submodules by setting all of the submodule.$name.url
lines in .git/config
:
$ git submodule init Submodule 'bitbake' (https://github.com/WindRiver-OpenSourceLabs/bitbake) registered for path 'layers/bitbake' Submodule 'external-binaries' (https://github.com/WindRiver-OpenSourceLabs/external-binaries) registered for path 'layers/external-binaries' Submodule 'fsl-ls10xx' (https://github.com/WindRiver-OpenSourceLabs/fsl-ls10xx) registered for path 'layers/fsl-ls10xx' ... etc etc ...
We're still not using any more space:
$ du -ks . 1228 $
but the repository's .git/config
file now knows about the submodules:
$ cat .git/config ... snip existing earlier content ... [submodule "bitbake"] active = true url = https://github.com/WindRiver-OpenSourceLabs/bitbake [submodule "external-binaries"] active = true url = https://github.com/WindRiver-OpenSourceLabs/external-binaries [submodule "fsl-ls10xx"] active = true url = https://github.com/WindRiver-OpenSourceLabs/fsl-ls10xx ... etc etc ...
Update all the initialized submodules:
$ du -ks layers/ 108 layers $
$ git submodule update Cloning into '/home/rpjday/github/wr_open_source_labs/wr-core/layers/bitbake'... Cloning into '/home/rpjday/github/wr_open_source_labs/wr-core/layers/external-binaries'... Cloning into '/home/rpjday/github/wr_open_source_labs/wr-core/layers/meta-flatpak'... ... etc etc ...
Now we have the submodules:
$ du -ks layers/ 153620 layers $ du -ks . 449876 $
One option:
$ git clone ... $ cd ... $ git submodule update --init --recursive
or simply, all in one step:
$ git clone --recurse-submodules ...
Using bitbake
as an example, the working tree is under layers/bitbake
:
$ cd layers/bitbake $ ls -F AUTHORS conf/ HEADER README bin/ contrib/ lib/ toaster-requirements.txt ChangeLog COPYING LICENSE TODO classes/ doc/ MANIFEST.in $
However, here is the .git
file:
$ cat .git gitdir: ../../.git/modules/bitbake $
The submodule Git directories themselvesare stored under .git/modules/
:
$ ls -F .git/modules/ bitbake/ meta-intel/ meta-virtualization/ external-binaries/ meta-iot-cloud/ oe-core/ fsl-ls10xx/ meta-java/ oe-meta-go/ intel-iot-refkit/ meta-openembedded/ overc-installer/ layers/ meta-overc/ qemuarm/ meta-browser/ meta-raspberrypi/ qemuarma9/ meta-cloud-services/ meta-secure-core/ vendor-proprietary/ meta-dpdk/ meta-security/ wrlabs-integration/ meta-gateway/ meta-up-board/ xilinx-zynq/ $
$ ls -F .git/modules/bitbake/ branches/ description hooks/ info/ objects/ refs/ config HEAD index logs/ packed-refs $
$ git submodule status 055865047c63b9c3b213b47a1884924ce0adeda0 layers/bitbake (1.38.0-10-g055865047) 78d4168b29efe5e812c36502a1c300d3d4c45935 layers/external-binaries (78d4168) 0a8c797caa70f517c327c67b15b0d55e902b7c44 layers/fsl-ls10xx (remotes/origin/pulsar-8) 4ca2e7474b046fa0bd7813cd6da7cf455733d61c layers/intel-iot-refkit (2017.07-258-g4ca2e74) 1cd38d701a49eade80a04140f70d3383117b9745 layers/meta-browser (remotes/origin/krogoth-266-g1cd38d7) 31e3086409b5055b8ce5962a614f1284003c513d layers/meta-cloud-services (31e3086) 8a0f847836671111915088d2838e3ff6f3c7642c layers/meta-dpdk (tmpfix-20180615) 6975858540552012d8082c74b5c5d9db8e260865 layers/meta-flatpak (6975858) 48dae0603506cff9bde674755d963e5cf3e75244 layers/meta-gateway (48dae06) fec5d45ec3978d4b7c9649bee45fe58f914bcee4 layers/meta-intel (9.0-sumo-2.5-8-gfec5d45e) 6039a7a8a9a0cad4e32736be3a05a529b65e126a layers/meta-iot-cloud (remotes/origin/pulsar-x) aa276b00d8a6bbffffd59d1980153c99521ce6ea layers/meta-java (tmpfix-20180615) a4056a96ecc857b1d675b89dc35cf9ed8a876f80 layers/meta-openembedded (tmpfix-20180618) 62f03f7d42f784923a8b02ce9d30d62d6552a22b layers/meta-overc (tmpfix-20180615) b2da4618b0ac2ad7dbc26387cad4c03796f8a06e layers/meta-raspberrypi (remotes/origin/krogoth-354-gb2da461) fb838242ad1b5c7caf5104cb3560dd6e655f1d92 layers/meta-secure-core (heads/master) f9c5e2022b54097474e46d9efb54080bd0e4d606 layers/meta-security (remotes/origin/pulsar-8-186-gf9c5e20) 024d0f5e404cb29789c2bba90aa72efd00f2a0e6 layers/meta-up-board (remotes/origin/pulsar-x) 8344dd9e9f55a5ea81f9c57c74bd9a8b9a04e8ad layers/meta-virtualization (tmpfix-20180615) b34e86b4ee13d53f09d558e613d5b66c845edde6 layers/oe-core (uninative-2.1-109-gb34e86b4ee) b29b384eae4a40933d07cc64c51e890c289d9621 layers/oe-meta-go (heads/ovp7-20150902) 3998487c37639a484fde00fd66121f7660e06c56 layers/qemuarm (remotes/origin/LB13_8.0_RCPL0001) d902f0eeec53662e30cc1ab98c5f296496d29086 layers/qemuarma9 (remotes/origin/master) 032d6a6f94297707acd228c96313a603c5702a38 layers/vendor-proprietary (032d6a6) 02e0f675db192294044ffbce6146ec25f34ffdba layers/wrlabs-integration (remotes/origin/master-oci-fli-100-g02e0f67) cd816cf2bfcf0e917e02f4cbd48800ee5a583f33 layers/xilinx-zynq (remotes/origin/pulsar-8) aabcd6ab2b69383ba4b41b7bebfc600dd3b166dd overc-installer (remotes/origin/pulsar-7.0-224-gaabcd6a) $
Add a submodule to the repo for the “Pro Git” book, which is clean and has no current submodules:
$ git status On branch master Your branch is up to date with 'rpjday/master'. nothing to commit, working tree clean $
$ git submodule status $
$ git submodule add https://github.com/boostorg/coroutine2 Cloning into '/home/rpjday/ebooks/progit/progit2/coroutine2'... remote: Enumerating objects: 2, done. remote: Counting objects: 100% (2/2), done. remote: Compressing objects: 100% (2/2), done. remote: Total 1189 (delta 0), reused 1 (delta 0), pack-reused 1187 Receiving objects: 100% (1189/1189), 292.30 KiB | 1.82 MiB/s, done. Resolving deltas: 100% (703/703), done. $
Note that this has not committed the new submodule yet. So what has it done?
$ git status On branch master Your branch is up to date with 'rpjday/master'. Changes to be committed: (use "git reset HEAD <file>..." to unstage) new file: .gitmodules new file: coroutine2 $
$ git submodule status 0233d35081de5b669c60ef8ec5a53854e1b2a577 coroutine2 (heads/develop) $
The new .gitmodules
file:
$ cat .gitmodules [submodule "coroutine2"] path = coroutine2 url = https://github.com/boostorg/coroutine2
The content of the new submodule working tree:
$ tree coroutine2/ coroutine2/ ├── doc │ ├── acknowledgements.qbk │ ├── architectures.qbk │ ├── asymmetric.qbk │ ├── coro.qbk │ ├── coroutine.qbk │ ├── images │ │ ├── event_model.dia │ │ ├── event_model.png ... etc etc ...
The content of the new submodule's Git directory:
$ ls -lF .git/modules/coroutine2/ total 48 drwxrwxr-x. 2 rpjday rpjday 4096 Feb 27 09:31 branches/ -rw-rw-r--. 1 rpjday rpjday 297 Feb 27 09:31 config -rw-rw-r--. 1 rpjday rpjday 73 Feb 27 09:31 description -rw-rw-r--. 1 rpjday rpjday 24 Feb 27 09:31 HEAD drwxrwxr-x. 2 rpjday rpjday 4096 Feb 27 09:31 hooks/ -rw-rw-r--. 1 rpjday rpjday 7299 Feb 27 09:33 index drwxrwxr-x. 2 rpjday rpjday 4096 Feb 27 09:31 info/ drwxrwxr-x. 3 rpjday rpjday 4096 Feb 27 09:31 logs/ drwxrwxr-x. 4 rpjday rpjday 4096 Feb 27 09:31 objects/ -rw-rw-r--. 1 rpjday rpjday 1161 Feb 27 09:31 packed-refs drwxrwxr-x. 5 rpjday rpjday 4096 Feb 27 09:31 refs/ $
$ git diff --cached diff --git a/.gitmodules b/.gitmodules new file mode 100644 index 0000000..d0564c5 --- /dev/null +++ b/.gitmodules @@ -0,0 +1,3 @@ +[submodule "coroutine2"] + path = coroutine2 + url = https://github.com/boostorg/coroutine2 diff --git a/coroutine2 b/coroutine2 new file mode 160000 index 0000000..0233d35 --- /dev/null +++ b/coroutine2 @@ -0,0 +1 @@ +Subproject commit 0233d35081de5b669c60ef8ec5a53854e1b2a577 $
Abbreviated form:
$ git diff --cached --submodule diff --git a/.gitmodules b/.gitmodules new file mode 100644 index 0000000..d0564c5 --- /dev/null +++ b/.gitmodules @@ -0,0 +1,3 @@ +[submodule "coroutine2"] + path = coroutine2 + url = https://github.com/boostorg/coroutine2 Submodule coroutine2 0000000...0233d35 (new submodule) $
$ git commit -a -m "added coroutine2 submodule" [master a7561cb] added coroutine2 submodule 2 files changed, 4 insertions(+) create mode 100644 .gitmodules create mode 160000 coroutine2 <--- weird mode representing submodule $
at which point you can push the result:
$ git push origin master