Git submodule update не работает

Git Submodule иногда ломаются, как их починить?

Наверняка если вы начали читать эту статью, то знаете, что такое git и для чего он нужен. НО не все используют такую интересную функциональность в git как submodules.

Это дает возможность прицепить к вашему проекту другие проекты и переиспользовать их в вашем. Для примера рассмотрим такую историю.

Есть репозиторий, в котором содержится groovy скрипт, который решает задачу по извлечению из системы хранения чувствительных данных Vault.

Git репозиторий с Vault

Также есть репозиторий CICD, в которых содержатся конвейеры конкретных приложений.

Git репозиторий с submodules

Для того, чтобы подключить репозиторий Vault к репозиторию CICD как submulde необходимо выполнить, зафиксировать commit и отправить push в ветку dev

в итоге появляется файл .gitsubmodules и в нем соответствующая запись

Vault локально в Intellij Idea

Если кстати решите отключить submudule , то необходимо сделать следующее

и удаляем из .gitmodules секцию [submodule «vault»]

Если вы что-то поменяли в репозитории Vault и хотите чтобы указатель в основном проекте был перемещен на самый новый коммит и изменения были отправлены в ветку master необходимо сделать следующее:

Чаше всего все работает штатно, но иногда обстоятельства складываются таким образом:

22:00 пятница и вы хотите получить именно сегодня результат, потому как потратили на этот pipeline почти целый день, просто уйти и бросить задачу — вы не только не получите свою дозу дофамина, а у вас еще выработается дополнительная доза кортизола,

в какой-то момент вы видите, что вот оно почти готово и своему коллеге по работе который почему-то тоже задержался вы говорите еще 30 минут и мы пойдем,

на 25 минуте у вас получилось наконец-то полностью доделать pipeline и протестировать deployment.yml и теперь осталось дело за малым раскомментировать stage с build и протестировать снова и в процессе внести косметическое изменение в проект в vault, вы нормально коммитите в соответствующие репозитории и на основном проекте выполняете

но что-то идет не так и когда вы запускаете Job в Jenkins возвращается ошибка

пробуете перейти в giltab по ссылке vaurt @ то вам возвращается 404 ошибка

404 ошибка при попытке перехода на git submodule vault

Для примера нашел подобные ошибки на toster.ru

https://qna.habr.com/q/131671

В ответе было написано что гуглится 10 секунд, но четкого скрипта как восстановить работоспособность найдено не было и я решил что моя история как это быстро починить репозиторий CICD будет полезна

Начинаем с клонировая репозитория

далее необходимо сделать инициализацию submodule

далее выполняем рекурсивный update

далее пробуем выполнить update именно sub модуля vault

получаем в ответ ошибку

переходим в каталог vault и проверяем та ли ветка которая нужна мы сейчас используем

ветка верная v1

далее выполняем актуализацию

мы обновили каталог с sub modules до последнего коммита repo с vault, теперь необходимо перейти в каталог с основным проектом и сохранить изменения

далее зафиксированные изменения мы отправляем в репозиторий с проектом

проверяем repo с проектом, что все сработало как нужно

commit Fix old commit with vault

переходим по ссылке после @

восстановленная ссылка git submodule на репозиторий с Vault

если до этого мы все сделали правильно, то ссылка должна открыться.

Источник

git submodule update not functioning

I have a repository with various nested submodules. Committing and pushing works pretty well and the changes are visible at GitHub as expected.

In the testing/production environments, new releases of this project are being deployed using these commands:

But this only updates the root project, none of the submodules are updated to the commits associated with the HEAD at GitHub. So far the only way I have found to update the whole project is to run git pull inside each individual submodule folder.

I understand that git submodule update is the method referenced in most places, but it is not really producing any results in this case. What could be the cause?

2 Answers 2

You need to make sure your submodules are following a branch, or they will only be checked out at a specific SHA1 (not at the latest of a branch, but the special entry of the index of your parent repo)

See «Git submodules: Specify a branch/tag» in order to make your submodule follow a branch.

Then a git submodule update —init —recursive —remote would be enough to check out the latest from that branch.

This ( git submodule update —remote ) requires git 1.8.2+ March 2013. The OP Luís de Sousa has a git 1.7.9.5 (March 2012) which doesn’t offer this feature.

You have a specific version committed to your repository. For git the submodules are «files» containing a sha1-hash of the version you provided. git submodule update —init —recursive ensures that your submodule is available in exactly that version.

• You make a git init on a directory and have a empty repo
• You add a submodule using git submodule add which will record the current masters sha1-hash of that repo you are adding in your own repository
• Multiple commits are made to the submodule but your repo will still contain that one hash you had as you added the submodule
• If you make a git pull in the submodule you will get the new commits and have an uncommitted change in your own repository (the new sha1-hash of the submodules master)
• As soon as you commit that change you are «pinning» the current version of the submodule
• git submodule update will now enforce the new version is there

So if some other contributor of your own repository updates his checkout and does a git submodule update he will get the version you pinned. So in general the submodule update will do some work only when the submodule is not checked out or if someone changed the associated hash in your repository.

Источник

Fatal Error when updating submodule using GIT

I am trying to update the submodules of this git repositary but I keep getting a fatal errors:

Any ideas of why this is happening withthe submodule? The repo is this one:

(The submodule is able to be cloned if I do not try to clone it as a submodule.)

8 Answers 8

The issue is that git can’t find the public key needed to download the repo from your server, the solution is to use the public url.

In the file .gitmodule you will find the following entry:

The URL need to be changed to the public URL for the module:

As you can see the prefix git@ has been changed to https:// and the infix : becomes /

EDIT: In your own repository you might need to use git:// rather than https://

The previous answer was unclear to me, so I added this.

EDIT 2: If you find you need to run git submodule sync or need to edit .git/config to get this to work, you have likely set up remotes for the submodules.

If it can help some people:

I update my .gitmodules

Then I update my .git/config too

Like some of you said it before (and I thank you).

Then I update my .git/modules/example/config

And to finish I do

You can manually pass in the key in Build —> «Execute shell» section of jenkins job :

ssh-agent bash -c ‘ssh-add ; git submodule update —init —recursive’

ssh-agent bash -c ‘ssh-add /var/lib/jenkins/.ssh/jenkins_rsa; git submodule update —init —recursive’

The following steps will fix the problem.

1. Delete the folder of the submodule on your local.
2. Do git submodule sync
3. then git submodule update —init

Hope this helps.

This is happened that many times for me that I put a function in my .bash_profile (works on BSD sed / GNU / Mac):

Underlying solution based on Daniël ‘s answer

I had this same issue. However, in my situation, the team wanted to use the SSH access from the .gitmodules, so modifying the URL to use http:// was not an option.

Ultimately, my issue was having an incorrect

/.ssh/config file. The config file had some erroneous settings, so I was actually trying to access the incorrect server every time I’d really want to access git@github.com. I found this out by executing the following command:

The third line or so should say this:

If you aren’t attempting to connect to github.com, then your config file is pointing you off course.

Turns out I didn’t need any of the stuff in my config file anyway, so I was safe to delete it. If you want to maintain a config file, here is a good article on that:

Also, these GitHub docs really helped me debug my issue:

Источник

Git will not init/sync/update new submodules

Here’s part of the contents of my .gitmodules file:

However, .git/config only contains the first:

The second submodule ( external/pyfacebook ) was added by another developer in a feature branch. I’ve inherited the development now, and have checked out the feature branch. However, Git will not pull the submodule for me. I’ve tried:

• git submodule init
• git submodule update
• git submodule update —init
• git submodule sync
• Removing all submodule definitions from .git/config and running git submodule init . It only copies over the previously existing submodule and ignores the new one.
• Entering new submodule definitions in .git/config manually and running git submodule update . Only the previously existing submodules bother to update.

in various combinations, but git simply will not update .git/config based on the new contents of .gitmodules , nor will it create the external/pyfacebook folder and pull the submodule’s contents.

What am I missing? Is manual intervention (adding a submodule entry by hand to .git/config ) truly required, and why?

Edit: Manual intervention does not work. Manually adding the new submodule entry to .git/config doesn’t do a thing. The new submodule is ignored.

21 Answers 21

I had this same problem — it turned out that the .gitmodules file was committed, but the actual submodule commit (i.e. the record of the submodule’s commit ID) wasn’t.

Adding it manually seemed to do the trick — e.g.:

(Even without removing anything from .git/config or .gitmodules.)

Then commit it to record the ID properly.

Adding some further comments to this working answer: If the git submodule init or git submodule update does’nt work, then as described above git submodule add url should do the trick. One can cross check this by

and one should get an entry of the submodule you want to pull in the result of the git config —list command. If there is an entry of your submodule in the config result, then now the usual git submodule update —init should pull your submodule. To test this step, you can manually rename the submodule and then updating the submodule.

To find out if you have local changes in the submodule, it can be seen via git status -u ( if you want to see changes in the submodule ) or git status —ignore-submodules ( if you dont want to see the changes in the submodule ).

git version 2.7.4. This command updates local code git submodule update —init —force —remote

Had the same issue, when git ignored init and update commands, and does nothing.

HOW TO FIX

1. Your submodule folder should be committed into git repo
2. It shouldn’t be in .gitignore

If that requirements met, it will work. Otherwise, all commands will execute without any messages and result.

If you did all that, and it still doesn’t work:

1. Add submodule manually, e.g. git submodule add git@. path/to
2. git submodule init
3. git submodule update
4. commit and push all files — .gitmodules and your module folder (note, that content of folder will not commit)
5. drop your local git repo
6. clone a new one
7. ensure that .git/config doesn’t have any submodules yet
8. Now, git submodule init — and you will see a message that module registered
9. git submodule update — will fetch module
10. Now look at .git/config and you will find registered submodule

There seems to be a lot of confusion here (also) in the answers.

git submodule init is not intended to magically generate stuff in .git/config (from .gitmodules). It is intended to set up something in an entirely empty subdirectory after cloning the parent project, or pulling a commit that adds a previously non-existing submodule.

In other words, you follow a git clone of a project that has submodules (which you will know by the fact that the clone checked out a .gitmodules file) by a git submodule update —init —recursive .

You do not follow git submodule add . with a git submodule init (or git submodule update —init ), that isn’t supposed to work. In fact, the add will already update the appropriate .git/config if things work.

EDIT

If a previously non-existing git submodule was added by someone else, and you do a git pull of that commit, then the directory of that submodule will be entirely empty (when you execute git submodule status the new submodule’s hash should be visible but will have a — in front of it.) In this case you need to follow your git pull also with a git submodule update —init (plus —recursive when it’s a submodule inside a submodule) in order to get the new, previously non-existing, submodule checked out; just like after an initial clone of a project with submodules (where obviously you didn’t have those submodules before either).

Thinking that manually setting up .gitmodules is enough is WRONG

My local git version 2.22.0 as of this writing.

So I came to this thread wondering why wasn’t git submodule init working; I setup the .gitmodules file and proceeded to do git submodule init .

IMPORTANT

git submodule add company/project.git includes/project is required (when adding the module for the first time), this will:

• add config to .git/config
• update the .gitmodules file
• track the submodule location ( includes/project in this example).

you must then git commit after you have added the submodule, this will commit .gitmodules and the tracked submodule location.

When the project is cloned again, it will have the .gitmodules and the empty submodules directory (e.g. includes/project in this example). At this point .git/config does not have submodule config yet, until git submodule init is run, and remember this only works because .gitmodules AND includes/project are tracked in the main git repo.

Also for reference see:

I had the same problem but none of the solutions above helped. The entries in the .gitmodules and in .git/config were right but the command git submodules update —init —recursive was doing nothing. I also removed the submodule directory and did run git submodules update —init —recursive and got the submodule directory back but with exactly the same commit as before.

I found the answer on this page. The command is: git submodule update —remote

Sort of magically, but today I ran git submodule init followed by git submodule sync followed by git submodule update and it started pulling my submodules. Magic? Perhaps! This is truly one of the most annoying experiences with Git…

Scratch that. I actually got it working by doing git submodule update —init —recursive . Hope this helps.

PS: Make sure you are in the root git directory, not the submodule’s.

According to the answer from Dave James Miller I can confirm that it worked for me. The important thing here was to commit the subprojects commit ID. Just to have the entry in .gitmodules was not enough.

Here is an appropriate commit:

I had the same problem.

.gitmodules had the submodule, but after a git submodule init command it wasn’t in .git/config .

Turns out the developer who added the submodule also added the submodule directory to the .gitignore file. That doesn’t work.

This means the submodules haven’t been set up correctly and a git submodule add command will have to be executed. A detailed explanation of how submodules work:

If submodules were not added correctly:

If the submodules are not committed correctly, and the submodules and their corresponding folders are not already recorded in the index, then first every submodule in the .gitmodules file needs to be individually added via git submodule add before one can proceed.

• Note: as mentioned here, Git does not currently have a command to do this at once for multiple submodules present in a .gitmodules file.

What does git submodule add do?

When a repo with submodules has been set up correctly and someone has already performed a git submodule add command, the command has done the following things:

1. create a folder for the submodule to reside in if it doesn’t already exist
2. clone the project as a submodule
3. set the submodule parameters in the shareable .gitmodules file
4. set the submodule parameters in your private .git/config file
5. create a .git folder for the submodule in your superproject’s private .git/modules folder
6. and also: record in your superproject’s index for each submodule in which folder it will reside, and in what state it should be (the hash commit code of the submodule.)

Point 3 and 6 are relevant when you clone the superproject, and point 6 indicates whether a git submodule add has been performed correctly and was committed. You can check point 6 by seeing whether the folders in which the submodules should reside are already there and empty (Note: this does not require a .keep or .gitignore mechanism to be in place, since this is part of a Git mechanism.) After cloning the superproject you can also perform a git submodule to see which submodules are expected.

You can then proceed by doing:

will show the submodules present in the tree and their corresponding commit hash code, can serve as an initial check to see which submodules are expected

git submodule init

copies the submodules parameters from the repo’s .gitmodules file to your private .git/config file (point 4)

git submodule update

clones the submodules to a commit determined by the superproject and creates the submodules’ .git folders under your superproject’s .git/modules/ folder (points 2 and 5)

git submodule update —remote

same as update but sets the submodule to the latest commit on the branch available by the remote repo, similar as going in each submodule’s folder afterwards and do a git pull

or: git submodule update —init —remote

which is all of the above combined.

Alternatively, when the repo is set up correctly, you can also do a git clone with the —recursive flag to include the submodules with the init and update performed on them automatically.

Источник