This article describes how to turn the subfolders in a git repository into new git repositories, keeping the history.
In our case we have a big git repository (2GB) with slides for a lot of courses. Every course was a subdirectory in the repository. So even if you hold just one course, you have to clone everything. That’s why we decided to put every course into a separate repository. We use SCM-Manager for our repository and could therefore use its REST api in the script.
The whole procedure is based on the git command filter-branch (see https://www.kernel.org/pub/software/scm/git/docs/git-filter-branch.html). We use the
--subdirectory-filter option to turn a subdirectory into a new project root. So here we go:
- Get the list of subdirectories. This could be done in several ways: Clone your repository and run a find/ls or use an api, if your git repository provides something like this. This is the first block in our script. If you like, you could run this as a separate step and change the directory list as you like.
- For every subdirectory
- clone the repo
- run the filter
- create a new repository with Access rights (owner and group write here)
- push your changes to the new repository
- clean up. After every
git filter-branchrun we have to clone the original repository again, because the filter changes your local repository.
NOTE: The sub directory name could contain spaces or even umlauts. So we have to replace that to get a valid repository name.
sedis our choice here.
So the complete script is:
#!/bin/sh #get directory list git clone https://git.spree.de/scm-manager/git/presentations/schulungen.git schulungen.git cd schulungen.git ls -1d */ | sed 's#/##' > ../dirs.txt cd .. rm schulungen.git # filenames with space - so dont use a for-loop cat dirs.txt | while read i do REPO=$i REPO_NAME=`echo $REPO | sed 's/\([\ä\ö\ü\Ä\Ü\Ö]\)/\1e/g;y/\ä\ö\ü\Ä\Ö\Ü/aouAOU/;s/\ß/ss/g;s/ /-/g'` #filter git clone https://git.spree.de/scm-manager/git/presentations/schulungen.git schulungen.git cd schulungen.git git filter-branch --subdirectory-filter "$REPO" -- --all #create new repository DATA="<repositories><name>presentations/Schulungen/$REPO_NAME.git</name><type>git</type><permissions><groupPermission>true</groupPermission><name>ats</name><type>WRITE</type></permissions><permissions><groupPermission>false</groupPermission><name>rmagnus</name><type>OWNER</type></permissions><description>Schulungen $REPO</description></repositories>" curl -s -XPOST -n -H 'content-type: application/xml' -d "$DATA" https://git.spree.de/scm-manager/api/rest/repositories.xml # add the new repository as remote git remote add schulung https://git.spree.de/scm-manager/git/presentations/schulungen/$REPO_NAME.git # and push git push schulung #clean up cd .. rm schulungen.git done