Compare Revisions - nigel.stanger/Wiki

nigel.stanger / Wiki

Compare Revisions
View Page Back to Page History

Freaky Git stuff.md
## LFS ### Migrate a large repository to LFS in GitBucket 1. Create a new repository `newrepo` on the server by cloning the old one. 1. Clone `newrepo` to local. 1. Initialise `git-lfs` in `newrepo` as per the [git-lfs tutorial](https://github.com/git-lfs/git-lfs/wiki/Tutorial#adding-git-lfs-to-a-pre-existing-repo): ```sh git lfs track ".thing1" ".thing2" git add .gitattributes git commit -m "Track thing1, thing2 files (LFS)" git rm --cached ".thing1" ".thing2" git add . git commit -m "Convert last commit to LFS" ``` 1. Migrate the history as per the [git-lfs tutorial](https://github.com/git-lfs/git-lfs/wiki/Tutorial#migrating-existing-repository-data-to-lfs): ```sh git lfs migrate import --everything --include=".thing1,.thing2" ``` If you specify multiple `--include` switches only the last one will take effect, so to migrate multiple file types, put the glob expressions inside the same string as shown above. 1. We now need to force push `newrepo` to remote, but it may be too large for nginx’s request size limits. If you happen to have release tags handily scattered throughout history (or you can identify some well-spaced commit hashes) you can reduce the request size by only [pushing up to a certain commit](https://coderwall.com/p/hexinq/git-push-up-to-a-certain-commit), e.g.: ```sh git push --force origin 9d42ad26508d94429ce2179358db5c2bfc1dc03d:master # ...etc... # one last push to catch everything since the last tag: git push --force ``` Do this for each of the tags/commits in chronological order. You can find the commit hashes for the tags using `git show-ref --tags`. 1. Also force push the rewritten tags to prevent push/pull errors due to the different commit hashes: ```sh git push --force --tags ``` 1. Clean up local as per the [git-lfs tutorial](https://github.com/git-lfs/git-lfs/wiki/Tutorial#cleaning-up-the-git-directory-after-migrating): ```sh git reflog expire --expire-unreachable=now --all git gc --prune=now ``` ### Migrate LFS repo to a different remote Normally you just clone the old remote, create the new remote on GitBucket, then `git push -u origin master` (and maybe `git push --tags`). However this isn’t aware of LFS files, so it will only push the pointers and all your LFS files will be missing on the remote. The correct procedure: 1. `git clone <old remote>` 1. `git lfs fetch --all` in the local clone to make sure the clone has all of the LFS objects (including historical versions). If you don’t do this the LFS push later on may fail due to missing objects. 1. `git remote set-url <new remote>` in the local clone. 1. `git push -u origin master` as usual in the local clone to push all the non-LFS objects to the remote. 1. `git push --tags` in the local clone if needed. 1. `git lfs push --all origin master` in the local clone to push all the LFS objects to the remote. If this is too large you can use the same tag-based partial push trick as in the previous section. It may or may not like having `:master` on the end — I found it worked fine with just a commit hash. ## Finding repositories that haven’t been pushed Something along these lines (from comments on <https://stackoverflow.com/a/12499489>): ```sh find . -name .git -type d -print -exec git --git-dir={} --work-tree={}/.. cherry -v \; ``` ## Migrate a wiki repo to another GitBucket server 1. Clone the original repository that contains the wiki (`old-repo`). 1. Clone the actual wiki itself (`old-wiki`). 1. Create empty `new-repo` on new server. 1. Transfer ownership of `new-repo` if necessary. Do this before any of the remaining steps. 1. In `old-repo` clone: ```sh git remote set-url origin <new-repo-url> git push ``` 1. In `old-wiki` clone: ```sh git remote set-url origin <new-wiki-url> git push -f ``` 1. Update wiki link in `README.md` of `new-repo`. 1. Profit! ## Ignoring local changes A common problem with PDF outputs under Gradle is that they frequently end up being marked as out of date even though the original source (e.g., PlantUML or SVG) hasn’t changed. (I think this is because the PDF embeds metadata like its creation date and is therefore technically different every time.) This is annoying if the output is under version control. To avoid accidentally committing and pushing such changes, it’s useful to tell Git to ignore the changes (<https://compiledsuccessfully.dev/git-skip-worktree/>): ```sh git update-index --skip-worktree <file> ``` This isn’t the intended use of `skip-worktree`; it’s really designed for sparse checkouts. However, it works quite well for the case of “I want to have this file checked out but not to worry about changes most of the time”. This is as opposed to `git update-index --assume-unchanged`, which assumes that the marked file shouldn’t be changed (<https://stackoverflow.com/a/13631525>.) According to the Compiled Successfully link `git pull` should still work. If you do need to commit the changes to the file, you will need to undo the ignorance: ```sh git update-index --no-skip-worktree <file> ``` To list all skipped files: ```sh git ls-files -v\|grep '^S' ``` ## Managing a master/release branch (e.g., for assignments) Consider the INFO 201 project, privately developed on `isgb/teaching` and publicly released on `isgb/info201`. Once the repos are set up, the simplest way to manage the release process without accidentally pushing to the wrong repo is to have local `master` track the remote `master` on `isgb/teaching` (which should already be the case if things are correctly configured) and a local `release` branch tracking remote `master` on `isgb/info201`. Use `git -vv` to check which branches are tracking which remotes. ```sh git push -u origin master git remote set-url origin https://isgb.otago.ac.nz/teaching/git/INFO201/project.git git remote set-url release https://isgb.otago.ac.nz/info201/git/shared/project.git git push -u origin master git branch release git push -u release release:master ``` Development occurs on on local `master` and local `master` is pushed only to `origin/master`. Release only occurs when local `master` is merged onto local `release`. A plain `git push` therefore can’t accidentally release things. You have to explicitly merge, checkout local `release` and push from there: ```sh git checkout release git merge master git push release HEAD:master ``` Having different branch names for the release version is an extra layer of defence, as a plain `git push` will yell at you when the local and remote branch names are different. ## LFS ### Migrate a large repository to LFS in GitBucket 1. Create a new repository `newrepo` on the server by cloning the old one. 1. Clone `newrepo` to local. 1. Initialise `git-lfs` in `newrepo` as per the [git-lfs tutorial](https://github.com/git-lfs/git-lfs/wiki/Tutorial#adding-git-lfs-to-a-pre-existing-repo): ```sh git lfs track ".thing1" ".thing2" git add .gitattributes git commit -m "Track thing1, thing2 files (LFS)" git rm --cached ".thing1" ".thing2" git add . git commit -m "Convert last commit to LFS" ``` 1. Migrate the history as per the [git-lfs tutorial](https://github.com/git-lfs/git-lfs/wiki/Tutorial#migrating-existing-repository-data-to-lfs): ```sh git lfs migrate import --everything --include=".thing1,.thing2" ``` If you specify multiple `--include` switches only the last one will take effect, so to migrate multiple file types, put the glob expressions inside the same string as shown above. 1. We now need to force push `newrepo` to remote, but it may be too large for nginx’s request size limits. If you happen to have release tags handily scattered throughout history (or you can identify some well-spaced commit hashes) you can reduce the request size by only [pushing up to a certain commit](https://coderwall.com/p/hexinq/git-push-up-to-a-certain-commit), e.g.: ```sh git push --force origin 9d42ad26508d94429ce2179358db5c2bfc1dc03d:master # ...etc... # one last push to catch everything since the last tag: git push --force ``` Do this for each of the tags/commits in chronological order. You can find the commit hashes for the tags using `git show-ref --tags`. 1. Also force push the rewritten tags to prevent push/pull errors due to the different commit hashes: ```sh git push --force --tags ``` 1. Clean up local as per the [git-lfs tutorial](https://github.com/git-lfs/git-lfs/wiki/Tutorial#cleaning-up-the-git-directory-after-migrating): ```sh git reflog expire --expire-unreachable=now --all git gc --prune=now ``` ### Migrate LFS repo to a different remote Normally you just clone the old remote, create the new remote on GitBucket, then `git push -u origin master` (and maybe `git push --tags`). However this isn’t aware of LFS files, so it will only push the pointers and all your LFS files will be missing on the remote. The correct procedure: 1. `git clone <old remote>` 1. `git lfs fetch --all` in the local clone to make sure the clone has all of the LFS objects (including historical versions). If you don’t do this the LFS push later on may fail due to missing objects. 1. `git remote set-url <new remote>` in the local clone. 1. `git push -u origin master` as usual in the local clone to push all the non-LFS objects to the remote. 1. `git push --tags` in the local clone if needed. 1. `git lfs push --all origin master` in the local clone to push all the LFS objects to the remote. If this is too large you can use the same tag-based partial push trick as in the previous section. It may or may not like having `:master` on the end — I found it worked fine with just a commit hash. ## Finding repositories that haven’t been pushed Something along these lines (from comments on <https://stackoverflow.com/a/12499489>): ```sh find . -name .git -type d -print -exec git --git-dir={} --work-tree={}/.. cherry -v \; ``` ## Migrate a wiki repo to another GitBucket server 1. Clone the original repository that contains the wiki (`old-repo`). 1. Clone the actual wiki itself (`old-wiki`). 1. Create empty `new-repo` on new server. 1. Transfer ownership of `new-repo` if necessary. Do this before any of the remaining steps. 1. In `old-repo` clone: ```sh git remote set-url origin <new-repo-url> git push ``` 1. In `old-wiki` clone: ```sh git remote set-url origin <new-wiki-url> git push -f ``` 1. Update wiki link in `README.md` of `new-repo`. 1. Profit! ## Ignoring local changes A common problem with PDF outputs under Gradle is that they frequently end up being marked as out of date even though the original source (e.g., PlantUML or SVG) hasn’t changed. (I think this is because the PDF embeds metadata like its creation date and is therefore technically different every time.) This is annoying if the output is under version control. To avoid accidentally committing and pushing such changes, it’s useful to tell Git to ignore the changes (<https://compiledsuccessfully.dev/git-skip-worktree/>): ```sh git update-index --skip-worktree <file> ``` This isn’t the intended use of `skip-worktree`; it’s really designed for sparse checkouts. However, it works quite well for the case of “I want to have this file checked out but not to worry about changes most of the time”. This is as opposed to `git update-index --assume-unchanged`, which assumes that the marked file shouldn’t be changed (<https://stackoverflow.com/a/13631525>.) According to the Compiled Successfully link `git pull` should still work. If you do need to commit the changes to the file, you will need to undo the ignorance: ```sh git update-index --no-skip-worktree <file> ``` To list all skipped files: ```sh git ls-files -v\|grep '^S' ``` ## Managing a master/release branch (e.g., for assignments) Consider the INFO 201 project, privately developed on `isgb/teaching` and publicly released on `isgb/info201`. Once the repos are set up, the simplest way to manage the release process without accidentally pushing to the wrong repo is to have local `master` track the remote `master` on `isgb/teaching` (which should already be the case if things are correctly configured) and a local `release` branch tracking remote `master` on `isgb/info201`. Use `git -vv` to check which branches are tracking which remotes. ```sh git push -u origin master git remote set-url origin https://isgb.otago.ac.nz/teaching/git/INFO201/project.git git remote set-url release https://isgb.otago.ac.nz/info201/git/shared/project.git git push -u origin master git branch release git push -u release release:master ``` Development occurs on on local `master` and local `master` is pushed only to `origin/master`. Release only occurs when local `master` is merged onto local `release`. A plain `git push` therefore can’t accidentally release things even though it will push to both remotes, as local `release` won’t change until explicitly merged. When it’s time to release: ```sh git checkout release git merge master git push ```

Freaky Git stuff.md

## LFS

### Migrate a large repository to LFS in GitBucket

1. Create a new repository `newrepo` on the server by cloning the old one.
1. Clone `newrepo` to local.
1. Initialise `git-lfs` in `newrepo` as per the [git-lfs tutorial](https://github.com/git-lfs/git-lfs/wiki/Tutorial#adding-git-lfs-to-a-pre-existing-repo):
```sh
git lfs track "*.thing1" "*.thing2"
git add .gitattributes
git commit -m "Track thing1, thing2 files (LFS)"
git rm --cached "*.thing1" "*.thing2"
git add .
git commit -m "Convert last commit to LFS"
```
1. Migrate the history as per the [git-lfs tutorial](https://github.com/git-lfs/git-lfs/wiki/Tutorial#migrating-existing-repository-data-to-lfs):
```sh
git lfs migrate import --everything --include="*.thing1,*.thing2"
```
  If you specify multiple `--include` switches only the last one will take effect, so to migrate multiple file types, put the glob expressions inside the same string as shown above.
1. We now need to force push `newrepo` to remote, but it may be too large for nginx’s request size limits. If you happen to have release tags handily scattered throughout history (or you can identify some well-spaced commit hashes) you can reduce the request size by only [pushing up to a certain commit](https://coderwall.com/p/hexinq/git-push-up-to-a-certain-commit), e.g.:
```sh
git push --force origin 9d42ad26508d94429ce2179358db5c2bfc1dc03d:master
# ...etc...
# one last push to catch everything since the last tag:
git push --force
```
  Do this for each of the tags/commits in chronological order. You can find the commit hashes for the tags using `git show-ref --tags`.
1. Also force push the rewritten tags to prevent push/pull errors due to the different commit hashes:
```sh
git push --force --tags
```
1. Clean up local as per the [git-lfs tutorial](https://github.com/git-lfs/git-lfs/wiki/Tutorial#cleaning-up-the-git-directory-after-migrating):
```sh
git reflog expire --expire-unreachable=now --all
git gc --prune=now
```

### Migrate LFS repo to a different remote

Normally you just clone the old remote, create the new remote on GitBucket, then `git push -u origin master` (and maybe `git push --tags`). However this isn’t aware of LFS files, so it will only push the pointers and all your LFS files will be missing on the remote. The correct procedure:

1. `git clone <old remote>`
1. `git lfs fetch --all` in the local clone to make sure the clone has **all** of the LFS objects (including historical versions). If you don’t do this the LFS push later on may fail due to missing objects.
1. `git remote set-url <new remote>` in the local clone.
1. `git push -u origin master` as usual in the local clone to push all the non-LFS objects to the remote.
1. `git push --tags` in the local clone if needed.
1. `git lfs push --all origin master` in the local clone to push all the LFS objects to the remote. If this is too large you can use the same tag-based partial push trick as in the previous section. It may or may not like having `:master` on the end — I found it worked fine with just a commit hash.

## Finding repositories that haven’t been pushed

Something along these lines (from comments on <https://stackoverflow.com/a/12499489>):

```sh
find . -name .git -type d -print -exec git --git-dir={} --work-tree={}/.. cherry -v \;
```

## Migrate a wiki repo to another GitBucket server

1. Clone the original repository that contains the wiki (`old-repo`).
1. Clone the actual wiki itself (`old-wiki`).
1. Create empty `new-repo` on new server.
1. Transfer ownership of `new-repo` if necessary. **Do this before any of the remaining steps.**
1. In `old-repo` clone:
```sh
git remote set-url origin <new-repo-url>
git push
```
1. In `old-wiki` clone:
```sh
git remote set-url origin <new-wiki-url>
git push -f
```
1. Update wiki link in `README.md` of `new-repo`.
1. Profit!

## Ignoring local changes

A common problem with PDF outputs under Gradle is that they frequently end up being marked as out of date even though the original source (e.g., PlantUML or SVG) hasn’t changed. (I think this is because the PDF embeds metadata like its creation date and is therefore technically different every time.) This is annoying if the output is under version control. To avoid accidentally committing and pushing such changes, it’s useful to tell Git to ignore the changes (<https://compiledsuccessfully.dev/git-skip-worktree/>):

```sh
git update-index --skip-worktree <file>
```

This isn’t the intended use of `skip-worktree`; it’s really designed for sparse checkouts. However, it works quite well for the case of “I want to have this file checked out but not to worry about changes most of the time”. This is as opposed to `git update-index --assume-unchanged`, which assumes that the marked file **shouldn’t** be changed (<https://stackoverflow.com/a/13631525>.)

According to the Compiled Successfully link `git pull` should still work.

**If you do need to commit the changes to the file,** you will need to undo the ignorance:

```sh
git update-index --no-skip-worktree <file>
```

To list all skipped files:

```sh
git ls-files -v|grep '^S'
```

## Managing a master/release branch (e.g., for assignments)

Consider the INFO 201 project, privately developed on `isgb/teaching` and publicly released on `isgb/info201`. Once the repos are set up, the simplest way to manage the release process without accidentally pushing to the wrong repo is to have local `master` track the remote `master` on `isgb/teaching` (which should already be the case if things are correctly configured) and a local `release` branch tracking remote `master` on `isgb/info201`. Use `git -vv` to check which branches are tracking which remotes.

```sh
git push -u origin master
git remote set-url origin https://isgb.otago.ac.nz/teaching/git/INFO201/project.git
git remote set-url release https://isgb.otago.ac.nz/info201/git/shared/project.git
git push -u origin master
git branch release
git push -u release release:master
```

Development occurs on on local `master` and local `master` is pushed only to `origin/master`. Release only occurs when local `master` is merged onto local `release`. A plain `git push` therefore can’t accidentally release things. You have to explicitly merge, checkout local `release` and push from there:

```sh
git checkout release
git merge master
git push release HEAD:master
```

Having different branch names for the release version is an extra layer of defence, as a plain `git push` will yell at you when the local and remote branch names are different.