This repository was archived by the owner on Sep 11, 2020. It is now read-only.
remote: speed up pushes when the "remote" repo is local #1066
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
strib
force-pushed
the
strib/909-git-push-speedup-when-local
branch
from
5cd912d to
f8e68ee
Compare
`ObjectsWithStorageForIgnores` is the same as `Objects`, but a secondary storage layer can be provided, to be used to finding the full set of objects to be ignored while finding the reachable objects. This is useful when the main `s` storage layer is slow and/or remote, while the ignore list is available somewhere local. Issue: src-d#909 Signed-off-by: Jeremy Stribling <[email protected]>
This factors out some URL-parsing code from the transport layer so it can be used by config as well. Issue: src-d#909 Signed-off-by: Jeremy Stribling <[email protected]>
Issue: src-d#909 Signed-off-by: Jeremy Stribling <[email protected]>
strib
force-pushed
the
strib/909-git-push-speedup-when-local
branch
from
f8e68ee to
3889c64
Compare
jfontan
approved these changes
Feb 13, 2019
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As explained in #909, a valid use case for
Remote.PushContext()is to set the storage field to something backed by a slow storage layer (i.e., a distributed file system), and push to a "remote" that's actually backed by a local git repo in the file system. The code currently gets the list of known advertised references from the remote, but then computes the entire set of ignored commits using the provided storage layer, which can be very slow.Instead, this PR introduces an optimization where, if the "remote" URL is actually local, the push code will construct a new filesystem-based storage layer for the URL and use it to construct the ignore list. This results in significant performance improvement in some cases.
When used with the Keybase git remote helper, this brought down the time to fetch/pull a single commit in an example repo with a large history from 3m48s to less than 5s.
This PR also moves around the URL-parsing code so that it's available in a more standard util library.
Issue: #909