Skip to content

Big refactor, API incompatible changes #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 139 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
139 commits
Select commit Hold shift + click to select a range
e2fe0f3
pyflakes/pep8 compliance fixes
Oct 31, 2013
a3299e1
WIP git-fat rewrite
Oct 31, 2013
4b41593
Updated gitfat to work with legacy added files
Oct 31, 2013
c878627
Baseline for new decode function
Oct 31, 2013
7e88793
Refactored decode to return stream and bool
Nov 1, 2013
ae3830d
Added new filter-smudge. Updated to use new decode
Nov 1, 2013
3f9042d
Flake8 fixes, remove unused function, use new cmd_filter_smudge
Nov 1, 2013
dfdaf72
File reorganize and cleanup
Nov 1, 2013
534ea95
Removed redundancy
Nov 1, 2013
7e56fec
WIP refactor
Nov 2, 2013
1964ff0
Cleanup names, add list_files function
Nov 4, 2013
a7dc2b3
Added git-fat list command
Nov 5, 2013
29aeb9c
Small formatting changes
Nov 5, 2013
6208ad1
Added a comprehensive argument parser.
Nov 5, 2013
e4365ef
Adding clarifying changes from review with @rmeadows
Nov 6, 2013
acbf601
inital testing framework
Nov 6, 2013
386c310
Fixed race condition, fix coverage collection.
Nov 7, 2013
b8a5f19
No unit tests yet
Nov 7, 2013
8ba24c4
Added more thorough testing, removed gc command
Nov 8, 2013
2c2a891
updated the readme
Nov 8, 2013
3397938
Updated makefile and tests
Nov 8, 2013
f076f83
Ignore virtual environment directory
Nov 8, 2013
48649fe
fixes a deadlock condition in can_clean_file
Nov 12, 2013
d65d9fa
Fixed StopIteration test failures in can_clean_file
Nov 12, 2013
08f0c24
Fixed issue with the handling of files with spaces
Nov 15, 2013
e129b34
Fixed issue with 0 byte files already commited
Nov 19, 2013
93f9a45
Added test for previous commit
Nov 19, 2013
6fa3658
Added anonymous cloning over http.
Dec 2, 2013
d5e0564
Move checkout out of http download loop
Dec 3, 2013
326c075
Better error handling
Dec 3, 2013
7852ce6
Added myself to license file
Dec 4, 2013
a21357c
Cleaned up errors
Dec 4, 2013
2960497
Removed legacy encode
Dec 4, 2013
068341d
Clarified things
Dec 4, 2013
d8dfe7b
Added downloading message
Dec 4, 2013
56f5a5f
fix flake8 tests
Dec 4, 2013
6c4c3cd
Formatting and notes about http cloning
Dec 4, 2013
907c43f
Changed http-get to pull-http before pushing
Dec 4, 2013
526218a
Removed old test files, added python comment
Dec 4, 2013
162d0d8
Setup python packaging
Dec 4, 2013
1a27723
Fixed pyflakes
Dec 4, 2013
aba3934
Added summary
Dec 4, 2013
6c30cc4
Update readme.md to reflect changes
Dec 4, 2013
e90b46e
Reflect terminology change
Dec 4, 2013
c5443ad
Added note on setting up git fat http remotes
Dec 4, 2013
537418a
wrap 120
Dec 4, 2013
104e309
Updated filters to work with or without %f
Dec 5, 2013
174e4eb
Added legacy fat-file format support
Dec 8, 2013
a2b4871
Merge pull request #1 from cyaninc/develop
Dec 8, 2013
f461160
Release 0.1.1
Dec 8, 2013
0b16437
wrote find command to get large files in history
Dec 8, 2013
52deaab
Reimplemented retroactive import via index-filter
Dec 10, 2013
f3d3347
Added version option and version string
Dec 10, 2013
5c938c5
Custom usage script, less verbose
Dec 10, 2013
c295a88
Added readme about index-filter
Dec 10, 2013
d375fcb
Update readme, Flake8 Fixes
Dec 10, 2013
7423917
Merge pull request #2 from cyaninc/develop
Dec 10, 2013
893389f
Release 0.2.0
Dec 10, 2013
8172354
Fixed regression in git-fat converting old files +test
Dec 17, 2013
dfe8a28
Release 0.2.1
Dec 17, 2013
f884fe3
Removed -g option to git-rev-list
Dec 17, 2013
d0f1b8d
Documentation improvements for filter-branch
Dec 17, 2013
6fe8c90
Release 0.2.2
Dec 17, 2013
b0155a4
fixes issue with running git-fat script under custom isolated envs or…
dmibaranov Jan 7, 2014
bd00ae1
Merge pull request #3 from d9frog9n/develop
Jan 8, 2014
906f4d9
Enable git-fat to act like system binary
Jan 9, 2014
2d30a7d
Use relative imports instead of implicit ones
Jan 20, 2014
a9f9bcd
Reworked format strings to support 2.6
Jan 23, 2014
cc354b5
Don't require git directory to print version
Jan 23, 2014
ac10962
Release 0.2.4
Jan 23, 2014
5507221
Fixed IndexError if no argv is present
madssj Apr 2, 2014
24fac87
Merge pull request #13 from madssj/fix-indexerror
Apr 2, 2014
ca3846f
Added better testing
Jun 17, 2014
8ec4ee2
Fixed some more pylint errors
Jun 18, 2014
1738ed2
WIP
Jun 19, 2014
364d706
Lots o cleanup. Rsync remote working but untested
Jun 19, 2014
79f0a60
Added logging, converted more to the interface
Jun 19, 2014
3743680
Simplify index-filter
Jun 20, 2014
49381d5
minor logic fix
Jun 20, 2014
45be626
Some test fixes
Jun 20, 2014
5103d85
Remove old files, update pylint cfg
Jun 20, 2014
56cbe15
Favor exceptions over sys.exit(1)
Jun 27, 2014
fad93b8
Added base test case for new tests
Jun 27, 2014
9dcd73b
Added test with new format
Jun 27, 2014
885c828
Added more new tests
Jun 28, 2014
bc0895e
More test cases
Jun 28, 2014
eec4acd
fix status test
Jun 28, 2014
2379b4b
Added list and find tests
Jun 28, 2014
8e7d0c6
Removed old test_git_fat, replaced with new
Jun 28, 2014
c0a3824
Test touchups
Jun 28, 2014
b1ef2b5
Merge pull request #15 from cyaninc/backend_interface
Jun 28, 2014
d5134c7
Docs
Jun 28, 2014
9ea2644
specify backend and config in args
Jun 29, 2014
fa0448c
Only set default arguments explicitly
Jun 29, 2014
3d4c598
Updated docs
Jun 29, 2014
9af0419
Merge pull request #16 from abraithwaite/develop
Jun 29, 2014
f1fbfe4
Coverage working again
Jun 29, 2014
15499d5
Merge pull request #17 from abraithwaite/develop
Jun 29, 2014
351d1d2
Release 0.3.0
Jul 10, 2014
4ec987b
Fixed regression with git fat status
Jul 11, 2014
bb075d2
Merge pull request #18 from abraithwaite/fix-regression
Jul 11, 2014
27eb78b
Release 0.3.1
Jul 11, 2014
840d6fe
HTTP push not implmented but fails silently
Aug 8, 2014
8d676aa
Merge pull request #19 from abraithwaite/fix-http-push-silentfail
Aug 8, 2014
bec59a5
Release 0.3.2
Aug 8, 2014
6f2beff
better error message when rsync isn't in your path
jkane Sep 3, 2014
e82a028
Merge pull request #20 from jason-kane/better_rsync_failmsg
Sep 8, 2014
7dd95b2
Only do backend checks for push and pull commands
abraithwaite Sep 27, 2014
00f9904
Remove noisy pylint option
abraithwaite Sep 27, 2014
ab3998f
Merge pull request #24 from abraithwaite/fix-config
Sep 27, 2014
c24fa42
Fix index-filter with test
abraithwaite Sep 27, 2014
284d058
Merge pull request #25 from cyaninc/fix-index-filter
Sep 27, 2014
bcad2b8
Merge pull request #26 from cyaninc/develop
Oct 3, 2014
fb4c2c2
Release 0.3.3
Oct 3, 2014
a06052c
Added note about fork status
Oct 3, 2014
b651c82
Release 0.3.4
Oct 3, 2014
7cb76a9
... Spelling.
Oct 3, 2014
35cde34
Fix python quirk with os.rename sometimes failing
Nov 23, 2014
88752c2
ignore irrelevant pylint warning
Nov 24, 2014
f00336b
Support for Windows.
cztomczak Dec 24, 2014
09053a4
Minor cleanup (pull request #31).
cztomczak Dec 26, 2014
57a0611
Merge pull request #31 from CzarekTomczak/develop
abraithwaite Dec 27, 2014
d8af912
rsync: fix command to handle spaces
abraithwaite Jan 25, 2015
dc588f0
Merge pull request #35 from abraithwaite/fix-rsync-command
abraithwaite Jan 25, 2015
97a5430
Release 0.4.0
abraithwaite Jan 25, 2015
ba9ad62
Add support for rsyncd shares to the rsync backend
AndrewJDR Jan 27, 2015
8c9e569
Merge pull request #36 from ajohnson23/rsyncd_support
abraithwaite Jan 27, 2015
621cea1
Fixed bug in CopyBackend that prevented git fat pull running subseque…
mp-w Jan 28, 2015
674dc66
Merge pull request #38 from mp-w/develop
abraithwaite Jan 30, 2015
789709e
resolves #43 to improve the performance when pulling large amount of …
Apr 9, 2015
17b33cc
Merge pull request #44 from demonyangyue/issue43
abraithwaite Apr 13, 2015
9d5c921
Support partial pulls and respect GIT_SSH
lpsinger Apr 14, 2015
55081a3
Merge pull request #46 from lpsinger/rsync_improvements_2
abraithwaite Apr 17, 2015
0c3d359
Fix tests before release
abraithwaite Apr 21, 2015
616bb1a
resolves #49 to make 'git fat pull' compatible with a sparse-checkout…
Apr 27, 2015
40ee972
Merge pull request #50 from demonyangyue/issue49
abraithwaite May 6, 2015
6102f22
Merge branch 'develop'
abraithwaite May 6, 2015
2b43a1b
checkout: have my cake and eat it too
abraithwaite May 6, 2015
a25555b
Release 0.5.0
abraithwaite May 6, 2015
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix tests before release
We should really have a CI be running these. :-)
  • Loading branch information
abraithwaite committed Apr 21, 2015
commit 0c3d359a6652b7f47f3a9cebc9ee0f3d7c8456a4
41 changes: 24 additions & 17 deletions git_fat/git_fat.py
Original file line number Diff line number Diff line change
Expand Up @@ -347,7 +347,7 @@ def __init__(self, base_dir, **kwargs):
# Allow support for rsyncd servers (Looks like "remote = example.org::mybins")
ssh_user = ''
ssh_port = ''
if "::" in remote_url:
if "::" in remote_url:
self.is_rsyncd_remote = True
else:
self.is_rsyncd_remote = False
Expand Down Expand Up @@ -845,7 +845,8 @@ def checkout_all_index(self, show_orphans=False, **unused_kwargs):
# avoid unstaged changed being overwritten
if sub.check_output(["git", "ls-files", "-m"]):
print('You have unstaged changes in working directory')
print('please use "git add <file>..." to stage those changes or use "git checkout -- <file>..." to discard changes ')
print('please use "git add <file>..." to stage those changes'
' or use "git checkout -- <file>..." to discard changes')
exit(1)

for digest, fname in self._orphan_files():
Expand All @@ -860,7 +861,6 @@ def checkout_all_index(self, show_orphans=False, **unused_kwargs):
# This re-smudge is essentially a copy that restores permissions.
sub.check_call(['git', 'checkout-index', '--index', '--force', '--all'])


def checkout(self, show_orphans=False, **unused_kwargs):
'''
Update any stale files in the present working tree
Expand All @@ -875,7 +875,6 @@ def checkout(self, show_orphans=False, **unused_kwargs):
elif show_orphans:
print('Data unavailable: %s %s' % (digest, fname))


def can_clean_file(self, filename):
'''
Checks to see if the current file exists in the local repo before filter-clean
Expand Down Expand Up @@ -920,7 +919,7 @@ def pull(self, patterns=None, **kwargs):
# Make sure they're up to date
if kwargs.pop("many_binaries", False):
print('in accelerating mode')
self.checkout_all_index()
self.checkout_all_index()
else:
self.checkout()

Expand Down Expand Up @@ -1030,6 +1029,24 @@ def _configure_logging(log_level):
logger.setLevel(log_level)


def _load_backend(kwargs):
needs_backend = ('pull', 'push')
backend_opt = kwargs.pop('backend', None)
config_file = kwargs.pop('config_file', None)
backend = None
if kwargs['func'] == 'pull':
# since pull can be of the form pull [backend] [patterns], we need to check
# the first argument and insert into file patterns if it's not a backend
# this means you can't use a file pattern which is an exact match with
# a backend name (e.g. you can't have a file named copy, rsync, http, etc)
if backend_opt and backend_opt not in BACKEND_MAP:
kwargs['patterns'].insert(0, backend_opt)
backend_opt = None
if kwargs['func'] in needs_backend:
backend = _parse_config(backend=backend_opt, cfg_file_path=config_file)
return backend


def main():

parser = argparse.ArgumentParser(
Expand Down Expand Up @@ -1071,7 +1088,7 @@ def main():

sp = subparser.add_parser('pull', help='pull fatfiles from remote git-fat server')
sp.add_argument("backend", nargs="?", help='pull using given backend')
sp.add_argument("--many-binaries", dest='many_binaries', action='store_true',
sp.add_argument("--many-binaries", dest='many_binaries', action='store_true',
help='accelerate pulling a repository which contains many binaries')
sp.add_argument("patterns", nargs="*", help='files or file patterns to pull')
sp.set_defaults(func='pull')
Expand Down Expand Up @@ -1115,18 +1132,8 @@ def main():
log_level = _logging.WARNING
_configure_logging(log_level)

require_backend = ('pull', 'push')

try:
backend_opt = kwargs.pop('backend', None)
config_file = kwargs.pop('config_file', None)
backend = None
if kwargs['func'] == 'pull':
if backend_opt and backend_opt not in BACKEND_MAP:
kwargs['patterns'].insert(0, backend_opt)
backend_opt = None
if kwargs['func'] in require_backend:
backend = _parse_config(backend=backend_opt, cfg_file_path=config_file)
backend = _load_backend(kwargs) # load_backend mutates kwargs
run(backend, **kwargs)
except RuntimeError as err:
logger.error(str(err))
Expand Down