Skip to content

Finish initial typing of Index and Submodule #1285

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 73 commits into from
Jul 11, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
16f0607
Improve typing of config_levels, add assert_never()
Yobmod Jul 5, 2021
9400246
Fix IndexFile forwardref
Yobmod Jul 5, 2021
e4caa80
put typing_extensions.get_types() behind python version guard
Yobmod Jul 5, 2021
0939e38
fix is_config_level for < 3.8
Yobmod Jul 5, 2021
0fc93b5
Rmv is_config_level() and get_args(), not worth the trouble
Yobmod Jul 5, 2021
53f1195
Add Literal_config_levels.__args__
Yobmod Jul 5, 2021
41e9781
Improve BlameEntry.commit typing
Yobmod Jul 5, 2021
23b5d6b
Add types to submodule.util.py
Yobmod Jul 5, 2021
c2317a7
Make bytesIO forwardref
Yobmod Jul 5, 2021
a935134
Add types to submodule.root.py
Yobmod Jul 5, 2021
3ce319f
Add types to submodule.update()
Yobmod Jul 6, 2021
6471018
Improve types of @unbare_repo and @git_working_dir decorators
Yobmod Jul 6, 2021
fb09bfa
Improve types of diff.py
Yobmod Jul 6, 2021
2a6a2e2
Improve types of diff.py
Yobmod Jul 6, 2021
3578355
Add cast(Repo, mrepo) in try block
Yobmod Jul 6, 2021
1bcccd5
Fix Literal Typeguards
Yobmod Jul 6, 2021
278a371
Fix for mrepo
Yobmod Jul 6, 2021
deafa6a
Fix for mrepo2
Yobmod Jul 6, 2021
e6f340c
Rmv runtime_checkable < py3.8
Yobmod Jul 6, 2021
ed58e2f
Rmv runtime_checkable < py3.8 pt2
Yobmod Jul 6, 2021
eecf148
Rmv root.py types
Yobmod Jul 6, 2021
a857b97
Merge branch 'gitpython-developers:main' into main
Yobmod Jul 6, 2021
b78cca1
Rmv base.py types
Yobmod Jul 6, 2021
8ef1adb
Merge branch 'main' of https://github.com/Yobmod/gitpython
Yobmod Jul 6, 2021
6aebb73
Rmv submodule types
Yobmod Jul 6, 2021
c0ab23e
Rmv submodule types2
Yobmod Jul 6, 2021
8d2a770
Rmv diff typeguard
Yobmod Jul 6, 2021
1fd9e8c
Re-add submodule.util.py types
Yobmod Jul 6, 2021
1eceb89
Fix submodule.util.py types
Yobmod Jul 6, 2021
215abfd
Readd typeguard to Diff.py
Yobmod Jul 6, 2021
94c2ae4
Readd submodule.base.py types
Yobmod Jul 6, 2021
06eca0b
Make subodule a forward ref in Index.base
Yobmod Jul 6, 2021
33ffd0b
Make subodule a forward ref in Index.base2
Yobmod Jul 6, 2021
af7cee5
Make Repo a forward ref in Submodule.base
Yobmod Jul 6, 2021
f372187
Make subodule a forward ref in Index.base3
Yobmod Jul 6, 2021
de36cb6
UnMake subodule a forward ref in Index.base
Yobmod Jul 6, 2021
3cc0edc
UnMake subodule a forward ref in Index.base2
Yobmod Jul 6, 2021
28bde39
Type index _items_to_rela_paths()
Yobmod Jul 6, 2021
1d0e666
Check change_levels (should fail)
Yobmod Jul 6, 2021
e985851
Add 'U' to change_levels (should pass)
Yobmod Jul 6, 2021
873ebe6
Make diff.DiffIndex generic List['Diff']
Yobmod Jul 6, 2021
2e2fe18
Increase mypy strictness (no_implicit_optional & warn_redundant_casts…
Yobmod Jul 8, 2021
5d3818e
Finish initial typing of index folder
Yobmod Jul 8, 2021
9f88796
Mak GitCmdObjectDB a froward ref
Yobmod Jul 8, 2021
1533596
Mak EntryTup a froward ref
Yobmod Jul 8, 2021
4333dcb
Mmmmm
Yobmod Jul 8, 2021
fe5fef9
Mmmmmm
Yobmod Jul 8, 2021
d344abf
Fix traverse_trees_recursive()
Yobmod Jul 8, 2021
dfbc0f4
Fix traverse_trees_recursive() again
Yobmod Jul 8, 2021
c27d2b0
Use Tuple not tuple
Yobmod Jul 8, 2021
4f13b4e
fix base,ours,theirs typing
Yobmod Jul 8, 2021
627deff
Change List to MutableSequence in fun.py _find_by_name()
Yobmod Jul 8, 2021
f271c58
tests TraversableIterableObj typeguard
Yobmod Jul 8, 2021
4802a36
improve TraversableIterableObj typeguard
Yobmod Jul 8, 2021
1faa25f
Rmv typeguard from list_traverse(), was wrong
Yobmod Jul 8, 2021
f4cb7db
Change type of list_traverse() again.
Yobmod Jul 9, 2021
030b1fd
Add list_traverse() to Tree and TraversableIterableObj.
Yobmod Jul 9, 2021
3710e24
Rmv circular import, create Has_id_attribute Protocol instead
Yobmod Jul 9, 2021
5eea891
Fix list_traverse() docstring for Autodoc
Yobmod Jul 9, 2021
9377462
Make has_repo protocol runtime checkable and use in Diffable
Yobmod Jul 9, 2021
3c6deb0
Flatten list_traverse()
Yobmod Jul 9, 2021
a024bdd
Move TraverseNT to global, cos mypy complained on testing
Yobmod Jul 9, 2021
6271660
Rmv submodule.base Repo assert
Yobmod Jul 9, 2021
7c6ae2b
Try to distinguation git.diff module from diff.Diff.diff and diff.Daf…
Yobmod Jul 9, 2021
f916c14
Improve Diffable method typing
Yobmod Jul 9, 2021
e7b685d
Rmv Diffable assert, add Remoote.url property
Yobmod Jul 9, 2021
9bb630f
Add remote.url type
Yobmod Jul 9, 2021
b03af05
Remove defsult_index decorator from diff() and do check within functi…
Yobmod Jul 9, 2021
797e962
Make IndexFile and Diffable .diff() types agree
Yobmod Jul 9, 2021
09053c5
Improve IndexFile_process_diff_args() to get checks to rerun
Yobmod Jul 9, 2021
2ea528e
Fix typing of index.fun.write_tree_from_cache()
Yobmod Jul 9, 2021
e6a27ad
Use TreeCacheTup type alias throughout
Yobmod Jul 9, 2021
94c6652
Make TreeCacheTup forward ref
Yobmod Jul 9, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Readd submodule.base.py types
  • Loading branch information
Yobmod committed Jul 6, 2021
commit 94c2ae405ba635e801ff7a1ea00300e51f3a70db
5 changes: 3 additions & 2 deletions git/diff.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,8 @@


def is_change_type(inp: str) -> TypeGuard[Lit_change_type]:
return inp in ['A', 'D', 'C', 'M', 'R', 'T']
return True
# return inp in ['A', 'D', 'C', 'M', 'R', 'T']

# ------------------------------------------------------------------------

Expand Down Expand Up @@ -511,7 +512,7 @@ def _handle_diff_line(lines_bytes: bytes, repo: 'Repo', index: DiffIndex) -> Non
# Change type can be R100
# R: status letter
# 100: score (in case of copy and rename)
assert is_change_type(_change_type[0])
assert is_change_type(_change_type[0]), f"Unexpected value for change_type received: {_change_type[0]}"
change_type: Lit_change_type = _change_type[0]
score_str = ''.join(_change_type[1:])
score = int(score_str) if score_str.isdigit() else None
Expand Down
88 changes: 51 additions & 37 deletions git/objects/submodule/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,15 +47,16 @@
find_first_remote_branch
)

from git.repo import Repo

# typing ----------------------------------------------------------------------
from typing import Dict, TYPE_CHECKING
from typing import Callable, Dict, Mapping, Sequence, TYPE_CHECKING
from typing import Any, Iterator, Union

from git.types import Commit_ish, PathLike
from git.types import Commit_ish, PathLike, TBD

if TYPE_CHECKING:
from git.repo import Repo
from git.index import IndexFile


# -----------------------------------------------------------------------------
Expand Down Expand Up @@ -131,14 +132,14 @@ def __init__(self, repo: 'Repo', binsha: bytes,
if url is not None:
self._url = url
if branch_path is not None:
assert isinstance(branch_path, str)
# assert isinstance(branch_path, str)
self._branch_path = branch_path
if name is not None:
self._name = name

def _set_cache_(self, attr: str) -> None:
if attr in ('path', '_url', '_branch_path'):
reader = self.config_reader()
reader: SectionConstraint = self.config_reader()
# default submodule values
try:
self.path = reader.get('path')
Expand Down Expand Up @@ -226,7 +227,7 @@ def _config_parser(cls, repo: 'Repo',

return SubmoduleConfigParser(fp_module, read_only=read_only)

def _clear_cache(self):
def _clear_cache(self) -> None:
# clear the possibly changed values
for name in self._cache_attrs:
try:
Expand All @@ -246,7 +247,7 @@ def _sio_modules(cls, parent_commit: Commit_ish) -> BytesIO:
def _config_parser_constrained(self, read_only: bool) -> SectionConstraint:
""":return: Config Parser constrained to our submodule in read or write mode"""
try:
pc = self.parent_commit
pc: Union['Commit_ish', None] = self.parent_commit
except ValueError:
pc = None
# end handle empty parent repository
Expand All @@ -255,10 +256,12 @@ def _config_parser_constrained(self, read_only: bool) -> SectionConstraint:
return SectionConstraint(parser, sm_section(self.name))

@classmethod
def _module_abspath(cls, parent_repo, path, name):
def _module_abspath(cls, parent_repo: 'Repo', path: PathLike, name: str) -> PathLike:
if cls._need_gitfile_submodules(parent_repo.git):
return osp.join(parent_repo.git_dir, 'modules', name)
return osp.join(parent_repo.working_tree_dir, path)
if parent_repo.working_tree_dir:
return osp.join(parent_repo.working_tree_dir, path)
raise NotADirectoryError()
# end

@classmethod
Expand Down Expand Up @@ -286,15 +289,15 @@ def _clone_repo(cls, repo, url, path, name, **kwargs):
return clone

@classmethod
def _to_relative_path(cls, parent_repo, path):
def _to_relative_path(cls, parent_repo: 'Repo', path: PathLike) -> PathLike:
""":return: a path guaranteed to be relative to the given parent - repository
:raise ValueError: if path is not contained in the parent repository's working tree"""
path = to_native_path_linux(path)
if path.endswith('/'):
path = path[:-1]
# END handle trailing slash

if osp.isabs(path):
if osp.isabs(path) and parent_repo.working_tree_dir:
working_tree_linux = to_native_path_linux(parent_repo.working_tree_dir)
if not path.startswith(working_tree_linux):
raise ValueError("Submodule checkout path '%s' needs to be within the parents repository at '%s'"
Expand All @@ -308,7 +311,7 @@ def _to_relative_path(cls, parent_repo, path):
return path

@classmethod
def _write_git_file_and_module_config(cls, working_tree_dir, module_abspath):
def _write_git_file_and_module_config(cls, working_tree_dir: PathLike, module_abspath: PathLike) -> None:
"""Writes a .git file containing a(preferably) relative path to the actual git module repository.
It is an error if the module_abspath cannot be made into a relative path, relative to the working_tree_dir
:note: will overwrite existing files !
Expand All @@ -335,7 +338,8 @@ def _write_git_file_and_module_config(cls, working_tree_dir, module_abspath):

@classmethod
def add(cls, repo: 'Repo', name: str, path: PathLike, url: Union[str, None] = None,
branch=None, no_checkout: bool = False, depth=None, env=None, clone_multi_options=None
branch: Union[str, None] = None, no_checkout: bool = False, depth: Union[int, None] = None,
env: Mapping[str, str] = None, clone_multi_options: Union[Sequence[TBD], None] = None
) -> 'Submodule':
"""Add a new submodule to the given repository. This will alter the index
as well as the .gitmodules file, but will not create a new commit.
Expand Down Expand Up @@ -391,7 +395,7 @@ def add(cls, repo: 'Repo', name: str, path: PathLike, url: Union[str, None] = No
if sm.exists():
# reretrieve submodule from tree
try:
sm = repo.head.commit.tree[path] # type: ignore
sm = repo.head.commit.tree[str(path)]
sm._name = name
return sm
except KeyError:
Expand All @@ -414,7 +418,8 @@ def add(cls, repo: 'Repo', name: str, path: PathLike, url: Union[str, None] = No
# END check url
# END verify urls match

mrepo = None
mrepo: Union[Repo, None] = None

if url is None:
if not has_module:
raise ValueError("A URL was not given and a repository did not exist at %s" % path)
Expand All @@ -427,7 +432,7 @@ def add(cls, repo: 'Repo', name: str, path: PathLike, url: Union[str, None] = No
url = urls[0]
else:
# clone new repo
kwargs: Dict[str, Union[bool, int]] = {'n': no_checkout}
kwargs: Dict[str, Union[bool, int, Sequence[TBD]]] = {'n': no_checkout}
if not branch_is_default:
kwargs['b'] = br.name
# END setup checkout-branch
Expand All @@ -451,6 +456,8 @@ def add(cls, repo: 'Repo', name: str, path: PathLike, url: Union[str, None] = No
# otherwise there is a '-' character in front of the submodule listing
# a38efa84daef914e4de58d1905a500d8d14aaf45 mymodule (v0.9.0-1-ga38efa8)
# -a38efa84daef914e4de58d1905a500d8d14aaf45 submodules/intermediate/one
writer: Union[GitConfigParser, SectionConstraint]

with sm.repo.config_writer() as writer:
writer.set_value(sm_section(name), 'url', url)

Expand All @@ -467,13 +474,15 @@ def add(cls, repo: 'Repo', name: str, path: PathLike, url: Union[str, None] = No
sm._branch_path = br.path

# we deliberately assume that our head matches our index !
sm.binsha = mrepo.head.commit.binsha
sm.binsha = mrepo.head.commit.binsha # type: ignore
index.add([sm], write=True)

return sm

def update(self, recursive=False, init=True, to_latest_revision=False, progress=None, dry_run=False,
force=False, keep_going=False, env=None, clone_multi_options=None):
def update(self, recursive: bool = False, init: bool = True, to_latest_revision: bool = False,
progress: Union['UpdateProgress', None] = None, dry_run: bool = False,
force: bool = False, keep_going: bool = False, env: Mapping[str, str] = None,
clone_multi_options: Union[Sequence[TBD], None] = None):
"""Update the repository of this submodule to point to the checkout
we point at with the binsha of this instance.

Expand Down Expand Up @@ -580,6 +589,7 @@ def update(self, recursive=False, init=True, to_latest_revision=False, progress=
if not dry_run:
# see whether we have a valid branch to checkout
try:
assert isinstance(mrepo, Repo)
# find a remote which has our branch - we try to be flexible
remote_branch = find_first_remote_branch(mrepo.remotes, self.branch_name)
local_branch = mkhead(mrepo, self.branch_path)
Expand Down Expand Up @@ -640,7 +650,7 @@ def update(self, recursive=False, init=True, to_latest_revision=False, progress=
may_reset = True
if mrepo.head.commit.binsha != self.NULL_BIN_SHA:
base_commit = mrepo.merge_base(mrepo.head.commit, hexsha)
if len(base_commit) == 0 or base_commit[0].hexsha == hexsha:
if len(base_commit) == 0 or (base_commit[0] is not None and base_commit[0].hexsha == hexsha):
if force:
msg = "Will force checkout or reset on local branch that is possibly in the future of"
msg += "the commit it will be checked out to, effectively 'forgetting' new commits"
Expand Down Expand Up @@ -807,7 +817,8 @@ def move(self, module_path, configuration=True, module=True):
return self

@unbare_repo
def remove(self, module=True, force=False, configuration=True, dry_run=False):
def remove(self, module: bool = True, force: bool = False,
configuration: bool = True, dry_run: bool = False) -> 'Submodule':
"""Remove this submodule from the repository. This will remove our entry
from the .gitmodules file and the entry in the .git / config file.

Expand Down Expand Up @@ -861,7 +872,7 @@ def remove(self, module=True, force=False, configuration=True, dry_run=False):
# TODO: If we run into permission problems, we have a highly inconsistent
# state. Delete the .git folders last, start with the submodules first
mp = self.abspath
method = None
method: Union[None, Callable[[PathLike], None]] = None
if osp.islink(mp):
method = os.remove
elif osp.isdir(mp):
Expand Down Expand Up @@ -914,7 +925,7 @@ def remove(self, module=True, force=False, configuration=True, dry_run=False):
import gc
gc.collect()
try:
rmtree(wtd)
rmtree(str(wtd))
except Exception as ex:
if HIDE_WINDOWS_KNOWN_ERRORS:
raise SkipTest("FIXME: fails with: PermissionError\n {}".format(ex)) from ex
Expand All @@ -928,7 +939,7 @@ def remove(self, module=True, force=False, configuration=True, dry_run=False):
rmtree(git_dir)
except Exception as ex:
if HIDE_WINDOWS_KNOWN_ERRORS:
raise SkipTest("FIXME: fails with: PermissionError\n %s", ex) from ex
raise SkipTest(f"FIXME: fails with: PermissionError\n {ex}") from ex
else:
raise
# end handle separate bare repository
Expand All @@ -952,6 +963,8 @@ def remove(self, module=True, force=False, configuration=True, dry_run=False):

# now git config - need the config intact, otherwise we can't query
# information anymore
writer: Union[GitConfigParser, SectionConstraint]

with self.repo.config_writer() as writer:
writer.remove_section(sm_section(self.name))

Expand All @@ -961,7 +974,7 @@ def remove(self, module=True, force=False, configuration=True, dry_run=False):

return self

def set_parent_commit(self, commit: Union[Commit_ish, None], check=True):
def set_parent_commit(self, commit: Union[Commit_ish, None], check: bool = True) -> 'Submodule':
"""Set this instance to use the given commit whose tree is supposed to
contain the .gitmodules blob.

Expand Down Expand Up @@ -1009,7 +1022,7 @@ def set_parent_commit(self, commit: Union[Commit_ish, None], check=True):
return self

@unbare_repo
def config_writer(self, index=None, write=True):
def config_writer(self, index: Union['IndexFile', None] = None, write: bool = True) -> SectionConstraint:
""":return: a config writer instance allowing you to read and write the data
belonging to this submodule into the .gitmodules file.

Expand All @@ -1030,7 +1043,7 @@ def config_writer(self, index=None, write=True):
return writer

@unbare_repo
def rename(self, new_name):
def rename(self, new_name: str) -> 'Submodule':
"""Rename this submodule
:note: This method takes care of renaming the submodule in various places, such as

Expand Down Expand Up @@ -1065,13 +1078,14 @@ def rename(self, new_name):
destination_module_abspath = self._module_abspath(self.repo, self.path, new_name)
source_dir = mod.git_dir
# Let's be sure the submodule name is not so obviously tied to a directory
if destination_module_abspath.startswith(mod.git_dir):
if str(destination_module_abspath).startswith(str(mod.git_dir)):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that the type checker wanted add these conversions here, but is there a way to do without them?
It's just that git_dir is a directory and its value might not be decodable by the encoding enforced by str.

It would be great if you could give it another look and if there is an alternative, you could push that to main directly.

tmp_dir = self._module_abspath(self.repo, self.path, str(uuid.uuid4()))
os.renames(source_dir, tmp_dir)
source_dir = tmp_dir
# end handle self-containment
os.renames(source_dir, destination_module_abspath)
self._write_git_file_and_module_config(mod.working_tree_dir, destination_module_abspath)
if mod.working_tree_dir:
self._write_git_file_and_module_config(mod.working_tree_dir, destination_module_abspath)
# end move separate git repository

return self
Expand All @@ -1081,7 +1095,7 @@ def rename(self, new_name):
#{ Query Interface

@unbare_repo
def module(self):
def module(self) -> 'Repo':
""":return: Repo instance initialized from the repository at our submodule path
:raise InvalidGitRepositoryError: if a repository was not available. This could
also mean that it was not yet initialized"""
Expand All @@ -1098,7 +1112,7 @@ def module(self):
raise InvalidGitRepositoryError("Repository at %r was not yet checked out" % module_checkout_abspath)
# END handle exceptions

def module_exists(self):
def module_exists(self) -> bool:
""":return: True if our module exists and is a valid git repository. See module() method"""
try:
self.module()
Expand All @@ -1107,7 +1121,7 @@ def module_exists(self):
return False
# END handle exception

def exists(self):
def exists(self) -> bool:
"""
:return: True if the submodule exists, False otherwise. Please note that
a submodule may exist ( in the .gitmodules file) even though its module
Expand Down Expand Up @@ -1148,34 +1162,34 @@ def branch(self):
return mkhead(self.module(), self._branch_path)

@property
def branch_path(self):
def branch_path(self) -> PathLike:
"""
:return: full(relative) path as string to the branch we would checkout
from the remote and track"""
return self._branch_path

@property
def branch_name(self):
def branch_name(self) -> str:
""":return: the name of the branch, which is the shortest possible branch name"""
# use an instance method, for this we create a temporary Head instance
# which uses a repository that is available at least ( it makes no difference )
return git.Head(self.repo, self._branch_path).name

@property
def url(self):
def url(self) -> str:
""":return: The url to the repository which our module - repository refers to"""
return self._url

@property
def parent_commit(self):
def parent_commit(self) -> 'Commit_ish':
""":return: Commit instance with the tree containing the .gitmodules file
:note: will always point to the current head's commit if it was not set explicitly"""
if self._parent_commit is None:
return self.repo.commit()
return self._parent_commit

@property
def name(self):
def name(self) -> str:
""":return: The name of this submodule. It is used to identify it within the
.gitmodules file.
:note: by default, the name is the path at which to find the submodule, but
Expand Down