Skip to content

Include 'timeout' parameter in Git execute #354

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Oct 16, 2015
Merged

Include 'timeout' parameter in Git execute #354

Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
cd4caf1
Include 'timeout' parameter in Git execute
OswinNathanial Sep 28, 2015
9e1c90e
Raise exception when timeout is used in execute command on Windows
OswinNathanial Oct 9, 2015
7cc0e6c
Only create watchdog and event if timeout is specified in execute com…
OswinNathanial Oct 9, 2015
4faf5cd
Rename execute param 'timeout' to 'kill_after_timeout'
OswinNathanial Oct 13, 2015
d06e76b
Update docstring for 'kill_after_timeout' parameter
OswinNathanial Oct 13, 2015
dbbcaf7
Run os.kill for all child pids even after some of them are down
OswinNathanial Oct 13, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 43 additions & 1 deletion git/cmd.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
import mmap

from contextlib import contextmanager
from signal import SIGKILL
from subprocess import (
call,
Popen,
Expand Down Expand Up @@ -41,7 +42,7 @@

execute_kwargs = ('istream', 'with_keep_cwd', 'with_extended_output',
'with_exceptions', 'as_process', 'stdout_as_string',
'output_stream', 'with_stdout')
'output_stream', 'with_stdout', 'timeout')

log = logging.getLogger('git.cmd')
log.addHandler(logging.NullHandler())
Expand Down Expand Up @@ -475,6 +476,7 @@ def execute(self, command,
as_process=False,
output_stream=None,
stdout_as_string=True,
timeout=None,
with_stdout=True,
**subprocess_kwargs
):
Expand Down Expand Up @@ -531,6 +533,12 @@ def execute(self, command,

:param with_stdout: If True, default True, we open stdout on the created process

:param timeout:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about renaming the parameter to something even more descriptive, like kill_after_timeout to be very clear about the SIGKILL nature of the operation.

To specify a timeout in seconds for the git command, after which the process
should be killed. This will have no effect if as_process is set to True. It is
set to None by default and will let the process run until the timeout is
explicitly specified.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would love to see that this feature is not supported on windows.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be good to make clear that SIGKILL can have negative side-effects on a repository. For example, stale locks in case of git gc could render the repository incapable of accepting changes until the lock is manually removed.


:return:
* str(output) if extended_output = False (Default)
* tuple(int(status), str(stdout), str(stderr)) if extended_output = True
Expand Down Expand Up @@ -568,6 +576,8 @@ def execute(self, command,

if sys.platform == 'win32':
cmd_not_found_exception = WindowsError
if timeout:
raise GitCommandError('"timeout" feature is not supported on Windows.')
else:
if sys.version_info[0] > 2:
cmd_not_found_exception = FileNotFoundError # NOQA # this is defined, but flake8 doesn't know
Expand All @@ -592,13 +602,45 @@ def execute(self, command,
if as_process:
return self.AutoInterrupt(proc, command)

def _kill_process(pid):
""" Callback method to kill a process. """
p = Popen(['ps', '--ppid', str(pid)], stdout=PIPE)
child_pids = []
for line in p.stdout:
if len(line.split()) > 0:
local_pid = (line.split())[0]
if local_pid.isdigit():
child_pids.append(int(local_pid))
try:
os.kill(pid, SIGKILL)
for child_pid in child_pids:
os.kill(child_pid, SIGKILL)
kill_check.set() # tell the main routine that the process was killed
except OSError:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If os.kill throws OSError if the process to be killed doesn't exist, I fear that after os.kill(pid, SIGKILL), some of the child processes might already be down as pid. Right now, failure to kill the first child process, will prevent the others from being downed as well. Maybe something like that would help:

for child_pid in child_pids:
    try:
        os.kill(child_pid, SIGKILL)
    except OSError:
        pass

# It is possible that the process gets completed in the duration after timeout
# happens and before we try to kill the process.
pass
return
# end

if timeout:
kill_check = threading.Event()
watchdog = threading.Timer(timeout, _kill_process, args=(proc.pid, ))

# Wait for the process to return
status = 0
stdout_value = b''
stderr_value = b''
try:
if output_stream is None:
if timeout:
watchdog.start()
stdout_value, stderr_value = proc.communicate()
if timeout:
watchdog.cancel()
if kill_check.isSet():
stderr_value = 'Timeout: the command "%s" did not complete in %d ' \
'secs.' % (" ".join(command), timeout)
# strip trailing "\n"
if stdout_value.endswith(b"\n"):
stdout_value = stdout_value[:-1]
Expand Down