[Windows] signal.signal and os.kill doc should better explain what is supported

[giampaolo]

BPO	26350
Nosy	@pfmoore, @vstinner, @giampaolo, @tjguk, @ezio-melotti, @merwok, @zware, @eryksun, @zooba, @willingc, @JulienPalard

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2016-02-12.14:01:43.013>
labels = ['3.8', '3.9', '3.10', 'extension-modules', 'type-feature', 'OS-windows']
title = '[Windows] signal.signal and os.kill doc should better explain what is supported'
updated_at = <Date 2021-03-20.03:49:58.378>
user = 'https://github.com/giampaolo'

bugs.python.org fields:

activity = <Date 2021-03-20.03:49:58.378>
actor = 'eryksun'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Extension Modules', 'Windows']
creation = <Date 2016-02-12.14:01:43.013>
creator = 'giampaolo.rodola'
dependencies = []
files = []
hgrepos = []
issue_num = 26350
keywords = []
message_count = 5.0
messages = ['260179', '260181', '260201', '299939', '389142']
nosy_count = 12.0
nosy_names = ['paul.moore', 'vstinner', 'giampaolo.rodola', 'tim.golden', 'ezio.melotti', 'eric.araujo', 'zach.ware', 'eryksun', 'steve.dower', 'willingc', 'mdk', 'Ross Rosen']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue26350'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

[giampaolo]

I'm not sure whether this is a bug with signal.signal doc or with the function itself, anyway, here goes. On UNIX I'm able to register a signal handler for SIGTERM which is executed when the signal is received. On Windows I'm able to register it but it will never be executed:

import os, signal

def foo(*args):
    print("foo")  # this never gets printed on Windows

signal.signal(signal.SIGTERM, foo)
os.kill(os.getpid(), signal.SIGTERM)

I think signal module doc should be more clear about this. In details, if it is possible to execute a function on SIGTERM if should explain how. If not (and AFAIK it's not possible) it should state that "signal.signal(signal.SIGTERM, foo)" on Windows is a NOOP.

Note: I'm probably missing something but the same thing applies for SIGINT and possibly (all) other signals, so I'm not sure why Windows has signal.signal in the first place. What's its use case on Windows?

[vstinner]

See also issue bpo-23948.

[eryksun]

All of the implemented signals can be registered, and the associated handler can at least be executed by calling the C raise() function. However, for pure-Python code only SIGINT and SIGBREAK are really useful.

FYI, Windows doesn't implement POSIX signals at the system level. The closest analog is an asynchronous procedure call (APC). You can queue a user-mode APC to a thread via QueueUserAPC. It executes when the thread enters an alertable wait (e.g. WaitForSingleObjectEx or SleepEx with bAlertable as TRUE). Kernel-mode APCs get dispatched immediately (when the IRQL is below APC_LEVEL). They're typically used by the I/O manager to complete asynchronous I/O Request Packets (e.g. copying data to a user-mode buffer).

In principle, NT could implement POSIX-like signals using APCs, but in practice Windows uses other means for the same ends, such as exceptions and structured exception handling (e.g. __try, __except, __finally, __leave, RaiseException, AddVectoredExceptionHandler), dispatch objects (e.g. SetEvent and SetWaitableTimer, which queues an APC), and window messages (e.g. WM_QUIT, WM_TIMER).

On Windows, the C runtime implements the six signals that are required by standard C: SIGINT, SIGABRT, SIGTERM, SIGSEGV, SIGILL, and SIGFPE.

SIGABRT and SIGTERM are implemented just for the current process. You can call the handler via C raise().

    >>> import signal, ctypes
    >>> ucrtbase = ctypes.CDLL('ucrtbase')
    >>> c_raise = ucrtbase['raise']
    >>> foo = lambda *a: print('foo')
    >>> signal.signal(signal.SIGTERM, foo)
    <Handlers.SIG_DFL: 0>
    >>> c_raise(signal.SIGTERM)
    foo
    0

SIGTERM is useless.

You also can't do much with SIGABRT using the signal module because the abort() function kills the process once the handler returns, which happens immediately when using the signal module's internal handler (it trips a flag for the Python callable to be called later). Instead use the faulthandler module. Or call the CRT's signal() function via ctypes to set a ctypes callback as the handler.

The CRT implements SIGSEGV, SIGILL, and SIGFPE by setting a Windows structured exception handler (SEH) for the corresponding Windows exceptions:

STATUS_ACCESS_VIOLATION          SIGSEGV
STATUS_ILLEGAL_INSTRUCTION       SIGILL
STATUS_PRIVILEGED_INSTRUCTION    SIGILL
STATUS_FLOAT_DENORMAL_OPERAND    SIGFPE
STATUS_FLOAT_DIVIDE_BY_ZERO      SIGFPE
STATUS_FLOAT_INEXACT_RESULT      SIGFPE
STATUS_FLOAT_INVALID_OPERATION   SIGFPE
STATUS_FLOAT_OVERFLOW            SIGFPE
STATUS_FLOAT_STACK_CHECK         SIGFPE
STATUS_FLOAT_UNDERFLOW           SIGFPE
STATUS_FLOAT_MULTIPLE_FAULTS     SIGFPE
STATUS_FLOAT_MULTIPLE_TRAPS      SIGFPE

Use the faulthandler module for these exception-based signals. The way they're implemented is incompatible with Python's signal handler. The exception filter calls the registered handler and then returns EXCEPTION_CONTINUE_EXECUTION. Python's handler only trips a flag for the interpreter to call the registered Python callable at a later time. So the code that triggered the exception will trigger again, and so on in an endless loop.

That leaves SIGINT, to which Windows adds the non-standard SIGBREAK. Both console and non-console processes can raise() one of these signals, but only a console process can receive them from another process.

The CRT sets a console control event handler via SetConsoleCtrlHandler. When the console sends the process a CTRL_C_EVENT or CTRL_BREAK_EVENT, the CRT's handler calls the associated SIGINT or SIGBREAK handler.

Note that this is implemented by creating a new thread in the process that begins executing at kernel32!CtrlRoutine. Unlike SIGINT on POSIX, the handler does not execute on the main thread (hijacking a thread is taboo in Windows). This can lead to synchronization problems that Python 3 attempts to work around by using a Windows event object.

You can send a control event to all processes attached to the current console via GenerateConsoleCtrlEvent. You can target a subset of processes that belong to a process group, or send the event to all processes by targeting process group 0.

What the console does when the target ID isn't a process group ID is undefined. It basically acts like the target is group 0, but that shouldn't be relied on. (It's most likely a bug.) It can also mess up the console's list of attached processes (i.e. GetConsoleProcessList) by adding non-console processes.

The docs for os.kill clearly state that you can only send signal.CTRL_C_EVENT and signal.CTRL_BREAK_EVENT on Windows. Any other value is passed to TerminateProcess as the exit code, which kills the process without notice (like POSIX SIGKILL).

It also states that "[t]he Windows version of kill() additionally takes process handles to be killed", which I don't think was ever true. That line needs to be removed.

It also fails to clarify that the target has to be a process group ID, and what that is. Every process in a Windows session belongs to a process group, even if it's just the wininit.exe group (services session) or winlogon.exe group (interactive session). A new group is created by passing the creation flag CREATE_NEW_PROCESS_GROUP when creating a new process. The group ID is the process ID of the created process. AFAIK, the console is the only system that uses the process group, and that's just for GenerateConsoleCtrlEvent.

Don't rely on being able to send CTRL_C_EVENT to anything but group 0, since it's initially disabled in a new process group. It's not impossible to send this event to a new group, but the target process first has to enable CTRL_C_EVENT by calling SetConsoleCtrlHandler(NULL, FALSE).

CTRL_BREAK_EVENT is all you can depend on since it can't be disabled. Sending this event is a simple way to gracefully kill a child process that was started with CREATE_NEW_PROCESS_GROUP, assuming it has a Windows CTRL_BREAK_EVENT or C SIGBREAK handler. If not, the default handler will terminate the process, setting the exit code to STATUS_CTRL_C_EXIT. For example:

    >>> import os, signal, subprocess
    >>> p = subprocess.Popen('python.exe',
    ...         stdin=subprocess.PIPE,
    ...         creationflags=subprocess.CREATE_NEW_PROCESS_GROUP)
    >>> os.kill(p.pid, signal.CTRL_BREAK_EVENT)
    >>> STATUS_CONTROL_C_EXIT = 0xC000013A
    >>> p.wait() == STATUS_CONTROL_C_EXIT
    True

Notice that CTRL_BREAK_EVENT wasn't sent to the current process, because I targeted the process group of the child process (including all of its child processes that are attached to the console, and so on). If I had used group 0, the current process would have been killed as well since I didn't define a SIGBREAK handler. Let's try that, but with a handler set:

    >>> c_break = lambda *a: print('^BREAK')
    >>> signal.signal(signal.SIGBREAK, c_break)
    <Handlers.SIG_DFL: 0>
    >>> os.kill(0, signal.CTRL_BREAK_EVENT)
    ^BREAK

[RossRosen]

I'm not sure if this is helpful, but I thought it might be useful for you to hear a non-expert user's perspective. (In summary, I'm agreeing with the OP.)

I spent a lot of time getting some signal handling working on OSX. Then finally in my process got to porting to Windows. I spent a bunch more time scratching my head trying to figure out why it wasn't working. Even after reading the os.kill and signal docs and searching for "Windows" I still wasn't clear what was wrong.

This thread was the answer.

So my suggestion at the least would be to put warnings in the signal doc, and clarify the language in the os.kill.

[eryksun]

The signal.signal() docs should explain that only SIGINT and SIGBREAK can be signaled from another process, and only if the current process is a console application. A link to os.kill() could be added, which should explain their use as signals in relation to console control events. Also, setting a handler for SIGABRT, SIGFPE, SIGILL, and SIGSEGV should be discouraged in the documentation because they are not compatible with how Python handles signals by tripping a flag for deferred processing on the main thread.

The design of os.kill() in Windows has some unfortunate mistakes, but it needs to be better documented at least. It should explain that the pid value, when used with CTRL_C_EVENT and CTRL_BREAK_EVENT, is a process-group ID, not a process ID. It should explain that CTRL_C_EVENT will be ignored by a process in the target group if the process calls SetConsoleCtrlHandler(NULL, TRUE), or if it's started with Ctrl+C ignored either due to inheritance or due to being created in a new process group. It should explain that these console control events get mapped to the C signals SIGINT and SIGBREAK.

Also the line "[t]he Windows version of kill() additionally takes process handles to be killed" needs to removed. os.kill() has never accepted handles in addition to PIDs. Implementing that reliably would require an additional parameter.

---

Ideally, the code path in os.kill() that generates a console control event would have been reserved for a negative PID value (i.e. a process group ID), with -1 reserved to send the event to all processes in the console session (i.e. console process group 0). Also, in this case, os.kill() would use SIGINT and SIGBREAK instead of WinAPI CTRL_C_EVENT and CTRL_BREAK_EVENT. Such a design would have conformed much better with POSIX kill().