Cannot infer type of lambda when using default arguments

[mxmlnkn]

Bug Report

I get error: Cannot infer type of lambda even though the code seems fine to me.

Currently, I don't see any other workaround other than using # type: ignore.

To Reproduce

from typing import Callable, List

def foo(factories: List[Callable[[], int]]):
    return sum(f() for f in factories)

names = ["0", "abc"]
# Inference works fine
assert foo([lambda: len(name) for name in names]) == 4
# Cannot infer type of lambda
assert foo([lambda name=name: len(name) for name in names]) == 4

def foo2(factories: List[Callable[[int], int]]):
    return sum(f(0) for f in factories)

# Inference works fine
assert foo2([lambda i: len(name) for name in names]) == 4
# Cannot infer type of lambda
assert foo2([lambda i, name=name: len(name) for name in names]) == 4

Checked with:

mypy mypy-lambda-test.py

Expected Behavior

There should be no error. Especially because the other variants without default arguments are deemed fine.

Furthermore, using default arguments like this is the standard workaround to capture loop variables to lambdas.

Your Environment

• Mypy version used: 0.942
• Python version used: 3.9.7
• Operating system and version: Ubuntu 21.10

[erictraut]

I agree that the error message is confusing, but mypy is correct in generating an error in these two cases.

It is a type violation because both foo and foo2 indicate that they accept a list of Callable values that accept zero input parameters (in the case of foo) and one input parameter (in the case of foo2). You are attempting to call the function with a list that includes callables with the wrong number of arguments.

Here's a simplified example.

# No type violation
v1: Callable[[], int] = lambda : 1

# Type violation because lambda has an input parameter
v2: Callable[[], int] = lambda name: len(name)

If your intent is to accept a list of callables that accept an arbitrary number of arguments, you would need to use Callable[..., int] instead of Callable[[], int] or Callable[[int], int].

[JelleZijlstra]

The OP's example has a default for the name argument. Here's a simple example:

from typing import Callable
v2: Callable[[], int] = lambda name="x": len(name)

Mypy produces Cannot infer type of lambda for this example, but pyright correctly allows it.

[mxmlnkn]

It is a type violation because both foo and foo2 indicate that they accept a list of Callable values that accept zero input parameters (in the case of foo) and one input parameter (in the case of foo2). You are attempting to call the function with a list that includes callables with the wrong number of arguments.

Ah, that kinda makes sense. I would be fine with a better error message, too.

But, even better, would it be possible to somehow match functions only on the required (non-default) parameters? E.g. for the case of foo2, I'd like something like Callable[[int, ...], int].

[erictraut]

Ah yes, I missed the fact that default arguments were provided in the sample code.

Pyright does not generate an error in "basic" type checking mode, but in "strict" mode it produces effectively the same error as mypy. The type of the lambda parameter cannot be inferred from bidirectional type inference because there's insufficient information provided by the supplied type annotation. So this is arguably not a bug in mypy.

I suppose a type checker could use the inferred type of the default argument to infer the type of the lambda parameter, but that would require a lot of special-case logic to handle what is arguably an extreme edge case.

@mxmlnkn, out of curiosity, why are you defining a lambda that accepts extra parameters that are not required in the context in which it is used? Or is your actual use case more complicated that the above sample implies?

[mxmlnkn]

@mxmlnkn, out of curiosity, why are you defining a lambda that accepts extra parameters that are not required in the context in which it is used? Or is your actual use case more complicated that the above sample implies?

The full example is here. As mentioned in the initial post, I'm using default arguments to effectively safe "freeze" the current loop variable.

Note that in the minimal example in the initial post, the first and third assert will fail while the other two will be successful! The result for those failing asserts is 6.

[rpdelaney]

Here's a case that sent me Googling and brought me to this issue:

dumbpw/engine.py:102:15: error: Cannot infer type of lambda  [misc]
        validator=lambda charset, length: len("".join(charset)) > 0,
                  ^
dumbpw/engine.py:110:15: error: Cannot infer type of lambda  [misc]
        validator=lambda charset, length, result: all(char in "".join(charset) for char in result),
                  ^
dumbpw/engine.py:110:15: error: Argument "validator" to "ensure" has incompatible type "Callable[[Any, Any, Any], bool]"; expected
"Callable[[NamedArg(str, 'charset'), NamedArg(int, 'length'), str], Union[bool, str]]"  [arg-type]
        validator=lambda charset, length, result: all(char in "".join(charset) for char in result),
                  ^
Found 3 errors in 1 file (checked 1 source file)

As a typing n00b, the third error feels much more informative than the first two. But I don't have the knowledge to explain why I didn't get a similar error on line 102.

[kkpattern]

@mxmlnkn, out of curiosity, why are you defining a lambda that accepts extra parameters that are not required in the context in which it is used? Or is your actual use case more complicated that the above sample implies?

The full example is here. As mentioned in the initial post, I'm using default arguments to effectively safe "freeze" the current loop variable.

We use this trick to "freeze" loop variables too. We actually use this pattern quite a lot.

[kkpattern]

We found that we can use functools.partial instead of lambda in this case.

[andersk]

functools.partial isn’t a good workaround because its argument types aren’t checked at all (python/typing#1372, python/typeshed#8703).

[0e4ef622]

Just ran into this issue, also using this trick to "freeze" loop variables. Whats weird is while lambda x=x: ... isn't checked, what does work is (lambda x: lambda: ...)(x).

def f(x: int):
    pass

x: list[int] = []

# no error
a = lambda x=x: f(x)

# error: Argument 1 to "f" has incompatible type "list[int]"; expected "int"  [arg-type]
b = (lambda x: lambda: f(x))(x)

[JamesHutchison]

Is there a way to squelch this error automatically? The "freeze variables" pattern is really common in reactpy