Function Types Do Not Support Dynamic

[RohitSaily]

This is derived from @MikePendo's issue dart-lang/sdk#60288, a practical instance of this problem being encountered.

Context

Suppose one had a function final f=(final num n)=>print(n);. The type is void Function(num). int is a subtype of num so the function can be casted to a void Function(int) as expected. This is good.

If the types were reversed, we have final f=(final int i)=>print(i); of type void Function(int). It is not a subtype of void Function(num), even though int is a subtype of num. This is because the function may rely on behaviour specific to int in its execution, and therefore cannot be generalized. Again this is good.

Problem

Suppose we again have final f=(final int i)=>print(i); of type void Function(int). I may want to type-erase the parameters to have code that handles a general kind of n-ary function (in this case, a single input one). I expect the function to be castable to void Function(dynamic) because whether the function will work or not will be determined at runtime by dynamic computations. However, it is not castable! I cannot do f as void Function(dynamic), it throws an exception.

This is (1) not intuitive because dynamic should work and be resolved at runtime as it does everywhere else, and (2) inconvenient for generic situations, requiring less safe code to be written which can therefore nonnecessarily lead to more errors. Again, see @MikePendo's issue dart-lang/sdk#60288 for a practical instance of this problem.

Solution

Allow function types to have input, and output parameters type-erased as dynamic. For example, String Function(int, bool) should be able to be casted to any one of the following:

• dynamic Function(int, bool)
• void Function(dynamic, bool)
• dynamic Function(dynamic, bool)
• dynamic Function(dynamic, dynamic)

[lrhn]

This was how dynamic worked in Dart 1. It stopped working that way with the sounder type system of Dart 2.

The reason is that Dart 2 made a distinction between dynamic invocations and typed invocations, distinguished at compile time, and only dynamic invocations need to do runtime type checks at the call point. (Or at least type checks that are not statically known.)

In Dart 2, dynamic is a top type, a supertype of all types.
That means that if you can cast void Function(int) to void Function(dynamic), you can then up-cast that to void Function(String), and up-casts do not need to be checked.

Can't the compiler just check at the point where it casts a type that contains dynamic?
And no, it can't, because of generics.

class Caster<Sup, Sub extends Sup> {
  Sup cast(Sub f) => f;
}
void Function(String) strCast(
  void Function(dynamic) f) =>
    Caster<
      void Function(String),
      void Function(dynamic)>()
    .cast(f);

These functions only do up-casts, and there is no place where a dynamic occurs statically in a type that is cast to something else. The place where that happens uses generics, and all it knows is that one type of a subtype of the other.

That means that if we allow casting void Function(int) to void Function(dynamic), then either that type is not a normal function type at all, and it needs to be handled specially everywhere, including not working with generics, or its unsound.

Neither is a good option.

[RohitSaily]

Thanks for the explanation. I am having difficulty conceptualizing this

That means that if you can cast void Function(int) to void Function(dynamic), you can then up-cast that to void Function(String), and up-casts do not need to be checked.

[...]

class Caster<Sup, Sub extends Sup>
{     Sup cast(final Sub f)=>
            f;
}
void Function(String) strCast(final void Function(dynamic) f)=>
      Caster<void Function(String), void Function(dynamic)>().cast(f);

I don't understand why the design is to treat void Function(dynamic) as up-casting. Doesn't it also have to consider the possibility that it is a down-cast that gets checked at runtime because of dynamic? I see that Caster defines the generic parameter such that Sub extends Sup, but why can't this mean that the dynamic must end up being the same type or a subtype of String?

[lrhn]

An up-cast is a cast to a supertype. That's generally safe to do because an instance of the subtype must be an instance of the supertype. So no runtime checks are needed or done for up-casts.

This issue requests that you can freely "cast" a function type to a subtype, from void Function(int) to the subtype void Function(dynamic). That's not a sound cast, because a void Function(dynamic) can be called with any value as argument, and a void Function(int) cannot, so an object with runtime type void Function(int) is-not-a void Function(dynamic).

As I read the request, it would not be blindly treating a void Function(int) as a void Function(dynamic), but would require that when called, the function can be called with any value, and will then throw if the value is not actually an int.

There are different ways to do that:

• Wrapping. The assignment effectively wraps f in (dynamic value) => f(value as int);. That gives the wrapped function a different identity. It's effectively a coercion which makes void Function(int) assignable to void Function(dynamic) the same way dynamic is assingable to int, lifting that coercion to the parameter. That's possible, but Dart has generally tried to reduce implicit coercions, so it's unlikely that it'll add this. It's just not useful enough, and it silently introduces a potential runtime error when assigning a function to a subtype that has dynamic as parameter type. You can just write the wrapper yourself, and make it explicit.
• Being unsound, and cathcing errors at runtime. Here the void Function(int) is directly given the type void Function(dynamic), which it doesn't satisfy from a subtyping perspective. It effectively treats void Function(dynamic) as if it was void Function(Never) (the actual supertype of all functions with one argument) plus it has a dynamic check at every invocation and up-cast, to makes sure the argument matches that actual parameter type. My comment above was to show that this is non-trivial, because you can't see all these down-casts at compile-time, some may be hidden by generics, and generics cannot distinguish Object? and dynamic internally (they're generic in the type, that means they don't care what the type is, the same code works in either case). That would likely mean every function invocation or cast which involves generics, and therefore might involve a type that is dynamic, has to have extra runtime checks. That's not good for performance.
• Treat dynamic as something other than a type, something that is both a top and a bottom type (like in Dart 1). A function type of void Function(dynamic) doesn't really have a type as parameter type, it has dynamic, which is not related to other types by the normal subtype/supertype relationship. It can be assigned to or from any function type, but that will include a runtime type check. When you try to call something of that type, it's a half-typed dynamic invocation that checks the argument value against the actual parameter type. You likely wouldn't be allowed to use dynamic as a type argument, whether for a generic class or a function. Those have bounds and bounds are based on subtyping, and this dynamic doesn't play well with subtyping. Which could actually be kind of neat 😁. But it's not what we have today, so that would be very breaking. Then it is possible to recognize all the places where a dynamic occurs directly or nested inside a type, where it's cast to something other than `dynamic, and add an extra runtime check. But again, a very different language than what is today, so not a likely change.

So:

Doesn't it also have to consider the possibility that it is a down-cast that gets checked at runtime because of dynamic?

If we introduce this feature, yes. In some way.
But then every such downcast has to do that check, which is an extra overhead.

The example here does a number of assignments, but let's boil it down to a simpler example.

void Function(int) f = (int x) {}; // Same type, completely safe.
void Function(dynamic) d = f; // Proposed allowed.
var dlist = <void Function(dynamic)>[d]; // Same type, should be allowed, right?
List<void Function(String)> slist = dlist; // Up-cast! Should be safe.
void Function(String) g = slist.first; // Same type.

The first line is trivial, the second line is what is proposed to be allowed.
The third line creates a list with the same type as the static type of its only element.
The fourth line up-casts the list. It should be safe because void Function(String) is a supertype of void Function(dynamic) as long as dynamic is a top type.
The fifth line reads a value from a list and assigns it to a variable with the same type. That should be safe.

Every operation here, other than the second line, is an assignment where the current language knows there is no need to do a type check, because it's assigning an expression with a static type to a variable with a declared type that is a supertype of (or the same as) that static type. In a sound language, it cannot fail.

But it's not sound because it ends up assigning a void Function(int) to void Function(String), which it isn't.

Where would you insert an extra check, which would throw in this case, to make this code sound?
And would there be any way to avoid that check, or will it have to happen on all assignments of that form, whether dynamic was involved or not.

To make this possible, it has to either change what dynamic means everywhere, or change what assignment to void Function(dynamic) means. Otherwise it'll just be unsound.

[ghost]

The question is not why this is an error, but why this is a runtime error.
The compiler has enough information to flag the cast statically:

main() {
  final f=(final num n)=>print(n);  
  (f as void Function(dynamic))(5); // runtime error. Why not a static error?
  5 as String; // not a static error either
  5 as int; // Warning: unnecessary cast
}

It seems as is treated like: "well, the user wants to cast in runtime - let him do it, hehe". But there's still an "unnecessary cast" warning for 5 as int.

Sometimes, the situation is reversed: the compiler knows for sure that the runtime check would be redundant, but there's no mechanism for the compiler to tell the runtime: don't check the type; I guarantee it's correct (that's what I gathered from the parallel thread). The check can be quite expensive, especially in the case of a function parameter.

My speculation is that the phenomenon is a heritage of dart 1, where the static types were not even preserved in runtime.

[RohitSaily]

@lrhn

An up-cast is a cast to a supertype. That's generally safe to do because an instance of the subtype must be an instance of the supertype. So no runtime checks are needed or done for up-casts.

I understand this, for example String can be up-casted to Object because String is an Object. Similarly, an int can be up-casted to dynamic because dynamic is one of multiple top-types.

void Function(int) f = (int x) {}; // Same type, completely safe.

This looks good to me, the annotated type is the strongest type of the right-hand-side.

void Function(dynamic) d = f; // Proposed allowed.

Yes, this is what I would like i.e. void Function(int) being casted to void Function(dynamic).

var dlist = <void Function(dynamic)>[d]; // Same type, should be allowed, right?

This looks good to me, the list's type parameter is identical to the item.

List<void Function(String)> slist = dlist; // Up-cast! Should be safe.

I believe this a part I did not understand prior to your example. If I understand correctly now, then this is an up-cast because normally whatever a function can do to a supertype it should be able to do to any of its subtypes, since any subtype is guaranteed to have that supertype's API. I believe I was getting confused because normally changing a subtype to its supertype in terms of extends/implements/mixin hierarchy is an up-cast.

The only way to avoid a breaking change would be to introduce another type that functions like dynamic but isn't a top-type like dynamic. For this discussion let's use the _ notation. I have no idea how the runtime type-checking is done and what information is kept track of, so as a start point what if we had

void Function(int) f = (int x) {}; // Same type, completely safe.
void Function(_) d = f; // Proposed allowed.
var dlist = <void Function(_)>[d]; // Same type, should be allowed, right?

But from there up-casting is not permitted (unless there is a way to reasonably allow as which performs a check and throws if needed). The function can still be called as if it were dynamic and when it is called the checks would be performed. Can these checks only be present when the function is of a type with _? If not, then perhaps the function does not even have to be allowed to be callable when it is that type either. This could still solve the issue I cited in the original post

if (widget is RadioListTile<_>)
{     widget.onChanged; // Accessing the member won't crash because there is no type issue
}

Or would this be pointless compared to the solution which ended up working:

if (widget is RadioListTile)
{     (widget as dynamic).onChanged; // Accessing the member doesn't crash
}