Record deferred accesses in kernel world impacts and use it for splitting only

under a flag.

This reapplies commit 70e1517, but adds a flag
to gradually migrate users before enabling it by default.

Patchset 1 matches the old CL

Change-Id: Iaf7ee3dec8d4aa658f0b4334549b507e5a610a68
Reviewed-on: https://dart-review.googlesource.com/c/86444
Reviewed-by: Stephen Adams <[email protected]>
Commit-Queue: Sigmund Cherem <[email protected]>

Author: sigmundch

CHANGELOG.md

@@ -21,6 +21,64 @@
 
 ### Tool Changes
 
+#### dart2js
+
+* We fixed a bug in how deferred constructor calls were incorrectly not
+  marked as deferred. The old behavior didn't cause breakages, but was imprecise
+  and pushed more code to the main output unit.
+
+* A new deferred split algorithm implementation was added.
+
+  This implementation fixes a soundness bug and addresses performance issues of
+  the previous implementation, because of that it can have a visible impact
+  on apps. In particular,
+
+    * We fixed a performance issue which was introduced when we migrated to the
+      Common front-end. On large apps, the fix can cut down 2/3 of the time
+      spent on this task.
+
+    * We fixed a bug in how inferred types were miscategorized (#35311). The old
+      behavior was unsound and could produce broken programs. The fix may cause
+      more code to be pulled into the main output unit.
+
+      This shows up frequently when returning deferred values from closures
+      since the closure's inferred return type is the deferred type.
+      For example, if you have:
+
+      ```dart
+      () async {
+        await deferred_prefix.loadLibrary();
+        return new deferred_prefix.Foo();
+      }
+      ```
+
+      The closure's return type is `Future<Foo>`. The old implementation defers
+      `Foo`, and incorrectly makes the return type `Future<dynamic>`. This may
+      break in places where the correct type is expected.
+
+      The new implementation will not defer `Foo`, and will place it in the main
+      output unit. If your intent is to defer it, then you need to ensure the
+      return type is not inferred to be `Foo`. For example, you can do so by
+      changing the code to a named closure with a declared type, or by ensuring
+      that the return expression has the type you want, like:
+
+      ```dart
+      () async {
+        await deferred_prefix.loadLibrary();
+        return new deferred_prefix.Foo() as dynamic;
+      }
+      ```
+
+    * Because the new implementation might require you to inspect and fix
+      your app, we exposed two temporary flags:
+
+        * `--report-invalid-deferred-types`: when provided, we will run
+          both the old and new algorithm and report where the issue was
+          detected.
+
+        * `--new-deferred-split`: enables the new algorithm.
+
+
 #### Pub
 
 #### Linter

pkg/compiler/lib/src/commandline_options.dart

@@ -78,6 +78,10 @@ class Flags {
 
   static const String serverMode = '--server-mode';
 
+  static const String newDeferredSplit = '--new-deferred-split';
+  static const String reportInvalidInferredDeferredTypes =
+      '--report-invalid-deferred-types';
+
   /// Flag for a combination of flags for 'production' mode.
   static const String benchmarkingProduction = '--benchmarking-production';
 

pkg/compiler/lib/src/compiler.dart

@@ -67,10 +67,9 @@ abstract class Compiler {
   /// Options provided from command-line arguments.
   final CompilerOptions options;
 
-  /**
-   * If true, stop compilation after type inference is complete. Used for
-   * debugging and testing purposes only.
-   */
+  // These internal flags are used to stop compilation after a specific phase.
+  // Used only for debugging and testing purposes only.
+  bool stopAfterClosedWorld = false;
   bool stopAfterTypeInference = false;
 
   /// Output provider from user of Compiler API.
@@ -251,6 +250,7 @@ abstract class Compiler {
       generateJavaScriptCode(results);
     } else {
       KernelResult result = await kernelLoader.load(uri);
+      reporter.log("Kernel load complete");
       if (result == null) return;
       if (compilationFailed && !options.generateCodeWithCompileTimeErrors) {
         return;
@@ -390,6 +390,7 @@ abstract class Compiler {
     selfTask.measureSubtask("compileFromKernel", () {
       JClosedWorld closedWorld = selfTask.measureSubtask("computeClosedWorld",
           () => computeClosedWorld(rootLibraryUri, libraries));
+      if (stopAfterClosedWorld) return;
       if (closedWorld != null) {
         GlobalTypeInferenceResults globalInferenceResults =
             performGlobalTypeInference(closedWorld);

pkg/compiler/lib/src/dart2js.dart

@@ -377,6 +377,8 @@ Future<api.CompilationResult> compile(List<String> argv,
     new OptionHandler(Flags.disableRtiOptimization, passThrough),
     new OptionHandler(Flags.terse, passThrough),
     new OptionHandler('--deferred-map=.+', passThrough),
+    new OptionHandler(Flags.newDeferredSplit, passThrough),
+    new OptionHandler(Flags.reportInvalidInferredDeferredTypes, passThrough),
     new OptionHandler(Flags.dumpInfo, passThrough),
     new OptionHandler('--disallow-unsafe-eval', ignoreOption),
     new OptionHandler(Option.showPackageWarnings, passThrough),