Use AOT mode in the Dart SDK binaries

[a-siva]

Use AOT mode in the Dart SDK binaries (faster startup and more predictable performance)
List of items to account for before this task is complete

• Switch frontend server invoked by dartdev to AOT mode
• Switch frontend server invoked by flutter tools to AOT mode
• Switch Analysis server to AOT mode
• Switch DDS which is launched as a separate process to AOT mode
• Switch DTD which is launched as a separate process to AOT mode
• Switch 'dartdev compile kernel' to use an AOT frontend
• Switch 'dartdev compile exe' to use an AOT frontend
• Switch 'dartdev compile wasm' to use an AOT frontend
• Switch dart2js launched from dartdev to use AOT mode
• Switch dartdevc launched from pkg/dev_compiler to use AOT mode
• Switch dartdev which includes a number of smaller utilities to AOT mode
• Switch gen_kernel to use an AOT snapshot when invoked from 'dart compile exe'
• Remove the kernel isolate from the VM, rename the VM binary to dartvm and have the VM only support running from kernel files

[a-siva]

CL to switch dds to run dds in AOT mode https://dart-review.googlesource.com/c/sdk/+/327140

[a-siva]

CL to switch gen_kerenl to run in AOT mode when invoked from 'dart compile exe' command https://dart-review.googlesource.com/c/sdk/+/327520?tab=checks

[derekxu16]

PR to add --frontend-server-starter-path option to flutter run and flutter test: flutter/flutter#135038

PR to modify flutter_tools to start frontend_server from an AOT snapshot: flutter/flutter#136282

[derekxu16]

PR to make FrontendServerClient start the frontend server from the AOT snapshot in the Dart SDK: dart-lang/webdev#2263

[mraleph]

On the topic of AOT compiling our tools:

$ tools/test.py -c dart2analyzer --use-sdk co19/Language
...
[01:20 | 100% | +20451 | -   96]

=== 20451 tests passed, 96 failed ===

________________________________________________________
Executed in   81.51 secs    fish           external
   usr time  560.09 secs  134.00 micros  560.09 secs
   sys time   57.52 secs  769.00 micros   57.52 secs

The same but changing to run from AOT snapshot (which takes ~12s to produce):

$ tools/test.py -c dart2analyzer --use-sdk co19
[00:52 | 100% | +20451 | -   96]

=== 20451 tests passed, 96 failed ===

________________________________________________________
Executed in   54.07 secs    fish           external
   usr time  334.59 secs  124.00 micros  334.59 secs
   sys time   32.50 secs  799.00 micros   32.49 secs

/cc @scheglov @bwilkerson @srawlins

[srawlins]

Very nice!

[stereotype441]

@a-siva looking at pkg/dartdev/lib/src/sdk.dart and pkg/dartdev/lib/src/analysis_server.dart, it looks like the analysis server still hasn't been switched over to AoT. But the box "Switch frontend server invoked by Analysis server to AOT mode" is checked above. Am I misreading things, or is it possible that this box got checked by accident?

I'm wondering because discussion at #56454 makes it seem like the analysis server is still being invoked in Jit mode.

[srawlins]

I think it's still in JIT mode because we don't support running plugins from AOT; we always run them from source (JIT); please forgive any misunderstandings I have here 😅 .

I did some prototyping with AOT-compiled plugins, for our plugins V2; so it's a thing we can do.

[scheglov]

Note also that the current approach to run macros in analyzer and DAS is to compile them into kernel files, and run in the same process.

[a-siva]

@a-siva looking at pkg/dartdev/lib/src/sdk.dart and pkg/dartdev/lib/src/analysis_server.dart, it looks like the analysis server still hasn't been switched over to AoT. But the box "Switch frontend server invoked by Analysis server to AOT mode" is checked above. Am I misreading things, or is it possible that this box got checked by accident?

I'm wondering because discussion at #56454 makes it seem like the analysis server is still being invoked in Jit mode.

Yes, the analysis server has not yet been switched to AOT yet. Not sure why it had the checked flag above. Fixed it.

[a-siva]

Note also that the current approach to run macros in analyzer and DAS is to compile them into kernel files, and run in the same process.

We have had this discussion with the CFE team too and they have an out of process mode for executing the kernel files generated from macros. maybe the Analyzer should also consider this.

[a-siva]

I think it's still in JIT mode because we don't support running plugins from AOT; we always run them from source (JIT); please forgive any misunderstandings I have here 😅 .

I did some prototyping with AOT-compiled plugins, for our plugins V2; so it's a thing we can do.

Yes, it is still JIT mode, there was some discussion regarding plugins here https://docs.google.com/document/d/1W0JmNwlWD9W8Jf1vzC-W7MSOR-l1XOquVUCaf_7gs3A/edit?resourcekey=0-3fhJDVXmmWr4gLkBcRB9Sg&disco=AAAAwkyH-sY

[mraleph]

cc @bwilkerson @mit-mit this is related to our discussions regarding prioritizing efforts which would allow us to switch analyzer to AOT mode targeting next stable.

[mkustermann]

@a-siva Is there any plan to also switch the formatter?

Developers often have the format-on-save enabled, which will make the IDE/Editor format the source before writing to disc.
I have noticed that there's a really noticeable (and annoying) delay when working on Dart files with format-on-save.

Assume we have a dart file of length 1029:

% wc -l sdk/lib/_internal/wasm/lib/js_string.dart
1029 sdk/lib/_internal/wasm/lib/js_string.dart

Compare dart format - taking half a second and 91MB

% measure out/ReleaseX64/dart-sdk/bin/dart format sdk/lib/_internal/wasm/lib/js_string.dart   
Formatted 1 file (0 changed) in 0.40 seconds.
        Memory: 91080 kB  -     0:00.53 real,   0.64 user,      0.11 sys

with AOT compiled package:dart_styles bin/format.dart:

% measure ./third_party/pkg/dart_style/bin/format.exe format sdk/lib/_internal/wasm/lib/js_string.dart
No file or directory found at "format".
Formatted 1 file (0 changed) in 0.04 seconds.
        Memory: 18008 kB  -     0:00.05 real,   0.02 user,      0.03 sys

=> that's 10x faster and 5x less memory.

Since the dart format is used by editors very often, I think it should be a priority on making it fast.

[stereotype441]

@mkustermann

@a-siva Is there any plan to also switch the formatter?

Developers often have the format-on-save enabled, which will make the IDE/Editor format the source before writing to disc. I have noticed that there's a really noticeable (and annoying) delay when working on Dart files with format-on-save.

I have seen this slowness too, and I agree that it's really noticeable and annoying. However, switching the formatter to AOT won't address it, because format-on-save doesn't use dart format directly; it works by asking the analysis server to format the file. The reason the analysis server frequently takes a long time to reformat the binary is because it's a single-threaded program, and if it's in the middle of re-analyzing a large library cycle at the time the reformat request arrives (which is common, because people usually save files shortly after making changes), it has to finish its re-analysis before it can respond.

@scheglov is doing some work to try to make the analyzer handle dependencies in a more fine-grained fashion, which should improve this a lot by reducing how often the analyzer has to re-analyze large library cycles in the first place.

[a-siva]

@a-siva Is there any plan to also switch the formatter?

My assumption was that the dart formatter is part of dartdev by importing it here

import 'package:dart_style/src/cli/format_command.dart';

and when dartdev itself becomes an AOT module that would take care of it.

[mkustermann]

I have seen this slowness too, and I agree that it's really noticeable and annoying. However, switching the formatter to AOT won't address it, because format-on-save doesn't use dart format directly; it works by asking the analysis server to format the file.

Well, that depends on the editor and plugins used. At least my editor uses dart format - and it seems in my measurements that switching to AOT would make it 10x faster and use 5x less memory.

(Though interesting to know that the editor delegates this sometimes to LSP plugin - not something I would expect)

My assumption was that the dart formatter is part of dartdev by importing it here
and when dartdev itself becomes an AOT module that would take care of it.

It's a choice. I'm not sure if people consciously thought about this. It seems dartdev has 3 variants now:

• Directly import & call the tool (IIRC formatter case)
• Spawning an app-jit snapshot via Isolate.spawnUri (IIRC dart2js case)
• Running aot snapshot as subprocess (IIRC dart2wasm case)

What determines which tool uses which mechanism?

Is the plan to eventually have all tools in one big AOT snapshot or still a mixture of isolate spawning, subprocess spawning or direct invocation (which seems rather messy)?

Depending on the timelines when dartdev itself moves to AOT, one could consider making the formatter call out to AOT compiled formatter as subprocess.

[mit-mit]

@srawlins can we mark the analyzer done in the checklist now that 8d62a1f landed?

[srawlins]

Hmm, I don't think so. Maybe @a-siva can say what the concrete goal is. I see there are many binaries that we ship as both a JIT snapshot and as an AOT snapshot (like dart_tooling_daemon, and dartdevc).

With 8d62a1f it is possible to build an AOT snapshot of DAS, via build.py analysis_server_aot_product. But it is not usable as dart analyze nor dart language-server yet. After https://dart-review.googlesource.com/c/sdk/+/424342 it will be usable via dart language-server, but not the default snapshot yet. It also isn't used in google3 yet. So I am planning on landing a few more CLs to get all of the common entrypoints using the AOT snapshot.

[srawlins]

An update for the analysis server:

• The current SDK provides both the current JIT snapshot of analysis_server, and now an AOT snapshot of analysis_server.
• I have code landed to support AOT in dartdev, dart language-server (the command IDEs use), dart analyze (the command line tool), and dart fix.
• Each of these is disabled by default, meaning each of the dartdev commands will use the JIT snapshot by default. To use the AOT snapshot of the analysis server, you can pass --use-aot-snapshot.

So this is more-or-less what is going out in the Dart 3.9 Beta 1. I should be able to land that dart fix CL. I'd like to ask folks, especially @rrousselGit, to experiment with using the AOT snapshot.

Next, I'd like to flip the flag in Dart 3.9 Beta 2, so that the AOT snapshot is the default in each of those commands. But the JIT snapshot is still available and can be used via --no-use-aot-snapshot as an escape hatch.

Later, we can remove the escape hatch, and the JIT snapshots. Maybe 3.9 final? Maybe 3.10?