Skip to content

[native_assets_builder] Benchmark hook overheads #2236

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dcharkes opened this issue Apr 24, 2025 · 0 comments
Open

[native_assets_builder] Benchmark hook overheads #2236

dcharkes opened this issue Apr 24, 2025 · 0 comments

Comments

@dcharkes
Copy link
Collaborator

When we want to remove the experimental flag for hooks we should check what the typical overhead is for hooks:

  1. If there are no hooks at all in the package graph, do dart and flutter commands not regress.
  2. The first-time hooks are run, what is the overhead of the hook compared to the actual work done in the hook. (Actual work: building/downloading a binary.)
  3. For the 2nd-time run, how much is the overhead of the sha256 file hash checking.
@dcharkes dcharkes added this to the Native Assets v1.0 milestone Apr 24, 2025
@dcharkes dcharkes self-assigned this May 2, 2025
@dcharkes dcharkes moved this to In Progress in Native Assets May 13, 2025
github-merge-queue bot pushed a commit to flutter/flutter that referenced this issue May 15, 2025
The link hooks (`hook/link.dart`) in Flutter have no access to the
resource identifier experiment for tree-shaking yet. That means we can
start running hooks earlier in the flutter build.

When we do get around to adding tree-shaking information, we should only
have a dependency on `KernelSnapshot` for AOT builds in which the Dart
tree-shaker runs. Likely achieved by splitting the `DartBuildForNative`
into two targets, one for Debug and one for Release. (Note, the names
for these targets are somewhat weird, so we should address that in
another PR as well. It should probably be called `RunDartHooks` or
something.)

I don't expect this PR to have any performance effect for fast hooks.
Other targets already run concurrently with this target:

```
$ flutter build -d macos --verbose
[        ] [ +278 ms] compile_macos_framework: Starting due to {InvalidatedReasonKind.inputChanged: The following inputs have updated contents: /Users/dacoharkes/flt/flutter/examples/hello_world/.dart_tool/flutter_build/ce2ba8b9e7204bce6beba11f20d6df36/app.dill,/Users/dacoharkes/flt/flutter/packages/flutter_tools/lib/src/build_system/targets/macos.dart,/Users/dacoharkes/flt/flutter/bin/cache/engine.stamp}
[...]
[        ] [        ] dart_build: Starting due to {}
[        ] [  +20 ms] No packages with native assets. Skipping native assets compilation.
[        ] [   +1 ms] dart_build: Complete
[        ] [   +1 ms] install_code_assets: Starting due to {}
[        ] [   +1 ms] Writing native assets json to file:///Users/dacoharkes/flt/flutter/examples/hello_world/.dart_tool/flutter_build/ce2ba8b9e7204bce6beba11f20d6df36/native_assets.json.
[        ] [   +1 ms] Writing /Users/dacoharkes/flt/flutter/examples/hello_world/.dart_tool/flutter_build/ce2ba8b9e7204bce6beba11f20d6df36/native_assets.json done.
[        ] [        ] install_code_assets: Complete
[        ] [+1376 ms] Building App.framework for x86_64...
[...]
[        ] [  +41 ms] compile_macos_framework: Complete
```

However, this PR makes the hooks already run concurrently with kernel
compilation, which always takes quite a while.

```
[        ] [   +2 ms] kernel_snapshot_program: Starting due to {}
[...]
[        ] [+2515 ms] release_unpack_macos: Complete
[        ] [+2425 ms] kernel_snapshot_program: Complete
```

This means that if the hooks do actual work, and enough cores on the
host are available for Dart compilation and hook execution, the result
will be quicker than before.

### Context

* dart-lang/native#2236

### Testing

Existing native assets integration tests

### Benchmarks

Currently _not_ benchmarked. This will be benchmarked by the `flutter
build` and flutter-time-to-first-frame benchmarks once the experiment
flag is removed.
auto-submit bot pushed a commit that referenced this issue May 16, 2025
Bug: #2236

Optimize the code paths for when there are no hooks (speeds up standalone Dart).

* Return early in various places, and check if there are hooks at all as first thing.
* Write JSON encoding instead of YAML encoding. 17 ms -> 1 ms on a small JSON object. (And JSON is valid YAML.)

After this the longest running part in `dartdev` is loading the `package_config.json`: 10 ms. We can't really avoid this because we need that info to check whether we have any hooks that need running. And the `package_config.json` isn't already loaded by dartdev (in contrast to `flutter_tools` where we have a `PackageConfig` object available). dartdev delegates compilation to `package:pub`(!) via `getExecutableForCommand` and that API uses the package config file path. dart-lang/pub#4067
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: In Progress
Development

No branches or pull requests

1 participant