a few hacks that improve the build time #1028
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Building the whole BAP repository from scratch is now more than twice as fast and takes about 8 minutes or even less with faster CPUs and SSD drives. What is more important, rebuilding BAP now takes less than a minute even if something was changed deep in the BAP's core, like disassembler or even regular or bap-types. Below is the list of tricks that made this possible. 1) Disable -short-paths. This option was responsible for nearly x2 slowdown. Since nowadays we mostly do typechecking with merlin having short paths doesn't really matter anymore. So screw them. Short paths are also disable in bapbuild. 2) Disable cross-module optimizations. When cross-module optimizations are enabled (by default) every time a module changes all dependent modules are recompiled. This behavior sort of defeats the purpose of separate compilation and leads to 20 minutes of recompilation when some module in disassembler is changed. Now it won't happen and when any module is changed no other modules will be recompiled. If the interface is changed only the modules that import this interface will be recompiled. Theoretically, it may lead to less optimal builds. We didn't notice any slowdown yet, but we may consider putting this particular trick under a compilation flag and disable during releases. 3) Hotfix ocamlbuild linking rule. There is a bug in ocamlbuild that prevents caching from working properly. The rule for linking library mentions .so as its product, but it never builds it (and can't (and shouldn't)). As a result the caching system notices a missing object and triggers relinking of all libraries. Ideally, it should be fixed on the upstream (I will PR to them) but we don't want to wait for upsteam, especially since the fix is simple - we just override the rule by our own, that didn't mention any .so files. 4) Disables compression when plugins are packed. Plugins are zip files underneath the hood and when we pack and unpack them it takes time to compress them. It goes much faster (especially for custom llvm builds with debugging symbols) when the compression is disabled. This trick is also a candidate to be made optional. 5) Packs a plugin in one step. Instead of doing 4 packings/repackings we now do it only once. 6) Unlimits the number of jobs tried by ocamlbuild. This may break on Travis, but let's try. We all know that ocamlbuild is not very good in parallelizing tasks, but it tries its best. I didn't see more than 12 jobs at once, so it should be allright.
Member
Author
|
For the reference, we can remove our own cmxs rule as soon as this issue [is resolved][1] in the upstream. |
It should depend on '.a' also, as `cmxa` is a collection of `cmx` so it doesn't change when cross-module optimizations are disabled, or when the set of imported modules is unchanged.
Now all built plugins are cached inside the _build folder, instead of being rebuilt every time in a temporary folder. And md5sum of the transitive closure of plugin dependencies is used as the cache validator. If any dependency is changed, then the plugin is relinked and repacked. Additionally, the script builds only those plugins that were migrated into the build tree, so when we're installing via opam it doesn't try to rebuild already installed plugins. This is an extperimental patch, I'm concerned with its portability. We use md5sum utility, which is not present on MacOS X. However, there is md5 there, with a slightly different interface, which we're trying to use instead. Probably, we should rewrite this in OCaml instead, for better portability and performance. Finally, it introduces new system dependency, md5[sum], so our docker builds may fail.
Adds a new variable called `development` to the configure script that is disabled by default. When enabled (with the `--enable-development` option) it will enable certain options and optimizations that are in general not suitable for the released version. Right now this option, when enabled, disables cross-module optimizations. We might later attach more build time optimizations and tweaks to that option. The cross-module optimizations, when disabled will significantly improve the rebuilding time (at no to neglible preformance degradation). Turning them off won't affect a lot, the initial build time so there is no need to disable them during the initial installation.
gitoleg
approved these changes
Jan 28, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Building the whole BAP repository from scratch is now more than twice
as fast and takes about 8 minutes or even less with faster CPUs and
SSD drives.
What is more important, rebuilding BAP now takes less than a minute
even if something was changed deep in the BAP's core, like
disassembler or even regular or bap-types.
Below is the list of tricks that made this possible.
Disable -short-paths. This option was responsible for nearly x2
slowdown. Since nowadays we mostly do type-checking with merlin having
short paths doesn't really matter anymore. So screw them. Short paths
are also disabled in bapbuild.
Disable cross-module optimizations. When cross-module optimizations
are enabled (by default) every time a module changes all dependent
modules are recompiled. This behavior sort of defeats the purpose of
separate compilation and leads to 20 minutes of recompilation when
some module in disassembler is changed. Now it won't happen and when
any module is changed no other modules will be recompiled. If the
interface is changed only the modules that import this interface will
be recompiled. Theoretically, it may lead to less optimal builds. We
didn't notice any slowdown yet, but we may consider putting this
particular trick under a compilation flag and disable during releases.
Hotfix ocamlbuild linking rule. There is a bug in ocamlbuild that
prevents caching from working properly. The rule for linking library
mentions .so as its product, but it never builds it (and can't (and
shouldn't)). As a result, the caching system notices a missing object
and triggers relinking of all libraries. Ideally, it should be fixed
on the upstream (I will PR to them) but we don't want to wait for
upstream, especially since the fix is simple - we just override the
rule by our own, that didn't mention any .so files.
Disables compression when plugins are packed. Plugins are zip files
underneath the hood and when we pack and unpack them it takes time to
compress them. It goes much faster (especially for custom llvm builds
with debugging symbols) when the compression is disabled. This trick
is also a candidate to be made optional.
Packs a plugin in one step. Instead of doing 4 packings/repackings
we now do it only once.
Unlimits the number of jobs tried by ocamlbuild. This may break on
Travis, but let's try. We all know that ocamlbuild is not very good in
parallelizing tasks, but it tries its best. I didn't see more than 12
jobs at once, so it should be all right.