Commits on Jun 17, 2025

1. Add SIGINT handler for process_bazel_build_log.py (#3200) …

   **Background**
   
   - **Issue** : Building from Xcode got slower because bazel server got
   killed by Xcode Index build SIGINT.
   - We've observed that iOS build performance from Xcode are getting
   slower over time. It's discovered that even the normal build after no
   code change was causing building analysis cache from scratch. It was
   because the bazel server was terminated while running index build and
   because the analysis cache is saved in memory, bazel server for normal
   build needs to construct analysis cache from scratch. As a result, every
   time developers try build action, the analysis cache had to be built
   from scratch (taking a couple minutes in the beginning).
    
   - **Hypothesis on cause** : SIGINT sent more than once or even
   SIGTERM/SIGKILL sent by Xcode
     - **Why?** 
   - Apple engineer told me that Xcode would send SIGINT to index build if
   the index build takes more than certain threshold amount of seconds, and
   if the process doesn't exit in another certain threshold amount of
   seconds, Xcode would then send SIGTERM or SIGKILL eventually. Bazel
   already has [SIGINT
   handler](https://sourcegraph.com/github.com/bazelbuild/bazel/-/blob/src/main/cpp/blaze_util_posix.cc?L152-154),
   **which gracefully handles interrupt signal and keeps the server
   alive**. However, that was not our case, and I suspected that SIGINT was
   being sent more than once or even SIGTERM/SIGKILL is being sent.
     - **Testing hypothesis** : 
   - I installed a signal handler to [python script in rules_xcodeproj that
   is used to invoke bazel build command] and the signal handler would
   simply print out the received signal and just ignore the signal.
   - It turns out that only one SIGINT is sent to the python script
   `process_bazel_build_log.py` and gets ignored by the python script and
   that Bazel would still terminate ungracefully nevertheless. Therefore,
   the hypothesis is wrong.
   
   - **Real cause** : One SIGINT is sent by Xcode index build and Bazel
   server is terminating ungracefully (expected to cancel invocations but
   not terminate the server).
   - Here's how Xcode index build action starts the bazel build command in
   chain:
       - [generate_index_build_bazel_dependencies.sh]
   - |-->[bazel_build.sh] (called with `source` command making
   `bazel_build.sh` use the same process as the parent process)
           - |-->[process_bazel_build_log.py]
             - |-->Bazel command is run with Python `subprocess.popen()`
     - **Why is Bazel server terminated ungracefully?**
   - `bazel_build.sh` (same process as
   `generate_index_build_bazel_dependencies.sh`) ,
   `process_bazel_build_log.py` and `Bazel process` all belong to the same
   process group. And `Bazel` has SIGINT handler.
   - When Xcode sends SIGINT to the process group and each one of
   `bazel_build.sh (generate_index_build_bazel_dependencies)`,
   `process_bazel_build_log.py`, and `Bazel` processes gets SIGINT.
         - `Bazel` receives and ignores the first three SIGINT. 
   - `bazel_build.sh` receives SIGINT and would wait until the process
   `process_bazel_build_log.py` finishes.
   - **`process_bazel_build_log.py` doesn't have SIGINT handler, so when
   python process receives SIGINT, python process terminates right away and
   doesn't wait for `Bazel` process to end.**
   
   - **How to fix so that Bazel handles SIGINT gracefully**
   - Add SIGINT handler to the python script `process_bazel_build_log.py`
   and make the Python process for the script to wait until process for
   `Bazel` exits. SIGINT handler behavior would be to trap and ignore the
   signal.
   
   **Changes**
   
   - Add SIGINT handler to the python script `process_bazel_build_log.py`
   and make the Python process for the script to wait until process for
   `Bazel` exits. SIGINT handler behavior would be to trap and ignore the
   signal.
   
   **Test**
   - Test deployed rules_xcodeproj on mobile
     1. Generate Xcode project
     2. Run unit test target once. 
     3. Wait until index build starts Bazel build 
     4. Run unit test while index Bazel build is running 
   5. Confirm the Bazel process from the index build exits gracefully
   (prints out interruption signal and normally exits bazel invocation)
   6. Confirm the Bazel build for unit test target re-uses the analysis
   cache and do not build from scratch.
   
   Signed-off-by: Yongjin Cho <[email protected]>

   

   yongjincho92 authored Jun 17, 2025
   Configuration menu

   • View commit details
   • Copy full SHA for ff66d76
   • Browse repository at this point

   Copy the full SHA

   ff66d76 View commit details

   Browse the repository at this point in the history