Travis-CI build failures

[akarnokd]

Lately, both the main and PR builds fail with error 137, which is an indication the process was killed by SIGKILL and likely due to resource exhaustion (disk or memory?). I thought reverting one of my test changes will fix it (#2831) but strangely, only the PR's build succeeded, the main did not.

[benjchristensen]

I see this error:

16:47:29.151 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Changing state to: FAILED
16:47:29.156 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Process 'Gradle Test Executor 1' finished with exit value 137 (state: FAILED)
16:47:29.169 [DEBUG] [TestEventLogger] 
16:47:29.170 [DEBUG] [TestEventLogger] Test Run PASSED
16:47:29.173 [DEBUG] [org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter] Finished executing task ':test'
16:47:29.173 [LIFECYCLE] [class org.gradle.TaskExecutionLogger] :test FAILED
16:47:29.173 [INFO] [org.gradle.execution.taskgraph.AbstractTaskPlanExecutor] :test (Thread[main,5,main]) completed. Took 1 mins 1.11 secs.
16:47:29.173 [DEBUG] [org.gradle.execution.taskgraph.AbstractTaskPlanExecutor] Task worker [Thread[main,5,main]] finished, busy: 1 mins 14.42 secs, idle: 0.014 secs
16:47:29.188 [ERROR] [org.gradle.BuildExceptionReporter] 
16:47:29.188 [ERROR] [org.gradle.BuildExceptionReporter] FAILURE: Build failed with an exception.
16:47:29.189 [ERROR] [org.gradle.BuildExceptionReporter] 
16:47:29.189 [ERROR] [org.gradle.BuildExceptionReporter] * What went wrong:
16:47:29.189 [ERROR] [org.gradle.BuildExceptionReporter] Execution failed for task ':test'.
16:47:29.189 [ERROR] [org.gradle.BuildExceptionReporter] > Process 'Gradle Test Executor 1' finished with non-zero exit value 137
16:47:29.189 [ERROR] [org.gradle.BuildExceptionReporter] 
16:47:29.189 [ERROR] [org.gradle.BuildExceptionReporter] * Exception is:
16:47:29.192 [ERROR] [org.gradle.BuildExceptionReporter] org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':test'.
16:47:29.193 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:69)
16:47:29.193 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46)
16:47:29.193 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:35)
16:47:29.193 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:64)
16:47:29.193 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
16:47:29.193 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:42)
16:47:29.193 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
16:47:29.193 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53)
16:47:29.193 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
16:47:29.193 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.AbstractTask.executeWithoutThrowingTaskFailure(AbstractTask.java:305)
16:47:29.193 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.executeTask(AbstractTaskPlanExecutor.java:79)
16:47:29.194 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:63)
16:47:29.194 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:51)
16:47:29.194 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:23)
16:47:29.194 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:88)
16:47:29.194 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:29)
16:47:29.194 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62)
16:47:29.194 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.DefaultBuildExecuter.access$200(DefaultBuildExecuter.java:23)
16:47:29.194 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.DefaultBuildExecuter$2.proceed(DefaultBuildExecuter.java:68)
16:47:29.194 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
16:47:29.194 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62)
16:47:29.194 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:55)
16:47:29.194 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:149)
16:47:29.195 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:106)
16:47:29.195 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:86)
16:47:29.195 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:80)
16:47:29.195 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:33)
16:47:29.195 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:24)
16:47:29.195 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:36)
16:47:29.195 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:26)
16:47:29.195 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:51)
16:47:29.195 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:171)
16:47:29.195 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:237)
16:47:29.195 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:210)
16:47:29.195 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:35)
16:47:29.195 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24)
16:47:29.196 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:206)
16:47:29.196 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169)
16:47:29.196 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)
16:47:29.196 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)
16:47:29.196 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.Main.doAction(Main.java:33)
16:47:29.196 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)
16:47:29.196 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:54)
16:47:29.196 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:35)
16:47:29.196 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.GradleMain.main(GradleMain.java:23)
16:47:29.196 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:30)
16:47:29.196 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:127)
16:47:29.196 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:56)
16:47:29.197 [ERROR] [org.gradle.BuildExceptionReporter] Caused by: org.gradle.process.internal.ExecException: Process 'Gradle Test Executor 1' finished with non-zero exit value 137
16:47:29.197 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.process.internal.DefaultExecHandle$ExecResultImpl.assertNormalExitValue(DefaultExecHandle.java:365)
16:47:29.197 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.process.internal.DefaultWorkerProcess.waitForStop(DefaultWorkerProcess.java:161)
16:47:29.197 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.testing.worker.ForkingTestClassProcessor.stop(ForkingTestClassProcessor.java:86)
16:47:29.197 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.testing.processors.RestartEveryNTestClassProcessor.endBatch(RestartEveryNTestClassProcessor.java:60)
16:47:29.197 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.testing.processors.RestartEveryNTestClassProcessor.stop(RestartEveryNTestClassProcessor.java:54)
16:47:29.197 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
16:47:29.197 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
16:47:29.197 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.messaging.dispatch.FailureHandlingDispatch.dispatch(FailureHandlingDispatch.java:29)
16:47:29.197 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.messaging.dispatch.AsyncDispatch.dispatchMessages(AsyncDispatch.java:132)
16:47:29.197 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.messaging.dispatch.AsyncDispatch.access$000(AsyncDispatch.java:33)
16:47:29.197 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.messaging.dispatch.AsyncDispatch$1.run(AsyncDispatch.java:72)
16:47:29.198 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.internal.concurrent.DefaultExecutorFactory$StoppableExecutorImpl$1.run(DefaultExecutorFactory.java:64)
16:47:29.198 [ERROR] [org.gradle.BuildExceptionReporter] 
16:47:29.198 [LIFECYCLE] [org.gradle.BuildResultLogger] 
16:47:29.198 [LIFECYCLE] [org.gradle.BuildResultLogger] BUILD FAILED

But I can't find any unit tests failing or other clear issues even with debug turned on.

[benjchristensen]

Looks like memory as you considered.

[akarnokd]

The error log just details the test runner got sigkill (137). In #2834, I've added various leak detection and forced GCs and could get the PR built.

[benjchristensen]

Looks like increasing memory fixed it.