FIX: Android can now start/stop while ios tests are running

[tylerqr]

Proposed changes

The problem is when running tests locally, if ios tests are already running and you try to start/stop/restart android tests, it fails. Here's the ai generated details of the problem and solution:

Debugging Maestro iOS/Android Simultaneous Run Issue

Problem Statement

When running Maestro tests, if an iOS test is started first, attempting to subsequently start an Android test fails immediately with a Connection refused error (initially on localhost/[::1]:7560, later on 127.0.0.1:<port>). Starting the Android test first, followed by the iOS test, works without issues. The goal is to allow both platforms to be started and stopped independently without causing failures.

Debugging Steps & Findings

[ {
"command" : {
"applyConfigurationCommand" : {
"config" : {
"appId" : "com.rumbly"
},
"optional" : false
}
},
"metadata" : {
"status" : "COMPLETED",
"timestamp" : 1743355225699,
"duration" : 1
}
}, {
"command" : {
"stopAppCommand" : {
"appId" : "com.rumbly",
"optional" : false
}
},
"metadata" : {
"status" : "COMPLETED",
"timestamp" : 1743355225700,
"duration" : 28
}
}, {
"command" : {
"defineVariablesCommand" : {
"env" : {
"MAESTRO_FILENAME" : "killAndRelaunch"
},
"optional" : false
}
},
"metadata" : {
"status" : "COMPLETED",
"timestamp" : 1743355225695,
"duration" : 3
}
}, {
"command" : {
"launchAppCommand" : {
"appId" : "com.rumbly",
"clearState" : false,
"stopApp" : true,
"launchArguments" : {
"isMaestroTest" : "true",
"IS_MAESTRO_TEST" : "true",
"maestroTestRunning" : "true",
"testFile" : "killAndRelaunch"
},
"optional" : false
}
},
"metadata" : {
"status" : "FAILED",
"timestamp" : 1743355225729,
"duration" : 6090,
"error" : {
"message" : "Unable to launch app com.rumbly: null",
"stackTrace" : [ {
"classLoaderName" : "app",
"methodName" : "launchAppCommand",
"fileName" : "Orchestra.kt",
"lineNumber" : 890,
"nativeMethod" : false,
"className" : "maestro.orchestra.Orchestra"
}, {
"classLoaderName" : "app",
"methodName" : "executeCommand",
"fileName" : "Orchestra.kt",
"lineNumber" : 299,
"nativeMethod" : false,
"className" : "maestro.orchestra.Orchestra"
}, {
"classLoaderName" : "app",
"methodName" : "executeCommands",
"fileName" : "Orchestra.kt",
"lineNumber" : 208,
"nativeMethod" : false,
"className" : "maestro.orchestra.Orchestra"
}, {
"classLoaderName" : "app",
"methodName" : "runFlow",
"fileName" : "Orchestra.kt",
"lineNumber" : 139,
"nativeMethod" : false,
"className" : "maestro.orchestra.Orchestra"
}, {
"classLoaderName" : "app",
"methodName" : "runCommands",
"fileName" : "MaestroCommandRunner.kt",
"lineNumber" : 187,
"nativeMethod" : false,
"className" : "maestro.cli.runner.MaestroCommandRunner"
}, {
"classLoaderName" : "app",
"methodName" : "invoke",
"fileName" : "TestRunner.kt",
"lineNumber" : 70,
"nativeMethod" : false,
"className" : "maestro.cli.runner.TestRunner$runSingle$result$1"
}, {
"classLoaderName" : "app",
"methodName" : "invoke",
"fileName" : "TestRunner.kt",
"lineNumber" : 61,
"nativeMethod" : false,
"className" : "maestro.cli.runner.TestRunner$runSingle$result$1"
}, {
"classLoaderName" : "app",
"methodName" : "runCatching",
"fileName" : "TestRunner.kt",
"lineNumber" : 175,
"nativeMethod" : false,
"className" : "maestro.cli.runner.TestRunner"
}, {
"classLoaderName" : "app",
"methodName" : "runSingle",
"fileName" : "TestRunner.kt",
"lineNumber" : 61,
"nativeMethod" : false,
"className" : "maestro.cli.runner.TestRunner"
}, {
"classLoaderName" : "app",
"methodName" : "runSingleFlow",
"fileName" : "TestCommand.kt",
"lineNumber" : 416,
"nativeMethod" : false,
"className" : "maestro.cli.command.TestCommand"
}, {
"classLoaderName" : "app",
"methodName" : "access$runSingleFlow",
"fileName" : "TestCommand.kt",
"lineNumber" : 63,
"nativeMethod" : false,
"className" : "maestro.cli.command.TestCommand"
}, {
"classLoaderName" : "app",
"methodName" : "invoke",
"fileName" : "TestCommand.kt",
"lineNumber" : 391,
"nativeMethod" : false,
"className" : "maestro.cli.command.TestCommand$runShardSuite$2"
}, {
"classLoaderName" : "app",
"methodName" : "invoke",
"fileName" : "TestCommand.kt",
"lineNumber" : 361,
"nativeMethod" : false,
"className" : "maestro.cli.command.TestCommand$runShardSuite$2"
}, {
"classLoaderName" : "app",
"methodName" : "newSession",
"fileName" : "MaestroSessionManager.kt",
"lineNumber" : 110,
"nativeMethod" : false,
"className" : "maestro.cli.session.MaestroSessionManager"
}, {
"classLoaderName" : "app",
"methodName" : "newSession$default",
"fileName" : "MaestroSessionManager.kt",
"lineNumber" : 54,
"nativeMethod" : false,
"className" : "maestro.cli.session.MaestroSessionManager"
}, {
"classLoaderName" : "app",
"methodName" : "runShardSuite",
"fileName" : "TestCommand.kt",
"lineNumber" : 361,
"nativeMethod" : false,
"className" : "maestro.cli.command.TestCommand"
}, {
"classLoaderName" : "app",
"methodName" : "access$runShardSuite",
"fileName" : "TestCommand.kt",
"lineNumber" : 63,
"nativeMethod" : false,
"className" : "maestro.cli.command.TestCommand"
}, {
"classLoaderName" : "app",
"methodName" : "invokeSuspend",
"fileName" : "TestCommand.kt",
"lineNumber" : 321,
"nativeMethod" : false,
"className" : "maestro.cli.command.TestCommand$handleSessions$1$results$1$1"
}, {
"classLoaderName" : "app",
"methodName" : "resumeWith",
"fileName" : "ContinuationImpl.kt",
"lineNumber" : 33,
"nativeMethod" : false,
"className" : "kotlin.coroutines.jvm.internal.BaseContinuationImpl"
}, {
"classLoaderName" : "app",
"methodName" : "run",
"fileName" : "DispatchedTask.kt",
"lineNumber" : 104,
"nativeMethod" : false,
"className" : "kotlinx.coroutines.DispatchedTask"
}, {
"classLoaderName" : "app",
"methodName" : "run",
"fileName" : "LimitedDispatcher.kt",
"lineNumber" : 111,
"nativeMethod" : false,
"className" : "kotlinx.coroutines.internal.LimitedDispatcher$Worker"
}, {
"classLoaderName" : "app",
"methodName" : "run",
"fileName" : "Tasks.kt",
"lineNumber" : 99,
"nativeMethod" : false,
"className" : "kotlinx.coroutines.scheduling.TaskImpl"
}, {
"classLoaderName" : "app",
"methodName" : "runSafely",
"fileName" : "CoroutineScheduler.kt",
"lineNumber" : 585,
"nativeMethod" : false,
"className" : "kotlinx.coroutines.scheduling.CoroutineScheduler"
}, {
"classLoaderName" : "app",
"methodName" : "executeTask",
"fileName" : "CoroutineScheduler.kt",
"lineNumber" : 802,
"nativeMethod" : false,
"className" : "kotlinx.coroutines.scheduling.CoroutineScheduler$Worker"
}, {
"classLoaderName" : "app",
"methodName" : "runWorker",
"fileName" : "CoroutineScheduler.kt",
"lineNumber" : 706,
"nativeMethod" : false,
"className" : "kotlinx.coroutines.scheduling.CoroutineScheduler$Worker"
}, {
"classLoaderName" : "app",
"methodName" : "run",
"fileName" : "CoroutineScheduler.kt",
"lineNumber" : 693,
"nativeMethod" : false,
"className" : "kotlinx.coroutines.scheduling.CoroutineScheduler$Worker"
} ],
"localizedMessage" : "Unable to launch app com.rumbly: null"
}
}
} ]

1. Initial Hypothesis: Port Conflict (IPv6)

   • Observation: Error message Connection refused: localhost/[0:0:0:0:0:0:0:1]:7560 suggested a potential conflict on the default port or an issue with IPv6.
   • Action 1: Searched for hardcoded port 7560. (Not found).
   • Action 2: Searched for gRPC server/port configuration. Found separate port selection logic in TestCommand.kt and MaestroSessionManager.kt.
   • Action 3: Unified port selection logic. Removed selectPort from TestCommand.kt and modified runShardSuite to pass driverHostPort = null to MaestroSessionManager.newSession, ensuring the manager's distinct port ranges (Android: 7100-7300, iOS: 8001-8200) and tracking were used.
   • Result 3: Error persisted, but changed slightly, still mentioning port 7001 (Connection refused: localhost/[::1]:7001). This indicated the port unification alone wasn't the fix and suggested a potential default fallback.

2. Hypothesis 2: Port Fallback / IPv4 Forcing

   • Observation: The error still involved port 7001, the default fallback in AndroidDriver. The MaestroSessionManager.createAndroid function (used for specific device IDs) wasn't selecting a port like pickAndroidDevice did; it passed the potentially null driverHostPort directly, causing the fallback. The connection was also still attempted over IPv6 (localhost).
   • Action 1: Modified AndroidDriver.kt's ManagedChannelBuilder to explicitly use "127.0.0.1" instead of "localhost" to force IPv4.
   • Action 2: Modified MaestroSessionManager.createAndroid to call selectAvailableAndroidPort() if driverHostPort was null, ensuring a port from the correct Android range was always selected and passed to AndroidDriver.
   • Result 2: Error persisted, but now correctly used IPv4 and a port from the Android range (e.g., Connection refused: /127.0.0.1:7107). This confirmed port selection and IP were fixed, but the underlying connection problem remained.

3. Hypothesis 3: adb forward Failure / Interference

   • Observation: Since the port and IP were correct, the issue likely lay in establishing the connection via adb forward or starting the on-device server, potentially due to interference from the active iOS driver tooling (idb/simctl).
   • Action 1: Added detailed logging around dadb.tcpForward (allocateForwarder) and am instrument (startInstrumentationSession) calls in AndroidDriver.kt.
   • Result 1 (Log Analysis): The logs showed the correct port being selected by MaestroSessionManager, but none of the new logs inside AndroidDriver.allocateForwarder or AndroidDriver.startInstrumentationSession appeared before the Connection refused error. This indicated the AndroidDriver.open() method was failing before these critical setup steps could log anything.
   • Action 2: Added logging around the driver.open() call itself within the Maestro.android factory method in Maestro.kt.
   • Result 2 (Log Analysis): Again, no logs from the factory method (neither "Attempting to call driver.open()" nor success/failure messages) appeared in the log file before the error.
   • Action 3: Attempted manual check of adb forward --list during the Android test startup.
   • Result 3: User reported the Android test failed too quickly (<1 second) to perform the manual check, strongly suggesting an immediate failure related to adb forward.
   • Action 4: Implemented a retry mechanism (2 attempts, 500ms delay) for the dadb.tcpForward call in AndroidDriver.allocateForwarder.
   • Result 4 (Log Analysis - /Users/tylerruff/.maestro/tests/2025-03-30_213651/maestro.log): The latest logs still show no log messages from within allocateForwarder (e.g., "Allocating ADB forward for hostPort... (Attempt 1/2)") before the Connection refused: /127.0.0.1:7226 error occurs.

Current Conclusion

The retry mechanism did not help because the failure occurs before or during the very first attempt to execute the dadb.tcpForward command inside the AndroidDriver.allocateForwarder method. Even the initial logging statement right before the try block in the retry loop is not reached.

This strongly indicates that the presence of an active iOS Maestro test run (and its associated tooling like idb or xcrun) causes an immediate, potentially silent failure within the adb/dadb system when Maestro tries to set up port forwarding for the Android device. The interference prevents the Android driver's initialization sequence from even beginning its core setup steps.

Potential Next Steps / Causes

• Investigate dadb library: The interaction between dadb and a potentially busy/interfered-with adb server needs closer examination. Is there a way dadb.tcpForward can fail without throwing a catchable exception immediately under specific adb server states?
• External adb command: As a diagnostic step (not a final fix), try replacing the dadb.tcpForward call temporarily with a direct execution of the adb forward tcp:<port> tcp:<port> command via Runtime.exec() or similar, to see if it yields a different error or behavior.
• Isolate iOS Interference: Identify precisely which part of the iOS driver startup/activity interferes with adb. Is it idb, xcrun simctl, or something else?
• Sequential Initialization: Explore if Maestro could internally sequence the driver initializations more carefully, perhaps ensuring adb operations complete fully before iOS-related tooling becomes highly active, though this might impact perceived startup time.

Resolution: Incorrect Initialization Order

• Observation: Further log analysis, after adding more granular logging and error handling, revealed that the Connection refused error occurred before the AndroidDriver.open() method (containing the allocateForwarder logic for adb forward) was explicitly called in the failing scenario (iOS started first).
• Root Cause: The stack trace showed the failure originated during a call to Maestro.getCachedDeviceInfo, which in turn calls AndroidDriver.deviceInfo(). This deviceInfo() method attempted a gRPC call to the Android driver without first ensuring the driver connection and port forwarding were established via AndroidDriver.open(). When an iOS test was already running, the interference likely prevented any implicit or preliminary connection/forwarding from succeeding, causing the immediate Connection refused when deviceInfo() tried to communicate.
• Solution: The AndroidDriver.deviceInfo() method was modified to explicitly check if the driver's open flag was true. If not, it now calls AndroidDriver.open() first, ensuring the necessary port forwarding (allocateForwarder) and instrumentation startup (startInstrumentationSession) complete before the deviceInfo gRPC call is attempted. This guarantees the connection is ready, regardless of whether deviceInfo() is called before methods like launchApp (which also call open()).

if (!open) {
    LOGGER.info("AndroidDriver.deviceInfo() called, but driver is not open. Calling open() first.")
    open()
}

This change allows Android and iOS tests to be started and stopped in any order without causing connection failures.

Testing

I can start/stop/restart android tests now no matter if ios tests are already running or not, on the same machine (local running tests). SUPER useful when writing new tests.

Issues fixed