Known audio issues in iOS Safari browser

[anna-vasilko]

This ticket describes a couple of known audio issues in the iOS Safari browser and provides most recent status and Twilio recommendations on how to handle them:
Note:

• Both described issues are rarely reproducible. If you hit an audio problem all the time, make sure
  to go through this guide to avoid most common pitfalls
• Both issues are not Twilio SDK bugs, but iOS Safari browser issues.

1. Remote audio does not play occasionally in iOS Safari browser

Webkit Bug: https://bugs.webkit.org/show_bug.cgi?id=198545
Status: This issue was fixed by Apple, the fix is released in iOS version 13.6.
Reproducibility of this issue before the fix

• Group rooms - occurs in 0-2 rooms out of 50
• Peer to peer rooms - occurs in 6-9 rooms out of 50

2. RemoteParticipants can't hear audio from iOS Safari after using another app (ex. Siri, Youtube etc)

Webkit Bug: https://bugs.webkit.org/show_bug.cgi?id=208516
Status: A workaround for this issue is shipped in SDK version 2.6.0. The workaround is expected to work on iOS versions 13.4.X and 13.5.x, and based on our testing it solves the problem in most of the interruption use cases. We will continue adding iterative improvements in the future for some more rare/edgy use cases.
Reproducibility This is 100% reproducible in particular scenarios when other applications like Siri or Youtube takes over the mic. Example: Join a video room, share audio/video -> Open Siri and talk to it -> come back to your video application -> now your mic will be sending silence to the room, remote party will not be able to hear you.
Recommendation:

• Update to the latest SDK version which contains a workaround 2.6.0.
• Add an audio indicator UI in your app so that your users can see if they are sending silent audio and react to it. More here.

Follow this guide with the best practices from our dev team to avoid media issues across browsers in your applications:

Other important bugs in iOS Safari:

• Improper audio routing
• Audio interruption issue
• Audio mute/unmute issue
• Audio volume inconsistency issue
• Previously captured track is muted when getUserMedia is called again

[bjm88]

+1. We are eagerly awaiting a fix for this from Twilio or Safari. We are able to reproduce #1 remote audio not playing on iOS Safari iPhone setups a lot, like 50% of the time. This is after doing the suggested work arounds of pause/play and avoiding rerendering of tag.

[manjeshbhargav]

@bjm88

We are in touch with Safari WebKit engineers and will keep you posted on any future updates from them regarding these issues.

Thanks,

Manjesh Malavalli
JSDK Team

[XuanyuHu]

@manjeshbhargav
Could you help to check for the iPadOS as well?
We have clients reported that the iPad Safari is losing audio as well.
Around 90% time, refreshing the Safari Browser will resolve the issue (audio will work).

[bjm88]

thanks @manjeshbhargav - ya fyi We escalated with Apple and put in developer ticket with them, 734340602, if you need to reference.

We had both the "option #2" pause/play and the "avoid rerender by adding directly to dom" audio track attachment workarounds in and were still able to reproduce the issue about 30-50% fo the time. We actually toook out the pause/play and just left the attachment change and are seeing better results, seems like more like 1 out of 10, but still early in the testing.

[manjeshbhargav]

@XuanyuHu , @bjm88 ,

In our best practices guide, we recommend that you provide an audio level indicator for your local and remote audio tracks, so that the application can recommend users to reload the app or restart their browsers. Please take a look at the recommendations and apply them where appropriate.

Thanks,

Manjesh Malavalli
JSDK Team

[bjm88]

thx @manjeshbhargav we are planning to do that. We did notice the issue happens more on older iPhones 6/7 and less on X. Also our QA eng found she could reproduce a LOT on normal iPhone settings with Siri enabled, but when disabling Siri could not reproduce at all. Perhaps something to rely to apple.

[anna-vasilko]

@bjm88 Thank you so much for sharing your observations, I relayed it to the Safari team at Apple

[elebumm]

I have the same issue where the entire calls works apart from audio being sent from the iOS device to the participant (Chrome usually but anything really.)

I was able to create this issue on my iPhone 11 Pro Max doing the following:

• Go to your web application on safari
• Go back to the home screen (while having safari in the background)
• Use an audio input of some sort. (I just said hi to Siri)
• End that audio input
• Go through the typical flow of how your application starts (accept permissions etc)
• So far, 100% of the time it detects audio device but doesn't capture audio (or send)

I can fix this by exiting safari and opening again.

I feel like this happens when Safari is open the entire time. I had it happen when safari was in the background for at least 5 minutes. This could be depending on what "type" of person you are:

• Someone who has all of their applications open in the background constantly (my dad)
• Someone who exits as soon as their done (me).

I am on iOS 13. Let me know if anyone finds issue running into it this way as well.

I have temporarily fixed this by adding tests mentioned by @anna-vasilko before video and audio is enabled and gives instructions as to how to fix it.

Let me know if this was helpful for anyone.

[chr4ss1]

I can produce audio issues in almost 99% of time in 1v1 direct rooms between ios devices, here is how:

1. I force everyone to go through mic / video check.

2. After they have gone through the mic check together, their browsers will automatically refresh by using location.reload(), (I thought I am clever: that would ensure there is no old garbage left behind, such as some streams floating around, basically start from completely clean sheet)

3. They now press Join room button and will be joined using Twilio.connect()

4. Neither party can hear each other. If they both refresh their browsers manually, they start hearing each other.

Basically this approach backfired. It looks like using location.reload() will almost guarantee that there is no audio anymore. Perhaps this method can be used by other people to get into the state of "there is no audio" very easily.

For now I have removed the window.location.reload() and added a button of "I can't hear the other person" which will show popup which tells both people to refresh their browser. I've also use only one single audio element throughout the app which is decoupled from react:

audioService.setRef(document.getElementById('audio-element'));

render(
  <App />,
  document.getElementById('root'),
);

[hmheng]

im trying to use the proposed workaround in #922
with pause() and play().
however, Mac Safari tell me it is NotAllowedError, similar to the issue : mediaelement/mediaelement#2410