WebRTC Safari Video Bug

2017/09/26 code web

Apple finally released iOS 11! With it, WebRTC finally comes to the iPhone.

My favourite aspect of my little chat app is that I can actually send a small stream. I don't care much for seeing my friends in HD, I merely need to be able to see basic features of their faces (something that machine learning will surely help with down the line, for better or worse). To this end, I usually set the width of my video stream at around 320px, to make it really small. You can tweak bitrates and framerates instead, but the small size works well for me.

When adapting my codebase to support both Safaris, I hit some pretty subtle issues. After making some necessary tweaks (add playsline attributes to video elements for iPhone playback, for example), I started testing.

The first is that iPhone Safari can't handle "ideal" suggestions. In theory, setting width to ideal: 320 should find the nearest actually available resolution. On an iPhone, it's just an invalid constraint. However, one can demand an ideal: 352 constraint, based on the 352x288 resolution that the iPhone actually supports. Firefox Desktop will take this recommendation, and give back 320x240 as expected.

The Safari iPhone + Firefox Desktop example worked fine, so I thought I was done.

Unfortunately, when doing a 4-way stream, Chrome (Mobile and Desktop) as well as Safari Desktop started failing. getUserMedia would return the next-bext resolution, like Firefox, and I could see it on-screen. However, when the stream was received on the other side of the connection, and plugged into another video element, it would be blank. I couldn't verify, but I think audio was still being transmitted. The s. It had nothing to do with the network, as even passing a stream locally within a single document using WebRTC failed. To complicate things, the behavior was subtly different depending on which system initiated the call — Firefox sets the sdpMid of its various streams to terms like "audio", "video", and "data", whereas Chrome and Safari seem to use more opaque terms? I never figured out the reason for this.

In both Safari Desktop and Safari Mobile, a call to getSettings() to verify resolution data is useless, as they both return nonsensical width and height values. This means it took me a bit longer than it should to realize the real culprit: Safari and Chrome were also struggling, in some way, with the ideal constraints. If I set the recommendation to 320, they started working again. The current solution seems to be to ignore ideal altogether.

If we try 352 first, iPhone will work, but everyone else fails silently and irrecoverably, streaming broken video. If we try 320 instead, we can catch the iPhone error and modify its stream. It's ugly, but it works.