Tackling video and audio on the mobile web

3 comments

Imagine that you’ve got this cool project where you’re building a multimedia scrollstory for the web. There’s some great video and audio content, the designer has made some really nice looking visuals, all you’ve got to do now is to bring it all together in a fantastic interactive extravaganza.

And then suddenly there’s a voice somewhere in the back of your head: wasn’t there something with audio and video on mobile webbrowsers?

The touch revolution has given us wonderful new possibilities, but also a lot of headaches on how to translate the desktop web experience to mobile and tablet. Audio and video might be one of the biggest hurdles.

So, what are those problems? And more importantly: how can we fix them?

Problems

Let’s start with the problems. Basically, there are three major problems with media on mobile.

  1. The biggie: on both iOS and Android (this includes both mobile and tablet devices) it is not possible to ‘autoplay’ a media file. What this means, in practice, is that you always need a user action to initiate an audio or video play, such as a button click. For ‘narrative video’, such as a YouTube embed, this is usually not a big problem, but it can be very frustrating when you’re trying to use video as a background layer (example).
  2. Only on iOS mobile (hence: iPhone and iPod touch) video will always start in a separate player, outside of the browser. This means that you can’t layer text or images over the video. It also means that the user gets taken out of the experience in the browser. Like the autoplay problem, this is mostly acceptable for ‘narrative video’, but awful for background video. Note that this problem does not apply to audio.
  3. On iOS (both tablet and mobile) you can only play one media file at the same time. When you have two videos on a page, you have one of them playing and you hit ‘play’ on the other, the first one will pause.

There are some minor problems too. I won’t delve too deep into these points, it’s just so that you know it when you encounter them:

Solutions

I could spend this whole article about why these restrictions exist on mobile, and whether they should be there or not.

I won’t. Let’s just say that there are some good reasons why you don’t want to have autoplay video on mobile (think sleazy advertisements) and some good reasons why you do (that wonderful multimedia project that you want to build). Instead of pointing fingers, let’s look at possible solutions.

First of all: there doesn’t exist one simple and easy solution that will fix all of these problems. There are some stable workarounds and others that are more experimental.

Some recent libraries have tried other hacks, especially for iPhone. All these libraries are experimental, so make sure you thoroughly test them out before using  in production.

Rethinking the problem

Another solution is to completely rethink the app. Video and audio might be problematic, but why use them at all? Instead of using video to give the user a nice experience, maybe you can get the same thing while using CSS animations or the <canvas> tag.

The elephant in the room is of course going native. All of these problems don’t exist when you’re building an Objective-C / Swift (iOS) or Java (Android) app. But going native brings it own problems, including the need to develop three times if you want to reach the widest audience (iOS, Android and web for desktop), the loss of easy sharing using URLs, the hassle of going through the App / Play store and the fact that mobile developers don’t come cheap.

When all else fails, there’s always the obvious and easiest solution: use click-to-play video when possible, and still images when there’s no other option. But it’s a bit sad that mobile users will get a watered-down version of the desktop experience.

In the end it all comes down to what you really think is important and which tradeoffs you’re willing to take.

Do you have any other solutions or workarounds? Add them in the comments.

Add a comment

3 comments