Platform/MediaSourceExtensions

Media Source Extensions

Timeline wise we are hoping to support enough of the spec to support the YouTube production player by the end of the year but will need to collaborate with YouTube to ensure that we're focussing on the right things. We also expect that dash.js compatibility will require a similar feature set.

2013 Q3

Adaptive streaming and simple ad insertion (multi-period approach)

Complete multiple decoder implementation. This is current work in progress and included as part of the Q3 goal to get the YouTube demo player working. We support webm but we need our own container parser to get fMP4 working outside of Windows 8.

Tasks:

• [DONE] Support multiple SourceBuffers with one decoder per SourceBuffer (multiple decoders per SourceBuffer requires container demuxing support)

At this point we don't support sending new initialisation segments to a decoder.

2013 Q4

Adaptive streaming for MP4 and SourceBuffer restarting

Reseting a SourceBuffer at the end of a logical stream requires information that is not available without a container parser. Being able to play multiple streams could be worked around by appending to a new SourceBuffer instead. In order to support fMP4 on some platforms (e.g. Windows 7, Vista) we also need a container parser.

Tasks:

• Extend nestegg parser for webm to support sourcebuffer reset
• [DONE] Container parser with fragmented mp4 support – Chris Pearce
• [DONE] MP4 playback on Windows Vista, 7 and 8
• [DONE] VP9 support on all platforms

2014 Q1 proposed

Memory management

Requesting of content through the JavaScript application is required for supporting both seeking and re-requesting of discarded time ranges. Discarding unneeded compressed data is needed to avoid wasting vast amounts of memory.

Tasks:

• Requesting data through JavaScript API [MediaSource.readyState SourceBuffer.buffered]
• Discarding of used coded frames [SourceBuffer.buffered]

YouTube integration

Testing and debugging YouTube playback.

Tasks:

• Bug fixing in MSE implementation
• Evangelism/upstreaming fixes

MP4 platform support

• MP4 playback on Firefox OS

2014 Q2 proposed

Seamless audio adaption

Implement audio cross fade to avoid noise when switching from one audio quality to another.

Tasks:

• Audio cross fading

Ad insertion (dynamic triggering approach)

Timeline management is needed for sequencing adverts using the dynamic triggering approach.

Tasks:

• Support segments mode [SourceBuffer.mode = AppendMode.segments]
• Timeline management [SourceBuffer.buffered]

MP4 platform support

• MP4 software decoding through OpenH264 - pending availability, high profile and AAC support

2014 Q3 suggested

MP4 platform support

• Hardware accelerated playback on Mac OSX
• Hardware accelerated playback on Android

Backlog

Track API

The Track API is a separate web spec that adds further functionality to MediaSource. It allows multilple languages, camera angles or aspect ratios.

Streams

There don't appear to be clear cases where SourceBuffer.appendStream() is necessary. This API needs more justification. YouTube expressed interest in using this API when it is available.

HTTP Live Streaming

It would be possible to support HLS in Javascript in terms on MSE although MSE is not specifically designed to enable this. We are not intending to support MPEG2-TS. Without this we would need to write also need t write a remuxer in Javascript.

MP4 platform support

• Hardware accelerated playback on Linux