• What did we do in the media team this year?
• What projects are ongoing?
• What can you expect in the (near or not too near) future?

• We added support for new codecs
• We changed quite a few internal things that matter
• We wrote new features

Written by Xiph.Org, standardized as RFC 6716, required for WebRTC.

• Software decoding on mobile ☹
• But we have a chip that can decode video ☺
• But the libraries are buggy ☹
• But now it works ☺ (mostly)

• H.264
• VP8 is kind of planned (Bug 821524)

Main goal: support H.264, AAC, mp3 without having to get a license, because we lost this war.

• Windows Vista+: WMF (Windows Media Foundation)
• Linux, Mac: GStreamer
• Mac: Core Video

• WMF: media.windows-media-foundation.enabled
• GStreamer: ac_add_option --enable-gstreamer
• Core Video: not yet implemented

• Main problem: UA sniffing
• HTMLVideoElement.canPlayType("video/webm")!
• The fact that it returns "maybe", "probably" or "" is not an excuse not to use the API

• Refactored massively content/media
  • Removed now useless class hierarchy
  • Clear interfaces, less strong-coupling
  • Removed unused code
• Prepare the code for others projects to plug in (WebRTC, WebAudio)
• It was a bit of a mess sometimes, the code is much more approcheable now

• Called libcubeb, not sure why
• Written in C, straightforward to use in other projets
• Abstracts away CoreAudio, WinMM, Alsa, PulseAudio, AudioTrack, sndio, OpenSL…
• Pull-based (uses callbacks instead of blocking write), hence less threads needed
• Opens the door for latency optimizations, needed for B2G, WebRTC, games
• Needs a last backend to be able to remove the old Sydneyaudio library

• DASH (Dynamic Adaptative Streaming over HTTP)
• Chained OGG support (at last!)
• playbackRate with pitch preservation
• Metadata access in Javascript
• Media sniffing for people that don't set Content-Type
• getUserMedia
• MediaStream framework to handle and synchronize All The Things™

• WebAudio
• WebVTT

• We lost at the spec game, and we now have to implement it
• Underspecified to death, hard to implement
• Behind a pref: media.webaudio.enabled
• A lot of DOM/refactoring work for now, nothing shiny to show
• We are close to actually outputing sound!

• Basically, like <canvas>, but for audio
• Games (spatial positionning of audio)
• Rich audio applications (sampler, synthetizer)
• Intended to be used with WebRTC, getUserMedia
• Will probably be used to polyfill everything we don't have

• Synchronized text tracks for media elements, e.g. subtitles for a video
• Uses a format very close to the the popular .srt files
• Written by David Humphrey's students, at Seneca College (Toronto), with the help of Mozilla folks

          00:00.000 --> 01:24.000
          Some subtitle
           
          02:04.000 --> 02:44.000
          Some other text right after
      

• MediaSource
• Latency
• (platform decoders)

• Allows author to pass in compressed audio buffers using Javascript to a media element
• Needed to implement custom buffering strategy, protocols, for media
• Usable to implement DRM stuff (☹)
• Essential for the success of Shumway
• Useful for massive hackery, very low-level

• Games
• WebRTC (getUserMedia)

• Lightstreams?
• Tighter integration?

• Slides: http://padenot.github.com/FOSDEM-13
• irc.mozilla.org #media
• dev-media@lists.mozilla.org