Agenda

What's new since WAC'16?

Where are we in the standardization process?

What's coming up?

commit time repartition

$ git log --format="%at" --since=2016-04-04 
          | python datehist.py `git rev-parse --show-toplevel`

Breakdown

New features

Breaking changes

Retro-specification :-(

destination has an output

var ac = new AudioContext();
[...]
// Get a MediaStreamDestinationNode
var msdn = ac.createMediaStreamDestination();
// NEW! connect the context destination to something!
ac.destination.connect(msdn);
// Feed it to a recorder, an RTCPeerConnection...
var rec = new MediaRecorder(msdn.stream);

Biquad Filter formula change

Rationale

https://github.com/WebAudio/web-audio-api/issues/771

Detection of old vs. new filters

https://github.com/GoogleChrome/web-audio-samples/wiki/Detection-of-lowpass-BiquadFilter-implementation

Test page

http://rtoy.github.io/webaudio-hacks/more/biquad/biquad-lowpass-q.html?usedB=true

AudioParam

• Nominal ranges

• value behaviour changed

• Introspection: minValue, maxValue, defaultValue

Doppler effect is gone

• Specification was fundamentaly broken

• Behaviour was weird

• Usage was minimal

• setVelocity on AudioListener and PannerNode

Latency and clock correlation

AudioContext.baseLatency

latency of the graph processing/buffering caused by sandboxing/etc.

AudioContext.outputLatency

usual audio output latency (audio callback -> ear)

AudioContext.getOutputTimestamp()

Correlation between Date.now() (system clock) & AudioContext.currentTime (audio subsystem clock)

Latency hint

new AudioContext({ latencyHint: "interactive" });
new AudioContext({ latencyHint: "playback" });
new AudioContext({ latencyHint: "balanced" });
new AudioContext({ latencyHint: 0.05 /* seconds */ });

Trade CPU/battery usage vs. audio latency

Authors can determine the actual latency:

var ac = new AudioContext({latencyHint: "playback"});
console.log("Base latency:" + ac.baseLatency);
console.log("Audio output  latency:" + ac.outputLatency);
console.log("Total latency:" + (ac.baseLatency + ac.outputLatency));

PannerNode AudioParam

A bunch of new AudioParams on the panner:

PannerNodeposition.{position{X,Y,Z}, orientation{X,Y,Z}}

... and on the AudioListener

AudioListener.{position{X,Y,Z},forward{X,Y,Z},up{X,Y,Z}}

a-rate parameters on "equal-power" PannerNode, k-rate on "HRTF" nodes.

  panner.setPosition(1,2,3);
  panner.positionX.value = 1;
  panner.positionY.value = 2;
  panner.positionZ.value = 3;

setValueCurveAtTime implicit setValueAtTime

ap.setValueCurveAtTime(array, startTime, duration);

now does the equivalent of:

ap.setValueCurveAtTime(array, startTime, duration);
ap.setValueAtTime(array[array.length - 1], startTime + duration);

currentTime progression

Incremented by a render quantum (128 sample-frames), atomically, without waiting for stable state, at the end of the render quantum.

Real constructors

  var ac = new AudioContext();
  var gain1 = ac.createGain();
  var gain2 = new GainNode(ac);
  var buf1 = ac.createBufferSource(2, 128, 44100);
  var buf2 = new AudioBuffer({ length: 128,
                               channels: 2,
                               sampleRate: 44100 });
  // Mono by default
  var buf2 = new AudioBuffer({ length: 128,
                               sampleRate: 44100});
  var pw = ac.createPeriodicWave([1,2,3], [3,2,1]);
  var pw2 = new PeriodicWave([1,2,3], [3,2,1]);

paves the way for subclassing AudioNodes.

AudioNode dictionnary initialisation

Allow to share initialization objects, available for all AudioNodes:

var ac = new AudioContext;
var gain = new GainNode(ac, { gain: 3.0 });
var delay = new DelayNode(ac, { delayTime: 0.3 });
var source = 
  new AudioBufferSourceNode(ac, { buffer: new AudioBuffer(...),
                                  playbackRate: 3.0
                                  detune: 700,
                                  loop: true });
var osc = new OscillatorNode(ac, { type: "square", detune: 700 });

ConstantSourceNode

The equivalent of:

var ac = new AudioContext();
var source = new AudioBufferSourceNode();
var gain = new GainNode();
var buffer =
  new AudioBuffer({length: 1, sampleRate: ac.sampleRate});
buffer.getChannelData(0)[0] = 1.0;
source.connect(gain);
source.buffer = buffer;

and modifying gain.gain.

var ac = new AudioContext();
var c = new ConstantSourceNode();
c.offset.value = 1000;
c.start(ac.currentTime + 3.5);
c.stop();

cancelAndHoldAtTime

New AudioParam method, that does what it says:

  var ac = new AudioContext;
  var gain = new GainNode(ac);
  gain.gain.setValueAtTime(1.0, ac.currentTime);
  gain.gain.setTargetAtTime(0.0, ac.currentTime, 0.1);
  gain.gain.cancelAndHoldAtTime(ac.currentTime + 0.1);
  // gain.gain.value is some value in between 0.0 and 1.0.

Allow delaying audio playback

In practice, the initial "suspended" => "running" transition is allowed to be delayed, this allow knowing there is no audio that is currently being output.

More-or-less standardization of Safari's behaviour on mobile, but with more discoverability.

MediaStreamTrackAudioSourceNode

New AudioNode, allowing to precisely decide which MediaStreamTrack of a MediaStream is going to be routed to an AudioContext:

// ms is some MediaStream
var ac = new AudioContext();
// route the second audio track of a given MediaStream
var mstan =
  new MediaStreamTrackAudioSourceNode(ac, ms.getAudioTracks()[1]);

Configurable sample-rate for AudioContext

var ac = new AudioContext({sampleRate: 8000});

Allows lower CPU usage, useful for emulation, lo-fi audio work, and for very specific jobs (like AudioBufferSourceNode stiching).

Specify AudioBufferSourceNode & DynamicsCompressorNode

AudioBufferSourceNode playback algorithm

• processing algorithm
• Subsample-accurate start and looping

DynamicsCompressorNode processing algorithm

• processing algorithm
• Hard or soft-knee, peak detecting, adaptive release, fixed-lookahead, automatic makeup-gain

sequence<float> instead of Float32Array

var ac = new AudioContext();
var real = new Float32Array(2);
real[0] = 0.0;
real[1] = 0.0;
var imag = new Float32Array(2);
imag[0] = 0.0;
imag[1] = 1.0;
var wave = ac.createPeriodicWave(real, imag):

vs.

var ac = new AudioContext();
var wave = new PeriodicWave({ real: [0,0], imag: [0,1] });

Convenience introspection attribute

MediaStreamAudioSourceNode.mediaStream and MediaElementAudioSourceNode.mediaElement, instead of having to set it as an expando or kept around elsewhere.

Route to candidate recommendation

• Raymond Toy (Google) is officially editor since April last year
• Few big blockers left (last big one was DynamicsCompressorNode)
• W3C Audio Working Group Re-chartered
• CR transition documents
• web-platform-tests are lacking a bit (good test suites in implementations, interoperable)
• Spec issues on Github are tagged v.next

V.NEXT

• Audio encoding/decoding consolidation
• In memory compressed audio asssets
• SharedArrayBuffer + AudioWorklet + wasm = ❤️
• AudioContext in Web Worker
• Configurable AudioParam rate
• Custom HRTF
• Pulse/noise oscillator
• Expose x-runs
• Oscillator phase offset as an audioparam
• Frequency domain nodes
• Side chain compression
• Expander/noise-gain

Thanks !

Slides

https://padenot.github.io/wac-17-slides
https://github.com/padenot/wac-17-slides

Editor's draft of the specification

https://webaudio.github.io/web-audio-api

Specification issue tracker

https://github.com/webaudio/web-audio-api/issues/

Contribute to tests (running on all implementations)

https://github.com/w3c/web-platform-tests/

Email

padenot@mozilla.com

Twitter

@padenot