firefox - Specifying codecs with MediaRecorder

Question

How can I specify the codecs used with the MediaRecorder API? The only option I see is for mimeType which isn't really sufficient. Cramming in the codecs in the mimeType option doesn't seem to work.

var mediaRecorder = new MediaRecorder(
    outputMediaStream
  ),
  {
    mimeType: 'video/webm; codecs="opus,vp8"'
  }
);

This results in a WebM stream with Vorbis and VP8:

FFMPEG STDERR: Input #0, matroska,webm, from 'pipe:':
  Metadata:
    encoder         : QTmuxingAppLibWebM-0.0.1
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0(eng): Video: vp8, yuv420p, 640x360, SAR 1:1 DAR 16:9, 30 fps, 30 tbr, 1k tbn, 1k tbc (default)
    Stream #0:1(eng): Audio: vorbis, 44100 Hz, stereo, fltp (default)

If I make a MediaStream that only has an audio track (no video), then the MediaRecorder outputs Opus audio in an Ogg container:

FFMPEG STDERR: Input #0, ogg, from 'pipe:':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Audio: opus, 48000 Hz, stereo, fltp
    Metadata:
      ENCODER         : Mozilla44.0.2

How can I get Opus audio with VP8 video in a WebM container with the MediaRecorder API?

Firefox v44.0.2

Answer 1

How can I get Opus audio with VP8 video in a WebM container with the MediaRecorder API?

Unfortunately this is apparently impossible at this time. In fact it seems not even the mimetype can currently be set, in addition to the audio/video stream codecs. Firefox decides what encoder to used from those that are available, based on the requested streams, and the JavaScript API doesn't offer much say in the matter.

As always, the proof is in the source.

Here is where the encoded is initialized from the mimetype.

Excerpt from dom/media/MediaRecorder.cpp:

    // Allocate encoder and bind with union stream.
    // At this stage, the API doesn't allow UA to choose the output mimeType format.

    // Make sure the application has permission to assign AUDIO_3GPP
    if (mRecorder->mMimeType.EqualsLiteral(AUDIO_3GPP) && CheckPermission("audio-capture:3gpp")) {
      mEncoder = MediaEncoder::CreateEncoder(NS_LITERAL_STRING(AUDIO_3GPP),
                                             mRecorder->GetAudioBitrate(),
                                             mRecorder->GetVideoBitrate(),
                                             mRecorder->GetBitrate(),
                                             aTrackTypes);
    } else if (mRecorder->mMimeType.EqualsLiteral(AUDIO_3GPP2) && CheckPermission("audio-capture:3gpp2")) {
      mEncoder = MediaEncoder::CreateEncoder(NS_LITERAL_STRING(AUDIO_3GPP2),
                                             mRecorder->GetAudioBitrate(),
                                             mRecorder->GetVideoBitrate(),
                                             mRecorder->GetBitrate(),
                                             aTrackTypes);
    } else {
      mEncoder = MediaEncoder::CreateEncoder(NS_LITERAL_STRING(""),
                                             mRecorder->GetAudioBitrate(),
                                             mRecorder->GetVideoBitrate(),
                                             mRecorder->GetBitrate(),
                                             aTrackTypes);
    }

A we can see, except for the 3GPP* mimes, the mimetype passed to CreateEncoded is an empty string NS_LITERAL_STRING("").

This comment does offer hope for the future though:

// At this stage, the API doesn't allow UA to choose the output mimeType format.

Excerpts from dom/media/encoder/MediaEncoder.cpp:

The first case for video with WebM enabled.

  else if (MediaEncoder::IsWebMEncoderEnabled() &&
          (aMIMEType.EqualsLiteral(VIDEO_WEBM) ||
          (aTrackTypes & ContainerWriter::CREATE_VIDEO_TRACK))) {
    if (aTrackTypes & ContainerWriter::CREATE_AUDIO_TRACK) {
      audioEncoder = new VorbisTrackEncoder();
      NS_ENSURE_TRUE(audioEncoder, nullptr);
    }
    videoEncoder = new VP8TrackEncoder();
    writer = new WebMWriter(aTrackTypes);
    NS_ENSURE_TRUE(writer, nullptr);
    NS_ENSURE_TRUE(videoEncoder, nullptr);
    mimeType = NS_LITERAL_STRING(VIDEO_WEBM);
  }

The final case for just audio with OGG enabled.

  else if (MediaDecoder::IsOggEnabled() && MediaDecoder::IsOpusEnabled() &&
           (aMIMEType.EqualsLiteral(AUDIO_OGG) ||
           (aTrackTypes & ContainerWriter::CREATE_AUDIO_TRACK))) {
    writer = new OggWriter();
    audioEncoder = new OpusTrackEncoder();
    NS_ENSURE_TRUE(writer, nullptr);
    NS_ENSURE_TRUE(audioEncoder, nullptr);
    mimeType = NS_LITERAL_STRING(AUDIO_OGG);
  }

Base on this code, I think we can conclude that at this point it's not possible, but it does appear to be on the roadmap.

@jib located the following related Mozilla bugs reports.

• https://bugzil.la/987568
• https://bugzil.la/963238