javascript - HTML5 Video: Streaming Video with Blob URLs

Question

Welcome To Ask or Share your Answers For Others

javascript - HTML5 Video: Streaming Video with Blob URLs

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

javascript - HTML5 Video: Streaming Video with Blob URLs

I have an array of Blobs (binary data, really -- I can express it however is most efficient. I'm using Blobs for now but maybe a Uint8Array or something would be better). Each Blob contains 1 second of audio/video data. Every second a new Blob is generated and appended to my array. So the code roughly looks like so:

var arrayOfBlobs = [];
setInterval(function() {
    arrayOfBlobs.append(nextChunk());
}, 1000);

My goal is to stream this audio/video data to an HTML5 element. I know that a Blob URL can be generated and played like so:

var src = URL.createObjectURL(arrayOfBlobs[0]);
var video = document.getElementsByTagName("video")[0];
video.src = src;

Of course this only plays the first 1 second of video. I also assume I can trivially concatenate all of the Blobs currently in my array somehow to play more than one second:

// Something like this (untested)
var concatenatedBlob = new Blob(arrayOfBlobs);
var src = ...

However this will still eventually run out of data. As Blobs are immutable, I don't know how to keep appending data as it's received.

I'm certain this should be possible because YouTube and many other video streaming services utilize Blob URLs for video playback. How do they do it?

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙；凝视深渊过久,深渊将回以凝视…

1 Answer

深蓝 · Answer 1 · 2021-10-17T00:13:25+0000

Solution

After some significant Googling I managed to find the missing piece to the puzzle: MediaSource

Effectively the process goes like this:

Create a MediaSource
Create an object URL from the MediaSource
Set the video's src to the object URL
On the sourceopen event, create a SourceBuffer
Use SourceBuffer.appendBuffer() to add all of your chunks to the video

This way you can keep adding new bits of video without changing the object URL.

Caveats

The SourceBuffer object is very picky about codecs. These have to be declared, and must be exact, or it won't work
You can only append one blob of video data to the SourceBuffer at a time, and you can't append a second blob until the first one has finished (asynchronously) processing
If you append too much data to the SourceBuffer without calling .remove() then you'll eventually run out of RAM and the video will stop playing. I hit this limit around 1 hour on my laptop

Example Code

Depending on your setup, some of this may be unnecessary (particularly the part where we build a queue of video data before we have a SourceBuffer then slowly append our queue using updateend). If you are able to wait until the SourceBuffer has been created to start grabbing video data, your code will look much nicer.

<html>
<head>
</head>
<body>
    <video id="video"></video>
    <script>
        // As before, I'm regularly grabbing blobs of video data
        // The implementation of "nextChunk" could be various things:
        //   - reading from a MediaRecorder
        //   - reading from an XMLHttpRequest
        //   - reading from a local webcam
        //   - generating the files on the fly in JavaScript
        //   - etc
        var arrayOfBlobs = [];
        setInterval(function() {
            arrayOfBlobs.append(nextChunk());
            // NEW: Try to flush our queue of video data to the video element
            appendToSourceBuffer();
        }, 1000);

        // 1. Create a `MediaSource`
        var mediaSource = new MediaSource();

        // 2. Create an object URL from the `MediaSource`
        var url = URL.createObjectURL(mediaSource);

        // 3. Set the video's `src` to the object URL
        var video = document.getElementById("video");
        video.src = url;

        // 4. On the `sourceopen` event, create a `SourceBuffer`
        var sourceBuffer = null;
        mediaSource.addEventListener("sourceopen", function()
        {
            // NOTE: Browsers are VERY picky about the codec being EXACTLY
            // right here. Make sure you know which codecs you're using!
            sourceBuffer = mediaSource.addSourceBuffer("video/webm; codecs="opus,vp8"");

            // If we requested any video data prior to setting up the SourceBuffer,
            // we want to make sure we only append one blob at a time
            sourceBuffer.addEventListener("updateend", appendToSourceBuffer);
        });

        // 5. Use `SourceBuffer.appendBuffer()` to add all of your chunks to the video
        function appendToSourceBuffer()
        {
            if (
                mediaSource.readyState === "open" &&
                sourceBuffer &&
                sourceBuffer.updating === false
            )
            {
                sourceBuffer.appendBuffer(arrayOfBlobs.shift());
            }

            // Limit the total buffer size to 20 minutes
            // This way we don't run out of RAM
            if (
                video.buffered.length &&
                video.buffered.end(0) - video.buffered.start(0) > 1200
            )
            {
                sourceBuffer.remove(0, video.buffered.end(0) - 1200)
            }
        }
    </script>
</body>
</html>

As an added bonus this automatically gives you DVR functionality for live streams, because you're retaining 20 minutes of video data in your buffer (you can seek by simply using video.currentTime = ...)

Categories

javascript - HTML5 Video: Streaming Video with Blob URLs

javascript - HTML5 Video: Streaming Video with Blob URLs

Please log in or register to add a comment.

Please log in or register to answer this question.

1 Answer

Solution

Caveats

Example Code

Please log in or register to add a comment.

Just Browsing Browsing

Most popular tags