encryption - Play AES encrypted video in ExoPlayer offline

Question

I am trying to play from local storage an encrypted video using ExoPlayer. The command used to encrypt the video using FFMPEG is as follows:

-i /storage/emulated/0/Download/20210125_193031.mp4 -vcodec copy -acodec copy -c:v libx264 -encryption_scheme cenc-aes-ctr -encryption_key b42ca3172ee4e69bf51848a59db9cd13 -encryption_kid 09e367028f33436ca5dd60ffe6671e70 /storage/emulated/0/Download/out_enc.mp4

Here it is the sourcecode of my player:

public class PlayerActivity extends AppCompatActivity {
    private SimpleExoPlayer player;
    private DefaultDrmSessionManager drmSessionManager;

    @Override
    protected void onCreate(final Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);

        setContentView(R.layout.activity_player);
        // Build the media item.
        PlayerView playerView = findViewById(R.id.video_view);
        player = new SimpleExoPlayer.Builder(this).build();
        playerView.setPlayer(player);
        //player.prepare();
        //FFMPEG command: -i /storage/emulated/0/Download/20210125_193031.mp4 -vf scale=-1:720 -c:v libx264 -encryption_scheme cenc-aes-ctr -encryption_key b42ca3172ee4e69bf51848a59db9cd13 -encryption_kid 09e367028f33436ca5dd60ffe6671e70 /storage/emulated/0/Download/out_enc.mp4
        //base 64 keys generated from: https://www.base64encode.org/
        //playVideo("/storage/emulated/0/Download/out_enc.mp4", "MDllMzY3MDI4ZjMzNDM2Y2E1ZGQ2MGZmZTY2NzFlNzA=", "YjQyY2EzMTcyZWU0ZTY5YmY1MTg0OGE1OWRiOWNkMTM=");
        playVideo("/storage/emulated/0/Download/out_enc.mp4", "CeNnAo8zQ2yl3WD/5mcecA", "tCyjFy7k5pv1GEilnbnNEw");
    }

    private void playVideo(String url, String keyID, String keyValue) {
        try {
            drmSessionManager = buildDrmSessionManager(Util.getDrmUuid(C.CLEARKEY_UUID.toString()), true, keyID, keyValue
            );
        } catch (Exception e) {
            e.printStackTrace();
        }
        player.setMediaSource(buildDashMediaSource(Uri.parse(url)));
        player.prepare();
        player.setPlayWhenReady(true);
    }

    private MediaSource buildDashMediaSource(Uri uri) {
        DefaultDataSourceFactory dashChunkSourceFactory = new DefaultDataSourceFactory(this, "agent");
        return new ProgressiveMediaSource.Factory(dashChunkSourceFactory)
                .setDrmSessionManager(drmSessionManager)
                .createMediaSource(uri);
    }

    private DefaultDrmSessionManager buildDrmSessionManager(UUID uuid, Boolean multiSession, String id, String value) {
/*        String base64Id = Base64.encodeToString(id.getBytes(), Base64.DEFAULT);
        String base64Value = Base64.encodeToString(value.getBytes(), Base64.DEFAULT);*/
        String keyString = "{"keys":[{"kty":"oct","k":""+value+"","kid":""+id+""}],"type":"temporary"}";;
        LocalMediaDrmCallback drmCallback = new LocalMediaDrmCallback(keyString.getBytes());
        FrameworkMediaDrm mediaDrm = null;
        try {
            mediaDrm = FrameworkMediaDrm.newInstance(uuid);
        } catch (UnsupportedDrmException e) {
            e.printStackTrace();
        }
        return new DefaultDrmSessionManager(uuid, mediaDrm, drmCallback, null, multiSession);
    }

    @Override
    protected void onDestroy() {
        player.release();
        super.onDestroy();
    }

Here it is the link for the encrypted video. The main issue: the video is playing but is not decrypted. What am I missing?

question from:https://stackoverflow.com/questions/65950956/play-aes-encrypted-video-in-exoplayer-offline

Answer 1

Looking at the logcat output there does not seem to be any DRM, AES or clearkey errors.

Looking then at the video file itself, it appears to report some issues:

However, checking against other sample files encrypted using he same ffmpeg approach you used, they show similar issues so this appear to be typical output from ffprobe for a file encrypted in this way.

Looking then at the video file structure itself with an MP4 parser to see the individual atoms, or the header blocks, there does not appear to be a PSSH box.

A PSSH box is a header area that contains the data about the encryption for an ISOBMFF mp4 file - this is actually an optional field in the CENC spec so your video is valid even without this.

The obvious question then is, how would a player know the video is encoded? The answer, according to the CENC spec is:

2. Detection

For a stream determined to be in the ISO Base Media File Format [ISOBMFF], this ISO Common Encryption ('cenc') Protection Scheme may be detected as follows.

Protection scheme signaling conforms with [ISOBMFF]. When protection has been applied, the stream type will be transformed to 'encv' for video or 'enca' for audio, with a Protection Scheme Information Box ('sinf') added to the sample entry in the Sample Description Box ('stsd'). The Protection Scheme Information Box ('sinf') will contain a Scheme Type Box ('schm') with a scheme_type field set to a value of 'cenc'

Looking at your video using a MP4 analyser (see below) shows that it does indeed have the stream type shown as 'encv' in the stud box (output from inspect tool below):

Testing playback with ffplay itself using the same encryption key, shows that the video does actually play successfully:

ffplay out_enc.mp4 -decryption_key b42ca3172ee4e69bf51848a59db9cd13

However, a regular player will not be able to play it unless you provide the decryption key. It would be reasonable to to expect the player to flag an error in this case, but that does not appear to be happening with some common players I checked including VLC, so it is quite possible ExoPlayer on Android is not flagging this also.

Looking at ExoPlayer specifically, as noted by Duna in the comments below and outlined in this GIT thread https://github.com/google/ExoPlayer/issues/8532#issuecomment-771811707 , ExoPlayer does not currently (Feb 2021) read the PSSH box for MP4, only for fragmented MP4. From that thread:

After looking deeper into this, I found that ExoPlayer's Mp4Extractor actually doesn't read pssh boxes. Currently we only read this info in fragmented MP4 files (using FragmentedMp4Extractor). This means even when playing the file with the pssh box, the drmInitData still ends up null - meaning playback fails. I didn't realise this limitation when you initially filed the issue, otherwise I would have flagged it earlier.

However, looking at the Mp4Extractor code it does check for 'encv' and it does also check for the default key_id, both of which are present in the video file produced when inspected. Again, it would be reasonable for ExoPlayer to flag an error if it either does not find these in a format is can understand, or if it finds them but is not supplied a corresponding key to play the file with.

So how could the video be encrypted and played reliably?

You could use ffplay on android although I suspect this would not be too straightforward, based on past experience using ffmpeg on Android.

There are some easier (appearing) examples leveraging ExoPlayer also that it would be worth looking at - e.g:

• https://stackoverflow.com/a/62678769/334402

You could also look at leveraging DASH. Most media that is streamed to mobile devices these days use a streaming protocol like DASH or HLS - these formats will nearly always have the encryption data included in the 'manifest' or 'index' file and this will definitely be recognised by ExoPlayer. There are online tutorials and free tools to allow you package videos into DASH, including adding encryption. The ExoPlayer team provide information on downloading and playing back such these streams (link correct at time of writing):

• https://exoplayer.dev/downloading-media.html

If you want to examine the mp4 files yourself in more detail there are various free tools like:

• https://gpac.github.io/mp4box.js/test/filereader.html (highlighted by Duna in comments and looks very good)
• https://www.bento4.com/documentation/mp4dump/
• https://inspect.eyevinn.technology