h.264 - FFMPEG keeping quality when reducing FPS and streaming over RTSP with rtsp-simple-server

Question

I'm using the rtsp-simple-server (https://github.com/aler9/rtsp-simple-server) and feed the RTSP Server with a FFMPEG stream.

I use a docker compose file to start the stream:

version: "3.8"

services:

  ffmpeg:
    container_name: ffmpeg-base
    restart: always
    image: "jenssgb/rtspffmpeg:base"
    depends_on:
      - rtsp-server
    volumes:
      - $PWD/:/video
    network_mode: "host"
    command: "ffmpeg -re -stream_loop -1 -i /video/footage-1-b.mp4 -c copy -f rtsp rtsp://localhost:8554/compose-rtsp"
  
  rtsp-server:
    container_name: rtsp-server-base
    restart: always
    image: "aler9/rtsp-simple-server"
    network_mode: "host"

Now I'm trying to reduce the FPS of my video with transcoding it:

command: -re -stream_loop -1 -i ${VIDEO_FILE} -vf "fps=${FPS_COMPOSE}" -f rtsp rtsp://localhost:8554/compose-rtsp

This is basically working, but the quality of the output video becomes pretty bad. I tried a lot of things like -c:v libx264 which did help for a minute but let ffmpeg crash then.

av_interleaved_write_frame(): Broken pipe0:00:09.99 bitrate=N/A speed=0.985x    
[rtsp @ 0x5563b1755640] Packets poorly interleaved, failed to avoid negative timestamp -33660 in stream 0.
Try -max_interleave_delta 0 as a possible workaround.
av_interleaved_write_frame(): Broken pipe
Error writing trailer of rtsp://localhost:8554/compose-rtsp: Broken pipe

Any idea how I can reduce the FPS send the stream to the server but keep the video quality? Later I'm going to reduce the resolution as well - but for now I want to keep resolution and quality but only reduce the FPS.

Full logs from my test with -c:v libx264:

    ffmpeg -re -stream_loop -1 -i footage-1-b.mp4 -vf "fps=5" -c:v libx264 -f rtsp rtsp://localhost:8554/compose-rtsp
ffmpeg version 4.2.4-1ubuntu0.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
  configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'footage-1-b.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    title           : Session streamed by "nessyMediaServer"
    encoder         : Lavf58.29.100
    comment         : h264_3
  Duration: 00:59:59.63, start: 0.000000, bitrate: 2099 kb/s
    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuvj420p(pc), 1280x720 [SAR 1:1 DAR 16:9], 2061 kb/s, 24.96 fps, 25 tbr, 12800 tbn, 25 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 8000 Hz, mono, fltp, 35 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[aac @ 0x56277bc7f840] Too many bits 8832.000000 > 6144 per frame requested, clamping to max
[libx264 @ 0x56277bbc33c0] using SAR=1/1
[libx264 @ 0x56277bbc33c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x56277bbc33c0] profile High, level 3.1
[libx264 @ 0x56277bbc33c0] 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=5 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, rtsp, to 'rtsp://localhost:8554/compose-rtsp':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    title           : Session streamed by "nessyMediaServer"
    comment         : h264_3
    encoder         : Lavf58.29.100
    Stream #0:0(und): Video: h264 (libx264), yuvj420p(pc), 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, 5 fps, 90k tbn, 5 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc58.54.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1(und): Audio: aac (LC), 8000 Hz, mono, fltp, 48 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      encoder         : Lavc58.54.100 aac
av_interleaved_write_frame(): Broken pipe0:00:09.87 bitrate=N/A speed=0.978x    
[rtsp @ 0x56277bba0640] Packets poorly interleaved, failed to avoid negative timestamp -33660 in stream 0.
Try -max_interleave_delta 0 as a possible workaround.
av_interleaved_write_frame(): Broken pipe
Error writing trailer of rtsp://localhost:8554/compose-rtsp: Broken pipe
frame=   50 fps=4.6 q=23.0 Lsize=N/A time=00:00:10.21 bitrate=N/A speed=0.947x    
video:162kB audio:8kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libx264 @ 0x56277bbc33c0] frame I:1     Avg QP:19.85  size:165667
[libx264 @ 0x56277bbc33c0] frame P:13    Avg QP:20.88  size:140481
[libx264 @ 0x56277bbc33c0] frame B:36    Avg QP:24.58  size: 55445
[libx264 @ 0x56277bbc33c0] consecutive B-frames:  4.0%  0.0%  0.0% 96.0%
[libx264 @ 0x56277bbc33c0] mb I  I16..4:  4.4% 30.8% 64.8%
[libx264 @ 0x56277bbc33c0] mb P  I16..4:  4.1% 10.6% 20.0%  P16..4: 24.4% 24.8% 13.3%  0.0%  0.0%    skip: 2.6%
[libx264 @ 0x56277bbc33c0] mb B  I16..4:  0.8%  2.0%  4.0%  B16..8: 40.3% 14.5%  5.2%  direct:11.8%  skip:21.4%  L0:77.1% L1: 7.9% BI:14.9%
[libx264 @ 0x56277bbc33c0] 8x8 transform intra:30.1% inter:11.9%
[libx264 @ 0x56277bbc33c0] coded y,uvDC,uvAC intra: 82.5% 60.9% 26.6% inter: 55.0% 42.4% 2.7%
[libx264 @ 0x56277bbc33c0] i16 v,h,dc,p: 17% 26% 34% 23%
[libx264 @ 0x56277bbc33c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 22% 33% 15%  3%  4%  5%  4%  3%  9%
[libx264 @ 0x56277bbc33c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 22% 13%  3% 17% 11%  5%  3%  6%
[libx264 @ 0x56277bbc33c0] i8c dc,h,v,p: 54% 25% 16%  5%
[libx264 @ 0x56277bbc33c0] Weighted P-Frames: Y:7.7% UV:7.7%
[libx264 @ 0x56277bbc33c0] ref P L0: 33.2% 11.6% 29.0% 23.9%  2.4%
[libx264 @ 0x56277bbc33c0] ref B L0: 79.6% 11.9%  8.5%
[libx264 @ 0x56277bbc33c0] ref B L1: 95.9%  4.1%
[libx264 @ 0x56277bbc33c0] kb/s:3190.34
[aac @ 0x56277bc7f840] Qavg: 65536.000
Conversion failed!

Thank you, J

question from:https://stackoverflow.com/questions/65852931/ffmpeg-keeping-quality-when-reducing-fps-and-streaming-over-rtsp-with-rtsp-simpl

Answer 1

When you say, "the quality of the video becomes pretty bad," I guess you mean your transcoded output video has a lot of block artifacts in it. That's generally because you haven't allocated enough bandwidth to your output video stream. Without enough output bandwidth to play with, the coder quantizes and eliminates higher-frequency stuff so it looks nasty.

You didn't mention what sort of program material you have. But it's worth mentioning this: in material with lots of motion (think James Bond flick) it doesn't save much bandwidth to reduce the frame rate: we're coding the difference between successive frames. The longer you wait between frames, the more differences there are to code (and the harder the motion estimator has to work). If you radically reduce the frame rate (from 24 to 2 for example) it gets much worse.

Talking-heads material is generally less sensitive to framerate.

You might try setting your bandwidth -- your output bitrate -- explicitly like this.

-b:v 1000K -maxrate 1200K -bufsize 500K 

These parameters tell ffmpeg to give you a one mbps constant-rate stream with some variation allowed. Something like this from your example ...

-re -stream_loop -1 -i ${VIDEO_FILE} -vf "fps=${FPS_COMPOSE}"  -b:v 1M  -maxrate 1M -bufsize 500K -f rtsp rtsp://localhost:8554/compose-rtsp

It might be easiest to try various combinations of ffmpeg parameters on your laptop so you can quick-cycle various attempts before you choose one to put in your docker image.