How could I achieve a good ratio of compression and time while converting video files with ffmpeg?

Question

So, I made 2 scripts that convert CCTV footage in mp4 videos. One of them is just a -vcodec copy and creates a mp4 with the same size of the footage (huge, btw) and my other alternative was tweak with some parameters and figure out what was the best I could do without sacrifice too much quality and make it "fast". Then I come up with c:v libx264 -crf 30 -preset veryfast -filter:v fps=fps=20 which took something like 2 secs in my machine to run an average 6MB file and transform into a 600kB file.

Happy with the results I decided to put it on AWS lambda (to avoid bottlenecks), and then people started to complain about missing files, so I increase the timeout and the memory to 380MB. And even after that, I am still getting a few lambda errors...

Anyway, the lambda is going to cost me too much compared to just store the file without compression, there is another way to decrease size without sacrificing time?

[UPDATE] I crunch some numbers and even tho using lambda is not what I expected, I am still saving a lot of cash monthly by reducing the file size 10x times.

As asked, this is the logs for the ffmpeg.

ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 10 (GCC)
  configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --docdir=/usr/share/doc/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' --extra-ldflags='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld ' --extra-cflags=' -I/usr/include/rav1e' --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-version3 --enable-bzlib --disable-crystalhd --enable-fontconfig --enable-frei0r --enable-gcrypt --enable-gnutls --enable-ladspa --enable-libaom --enable-libdav1d --enable-libass --enable-libbluray --enable-libcdio --enable-libdrm --enable-libjack --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-liblensfun --enable-libmp3lame --enable-libmysofa --enable-nvenc --enable-openal --enable-opencl --enable-opengl --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librav1e --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libvorbis --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-version3 --enable-vapoursynth --enable-libvpx --enable-vulkan --enable-libglslang --enable-libx264 --enable-libx265 --enable-libxvid --enable-libxml2 --enable-libzimg --enable-libzvbi --enable-lv2 --enable-avfilter --enable-avresample --enable-libmodplug --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-lto --enable-libmfx --enable-runtime-cpudetect
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, dhav, from 'ch1_principal_28122020030156_28122020030600.dav':
  Duration: N/A, start: 1609124514.000000, bitrate: N/A
    Stream #0:0: Video: hevc (Main), yuvj420p(pc, bt709), 960x480, 30 fps, 30 tbr, 1k tbn
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x556ce99e7100] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x556ce99e7100] profile High, level 3.1, 4:2:0, 8-bit
[libx264 @ 0x556ce99e7100] 264 - core 160 r3011 cde9a93 - H.264/MPEG-4 AVC codec - Copyleft 2003-2020 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=2 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=6 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=20 scenecut=40 intra_refresh=0 rc_lookahead=10 rc=crf mbtree=1 crf=30.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to '_test1.mp4':
  Metadata:
    encoder         : Lavf58.45.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuvj420p(pc), 960x480, q=-1--1, 20 fps, 10240 tbn, 20 tbc
    Metadata:
      encoder         : Lavc58.91.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
[dhav @ 0x556ce99b8400] Packet corrupt (stream = 0, dts = NOPTS).te= 127.1kbits/s speed=10.9x    
ch1_principal_28122020030156_28122020030600.dav: corrupt input packet in stream 0
frame=  805 fps=226 q=-1.0 Lsize=     682kB time=00:00:40.10 bitrate= 139.4kbits/s speed=11.2x    
video:672kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.521397%
[libx264 @ 0x556ce99e7100] frame I:4     Avg QP:27.76  size: 31382
[libx264 @ 0x556ce99e7100] frame P:205   Avg QP:30.62  size:  2110
[libx264 @ 0x556ce99e7100] frame B:596   Avg QP:33.91  size:   217
[libx264 @ 0x556ce99e7100] consecutive B-frames:  1.1%  0.2%  0.7% 97.9%
[libx264 @ 0x556ce99e7100] mb I  I16..4:  8.3% 41.1% 50.6%
[libx264 @ 0x556ce99e7100] mb P  I16..4:  0.9%  3.0%  0.4%  P16..4: 16.9%  4.1%  1.3%  0.0%  0.0%    skip:73.4%
[libx264 @ 0x556ce99e7100] mb B  I16..4:  0.1%  0.2%  0.0%  B16..8:  1.4%  0.4%  0.0%  direct: 0.9%  skip:96.9%  L0:42.0% L1:48.0% BI: 9.9%
[libx264 @ 0x556ce99e7100] 8x8 transform intra:63.1% inter:40.6%
[libx264 @ 0x556ce99e7100] coded y,uvDC,uvAC intra: 55.6% 40.9% 5.0% inter: 1.4% 2.1% 0.0%
[libx264 @ 0x556ce99e7100] i16 v,h,dc,p: 27% 28% 24% 21%
[libx264 @ 0x556ce99e7100] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 16% 26%  6%  6%  6%  7% 11%  7%
[libx264 @ 0x556ce99e7100] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 17% 15%  9%  6%  6%  7% 13%  6%
[libx264 @ 0x556ce99e7100] i8c dc,h,v,p: 63% 16% 16%  5%
[libx264 @ 0x556ce99e7100] Weighted P-Frames: Y:8.8% UV:1.0%
[libx264 @ 0x556ce99e7100] kb/s:136.66

question from:https://stackoverflow.com/questions/65891087/how-could-i-achieve-a-good-ratio-of-compression-and-time-while-converting-video

Answer 1

libx264

You have to choose a balance between encoding speed and encoding efficiency.

1. Choose the slowest -preset you have patience for.
2. Choose the highest -crf value that provides an acceptable quality.

See FFmpeg Wiki: H.264 for more info.

libx265

If libx264 does not make a small enough file try libx265, but it takes longer to encode.

See FFmpeg Wiki: HEVC/H.265 for more info.

Hardware accelerated encoder

If you have the proper hardware, then you can use NVENC, QuickSync, or some other implementation.

Encoding will be fast, but it will not match the quality per bit provided by libx264 or libx265.

See FFmpeg Wiki: Hardware for more info.