Use :cquit to exit vim with an error code. Git will detect the error and stop opening new vimdiffs. You'll probably want to create a mapping for it in your .vimrc:
if &diff
map Q :cquit<CR>
endif
Then just hit Q to early abort from a git diff run.
In order for this to work you must edit your gitconfig:
git config --global difftool.trustExitCode true
git config --global mergetool.trustExitCode true
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…