If you delete the application registration in the Azure portal, it is actually a kind of "soft-deleted" and it can be restored. This is why you keep encountering this error.
The solution is that you can try to permanently delete the application using MS graph api.
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…