These optimization algorithms typically work by choosing a descent direction, and then performing a line search to that direction. I think this message means that the optimizer got into a position where it did not manage to find a direction where the value of the objective function decreases (fast enough), but could also not verify that the current position is a minimum.
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…