python - TFLite outputs different results for the same input

Question

The model was converted successfully, but the output differs for each execution. The same does not happen for the non-converted TensorFlow model, which always yields the same results. I've attached a notebook that shows how to reproduce the problem and how to detect it. After a few runs, the output gets more consistent. It only happens when using LSTMs and seems to get even worse when using return_sequences=True.

Notebook Link: https://github.com/BernardoGO/tflite_convert_bug/blob/main/BiLSTM%20Conversion%20Test.ipynb

My model:

new_model = Sequential()
new_model.add(Input(name='input', batch_size=1, shape=(925, 3)) )
new_model.add(Dense(8))
new_model.add(LSTM(64, return_sequences=True))
new_model.add(LSTM(64, return_sequences=True))
new_model.add(LSTM(64, return_sequences=True))
new_model.add(LSTM(64, return_sequences=True))
new_model.add(Activation('softmax', name='softmax'))

new_model.summary()

Command used to run the converter

converter = tf.lite.TFLiteConverter.from_keras_model(new_model)
#converter.experimental_new_converter = True
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]
#converter.allow_custom_ops=True
tflite_model = converter.convert()

open("tf_test.tflite", "wb").write(tflite_model)

The output from the converter invocation

INFO:tensorflow:Assets written to: /tmp/tmp4xjrqwpb/assets
Out[9]: 476052

• I've used this function to check if the output is the same after each execution:

def equal(tensor1, tensor2):
    for i, j in zip(tensor1.reshape(-1), tensor2.reshape(-1)):
        if abs(i - j) > 0.001:
            return False
    return True

System information

• OS Platform: Ubuntu 20.04
• TensorFlow installed Binary
• TensorFlow version: 2.3.1

question from:https://stackoverflow.com/questions/65855602/tflite-outputs-different-results-for-the-same-input

Answer 1

Waitting for answers