Let us say we are using tf.data
and tf.estimator
. The data_input_fn()
returns dataset.shuffle(SHUFFLE_BUFFER_SIZE).repeat().batch(BATCH_SIZE)
for training. Note the dataset will repeat indefinitely unless there is a stopping training event.
For example, one complete traversal of training dataset (epoch) contains 11 iteration steps [1, 2, 3, ..., 11]
, and tf.estimator
has configuration save_checkpoints_steps=3
. Then, parameter updates of the last two iteration steps [10, 11]
won't be able to be saved as checkpoints. Since the tf.estimator
creates new session
for each new epoch, will this lose parameter updates of the training samples corresponding to the iteration steps [10, 11]
?
question from:
https://stackoverflow.com/questions/65623583/does-tf-estimator-discard-data-partially 与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…