TensorFlow Version: 1.15.4
Keras in TensorFlow:2.2.4-tf
I’m training a keras model using fit and 4 callbacks, Here is code:
mode = 'max'
verbose = 1
ckpt_name = os.path.join(
self.context.ckpt_path, "{epoch:03d}-{%s:.4f}.h5" % monitor_name)
callbacks_list = [
LogCallback(log_batch_num=self.context.metric_size),
ReduceLROnPlateau(monitor=monitor_name,
factor=self.context.lr_decay_factor,
patience=self.context.lr_decay_step,
cooldown=1,
verbose=verbose,
min_lr=0.0001),
TensorBoard(log_dir=self.context.tensorboard_path, write_graph=False),
ModelCheckpoint(ckpt_name,
monitor=monitor_name,
verbose=verbose,
save_best_only=True,
save_weights_only=True,
mode=mode),
EarlyStopping(monitor=monitor_name,
min_delta=self.context.min_delta,
patience=self.context.patience,
verbose=verbose,
mode=mode,
baseline=self.context.base_metric,
restore_best_weights=True)
]
It will cause the Error
I think maybe OutOfData->Train stop->Save model->Metric not available->metric not in Logs->_get_file_path Error
I want to know how to fix it, make the training stop and save model ok even if Out of Data?