tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph execution error:

TensorFlow
,
1

I am training an object detection model. I’m using the already pretrained center net model config. Everything has been configured to the paths of my training and testing data(in the tfrecord format). But when I run the script I get the following error

C:\Users\SHOCKER\anaconda\envs\tf_gpu\lib\site-packages\keras\src\backend.py:452: UserWarning: tf.keras.backend.set_learning_phase is deprecated and will be removed after 2020-10-11. To update it, simply pass a True/False value to the training argument of the __call__ method of your layer or model.
warnings.warn(
Traceback (most recent call last):
File “C:\Users\SHOCKER\tensorflow_projects\models-master\research\object_detection\model_main_tf2.py”, line 114, in
tf.compat.v1.app.run()
File “C:\Users\SHOCKER\anaconda\envs\tf_gpu\lib\site-packages\tensorflow\python\platform\app.py”, line 36, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File “C:\Users\SHOCKER\anaconda\envs\tf_gpu\lib\site-packages\absl\app.py”, line 308, in run
_run_main(main, args)
File “C:\Users\SHOCKER\anaconda\envs\tf_gpu\lib\site-packages\absl\app.py”, line 254, in _run_main
sys.exit(main(argv))
File “C:\Users\SHOCKER\tensorflow_projects\models-master\research\object_detection\model_main_tf2.py”, line 105, in main
model_lib_v2.train_loop(
File “C:\Users\SHOCKER\AppData\Roaming\Python\Python39\site-packages\object_detection\model_lib_v2.py”, line 685, in train_loop
losses_dict = _dist_train_step(train_input_iter)
File “C:\Users\SHOCKER\anaconda\envs\tf_gpu\lib\site-packages\tensorflow\python\util\traceback_utils.py”, line 153, in error_handler
raise e.with_traceback(filtered_tb) from None
File “C:\Users\SHOCKER\anaconda\envs\tf_gpu\lib\site-packages\tensorflow\python\eager\execute.py”, line 53, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph execution error:

indices[0] = 0 is not in [0, 0)
[[{{node GatherV2_8}}]]
[[MultiDeviceIteratorGetNextFromShard]]
[[RemoteCall]]
[[while/body/_1/IteratorGetNext]] [Op:__inference__dist_train_step_35314]

I’m not sure why this is happening, could I please get some help

3

Hi @X_Life

Welcome to the TensorFlow Forum!

The reason behind this error could be that model has used older deprecated apis which are not supported with the latest Tensorflow version and throwing the error. As mentioned in error log " tf.keras.backend.set_learning_phase is deprecated and will be removed after 2020-10-11".

Please check the Tensorflow or other libraries version requirement for the above code to be supported to its used APIs. You can also refer to the Object detection with Model Garden which supports latest Tensorflow version. Thank you.