InvalidArgumentError: Graph execution error (NLP)

TensorFlow
1 
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit([input_sequences, target_sequences[:, :-1]], target_sequences[:, 1:], epochs=10, batch_size=32, validation_split=0.2)

I have used this code for Compile and Train the Model. But I am facing “Graph execution error”. After doing some staff I found some solution like batch_size issue. I have tried batch_size as 64&32 both but find similar issue. For your kind concern, I have used text dataset.

3

Hi @MD_Raihanul_Islam_To, Could you please provide more information about the type of NLP model you are building like a classification task, text generation etc. If possible could you please provide the stand alone code to reproduce the issue. Thank You.

5

Thanks for your response.
Actually I am trying to build a model to translate one langulage to another. But I am facing this error to compiled & fit the model.

7

Hi @MD_Raihanul_Islam_To, If possible could you please provide the stand alone code to reproduce the issue. Thank You.

9 
import pandas as pd
import numpy as np
import tensorflow as tf
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, Embedding, LSTM, Dense

Load and Preprocess Data:

# Load data from CSV
df = pd.read_csv('dataset.csv')

# Tokenize the sentences
tokenizer_eng = tf.keras.preprocessing.text.Tokenizer(filters='')
tokenizer_eng.fit_on_texts(df['english'])
input_sequences = tokenizer_eng.texts_to_sequences(df['english'])

tokenizer_frn = tf.keras.preprocessing.text.Tokenizer(filters='')
tokenizer_frn.fit_on_texts(df['french'])
target_sequences = tokenizer_frn.texts_to_sequences(df['french'])

# Pad sequences to the same length
input_sequences = tf.keras.preprocessing.sequence.pad_sequences(input_sequences)
target_sequences = tf.keras.preprocessing.sequence.pad_sequences(target_sequences)

Build the Model:

# Set hyperparameters
embedding_dim = 256
units = 512
input_vocab_size = len(tokenizer_eng.word_index) + 1
target_vocab_size = len(tokenizer_frn.word_index) + 1

# Define the Encoder
encoder_inputs = Input(shape=(None,))
encoder_embedding = Embedding(input_vocab_size, embedding_dim)(encoder_inputs)
encoder_lstm = LSTM(units, return_state=True)
encoder_outputs, state_h, state_c = encoder_lstm(encoder_embedding)
encoder_states = [state_h, state_c]

# Define the Decoder
decoder_inputs = Input(shape=(None,))
decoder_embedding = Embedding(target_vocab_size, embedding_dim)(decoder_inputs)
decoder_lstm = LSTM(units, return_sequences=True, return_state=True)
decoder_outputs, _, _ = decoder_lstm(decoder_embedding, initial_state=encoder_states)
decoder_dense = Dense(target_vocab_size, activation='softmax')
decoder_outputs = decoder_dense(decoder_outputs)

# Build the model
model = Model([encoder_inputs, decoder_inputs], decoder_outputs)

Compile and Train the Model:

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit([input_sequences, target_sequences[:, :-1]], target_sequences[:, 1:], epochs=10, batch_size=64, validation_split=0.2)

this is the entire code I have used.

And the error I have faced:

Epoch 1/10
---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-12-235fc25bdbdc> in <module>
      1 model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
----> 2 model.fit([input_sequences, target_sequences[:, :-1]], target_sequences[:, 1:], epochs=10, batch_size=32, validation_split=0.2)

D:\Python\lib\site-packages\keras\utils\traceback_utils.py in error_handler(*args, **kwargs)
     65     except Exception as e:  # pylint: disable=broad-except
     66       filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67       raise e.with_traceback(filtered_tb) from None
     68     finally:
     69       del filtered_tb

D:\Python\lib\site-packages\tensorflow\python\eager\execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     52   try:
     53     ctx.ensure_initialized()
---> 54     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
     55                                         inputs, attrs, num_outputs)
     56   except core._NotOkStatusException as e:

InvalidArgumentError: Graph execution error:

Detected at node 'model/embedding_1/embedding_lookup' defined at (most recent call last):
    File "D:\Python\lib\runpy.py", line 194, in _run_module_as_main
      return _run_code(code, main_globals, None,
    File "D:\Python\lib\runpy.py", line 87, in _run_code
      exec(code, run_globals)
    File "D:\Python\lib\site-packages\ipykernel_launcher.py", line 16, in <module>
      app.launch_new_instance()
    File "D:\Python\lib\site-packages\traitlets\config\application.py", line 664, in launch_instance
      app.start()
    File "D:\Python\lib\site-packages\ipykernel\kernelapp.py", line 612, in start
      self.io_loop.start()
    File "D:\Python\lib\site-packages\tornado\platform\asyncio.py", line 149, in start
      self.asyncio_loop.run_forever()
    File "D:\Python\lib\asyncio\base_events.py", line 570, in run_forever
      self._run_once()
    File "D:\Python\lib\asyncio\base_events.py", line 1859, in _run_once
      handle._run()
    File "D:\Python\lib\asyncio\events.py", line 81, in _run
      self._context.run(self._callback, *self._args)
    File "D:\Python\lib\site-packages\tornado\ioloop.py", line 690, in <lambda>
      lambda f: self._run_callback(functools.partial(callback, future))
    File "D:\Python\lib\site-packages\tornado\ioloop.py", line 743, in _run_callback
      ret = callback()
    File "D:\Python\lib\site-packages\tornado\gen.py", line 787, in inner
      self.run()
    File "D:\Python\lib\site-packages\tornado\gen.py", line 748, in run
      yielded = self.gen.send(value)
    File "D:\Python\lib\site-packages\ipykernel\kernelbase.py", line 381, in dispatch_queue
      yield self.process_one()
    File "D:\Python\lib\site-packages\tornado\gen.py", line 225, in wrapper
      runner = Runner(result, future, yielded)
    File "D:\Python\lib\site-packages\tornado\gen.py", line 714, in __init__
      self.run()
    File "D:\Python\lib\site-packages\tornado\gen.py", line 748, in run
      yielded = self.gen.send(value)
    File "D:\Python\lib\site-packages\ipykernel\kernelbase.py", line 365, in process_one
      yield gen.maybe_future(dispatch(*args))
    File "D:\Python\lib\site-packages\tornado\gen.py", line 209, in wrapper
      yielded = next(result)
    File "D:\Python\lib\site-packages\ipykernel\kernelbase.py", line 268, in dispatch_shell
      yield gen.maybe_future(handler(stream, idents, msg))
    File "D:\Python\lib\site-packages\tornado\gen.py", line 209, in wrapper
      yielded = next(result)
    File "D:\Python\lib\site-packages\ipykernel\kernelbase.py", line 543, in execute_request
      self.do_execute(
    File "D:\Python\lib\site-packages\tornado\gen.py", line 209, in wrapper
      yielded = next(result)
    File "D:\Python\lib\site-packages\ipykernel\ipkernel.py", line 306, in do_execute
      res = shell.run_cell(code, store_history=store_history, silent=silent)
    File "D:\Python\lib\site-packages\ipykernel\zmqshell.py", line 536, in run_cell
      return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
    File "D:\Python\lib\site-packages\IPython\core\interactiveshell.py", line 2866, in run_cell
      result = self._run_cell(
    File "D:\Python\lib\site-packages\IPython\core\interactiveshell.py", line 2895, in _run_cell
      return runner(coro)
    File "D:\Python\lib\site-packages\IPython\core\async_helpers.py", line 68, in _pseudo_sync_runner
      coro.send(None)
    File "D:\Python\lib\site-packages\IPython\core\interactiveshell.py", line 3071, in run_cell_async
      has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
    File "D:\Python\lib\site-packages\IPython\core\interactiveshell.py", line 3263, in run_ast_nodes
      if (await self.run_code(code, result,  async_=asy)):
    File "D:\Python\lib\site-packages\IPython\core\interactiveshell.py", line 3343, in run_code
      exec(code_obj, self.user_global_ns, self.user_ns)
    File "<ipython-input-12-235fc25bdbdc>", line 2, in <module>
      model.fit([input_sequences, target_sequences[:, :-1]], target_sequences[:, 1:], epochs=10, batch_size=32, validation_split=0.2)
    File "D:\Python\lib\site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
      return fn(*args, **kwargs)
    File "D:\Python\lib\site-packages\keras\engine\training.py", line 1384, in fit
      tmp_logs = self.train_function(iterator)
    File "D:\Python\lib\site-packages\keras\engine\training.py", line 1021, in train_function
      return step_function(self, iterator)
    File "D:\Python\lib\site-packages\keras\engine\training.py", line 1010, in step_function
      outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "D:\Python\lib\site-packages\keras\engine\training.py", line 1000, in run_step
      outputs = model.train_step(data)
    File "D:\Python\lib\site-packages\keras\engine\training.py", line 859, in train_step
      y_pred = self(x, training=True)
    File "D:\Python\lib\site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
      return fn(*args, **kwargs)
    File "D:\Python\lib\site-packages\keras\engine\base_layer.py", line 1096, in __call__
      outputs = call_fn(inputs, *args, **kwargs)
    File "D:\Python\lib\site-packages\keras\utils\traceback_utils.py", line 92, in error_handler
      return fn(*args, **kwargs)
    File "D:\Python\lib\site-packages\keras\engine\functional.py", line 451, in call
      return self._run_internal_graph(
    File "D:\Python\lib\site-packages\keras\engine\functional.py", line 589, in _run_internal_graph
      outputs = node.layer(*args, **kwargs)
    File "D:\Python\lib\site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
      return fn(*args, **kwargs)
    File "D:\Python\lib\site-packages\keras\engine\base_layer.py", line 1096, in __call__
      outputs = call_fn(inputs, *args, **kwargs)
    File "D:\Python\lib\site-packages\keras\utils\traceback_utils.py", line 92, in error_handler
      return fn(*args, **kwargs)
    File "D:\Python\lib\site-packages\keras\layers\embeddings.py", line 197, in call
      out = tf.nn.embedding_lookup(self.embeddings, inputs)
Node: 'model/embedding_1/embedding_lookup'
indices[26,129] = 6453 is not in [0, 5002)
	 [[{{node model/embedding_1/embedding_lookup}}]] [Op:__inference_train_function_5695]
11

Hi @MD_Raihanul_Islam_To, The error is due to the while creating a embeddings for index values, the index value is not present in the model vocabulary. while defining the vocab size make sure that the vocab size is maximum integer index + 1. Thank You.