AttributeError: 'Variable' object has no attribute '_distribute_strategy'

TensorFlow General Discussion
, ,
1

I got an error from this code

# Intialize the model
model = TFBertForSequenceClassification.from_pretrained('bert-base-uncased', num_labels=2)

# Compile the model with an appropriate optimizer, loss function, and metrics
optimizer = tf.keras.optimizers.Adam(learning_rate=2e-5)
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy')
model.compile(optimizer=optimizer, loss=loss, metrics=[metric])

This is the error

AttributeError                            Traceback (most recent call last)
Cell In[38], line 5
      3 loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
      4 metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy')
----> 5 model.compile(optimizer=optimizer, loss=loss, metrics=[metric])

File /opt/conda/lib/python3.10/site-packages/transformers/modeling_tf_utils.py:1495, in TFPreTrainedModel.compile(self, optimizer, loss, metrics, loss_weights, weighted_metrics, run_eagerly, steps_per_execution, **kwargs)
   1493 # This argument got renamed, we need to support both versions
   1494 if "steps_per_execution" in parent_args:
-> 1495     super().compile(
   1496         optimizer=optimizer,
   1497         loss=loss,
   1498         metrics=metrics,
   1499         loss_weights=loss_weights,
   1500         weighted_metrics=weighted_metrics,
   1501         run_eagerly=run_eagerly,
   1502         steps_per_execution=steps_per_execution,
   1503         **kwargs,
   1504     )
   1505 else:
   1506     super().compile(
   1507         optimizer=optimizer,
   1508         loss=loss,
   (...)
   1514         **kwargs,
   1515     )

File /opt/conda/lib/python3.10/site-packages/tf_keras/src/utils/traceback_utils.py:70, in filter_traceback.<locals>.error_handler(*args, **kwargs)
     67     filtered_tb = _process_traceback_frames(e.__traceback__)
     68     # To get the full stack trace, call:
     69     # `tf.debugging.disable_traceback_filtering()`
---> 70     raise e.with_traceback(filtered_tb) from None
     71 finally:
     72     del filtered_tb

File /opt/conda/lib/python3.10/site-packages/tensorflow/python/distribute/distribute_lib.py:4029, in _DefaultDistributionExtended.variable_created_in_scope(self, v)
   4028 def variable_created_in_scope(self, v):
-> 4029   return v._distribute_strategy is None

AttributeError: 'Variable' object has no attribute '_distribute_strategy'
3

Hi @Victor_Umunna, Before executing the above code part please try to run these lines of code

import os
os.environ["TF_USE_LEGACY_KERAS"] ="1"

I have tried with this work around and did not face any error. please refer to this gist for working code example. Thank You.