How extract loss along with the predicted label from transfomrer model

TensorFlow General Discussion
,
1

HI
how I can extract loss along with the predicted label from trained transformer based model?
for example in pytorch we can extarct logits from tokenized inputs:

with torch.no_grad():
output = model(**inputs)
pred_label = torch.argmax(output.logits, axis=-1)
loss = cross_entropy(output.logits, batch[“label”].to(device), reduction=“none”)

how we can do that in TF2?
thanks in advantage

3

Hi @caprone ,

It’ll be the same above.Get the logits from the model, calculate the loss, extract predicted labels and print the loss and predicted labels.

Example:

with tf.GradientTape() as tape:
outputs = model(inputs)
logits = outputs.logits
loss = sparse_categorical_crossentropy(true_labels, logits)
predicted_labels = tf.argmax(logits, axis=-1)
print(“Loss:”, loss.numpy())
print(“Predicted Labels:”, predicted_labels.numpy())

I hope this helps.

Thanks.