Model evaluation fails with: too many values to unpack (expected 2)

TensorFlow General Discussion
, ,
1

Can get the model evaluation to succeed thinking there is something wrong with the trained model.

Followed the turorial on:

Ran:

train_data, validation_data, test_data = object_detector.DataLoader.from_csv('my-dataset.csv')

model_arch = 'efficientdet_lite0'
spec = model_spec.get(model_arch)

EPOCHS = 20
model = object_detector.create(train_data, model_spec=spec, epochs=EPOCHS, batch_size=64, train_whole_model=True, validation_data=validation_data)

.
.
.
Epoch 20/20
513/513 [========================] - 108s 211ms/step - det_loss: 0.0851 - cls_loss: 0.0691 - box_loss: 3.1992e-04 - reg_l2_loss: 0.0627 - loss: 0.1477 - learning_rate: 2.2774e-05 - gradient_norm: 1.1593 - val_det_loss: 0.0648 - val_cls_loss: 0.0523 - val_box_loss: 2.5131e-04 - val_reg_l2_loss: 0.0627 - val_loss: 0.1275

And then:

loss, accuracy = model.evaluate(test_data)

*ValueError: too many values to unpack (expected 2)*

Dataset splits (image size 500x300):

  • Training Set - 4.8k images 92%
  • Validation Set 271 images 5%
  • Testing Set 135 images 3%
3

Can you print model.evaluate(test_data) directly?

2 Likes
4

the valuate method from model maker returns a dictionary with many metrics:
eg:

{'AP': 0.11919018,
 'AP50': 0.21924835,
 'AP75': 0.11994627,
 'AP_/Baked Goods': 0.012772277,
 'AP_/Cheese': 0.011738928,
 'AP_/Salad': 0.386223,
 'AP_/Seafood': 0.0024752475,
 'AP_/Tomato': 0.18274146,
 'APl': 0.12009244,
 'APm': 0.095719256,
 'APs': -1.0,
 'ARl': 0.24987727,
 'ARm': 0.55833334,
 'ARmax1': 0.09158534,
 'ARmax10': 0.20143048,
 'ARmax100': 0.25207776,
 'ARs': -1.0}

that’s why it can’t unpack into two values.
I’d assign it to a variable and just print the variable or use the right key

2 Likes
5

Thanks @Bhack and @lgusm!

Strange that ALL turorials use the unpack code snippet I tried. A lot of people should be seeing the same issue.

But now I’m getting:

{'AP': 0.8345967,
 'AP50': 0.9843931,
 'AP75': 0.9154825,
 'AP_/blue-landmat': 0.8039687,
 'AP_/class': -1.0,
 'AP_/red-landmat': 0.8652246,
 'APl': 0.88267785,
 'APm': 0.8323183,
 'APs': 0.41428608,
 'ARl': 0.9151628,
 'ARm': 0.88222224,
 'ARmax1': 0.8433333,
 'ARmax10': 0.8713492,
 'ARmax100': 0.8763492,
 'ARs': 0.5214286}
6

Are you talking about this page?

7

Yes, but the other tutorials are using actual keras models. This tutorial uses the wrapper object created by model maker. Personally, I don’t entirely understand why these are implemented as a wrapper class instead of just as keras models.

@khanhlvg

2 Likes
9

Model Maker targets users who are new to ML so we creates a wrapper around the Keras model. That’s why the behavior of a Model Maker model is different from a Keras model.

1 Like
11

@khanhlvg, which makes it more important to update tutorials. The link @markdaoust provided has the same unpack-code-snippet. Correct?

# Evaluate the model.loss, accuracy = model.evaluate(test_data)
1 Like
12

I see. Thanks for pointing out. I’ve sent an internal pull request to update the doc.

2 Likes