Unexpected output shape from NN

TensorFlow Special Interest Groups TF.js
1

I am using a few layers on top of VGG16, to tune the net for recognizing only if there is or not a specific object on an image.

const outputLayer = tf.layers
  .dense({ units: 1, activation: "sigmoid", dtype: "float32" })
  .apply(dense2);

const detection = tf.model({
  inputs: baseModel.inputs,
  outputs: outputLayer as tf.SymbolicTensor[],
});

detection.compile({
  optimizer: tf.train.adam(0.0001), //could be a parameter as well.
  loss: tf.losses.sigmoidCrossEntropy,
  metrics: ["mse"],
});

The output is [0,1] when I do a prediction. Why isn’t the output a single number, when the last layer has a single unit ?

Just in case this helps to solve, the labels have this format:

[
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
  0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1,
  1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
  1, 1, 1, 1,
  ... more items
]
2

My bad, at some point changed the directories and it was reading from an old model, by directing it to the right model it outputs the expected single value 0-1.

However, the network needs a modification

The output layer should be changed to use 2 units

const outputLayer = tf.layers
  .dense({ units: 2, activation: "softmax" }) // or sigmoid
  .apply(dense2);

And the compilation as well:

detection.compile({
  optimizer: tf.train.adam(0.0001), //could be a parameter as well.
  loss: "categoricalCrossEntropy",  // ------ this line
  metrics: ["mse"],
});

And the labels should be passed as a oneHot vector, in my case a 0,1 array when there is an airplane, or 1,0 array when there is a stop sign, and could also be passed the remaining 2 combinations if we have training set images.

Now the output makes sense to some extent.

1 Like