Stanford Background Scene Understanding in Keras

[innat]

We’ve shared a beginner-friendly code example on the “Stanford Background Scene Understanding” dataset; a multi-class segmentation task that consists of eight different types of objects (thing + stuff).

Code.
Dataset.

[Jimut_Bahan_Pal1]

Hi @innat, could you suggest ways to implement multi class dice loss, and maybe track other metrics like IoU, or Dice coefficient using the current code?

[innat]

You can use this REPO for this.

import segmentation_models as sm

model.compile(
    ...,
    loss=sm.losses.bce_jaccard_loss,
    metrics=[sm.metrics.iou_score],
)

Also, note that you may need to transform the target label to a one-hot encoded vector (tf.one_hot). Currently, in the above code example, I’ve used integer encoding.

[Jimut_Bahan_Pal1]

Hi @innat, thanks for sharing the repo, I was wondering that some images might not contain all the segmentation classes, like in this dataset for instance segmentation, CIHP Dataset | Papers With Code . Say I have a dataset with 30 classes to segment, but most of the images have say, 4-10 classes present, in that case the metrics calculation will be wrong right? since we are going over all the classes and most of the values will be 0 since the classes are not present.

Like for the repo you shared, on the metrics.py (qubvel/segmentation_models/blob/master/segmentation_models/metrics.py) file, how can we consider only the important classes present in the image and ignore the classes that are not present. Though there is a smoothing factor, but can there be much more concrete stuff? For example the calculation will go through all the classes of y_true and perhaps if a pixel from that particular class is found, we will consider the corresponding class for the y_pred and this will go on till say n_classes? Like a minor modification of this function image-segmentation-keras/keras_segmentation/metrics.py at master · divamgupta/image-segmentation-keras · GitHub

Will this not create a bottleneck in running (speed) and logging the metrics in model.compile, when we are logging the history? For your dataset, the method might work as it is, but when we are considering the dice coefficient as metrics in say k-fold cross validation, shall we not only take the important classes of an image rather than all the classes which are not even present? This might lower the metrics in general even if it has an epsilon smooth factor. Would love to hear your opinion in this… I am not sure if he is using this technique or not, but correct me if I am wrong. If he is using this technique then will passing metrics=[sm.metrics.iou_score] to model.compile will work?