I have a multi-class classification problem and the model I train gives an overall reasonable accuracy but fails in one or a few smaller classes (very low performance only on that class). I think part of the reason is that I use regular CrossEntropy as my loss function so I am optimizing for overall accuracy and not enforcing minimum per-class performance in any way. Is there any loss function in Keras that can do that for me? In other words, how do I force my model to not just focus on overall accuracy but ensure that I will have a reasonable performance in all classes?
Use sigmoid activation on your output layer and binary_crossentropy as loss function.
softmax gives probability distribution around n-classes and it works well when the classes are mutually exclusive and not their probabilities.
Where as sigmiod provides independent probabilities.
1 Like
Check also if these classes are imbalanced:
Other then these you can also use some imbalanced loss.
1 Like
Thanks but my classes are mutually exclusive so I don’t think this works. Each observation belongs to one class only (out of 9 classes). It’s just that the model works well on 8 classes and sacrifices performance in one or a few classes.
Thanks. I have tried re-weighting and it slightly helps the problematic classes but doesn’t solve the problem. I am thinking maybe this is an indication that the problematic classes have an underlying issue and are just difficult to differentiate from the rest but don’t know how to test this hypothesis.
You can try to check if your model could overfit you data training on your current train+test data.
They you can check if you train data points for these specific classes are representative enough of the test dataset for the same classes.