Resnet152 with 4 input channels (RGB + NIR)

TensorFlow General Discussion
, ,
1

Hey there!

I used the model provided by Keras (Resnet152) to train a classification model using the RGB images. It worked so so, reaching a f1 score of 0.7. Thing is I also got a NIR (near infrared) channel which I simply ignored, because I couldnt load it into the ready provided model of keras.

It should have exactly 3 inputs channels, and width and height should be no smaller than 32. E.g. (200, 200, 3) would be one valid value. (from the docs)

However now I want to do use the resnet152 model with four channels and I have some questions:

  • Is it correct, that I have to manually implement the resnet152 model to use the 4 channels (and that there is no way to use the ones provided by TF)?
  • Do you know of existing implementations, so that I can have a look at them or even use em?
  • What should I watch out when creating a Resnet with 4 input channels?
  • To preprocess the 4 channeld input to yield a 3 channeled one is not an option for me

Thank you very much!

3

I might found a solution using get_config and model.from_config. You can find the approach in the following article: Implementing Transfer Learning from RGB to Multi-channel Imagery | by Sijuade Oguntayo | Towards Data Science

I will try it with this. If you have thoughts // other suggestions, let me know.