# None gradient using custom loss function that uses MSSIM

**URL:** <https://discuss.ai.google.dev/t/none-gradient-using-custom-loss-function-that-uses-mssim/31569>\
**Category:** General Discussion\
**Tags:** gradienttape, tfgradient, tf-train\
**Created:** [March 4, 2024, 4:03am UTC](https://discuss.ai.google.dev/t/none-gradient-using-custom-loss-function-that-uses-mssim/31569 "2024-03-04T04:03:34Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ahmad\_Hazem](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/ahmad_hazem/32/9429_2.png) [@Ahmad\_Hazem](https://discuss.ai.google.dev/u/Ahmad_Hazem)\
**Post date:** [March 4, 2024, 4:03am UTC](https://discuss.ai.google.dev/t/none-gradient-using-custom-loss-function-that-uses-mssim/31569/1 "2024-03-04T04:03:34Z")

</div>

Hello, I am a novice with tensorflow, I am trying create a custom loss function that uses MSSIM. I am trying to calculate the MSSIM of a ground truth with the gradcam++ sailancy map of the input image.  
Bellow is some of the code snipits.

Map calculation:  
def calc\_maps(model, HE\_images, HE\_labels):  
HE\_labels = list(tf.argmax(HE\_labels, axis=1))  
replace2linear = ReplaceToLinear()  
score = CategoricalScore(HE\_labels)  
gradcam = GradcamPlusPlus(model, model\_modifier=replace2linear, clone=True)  
cam = tf.convert\_to\_tensor(gradcam(score,HE\_images,penultimate\_layer=-1))

```
return cam

```

A trail I did:  
HE, IHC = train\_data\_loader[0]  
LABELS = tf.convert\_to\_tensor(HE[-1])  
HE = tf.convert\_to\_tensor(HE[:-1][0])  
IHC = tf.convert\_to\_tensor(IHC[:-1][0])

MAPS = 1 - calc\_maps(model, HE, LABELS)

with tf.GradientTape() as tape:  
predect = model(HE) #forward

```
im1 =tf.image.convert_image_dtype(IHC, tf.float32)
im2 =tf.image.convert_image_dtype(MAPS, tf.float32)

im2 = tf.expand_dims(im2, axis=-1)

ssim = tf.image.ssim_multiscale(im1, im2, 1.0)

```

gradients = tape.gradient(ssim, model.trainable\_variables)

print(gradients)

---

<div class="post-metadata">

**Author:** ![aniruthraj](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/aniruthraj/32/660_2.png) [@aniruthraj](https://discuss.ai.google.dev/u/aniruthraj)\
**Post date:** [January 3, 2025, 2:58pm UTC](https://discuss.ai.google.dev/t/none-gradient-using-custom-loss-function-that-uses-mssim/31569/3 "2025-01-03T14:58:58Z")

</div>

Hi @Ahmad_Hazem,

Sorry for the delay in response.  
To create a custom loss function using MSSIM between the ground truth (`IHC`) and GradCAM++ saliency maps (`MAPS`), first ensure both images are 4D tensors (batch\_size, height, width, channels) and then use `tf.image.ssim_multiscale()` to calculate the MSSIM, and define the loss as `1 - tf.reduce_mean(ssim)` to maximize similarity. You can calculate the gradients for backpropagation using `tf.GradientTape()`. Ensure proper dtype (`tf.float32`) for the images and other custom components are correctly implemented.Kindly refer this [documentation](https://www.tensorflow.org/api_docs/python/tf/GradientTape) about tf.GradientTape for more information.

Please let us know if this helps.Thank You.