# Extracting KL & NLL separately from compiler

**URL:** <https://discuss.ai.google.dev/t/extracting-kl-nll-separately-from-compiler/30662>\
**Category:** General Discussion\
**Tags:** tf-probability, help\_request\
**Created:** [July 11, 2022, 9:21am UTC](https://discuss.ai.google.dev/t/extracting-kl-nll-separately-from-compiler/30662 "2022-07-11T09:21:17Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![benjamin\_vollebregt](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/benjamin_vollebregt/32/11457_2.png) [@benjamin\_vollebregt](https://discuss.ai.google.dev/u/benjamin_vollebregt)\
**Post date:** [July 11, 2022, 9:21am UTC](https://discuss.ai.google.dev/t/extracting-kl-nll-separately-from-compiler/30662/1 "2022-07-11T09:21:17Z")

</div>

Dear,

I am currently building Bayesian Neural Networks and I managed to train my networks quite good using the negative log likelihood as loss in the model compiler. As I am using densevariational layers instead of regular dense layers, the KL divergence is added as loss to the compiler (at least that is what I understood).  
It is clearly visible that the loss (KL+NLL) converges to a minimum value, however I would like to plot the KL and NLL separately as well, so I must extract them from the compiler, or use a different approach.

Does anyone have experience with this / have any tips?

Thanks in advance!  
Benjamin

* * *

Model code (MWE):

# Define the model

def get\_full\_model(x\_train\_shape):

```
model = tf.keras.Sequential([

    # Epistemic uncertainty
    tfpl.DenseVariational(...),
    tfpl.DenseVariational(...),

    # Aleatoric uncertainty
    tfpl.IndependentNormal(1)
])

def neg_loglik(y_true, y_pred):
    return -y_pred.log_prob(y_true)

model.compile(loss=neg_loglik, optimizer='rmsprop',metrics=[tf.keras.metrics.MeanAbsoluteError()])

return model

```