Implementing custom loss function in Tensorflow leading to “ValueError: 'outputs' must be defined before the loop.”

TensorFlow General Discussion
, , ,
1

I’m working on my first machine learning project in Python - using TensorFlow to try and syllabify words using the Moby Hyphenator II dataset.

I am treating this as a multi-label classification problem in which words and their syllables are encoded in the following format:

T e n - s o r - f l o w
0 0 1   0 0 1   0 0 0 0

When reading through this guide as a starting point, I saw that the author used a custom function - they averaged weighted binary cross-entropy with the root mean squared error in PyTorch as such:

def bce_rmse(pred, target, pos_weight = 1.3, epsilon = 1e-12):
    # Weighted binary cross entropy
    loss_pos = target * torch.log(pred + epsilon)
    loss_neg = (1 - target) * torch.log(1 - pred + epsilon)
    bce = torch.mean(torch.neg(pos_weight * loss_pos + loss_neg))

    # Root mean squared error
    mse = (torch.sum(pred, dim = 0) - torch.sum(target, dim = 0)) ** 2
    rmse = torch.mean(torch.sqrt(mse + epsilon))

    return (bce + rmse) / 2

I have tried to implement this in TensorFlow in the following way:

def weighted_bce_mse(y_true, y_prediction):
    # Binary crossentropy with weighting
    epsilon = 1e-12
    positive_weight = 4.108897148948174
    loss_positive = y_true * tf.math.log(y_prediction + epsilon)
    loss_negative = (1 - y_true) * tf.math.log(1 - y_prediction + epsilon)
    bce_loss = np.mean(tf.math.negative(positive_weight * loss_positive + loss_negative))
    
    # Mean squared error
    mse = tf.keras.losses.MeanSquaredError()
    mse_loss = mse(y_true, y_prediction)

    averaged_bce_mse = (bce_loss + mse_loss) / 2
    return averaged_bce_mse

On doing so, I receive the error ValueError: 'outputs' must be defined before the loop. and I’m not sure why as I define this function before I build and compile my model.

I’m using the Keras Functional API, and my compilation and fit stages are:

model.compile(optimizer="adam", loss=weighted_bce_mse, metrics=["accuracy"], steps_per_execution=64)
history = model.fit(padded_inputs, padded_outputs, validation_data=(validation_inputs, validation_outputs), epochs=10, verbose=2)
3

I appreciate what you did.

5

weighted_bce_mse should be passed as in a string. If you want to pass it in as a function, consider using a wrapper function or use subclass tf.keras.losses.Loss