How to print the outputs of intermediate layers?

TensorFlow
, , ,
1

I am interested to see the output of intermediate layer, how do I do it?

I have to do it by following way:
import tensorflow as tf
import matplotlib.pyplot as plt
from tensorflow.keras import layers, models
from tensorflow.keras.callbacks import Callback

# Custom Callback to extract Dense layer values
class DenseLayerOutputCallback(Callback):
    def __init__(self, model, layer_name):
        super(DenseLayerOutputCallback, self).__init__()
        self.model = model
        self.layer_name = layer_name
        self.outputs = []

    def on_epoch_end(self, epoch, logs=None):
        intermediate_layer_model = models.Model(inputs=self.model.input, outputs=self.model.get_layer(self.layer_name).output)
        intermediate_outputs = intermediate_layer_model.predict(train_images)
        self.outputs.append(intermediate_outputs)

# Load the MNIST dataset
mnist = tf.keras.datasets.mnist
(train_images, train_labels), _ = mnist.load_data()

# Normalize the pixel values to the range [0, 1]
train_images = train_images / 255.0

# Define the input shape
input_shape = train_images[0].shape

# Define the model using the Functional API
input_layer = layers.Input(shape=input_shape)
x = layers.Flatten()(input_layer)
x = layers.Dense(128, activation='relu')(x)
x = layers.Dropout(0.2)(x)
output_layer = layers.Dense(10, activation='softmax')(x)

model = tf.keras.Model(inputs=input_layer, outputs=output_layer)

# Compile the model
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

# Create the custom callback to extract Dense layer values
#dense_layer_name = 'dense'  # Name of the Dense layer in the model
dense_layer_name = 'dense_6'  # Name of the Dense layer in the model
dense_output_callback = DenseLayerOutputCallback(model, layer_name=dense_layer_name)

# Train the model with the custom callback
history = model.fit(train_images, train_labels, epochs=5, batch_size=32, validation_split=0.1, callbacks=[dense_output_callback])

but I tried using

# Define the model using the Functional API
input_layer = layers.Input(shape=input_shape)
x = layers.Flatten()(input_layer)
tf.Print(x)
x = layers.Dense(128, activation='relu')(x)
x = layers.Dropout(0.2)(x)
output_layer = layers.Dense(10, activation='softmax')(x)

That doesn’t worked out. I would appreciate reply.

I am using
tf.__version__ --> 2.12.0

3

Hi @Abhijeet ,

Could you please check this keras article for printing output of intermediate layers.

I hope this hepls!

Thanks.

1 Like
4

Thanks for your reply.

I tried that, however, I was hoping to see instantaneous output as seen in pytorch.

I tried with tf.sessions but that isnt working with newer versions.

6

The technique I have used is to make a Functional model and add an extra output layer to the intermediate layer that I want to instrument.

You cannot see output from inside a Model because a Model is compiled from beginning to end for GPU or CPU execution. The individual layers do not exist as Python objects.

Tensorflow & Pytorch are not just simple libraries. They are presented as “libraries”, but they are partly operating system bolt-ons.