Converting channel_last data to channel_first

rishit_dagli · January 19, 2022, 9:44am

I was writing a tutorial for one of my paper implementations quite similar to the Image classification tutorial and the model I use expects data in the channels_first format. Of course, while using tf.data.Dataset we can simply use tf.transpose.

I was wondering if there was any built-in way to load data with tf.keras.utils.image_dataset_from_directory in the channels_first format or a layer to do so?

Right now I do so in my model adding a lambda layer to do so:

model = tf.keras.Sequential([tf.keras.layers.Lambda(lambda x: tf.einsum("...ijk->...kij", x)),
                                                   ...
                                                   ...])

Here is my complete Colab notebook if required (created in a similar fashion to the classification guide on TensorFlow website):

github.com

Rishit-dagli/Transformer-in-Transformer/blob/main/training.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/Rishit-dagli/Transformer-in-Transformer/blob/main/training.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FE7KNzPPVrVV"
   },
   "source": [
    "# Image classification with Transformer iN Transformer\n",
    "\n",

This file has been truncated. show original

ariG23498 · January 19, 2022, 10:23am

Another option is to use the tf.keras.layers.Permute

Link: tf.keras.layers.Permute | TensorFlow v2.16.1

Bhack · January 19, 2022, 12:07pm

In the end this API is just going to dataset map over load_image that it is a multiple call over tf.image.decode_image so you don’t have any parameter to do it internally.

github.com

keras-team/keras/blob/41c992105fdc0f0d00e7618752f36e0884f8e5c9/keras/preprocessing/image_dataset.py#L251


      
                                          labels,
                                          label_mode,
                                          num_classes,
                                          interpolation,
                                          crop_to_aspect_ratio=False):
            """Constructs a dataset of images and labels."""
            # TODO(fchollet): consider making num_parallel_calls settable
            path_ds = tf.data.Dataset.from_tensor_slices(image_paths)
            args = (image_size, num_channels, interpolation, crop_to_aspect_ratio)
            img_ds = path_ds.map(
                lambda x: load_image(x, *args), num_parallel_calls=tf.data.AUTOTUNE)
            if label_mode:
              label_ds = dataset_utils.labels_to_dataset(labels, label_mode, num_classes)
              img_ds = tf.data.Dataset.zip((img_ds, label_ds))
            return img_ds
          
          
          def load_image(path, image_size, num_channels, interpolation,
                         crop_to_aspect_ratio=False):
            """Load an image from a path and resize it."""
            img = tf.io.read_file(path)

rishit_dagli · January 20, 2022, 3:25am

Got it, thanks so much @ariG23498 and @Bhack ! Also surprisingly enough (I haven’t checked the source code for Permute layer yet and I will get to doing that) while my experiments I noticed the exact same operation with tf.keras.layers.Permute is faster than tf.einsum.

Bhack · January 20, 2022, 6:51am

permute is a tf.transpose