ValueError: Input 0 of layer "sequential_1" is incompatible with the layer: expected shape=(None, 6, 216, 1), found shape=(None, 216, 1)

TensorFlow General Discussion
1

Hey Tensorflow community :wave:

I’m new to this ML package and am stuck on a bug related to the shapes of my tensors.

I’m trying to create a CNN model for speech emotion classification but I am getting the error at the model.fit stage.

The shapes of the 2 tensors being used are:
x_testcnn shape: (9, 216, 1)
y_test shape: (9, 4)

The full error that I’m getting is:

ValueError                                Traceback (most recent call last)
<ipython-input-58-bd5fc6744319> in <module>
----> 1 cnnhistory = model.fit(x_traincnn, y_train, batch_size=16, epochs=700, validation_data=(x_testcnn, y_test))

1 frames
/usr/local/lib/python3.8/dist-packages/keras/engine/training.py in tf__train_function(iterator)
     13                 try:
     14                     do_return = True
---> 15                     retval_ = ag__.converted_call(ag__.ld(step_function), (ag__.ld(self), ag__.ld(iterator)), None, fscope)
     16                 except:
     17                     do_return = False

ValueError: in user code:

    File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1249, in train_function  *
        return step_function(self, iterator)
    File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1233, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1222, in run_step  **
        outputs = model.train_step(data)
    File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1023, in train_step
        y_pred = self(x, training=True)
    File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 70, in error_handler
        raise e.with_traceback(filtered_tb) from None
    File "/usr/local/lib/python3.8/dist-packages/keras/engine/input_spec.py", line 295, in assert_input_compatibility
        raise ValueError(

    ValueError: Input 0 of layer "sequential_1" is incompatible with the layer: expected shape=(None, 6, 216, 1), found shape=(None, 216, 1)

Any help would be appreciated!

Thanks a lot

1 Like
2

can you include the net definition (you can just print the summary)

1 Like
3

Hi @Mah_Neh

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv1d (Conv1D)             (None, 216, 256)          1536      
                                                                 
 activation (Activation)     (None, 216, 256)          0         
                                                                 
 conv1d_1 (Conv1D)           (None, 216, 128)          163968    
                                                                 
 activation_1 (Activation)   (None, 216, 128)          0         
                                                                 
 dropout (Dropout)           (None, 216, 128)          0         
                                                                 
 max_pooling1d (MaxPooling1D  (None, 27, 128)          0         
 )                                                               
                                                                 
 conv1d_2 (Conv1D)           (None, 27, 128)           82048     
                                                                 
 activation_2 (Activation)   (None, 27, 128)           0         
                                                                 
 conv1d_3 (Conv1D)           (None, 27, 128)           82048     
                                                                 
 activation_3 (Activation)   (None, 27, 128)           0         
                                                                 
 flatten (Flatten)           (None, 3456)              0         
                                                                 
 dense (Dense)               (None, 10)                34570     
                                                                 
 activation_4 (Activation)   (None, 10)                0         
                                                                 
=================================================================
Total params: 364,170
Trainable params: 364,170
Non-trainable params: 0

Is this what you wanted?

4

Yes but I wonder why there isnt an input shape, can you add also how you defined the model? Or a link to whatever you use

5

This is how I defined the model:
model.compile(loss='categorical_crossentropy', optimizer=opt,metrics=['accuracy'])

And this is the link to my GitHub repo where the code is. You can go through the code if you’d like.

7

From a quick look, it seems as if the labels you have in cell 57 are 10, but then you pass 5 labels, I think you have repeated labels somewhere, do you notice the same? Or maybe I didnt understand @Sankalp_20

9

Thanks for the reply @Mah_Neh

I see I have 10 labels in cell 57 but could you elaborate a little more on the 5 label part. I’m not quite sure what to do here.

from tensorflow import keras
#from keras import optimizers

model = Sequential()


model.add(Conv1D(256, 5,padding='same',
                 input_shape=(216,1)))
model.add(Activation('relu'))
model.add(Conv1D(128, 5,padding='same'))
model.add(Activation('relu'))
model.add(Dropout(0.1))
model.add(MaxPooling1D(pool_size=(8)))
model.add(Conv1D(128, 5,padding='same',))
model.add(Activation('relu'))
#model.add(Conv1D(128, 5,padding='same',))
#model.add(Activation('relu'))
#model.add(Conv1D(128, 5,padding='same',))
#model.add(Activation('relu'))
#model.add(Dropout(0.2))
model.add(Conv1D(128, 5,padding='same',))
model.add(Activation('relu'))
model.add(Flatten())
model.add(Dense(10))
model.add(Activation('softmax'))
#opt = keras.optimizers.RMSprop(lr=0.00001, decay=1e-6)
opt = keras.optimizers.legacy.Adam(lr=0.00001, decay=1e-6)

I suppose you are referring to the "5"s I have entered as the argument for Conv1D?

If so, I tried changing that to 10 but I am still getting the error.

Would be great if you could suggest a change :slight_smile:

1 Like
10

I will think about it, I tried to run it but I do not have the dataset, I am not sure if there is a secure way to share this. I am also learning and trying to solve some problems, understanding the dimensions is essential. What I would to is try to remove stuff, to scope the error.

For example, does the same happen is the evaluation data is removed from the fit fn (to know if the problem is already present in the training data.)

I thought that maybe the labels should actually be five rather than the input 10, for some reason.

12

Here is a link to the audio files I’m using to train the model. (I’m using the RAVDESS dataset)
https://drive.google.com/drive/folders/1TFDdJM14ze4qnRFh9nDLIpnodOYjLBrU?usp=sharing

1 Like
13

Can you show the full code, please or at least the line with the .fit() command as we may be missing some bits here?

If you are a starter with Tensorflow/ML/NN, and if I may advise, you should:

  • It is critical you understand tensors shape, e.g. (batch, timesteps, input_features) Introduction to Tensors  |  TensorFlow Core as hinted by @Mah_Neh
  • You also need to understand the layers you use: data that get in, operations within that layer and output
  • Add one layer another. check if this works. If it does, add the next layer, etc…
  • Use the same import(s) as in keras/Tensorflow documentation e.g. from keras import layers, and when using a layer layers.[that layer]
  • Use variables as much as possible instead of hard coding tensors shapes

I mean otherwise you will get stuck all the time whenever you create a new model.

Thank you.

2 Likes
14

Great then I will reproduce it from scratch in colab

1 Like
15

Thanks @tagoma,

I’ll keep these things in mind as I go forward on my path to learn Tensorflow in more detail.

To answer the first part of your question, this is the model and .fit() command I used:

model = Sequential()


model.add(Conv1D(256, 5,padding='same',
                 input_shape=(216,1)))
model.add(Activation('relu'))
model.add(Conv1D(128, 5,padding='same'))
model.add(Activation('relu'))
model.add(Dropout(0.1))
model.add(MaxPooling1D(pool_size=(8)))
model.add(Conv1D(128, 5,padding='same',))
model.add(Activation('relu'))
#model.add(Conv1D(128, 5,padding='same',))
#model.add(Activation('relu'))
#model.add(Conv1D(128, 5,padding='same',))
#model.add(Activation('relu'))
#model.add(Dropout(0.2))
model.add(Conv1D(128, 5,padding='same',))
model.add(Activation('relu'))
model.add(Flatten())
model.add(Dense(10))
model.add(Activation('softmax'))
#opt = keras.optimizers.RMSprop(lr=0.00001, decay=1e-6)
opt = keras.optimizers.legacy.Adam(lr=0.00001, decay=1e-6)

cnnhistory = model.fit(x_traincnn, y_train, batch_size=16, epochs=700, validation_data=(x_testcnn, y_test))

Let me know if there is some other code snippet that you require and I’ll add it

Thanks in advance

16

I’m a little confused to be honest. I went through your Colab now (using Actor_01/03-01-01-01-01-01-01.wav as you did) and the tensors shapes and errors messages are often different from what you wrote down here, and also from what was in your Colab as I opened it the first time and what I get when I ran the Colab cells. The shape of your training and test datasets also don’t look quite consistent with one another. Can you please check?

17

Not sure how this works really because I cant see a single file. NVM I found the files elsewhere

18

Took me a while to figure this out but the main error is that you have 5 labels and your output layer has 10 units.

So replace model.add(Dense(10)) with model.add(Dense(5))

I actually re wrote the whole notebook apart from parts I didnt get and think your idea is fantastic.

Comments

Basically you are representing sound as an image, and trying to find patterns on it. I am a newcomer as well.

Now you pass 32,216 but as far as I can see this consumes all your samples,

What I did is to reshape each sample into an image of 27x8 (216) just to see what happens.

Here is the code.

1 Like
19

Thanks a lot for reproducing the issue @Mah_Neh

I ran the notebook you provided but I am facing the same error with the shapes again. I’m not sure why’s that happening.

The shapes I’m using:
X_test: (6, 27, 8)
Y_test: (6, 4)

This shouldn’t really throw an error considering I reshaped the sample like you described.

The full error I’m getting:

ValueError: in user code:

    File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1820, in test_function  *
        return step_function(self, iterator)
    File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1804, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1792, in run_step  **
        outputs = model.test_step(data)
    File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1758, in test_step
        self.compute_loss(x, y, y_pred, sample_weight)
    File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1082, in compute_loss
        return self.compiled_loss(
    File "/usr/local/lib/python3.8/dist-packages/keras/engine/compile_utils.py", line 265, in __call__
        loss_value = loss_obj(y_t, y_p, sample_weight=sw)
    File "/usr/local/lib/python3.8/dist-packages/keras/losses.py", line 152, in __call__
        losses = call_fn(y_true, y_pred)
    File "/usr/local/lib/python3.8/dist-packages/keras/losses.py", line 284, in call  **
        return ag_fn(y_true, y_pred, **self._fn_kwargs)
    File "/usr/local/lib/python3.8/dist-packages/keras/losses.py", line 2004, in categorical_crossentropy
        return backend.categorical_crossentropy(
    File "/usr/local/lib/python3.8/dist-packages/keras/backend.py", line 5532, in categorical_crossentropy
        target.shape.assert_is_compatible_with(output.shape)

    ValueError: Shapes (None, 4) and (None, 5) are incompatible
21

Can you try again Google Colab.

It would be nice if you can explain the main idea of your Notebook, so that we can also understand your goal. I think there may be something missing yet, like, is representing just a series of values (intensities of sound) in time, as a matrix instead, i.e just splitting the array, enough for sentient classification?

So we would ideally understand well how that process is thought out. Currently I think it is not enough. You will notice, for example, that the validation loss barely changes, which is also an indication of something missing…

@Sankalp_20

1 Like
23

@Mah_Neh Thanks a lot for all the efforts you put in! Your notebook worked for me! Although the data is underfitting right now but I will fix that

To answer your question on what I am trying to achieve from this notebook: I’m trying to create an audio classifier that I’m training on the RAVDESS dataset. I then need to use that model to classify a bunch of audio files that I’ll feed it

2 Likes
24

Great, could be nice to see your results in a while, looks like an interesting project.

Good luck!

2 Likes