ValueError: `logits` and `labels` must have the same shape, received ((None, 256, 256, 4) vs (None,)) ValueError: `logits` and `labels` must have the same shape, received ((None, 256, 256, 4) vs (None,))

[Amisha_Gouda]

def conv_block(input, num_filters):
x = Conv2D(num_filters, 3, padding=“same”)(input)
x = BatchNormalization()(x) #Not in the original network.
x = Activation(“relu”)(x)

x = Conv2D(num_filters, 3, padding="same")(x)
x = BatchNormalization()(x)  #Not in the original network
x = Activation("relu")(x)

return x

def encoder_block(input, num_filters):
x = conv_block(input, num_filters)
p = MaxPool2D((2, 2))(x)
return x, p

def decoder_block(input, skip_features, num_filters):
x = Conv2DTranspose(num_filters, (2, 2),strides=2,padding=“same”)(input)
x = Concatenate()([x, skip_features])
x = conv_block(x, num_filters)
return x

def build_unet(input_shape, n_classes):
inputs = Input(input_shape)

s1, p1 = encoder_block(inputs, 64)
s2, p2 = encoder_block(p1, 128)
s3, p3 = encoder_block(p2, 256)
s4, p4 = encoder_block(p3, 512)

b1 = conv_block(p4, 1024) #Bridge

d1 = decoder_block(b1, s4, 512)
d2 = decoder_block(d1, s3, 256)
d3 = decoder_block(d2, s2, 128)
d4 = decoder_block(d3, s1, 64)

if n_classes == 1:
  activation = 'sigmoid'
else:
  activation = 'softmax'

outputs = Conv2D(n_classes, 1, padding="same", activation=activation)(d4)  


model = Model(inputs, outputs, name="U-Net")


return model

my_unet = build_unet(input_shape=(256,256,3), n_classes=4)

print(my_unet.summary())

my_unet.compile(optimizer=tf.keras.optimizers.Adam(), loss=‘binary_crossentropy’, metrics=[‘accuracy’])

history = my_unet.fit(
train_ds,
batch_size=BATCH_SIZE,
validation_data=val_ds,
verbose=1,
epochs=50,

)

can somebody help me with this code

[Kiran_Sai_Ramineni]

Hi @Amisha_Gouda, While passing labels during model.fit the labels should have the same shape as the input. But you are passing the shape as (None,). Please refer to this gist for the working code example. Thank You.

[Ayushi_Tripathi]

I have a similar error:

ValueError: 'logits' and 'labels' must have the same shape, received ((None, 2) vs (None, 1)).

My dataframe has two columns: Links (to string type, has titles of new articles) and Shortlisted (of numeric type, has value either 0 (not shortlisted) or 1 (shortlisted)).

I am working on a binary classification (nlp) project using BERT (by Google)
My code is throwing this error:
ValueError: logits and labels must have the same shape, received ((None, 2) vs (None, 1)).

And I am not understanding what this means and how to fix this.

1. Splitting into training and testing data.

df['Length'].max()train_df=df[~(df['Sheet'].str.contains('April2024', regex=True) | df['Sheet'].str.contains('May2024', regex=True))]
test_df=df[len(train_df):]

2. Importing libraries and classes

from transformers import BertTokenizer, create_optimizer, TFBertForSequenceClassification
from sklearn.model_selection import train_test_split
import tensorflow as tf

3. Split the data into training, validation, and test sets

train_texts, train_labels = train_df['Links'].tolist(), train_df['Shortlisted'].tolist()
val_texts, test_texts, val_labels, test_labels = train_test_split(
    test_df['Links'].tolist(), test_df['Shortlisted'].tolist(), test_size=0.5, random_state=42)

4. BERT Tokenizer (the max_length is 163 as the longest title that i have in ‘Links’ column is of length 163)

tokenizer = BertTokenizer.from_pretrained('bert-base-cased')

def tokenize_function(texts):
    return tokenizer(texts, padding='max_length', truncation=True, max_length=163, return_tensors="tf")

train_encodings = tokenize_function(train_texts)
val_encodings = tokenize_function(val_texts)
test_encodings = tokenize_function(test_texts)

5. Creating a Tensorflow dataset

train_dataset = tf.data.Dataset.from_tensor_slices((
    dict(train_encodings),
    train_labels
)).shuffle(len(train_texts)).batch(8)

val_dataset = tf.data.Dataset.from_tensor_slices((
    dict(val_encodings),
    val_labels
)).batch(16)

test_dataset = tf.data.Dataset.from_tensor_slices((
    dict(test_encodings),
    test_labels
)).batch(16)

6. Fetching pre-trained model

model = TFBertForSequenceClassification.from_pretrained('bert-base-cased', num_labels=2)

7. The error is here!!! (when training and validating the model, i have set epochs=1 as i have around 9700 records only to train the model)

model.compile(optimizer=tf.keras.optimizers.Adam(),
              loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
              metrics=tf.keras.metrics.CategoricalAccuracy())

history = model.fit(train_dataset, epochs=1, validation_data=val_dataset)

8. Testing model

results = model.evaluate(test_dataset)

print(results)

9. Saving model

model.save_pretrained('./fine-tuned-bert')

tokenizer.save_pretrained('./fine-tuned-bert')

10. Testing model on a new article

model = TFBertForSequenceClassification.from_pretrained('./fine-tuned-bert')
tokenizer = BertTokenizer.from_pretrained('./fine-tuned-bert')

def predict(text):
    inputs = tokenizer(text, return_tensors='tf', padding=True, truncation=True, max_length=128)
    outputs = model(**inputs)
    predictions = tf.argmax(outputs.logits, axis=-1)
    return predictions

new_text = "Green energy to drive power sector investment, coal to remain significant: Moody's"
predicted_label = predict(new_text)
print(predicted_label)

Also, this is my first time working on an NLP project. Any suggestions on improvement in this code will be highly appreciated!

P.S.: below snippet of code did not throw an error and gave an accuracy of above 95%. The test data also had 100% accuracy.

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=5e-5),
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])

history = model.fit(train_dataset, epochs=1, validation_data=val_dataset)