# ERROR: tensorflow model.compile() with imbalanced dataset

**URL:** <https://discuss.ai.google.dev/t/error-tensorflow-model-compile-with-imbalanced-dataset/31766>\
**Category:** General Discussion\
**Tags:** model, datasets\
**Created:** [February 12, 2024, 5:14am UTC](https://discuss.ai.google.dev/t/error-tensorflow-model-compile-with-imbalanced-dataset/31766 "2024-02-12T05:14:08Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Wael\_AbuRezeq](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/wael_aburezeq/32/8249_2.png) [@Wael\_AbuRezeq](https://discuss.ai.google.dev/u/Wael_AbuRezeq)\
**Post date:** [February 12, 2024, 5:14am UTC](https://discuss.ai.google.dev/t/error-tensorflow-model-compile-with-imbalanced-dataset/31766/1 "2024-02-12T05:14:08Z")

</div>

hi community 🙂

I am training a simple LSTM 3 layers network on a small dataset size of around 6K records, with 6 imbalanced classess.

My network is overfitting , I tried early stopping, different batch size, and learning rate scheduler . I am now exploring class\_weight option but I am getting error when compiling. Here is my code:

**Identifiying my network**

```
from keras.layers import Embedding, Bidirectional, LSTM, GlobalMaxPooling1D, Dense
def get_model_text_lstm_embedding(input_shape=False):
model = Sequential()
model.add(
    Embedding(
        input_dim=len(vocab_list),
        output_dim=embedding_matrix.shape[1],
        input_length=train_features_indexed_padding.shape[1],
    )
)
model.add(Bidirectional(LSTM(units=128, return_sequences=True)))
model.add(GlobalMaxPooling1D())
model.add(Dense(units=HP.n_classes, activation="softmax"))
return model

```

**Class Weights**

```
from sklearn.utils.class_weight import compute_class_weight
train_classes = np.argmax(y_train_text, axis=1)
class_weights = compute_class_weight(class_weight="balanced", classes=np.unique(train_classes), y=train_classes)

```

**Compling and Training**

```
text_model_lstm_embedding.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
loss="categorical_crossentropy", metrics=["accuracy"], class_weight=class_weights)
history_text_model_lstm_embedding = text_model_lstm_embedding.fit(
x_train_text, y_train_text,validation_data=(x_test_text, y_test_text), 
callbacks=callback_f(nameof(text_model_lstm_embedding)),
epochs=HP.epochs,verbose=3,shuffle=True,batch_size=32)

```

`

**The error message**

> TypeError: Invalid keyword argument(s) in `compile()`: ({‘class\_weight’},). Valid keyword arguments include “cloning”, “experimental\_run\_tf\_function”, “distribute”, “target\_tensors”, or “sample\_weight\_mode”.

---

<div class="post-metadata">

**Author:** ![Kiran\_Sai\_Ramineni](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/kiran_sai_ramineni/32/8859_2.png) [@Kiran\_Sai\_Ramineni](https://discuss.ai.google.dev/u/Kiran_Sai_Ramineni)\
**Post date:** [February 12, 2024, 6:52am UTC](https://discuss.ai.google.dev/t/error-tensorflow-model-compile-with-imbalanced-dataset/31766/3 "2024-02-12T06:52:02Z")

</div>

> [@Wael\_AbuRezeq](#):
>
> ```auto
> train_classes = np.argmax(y_train_text, axis=1)
> 
> ```

Hi @Wael_AbuRezeq, I have few suggestions in your code, while calculating the class weights you have to get the all unique classes for which you have to use np.unique if you use argmax you only get the highest class number

```auto
import numpy as np
from sklearn.utils.class_weight import compute_class_weight
y = [1, 1, 1, 1, 0, 0]
c_weights=compute_class_weight(class_weight="balanced", classes=np.unique(y), y=y)

```

If you compute class\_weights in this way you will get the class weights in an array, but while passing class\_weights in the compile method it should be in dictionary format. For example `class_weight = {0: weight_for_0, 1: weight_for_1, ...........}`.

Thank You.

---

<div class="post-metadata">

**Author:** ![Wael\_AbuRezeq](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/wael_aburezeq/32/8249_2.png) [@Wael\_AbuRezeq](https://discuss.ai.google.dev/u/Wael_AbuRezeq)\
**Post date:** [February 12, 2024, 9:59am UTC](https://discuss.ai.google.dev/t/error-tensorflow-model-compile-with-imbalanced-dataset/31766/5 "2024-02-12T09:59:02Z")

</div>

hi Kiran,  
thank you for your reply!

I tried it as a dictionary format, the error presists. I believe the error is more into the compatability of my tf compile with the keyword class\_weight

TypeError: Invalid keyword argument(s) in `compile()`: ({‘class\_weight’},). Valid keyword arguments include “cloning”, “experimental\_run\_tf\_function”, “distribute”, “target\_tensors”, or “sample\_weight\_mode”.

---

<div class="post-metadata">

**Author:** ![Kiran\_Sai\_Ramineni](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/kiran_sai_ramineni/32/8859_2.png) [@Kiran\_Sai\_Ramineni](https://discuss.ai.google.dev/u/Kiran_Sai_Ramineni)\
**Post date:** [February 16, 2024, 4:33am UTC](https://discuss.ai.google.dev/t/error-tensorflow-model-compile-with-imbalanced-dataset/31766/8 "2024-02-16T04:33:03Z")

</div>

Hi @Wael_AbuRezeq, Could you please stand alone code to reproduce the issue. Thank You.

---

<div class="post-metadata">

**Author:** ![Wael\_AbuRezeq](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/wael_aburezeq/32/8249_2.png) [@Wael\_AbuRezeq](https://discuss.ai.google.dev/u/Wael_AbuRezeq)\
**Post date:** [February 21, 2024, 12:10pm UTC](https://discuss.ai.google.dev/t/error-tensorflow-model-compile-with-imbalanced-dataset/31766/10 "2024-02-21T12:10:08Z")

</div>

hi @Kiran,

Here is the code

> from sklearn.utils.class\_weight import compute\_class\_weight  
> train\_classes = np.argmax(y\_train\_text, axis=1)  
> class\_weights = compute\_class\_weight(class\_weight = “balanced”, classes= np.unique(train\_classes), y= train\_classes)  
> class\_weights\_dict = dict(zip(np.unique(train\_classes), class\_weights))  
> class\_weights\_dict  
> text\_model\_lstm\_embedding = get\_model\_text\_lstm\_embedding()

# Compile the model with class weights

> text\_model\_lstm\_embedding.compile(optimizer=tf.keras.optimizers.Adam(learning\_rate=0.0001),  
> loss= ‘categorical\_crossentropy’ ,  
> class\_weight=class\_weights\_dict  
> )

> TypeError: Invalid keyword argument(s) in `compile()` : ({‘class\_weight’},). Valid keyword arguments include “cloning”, “experimental\_run\_tf\_function”, “distribute”, “target\_tensors”, or “sample\_weight\_mode”.

---

<div class="post-metadata">

**Author:** ![Kiran\_Sai\_Ramineni](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/kiran_sai_ramineni/32/8859_2.png) [@Kiran\_Sai\_Ramineni](https://discuss.ai.google.dev/u/Kiran_Sai_Ramineni)\
**Post date:** [February 22, 2024, 6:02am UTC](https://discuss.ai.google.dev/t/error-tensorflow-model-compile-with-imbalanced-dataset/31766/12 "2024-02-22T06:02:56Z")

</div>

Hi @Wael_AbuRezeq, The class weights argument should be used in model.fit, but i can see that you are using that argument in the model.compile. For example,

```auto
model.fit(
    train_features,
    train_labels,
    epochs=10,
   # The class weights go here
    class_weight=class_weight)

```

Thank You.

---

<div class="post-metadata">

**Author:** ![Wael\_AbuRezeq](https://d1dlmcr85iqnpo.cloudfront.net/user_avatar/discuss.ai.google.dev/wael_aburezeq/32/8249_2.png) [@Wael\_AbuRezeq](https://discuss.ai.google.dev/u/Wael_AbuRezeq)\
**Post date:** [March 14, 2024, 6:33am UTC](https://discuss.ai.google.dev/t/error-tensorflow-model-compile-with-imbalanced-dataset/31766/14 "2024-03-14T06:33:25Z")

</div>

> [@Wael\_AbuRezeq](#):
>
> class\_weights\_dict

Thanks a million 🙂 it finally worked!

Just one correction, the class\_weight gets the input as dictionary (as you originally stated but you used the array version in your response