I have an autoencoder that is supposed to create variation of the input text but this is the output: what went wrong. during evaluation it had an accuracy of 76%. Below is the code

TensorFlow General Discussion
,
1

loaded_model = load_model(“vec_drug”)
loaded_layer = loaded_model.layers[0]
vocab = loaded_layer.get_vocabulary()

def get_word(v):

v = tf.convert_to_tensor(v)
transformed = ""
for each in tf.squeeze(v):
    transformed= transformed+ " "+ vocab[each]

return transformed

inputs = Input(shape=(max_len,))
encoder1 = Embedding(vocab_size, 128)(inputs)
encoder2 = LSTM(128)(encoder1)
encoder3 = RepeatVector(max_len)(encoder2)

decoder1 = LSTM(128, return_sequences=True)(encoder3)
outputs = TimeDistributed(Dense(vocab_size, activation=‘softmax’))(decoder1)

model = keras.Model(inputs=inputs, outputs=outputs)
model.load_weights(“weights-improvement-01-0.76.hdf5”)
model.compile(optimizer=“adam”, loss=“sparse_categorical_crossentropy”, metrics=[“accuracy”])

model.save(“drugs_model”, save_format=“tf”)
saved_model = load_model(“drugs_model”)

inputs = Input(shape=(1,), dtype=‘string’)
x = vectorize_text(inputs)
x = tf.reshape(x,[1,120])
output = saved_model(x)

end_model = keras.Model(inputs, output)
end_model.compile(loss=“sparse_categorical_crossentropy”, optimizer=“adam”, metrics=[‘accuracy’])
end_model.save(“full_model”, save_format=“tf”)

full_model = load_model(“full_model”)

def generate_text(seed_text, next_words, model,max_sequence_len,temp):

words = []
prediction = model.predict([seed_text])
for i in np.round(prediction[0],3):
    words.append(np.argmax(i))
    
words = np.array(words)
output_text = get_word(words)

return output_text

seed = “Do not use in larger or smaller amounts or for longer than recommended.”

created_text = generate_text(seed,5,end_model,20,1)

print(created_text)

output : or or or or or or or or or or or or or or or or or or or or or or or or

3

Hi @Progress_Munoriarwa

Thank you for using TensorFlow,
In the above problem, we are getting repetitive answer if you generate words iteratively by feeding back the previous output into the model, then by sampling from output probability distribution, makes the text prediction more varied and generalized.
For example,

predicted_idx = np.random.choice(range(len(prediction[0])), p=prediction[0])
predicted_word = vocab[predicted_idx]
input_text += " " + predicted_word #for next iteration