Error when creating DQNAgent with Keras-RL: "Keras symbolic inputs/outputs do not implement __len__"

TensorFlow General Discussion
1

I’m trying to create a DQNAgent using Keras-RL to train a CartPole-v0 environment. However, I’m encountering an issue related to the Keras symbolic inputs/outputs not implementing len . Here’s the relevant part of the code:

import gym
import numpy as np
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Flatten
from tensorflow.keras.optimizers import Adam
from rl.agents import DQNAgent
from rl.policy import BoltzmannQPolicy
from rl.memory import SequentialMemory

env = gym.make('CartPole-v0')
states = env.observation_space.shape[0]
actions = env.action_space.n

def build_model(states, actions):
    model = Sequential()
    model.add(Flatten(input_shape=(1,states)))
    model.add(Dense(24, activation='relu'))
    model.add(Dense(24, activation='relu'))
    model.add(Dense(actions, activation='linear'))
    return model

model = build_model(states, actions)

def build_agent(model, actions):
    policy = BoltzmannQPolicy()
    memory = SequentialMemory(limit=50000, window_length=1)
    dqn = DQNAgent(model=model, memory=memory, policy=policy, 
                  nb_actions=actions, nb_steps_warmup=10, target_model_update=1e-2)
    return dqn

dqn = build_agent(model, actions)

The error I’m encountering is:

TypeError: Keras symbolic inputs/outputs do not implement `__len__`. You may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model. This error will also get raised if you try asserting a symbolic input/output directly.

I’ve encountered the mentioned error while trying to create a DQNAgent using Keras-RL for training a CartPole-v0 environment. In an attempt to resolve the issue, I followed some online recommendations suggesting updates to TensorFlow, Keras, or Gym to specific versions. Despite these updates, the problem persists. I expected that the suggested version updates would resolve the error, but unfortunately, it did not.