python - how to get the dimention of Openai gym spaces.Tuple to be used in DQN when building neural network with Keras

Question

I built a cumstom environment with Openai Gym spaces.Tuple because my observation is made up of: hour(0-23), day(1-7), month(1-12), which are discrete; four continuous numbers, which are from a csv file; and an array of shape (4*24), which are also from a csv file.

self.observation_space = spaces.Tuple(spaces=(
                                             spaces.Box(low=-high, high=high, shape=(4,), dtype=np.float16),
                                             spaces.Box(low=-high, high=high, shape=(4, 24), dtype=np.float16),
                                             spaces.Discrete(24),
                                             spaces.Discrete(7),
                                             spaces.Discrete(12)))

This is my reset() function to read data from the csv file:

    def reset(self):
        index = 0
        hour = 0
        day = 1
        month = 6
        array1 = np.array([
            self.df.loc[index, 'A'],
            self.df.loc[index, 'B'],
            self.df.loc[index, 'C'],
            self.df.loc[index, 'D'],
        ], dtype=np.float16)
        array2 = np.array([
            self.df.loc[index: index+23, 'A'],
            self.df.loc[index: index+23, 'B'],
            self.df.loc[index: index+23, 'C'],
            self.df.loc[index: index+23, 'D'],
        ], dtype=np.float16)
        tup = (array1, array2, hour, day, month)
        return tup 

To train the agent, I want to use DQN algorithm, which is the DQNAgent from keras-rl library Here is my code to build the neural network model:

model = Sequential()
model.add(Flatten(input_shape=(1,) + env.observation_space.shape))
model.add(Dense(16))
model.add(Activation('relu'))
model.add(Dense(nb_actions))
model.add(Activation('linear'))

According to my understanding, spaces.Tuple instances don't have shape() method, and the len method returns the number of spaces in the tuple. e.g. len = 5 in my environment

state = env.reset()
len = state.__len__()

But to build the neural network, it seems that I need 4 + 4*24 + 3 = 103 input neuron. Is there any method to get the input number for the DQN model?

I came up with a method that use Spaces.Box instead of Spaces.Tuple:

self.observation_space = spaces.Box(low=-high, high=high, shape=(103,), dtype=np.float16) 

But this seems not be the most ideal way.

Thanks in advance!

Answer 1

等待大神解答