I built a cumstom environment with Openai Gym spaces.Tuple because my observation is made up of: hour(0-23), day(1-7), month(1-12), which are discrete; four continuous numbers, which are from a csv file; and an array of shape (4*24), which are also from a csv file.
self.observation_space = spaces.Tuple(spaces=(
spaces.Box(low=-high, high=high, shape=(4,), dtype=np.float16),
spaces.Box(low=-high, high=high, shape=(4, 24), dtype=np.float16),
spaces.Discrete(24),
spaces.Discrete(7),
spaces.Discrete(12)))
This is my reset() function to read data from the csv file:
def reset(self):
index = 0
hour = 0
day = 1
month = 6
array1 = np.array([
self.df.loc[index, 'A'],
self.df.loc[index, 'B'],
self.df.loc[index, 'C'],
self.df.loc[index, 'D'],
], dtype=np.float16)
array2 = np.array([
self.df.loc[index: index+23, 'A'],
self.df.loc[index: index+23, 'B'],
self.df.loc[index: index+23, 'C'],
self.df.loc[index: index+23, 'D'],
], dtype=np.float16)
tup = (array1, array2, hour, day, month)
return tup
To train the agent, I want to use DQN algorithm, which is the DQNAgent from keras-rl library
Here is my code to build the neural network model:
model = Sequential()
model.add(Flatten(input_shape=(1,) + env.observation_space.shape))
model.add(Dense(16))
model.add(Activation('relu'))
model.add(Dense(nb_actions))
model.add(Activation('linear'))
According to my understanding, spaces.Tuple instances don't have shape() method, and the len method returns the number of spaces in the tuple. e.g. len = 5 in my environment
state = env.reset()
len = state.__len__()
But to build the neural network, it seems that I need 4 + 4*24 + 3 = 103 input neuron.
Is there any method to get the input number for the DQN model?
I came up with a method that use Spaces.Box instead of Spaces.Tuple:
self.observation_space = spaces.Box(low=-high, high=high, shape=(103,), dtype=np.float16)
But this seems not be the most ideal way.
Thanks in advance!