ValueError：无法将大小为 1 的数组重塑为形状 (1,4)

Question

注释掉有问题的代码也会给我这个错误：AssertionError: Cannot call env.step() before calling reset()

尝试按照 openai 健身房的教程进行操作。重塑环境状态时出现 numpy 错误。我观看的教程中不存在这两个错误。我在 Ubuntu 上使用了我自己的 jupyter notebook，也在 Kaggle 的 notebook 上使用了它。我在两种环境中都遇到了错误。所以要么我的代码有问题，要么有一个不推荐使用的方法没有被标记。

'''

env = gym.make("CartPole-v1")

state_size = env.observation_space.shape[0]
action_size = env.action_space.n

# hyperparameter for gradient descent (vary by powers of 2)
batch_size = 32
n_episodes = 1001
output_dir = "model_output/cartpole"

if not os.path.exists(output_dir):
    os.makedirs(output_dir)

agent = Agent(state_size, action_size)

done = False
for e in range(n_episodes):
    
    # start each episode at beginning state
    state = env.reset
    
    # transpose state to fit nicely with DL network
    state = np.reshape(state, [1, state_size])
    
    # iterate over time steps of game
    for time in range(5000):
        
        env.render()
        action = agent.act(state)
        # returned values from taking a step forward
        next_state, reward, done, _ = env.step(action)
        # if we hit the time step 5000, reward is normal, else if we hit end, our reward is -10 for dying
        reward = reward if not done else -10
        
        next_state = np.reshape(next_state, [1, state_size])
        
        #
        agent.remember(state, action, reward, next_state, done)
        
        # moved into next state
        state = next_state
        
        if done:
            print("episode: {}/{}, score: {}, e: {:.2}").format(e, n_episodes, time, agent.epsilon)
            break
        
    if len(agent.memory) > batch_size:
        # train our theta
        agent.replay(batch_size)
        
    if e % 50 == 0:
        # save our model params at different chapters to hold onto if agent experiences regression
        agent.save(output_dir + "weights_" + "{:04d}".format(e) + ".hdf5")

'''

以下是错误：

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-22-1765494e4d10> in <module>
  6 
  7     # transpose state to fit nicely with DL network
----> 8     state = np.reshape(state, [1, 4])
  9 
 10     # iterate over time steps of game

<__array_function__ internals> in reshape(*args, **kwargs)

/opt/conda/lib/python3.7/site-packages/numpy/core/fromnumeric.py in reshape(a, newshaporder)
299            [5, 6]])
300     """
--> 301     return _wrapfunc(a, 'reshape', newshape, order=order)
302 
303 

/opt/conda/lib/python3.7/site-packages/numpy/core/fromnumeric.py in _wrapfunc(obj, method, *args, **kwds)
 56     bound = getattr(obj, method, None)
 57     if bound is None:
 ---> 58         return _wrapit(obj, method, *args, **kwds)
 59 
 60     try:

/opt/conda/lib/python3.7/site-packages/numpy/core/fromnumeric.py in _wrapit(obj, method, *args, **kwds)
 45     except AttributeError:
 46         wrap = None
 ---> 47     result = getattr(asarray(obj), method)(*args, **kwds)
 48     if wrap:
 49         if not isinstance(result, mu.ndarray):

 ValueError: cannot reshape array of size 1 into shape (1,4)

Answer 1

我刚刚想通了，我在“重置”周围缺少括号