Hindsight Experience Replay (HER) is a technique used in Reinforcement Learning to improve exploration and learning efficiency. It works by storing experiences and then using them to train the agent as if the desired outcome had occurred. For example, if an agent is trying to pick up an apple, but it knocks over a vase instead, HER would use the experience to train the agent as if it had picked up the apple. Your task is to implement the HER algorithm.
**Example:** Suppose we have a 2D grid world where the agent can move up, down, left, or right. The goal is to reach the target. The agent's experience is stored in a replay buffer, and the HER algorithm is used to train the agent.
**Constraints:** The grid size is 5x5, the target is at position (4,4), and the agent starts at position (0,0).
**Note:** This problem requires a good understanding of Reinforcement Learning and the HER algorithm.
Test Cases
[{'state': [0, 0], 'action': 0, 'reward': 0.0, 'done': False}, {'state': [0, 1], 'action': 1, 'reward': 0.0, 'done': False}][{'state': [0, 0], 'action': 0, 'reward': 1.0, 'done': False}, {'state': [0, 1], 'action': 1, 'reward': 1.0, 'done': False}][{'state': [4, 4], 'action': 3, 'reward': 0.0, 'done': False}, {'state': [4, 3], 'action': 2, 'reward': 0.0, 'done': False}][{'state': [4, 4], 'action': 3, 'reward': 1.0, 'done': False}, {'state': [4, 3], 'action': 2, 'reward': 1.0, 'done': False}]