Google Deepmind’s Dream-RSI helps AI agents improve by “dreaming” about past attempts

Researchers at Google and Deepmind have developed a method that helps AI agents tackle difficult search tasks more efficiently. It uses past search runs to test new strategies without repeating costly computations.

Self-improving AI agents are supposed to one day discover new algorithms, solutions to math problems, or faster code on their own. They follow the same basic process of proposing a solution, evaluating the result, learning from it, and trying again. Over thousands of attempts, they gradually work toward a good result. For complex tasks, the search space can grow enormous. The agent must constantly decide which promising approaches to pursue, which to try in parallel, and which to abandon. This process, called exploration, can determine whether the search succeeds or wastes compute chasing the wrong ideas. A research team from Google and Deepmind has introduced “Dream-RSI” to improve those decisions. The method changes how the agent searches, not the underlying AI model. Existing approaches generally handle exploration in two ways. A fixed search strategy can’t learn from experience, so the agent may repeatedly hit the same dead ends. Adapting the strategy during a search avoids that rigidity but comes at a cost. It takes many attempts to find out whether a strategy works, and testing countless alternatives would mean repeating long, expensive runs. Replaying past searches makes new strategies cheaper to test The researchers propose reusing data from a completed search to test alternative strategies within the space the agent has already explored. The agent records its attempts and their results as it searches, providing the data needed to replay those decisions later. The researchers compare this to finding your way through an unfamiliar area.