Recognizing The Enemy: Combining Reinforcement Learning with Strategy Selection using Case-Based Reasoning
This paper presents CBRetaliate, an agent that combines Case-Based Reasoning (CBR) and Reinforcement Learning (RL) algorithms. Unlike most previous work where RL is used to improve accuracy in the action selection process, CBRetaliate uses CBR to allow RL to respond more quickly to changing conditions. CBRetaliate combines two key features: it uses a time window to compute similarity and stores and reuses complete Q-tables for continuous problem solving.We demonstrate CBRetaliate on a team-based first-person shooter game, where our combined CBR+RL approach adapts quicker to changing tactics by an opponent than standalone RL.
HTN-MAKER: Learning HTNs with Minimal Additional Knowledge Engineering Required
We describe HTN-MAKER, an algorithm for learning hierarchical planning knowledge in the form of decomposition methods for Hierarchical Task Networks (HTNs). HTN-MAKER takes as input the initial states from a set of classical planning problems in a planning domain and solutions to those problems, as well as a set of semantically-annotated tasks to be accomplished. The algorithm analyzes this semantic information in order to determine which portions of the input plans accomplish a particular task and constructs HTN methods based on those analyses.Our theoretical results show that HTN-MAKER is sound and complete. We also present a formalism for a class of planning problems that are more expressive than classical planning. These planning problems can be represented as HTN planning problems. We show that the methods learned by HTN-MAKER enable an HTN planner to solve those problems. Our experiments confirm the theoretical results and demonstrate convergence in three well-known planning domains toward a set of HTN methods that can be used to solve nearly any problem expressible as a classical planning problem in that domain, relative to a set of goals.
Adaptation of Hierarchical Task Network Plans
This paper presents RepairSHOP, a system capable of performing plan adaptation and plan repair. RepairSHOP is built on top of the HTN planner SHOP. RepairSHOP has three properties. The first property is its design modularity, which makes it is straightforward to apply the same process discussed in this paper to build plan adaptation capabilities in other HTN planners. Second, RepairSHOP can perform plan repair. Third, RepairSHOP takes into account failed traces during plan adaptation/repair. As a result, it can result in improvements in running time performance. We performed experiments demonstrating performance gains of plan adaptation over plan generation from the scratch, measured in CPU time for problem solving.