Recognizing The Enemy: Combining Reinforcement Learning with Strategy Selection using Case-Based Reasoning
This paper presents CBRetaliate, an agent that combines Case-Based Reasoning (CBR) and Reinforcement Learning (RL) algorithms. Unlike most previous work where RL is used to improve accuracy in the action selection process, CBRetaliate uses CBR to allow RL to respond more quickly to changing conditions. CBRetaliate combines two key features: it uses a time window to compute similarity and stores and reuses complete Q-tables for continuous problem solving.We demonstrate CBRetaliate on a team-based first-person shooter game, where our combined CBR+RL approach adapts quicker to changing tactics by an opponent than standalone RL.
HTN-MAKER: Learning HTNs with Minimal Additional Knowledge Engineering Required
We describe HTN-MAKER, an algorithm for learning hierarchical planning knowledge in the form of decomposition methods for Hierarchical Task Networks (HTNs). HTN-MAKER takes as input the initial states from a set of classical planning problems in a planning domain and solutions to those problems, as well as a set of semantically-annotated tasks to be accomplished. The algorithm analyzes this semantic information in order to determine which portions of the input plans accomplish a particular task and constructs HTN methods based on those analyses.Our theoretical results show that HTN-MAKER is sound and complete. We also present a formalism for a class of planning problems that are more expressive than classical planning. These planning problems can be represented as HTN planning problems. We show that the methods learned by HTN-MAKER enable an HTN planner to solve those problems. Our experiments confirm the theoretical results and demonstrate convergence in three well-known planning domains toward a set of HTN methods that can be used to solve nearly any problem expressible as a classical planning problem in that domain, relative to a set of goals.
Game AI for a Turn-based Strategy Game with Plan Adaptation and Ontology-based Retrieval
In this paper we present a novel approach for developing adaptive game AI by combining case based planning techniques and ontological knowledge from the game environment. The proposed architecture combines several components: a case-based hierarchical planner (Repair-SHOP), a bridge to connect and reason with Ontologies formalized in Description Logics (DLs) based languages (OntoBridge), a DLs reasoner (Pellet) and a framework to develop Case-Based Reasoning (CBR) systems (jCOLIBRI ). In our ongoing work we are applying this approach to a commercial Civilization clone turn-based strategy game (CTP2) where game AI is in charge of planning the strategies for automated players. Our goal is to demonstrate that ontology-based retrieval will result in the retrieval of strategies that are easier to adapt than those plans returned by other classical retrieval mechanisms traditionally used in case-based planning.
Transfer Learning of Hierarchical Task-Network Planning Methods in a Real-Time Strategy Game
We describe a new integrated and automated AI planning and learning architecture, called Learn2SHOP. Learn2SHOP departs significantly from the previous works on AI planning and learning in that its modular architecture integrates Hierarchical Task Network (HTN) planning, concept learning, and computer simulations. Using simulations during the planning and learning process enables the system to get information about the outcomes of the actions. We have implemented Learn2SHOP and tested it on a transfer-learning task. The objective of transfer learning is transferring knowledge and skills learned from a wide variety of previous situations to the current, and likely different, previously unencountered problems(s). The experiments with Learn2SHOP have demonstrated the advantages of integrating planning, learning, and simulation in a real-time strategy game engine.
RETALIATE: Learning Winning Policies in First-Person Shooter Games
In this paper we present RETALIATE, an online reinforcement learning algorithm for developing winning policies in team first-person shooter games. RETALIATE has three crucial characteristics: (1) individual BOT behavior is fixed although not known in advance, therefore individual BOTS work asplug-ins, (2) RETALIATE models the problem of learning team tactics through a simple state formulation, (3) discount rates commonly used in Q-learning are not used. As a result of these characteristics, the application of the Q-learning algorithm results in the rapid exploration towards a winning policy against an opponent team. In our empirical evaluation we demonstrate that RETALIATE adapts well when the environment changes.