Recognizing The Enemy: Combining Reinforcement Learning with Strategy Selection using Case-Based Reasoning
This paper presents CBRetaliate, an agent that combines Case-Based Reasoning (CBR) and Reinforcement Learning (RL) algorithms. Unlike most previous work where RL is used to improve accuracy in the action selection process, CBRetaliate uses CBR to allow RL to respond more quickly to changing conditions. CBRetaliate combines two key features: it uses a time window to compute similarity and stores and reuses complete Q-tables for continuous problem solving.We demonstrate CBRetaliate on a team-based first-person shooter game, where our combined CBR+RL approach adapts quicker to changing tactics by an opponent than standalone RL.
Game AI for a Turn-based Strategy Game with Plan Adaptation and Ontology-based Retrieval
In this paper we present a novel approach for developing adaptive game AI by combining case based planning techniques and ontological knowledge from the game environment. The proposed architecture combines several components: a case-based hierarchical planner (Repair-SHOP), a bridge to connect and reason with Ontologies formalized in Description Logics (DLs) based languages (OntoBridge), a DLs reasoner (Pellet) and a framework to develop Case-Based Reasoning (CBR) systems (jCOLIBRI ). In our ongoing work we are applying this approach to a commercial Civilization clone turn-based strategy game (CTP2) where game AI is in charge of planning the strategies for automated players. Our goal is to demonstrate that ontology-based retrieval will result in the retrieval of strategies that are easier to adapt than those plans returned by other classical retrieval mechanisms traditionally used in case-based planning.
Transfer Learning of Hierarchical Task-Network Planning Methods in a Real-Time Strategy Game
We describe a new integrated and automated AI planning and learning architecture, called Learn2SHOP. Learn2SHOP departs significantly from the previous works on AI planning and learning in that its modular architecture integrates Hierarchical Task Network (HTN) planning, concept learning, and computer simulations. Using simulations during the planning and learning process enables the system to get information about the outcomes of the actions. We have implemented Learn2SHOP and tested it on a transfer-learning task. The objective of transfer learning is transferring knowledge and skills learned from a wide variety of previous situations to the current, and likely different, previously unencountered problems(s). The experiments with Learn2SHOP have demonstrated the advantages of integrating planning, learning, and simulation in a real-time strategy game engine.
RETALIATE: Learning Winning Policies in First-Person Shooter Games
In this paper we present RETALIATE, an online reinforcement learning algorithm for developing winning policies in team first-person shooter games. RETALIATE has three crucial characteristics: (1) individual BOT behavior is fixed although not known in advance, therefore individual BOTS work asplug-ins, (2) RETALIATE models the problem of learning team tactics through a simple state formulation, (3) discount rates commonly used in Q-learning are not used. As a result of these characteristics, the application of the Q-learning algorithm results in the rapid exploration towards a winning policy against an opponent team. In our empirical evaluation we demonstrate that RETALIATE adapts well when the environment changes.
Adaptation of Hierarchical Task Network Plans
This paper presents RepairSHOP, a system capable of performing plan adaptation and plan repair. RepairSHOP is built on top of the HTN planner SHOP. RepairSHOP has three properties. The first property is its design modularity, which makes it is straightforward to apply the same process discussed in this paper to build plan adaptation capabilities in other HTN planners. Second, RepairSHOP can perform plan repair. Third, RepairSHOP takes into account failed traces during plan adaptation/repair. As a result, it can result in improvements in running time performance. We performed experiments demonstrating performance gains of plan adaptation over plan generation from the scratch, measured in CPU time for problem solving.