Computer Science & Engineering
Recognizing The Enemy: Combining Reinforcement Learning with Strategy Selection using Case-Based Reasoning
This paper presents CBRetaliate, an agent that combines Case-Based Reasoning (CBR) and Reinforcement Learning (RL) algorithms. Unlike most previous work where RL is used to improve accuracy in the action selection process, CBRetaliate uses CBR to allow RL to respond more quickly to changing conditions. CBRetaliate combines two key features: it uses a time window to compute similarity and stores and reuses complete Q-tables for continuous problem solving.We demonstrate CBRetaliate on a team-based first-person shooter game, where our combined CBR+RL approach adapts quicker to changing tactics by an opponent than standalone RL.
HTN-MAKER: Learning HTNs with Minimal Additional Knowledge Engineering Required
We describe HTN-MAKER, an algorithm for learning hierarchical planning knowledge in the form of decomposition methods for Hierarchical Task Networks (HTNs). HTN-MAKER takes as input the initial states from a set of classical planning problems in a planning domain and solutions to those problems, as well as a set of semantically-annotated tasks to be accomplished. The algorithm analyzes this semantic information in order to determine which portions of the input plans accomplish a particular task and constructs HTN methods based on those analyses.Our theoretical results show that HTN-MAKER is sound and complete. We also present a formalism for a class of planning problems that are more expressive than classical planning. These planning problems can be represented as HTN planning problems. We show that the methods learned by HTN-MAKER enable an HTN planner to solve those problems. Our experiments confirm the theoretical results and demonstrate convergence in three well-known planning domains toward a set of HTN methods that can be used to solve nearly any problem expressible as a classical planning problem in that domain, relative to a set of goals.
Boundary Feedback Control for Mixing Enhancement of 2D Magnetohydrodynamic Channel Flow by Extremum Seeking
The interaction between electrically conducting fluids and magnetic fields in channel flows generates significant magnetohydrodynamics (MHD) effects, which often result in the need of higher pressure gradients to drive the fluid and lower heat transfer rates due to the laminarization of the flow. Active boundary control, either open-loop or closed-loop, can be employed to overcome this limitation. However, open-loop controllers generally have worse performance due to the uncertainties of the system. The extremum seeking scheme is a powerful tool to build feedback controllers based on existing open-loop controllers. In this work we demonstrate that by carefully tuning the extremum seeking the modified open-loop control scheme can be as good as the closed-loop control scheme presented in our earlier publications.
Mars Rovers in Middle School
We have developed an innovative curriculum using one sixth replicas of the rovers currently on Mars. Each student receives hands-on experience by performing missions in a simulation program. These missions allow students to relate to the process of controlling the actual rovers. Furthermore, students have an opportunity to remotely program and control the replicas in a realistic Martian landscape created in the basement of a middle school. Programming robots in this landscape is the centerpiece of a technology curriculum in all four middle schools in the Allentown School District as well as a summer and Saturday program at Lehigh University.
Segmentation-Based Retrieval of Document Images from Diverse Collections
We describe a methodology for retrieving document images from large extremely diverse collections. First we perform content extraction, that is the location and measurement of regions containing handwriting, machine-printed text, photographs, blank space, etc, in documents represented as bilevel, greylevel, or color images. Recent experiments have shown that even modest per-pixel content classication accuracies can support usefully high recall and precision rates (of, e.g., 8090%) for retrieval queries within document collections seeking pages that contain a fraction of a certain type of content. When the distribution of content and error rates are uniform across the entire collection, it is possible to derive IR measures from classication measures and vice versa. Our largest experiments to date, consisting of 80 training images totaling over 416 million pixels, are presented to illustrate these conclusions. This data set is more representative than previous experiments, containing a more balanced distribution of content types. Contained in this data set are also images of text obtained from handheld digital cameras and the success of existing methods (with no modication) in classifying these images with are discussed. Initial experiments in discriminating line art from the four classes mentioned above are also described. We also discuss methodological issues that aect both ground-truthing and evaluation measures.