Archive for July, 2007
RETALIATE: Learning Winning Policies in First-Person Shooter Games
In this paper we present RETALIATE, an online reinforcement learning algorithm for developing winning policies in team first-person shooter games. RETALIATE has three crucial characteristics: (1) individual BOT behavior is fixed although not known in advance, therefore individual BOTS work asplug-ins, (2) RETALIATE models the problem of learning team tactics through a simple state formulation, (3) discount rates commonly used in Q-learning are not used. As a result of these characteristics, the application of the Q-learning algorithm results in the rapid exploration towards a winning policy against an opponent team. In our empirical evaluation we demonstrate that RETALIATE adapts well when the environment changes.
An Examination of Biparental Care by Convict Cichlids: Sex Differences in the Retrieval of Offspring
Parents do not always share offspring care equally and each parent often has a specialized role. The most commonly observed sex-typical roles are defense by males and direct care of the offspring by females. In our study of convict cichlids (*Archocentrus nigrofasciatus*) we examined the behaviors of each parent when non-swimming young were displaced from the nest. Parental care was measured using retrieval of offspring and agonistic behaviors toward potential offspring predators. We found that males were more likely to retrieve young displaced far from the nest, while females retrieved young at closer distances. Additionally, although we found that both parents do attack the potential predator (a conspecific individual), retrieval behaviors did not change for either parent. We also examined retrieval behavior when one parent was removed and found that the remaining male or female parent was able to perform all behaviors associated with parental care. These preliminary studies suggest that while there is plasticity in parental behavior of convict cichlids, when together, parents often assume sex-typical roles.
Does past experience with a male predict female mate choice in the convict cichlid?
In many monogamous species, both sexes compete for mates and demonstrate mate choice. However, the degree to which intrasexual competition limits the choice of the other sex may not be well known. In this study, we monitored the effects of past experience of male-male competition on the mate preference of female convict cichlids, Archocentrus nigrofasciatus. Females were allowed to interact with two size-matched males while they competed until a winner and loser could be determined. Females were then given a choice between a male she had had past experience with and a novel male. Females were placed in two treatment groups and given a choice between a novel male and 1) a male that had successfully competed against another male (winner) or 2) a male that had unsuccessfully competed against another male (loser). Males were isolated from each other in the experimental aquarium so that intrasexual selection after the initial interaction could not affect the females preference. Mate preference was determined by observing a time based preference of the female for a specific male. Over a ten day period, females showed a significant preference for a winner male as compared to a novel male. Females did not show a decreased preference for a loser male over a novel male. This study suggests that positive past experience may impact female mate choice, whereas negative past experience may have little or no affect.
Neighbor interactions among male Cyprinodon elegans: Does size influence the development of “dear enemy” recognition?
Territorial males of many pupfish species exhibit dear enemy recognition, displaying less aggression to neighbors than to unfamiliar intruders. Familiarity may play a role in the development of dear enemy recognition if established neighbors save time and energy through decreased aggression. This hypothesis predicts that a territory owner decreases the intensity of aggressive responses to neighbors over time. I tested this prediction in male Cyprinodon elegans. Males have conditional alternative reproductive tactics with larger males defending territories around spawning sites, intermediate males as non-territorial satellites and smaller males as sneakers with female-like morphologies. The largest males in a population are territorial, but in absolute size; they may be intermediate or small. Therefore, aggressive behaviors and neighbor interactions were assessed across the time period of territory establishment for large, intermediate and small males. Selection for dear enemy recognition may be stronger on large males compared to intermediate and small males that do not typically hold territories in natural populations.
Multivariable Anti-Windup Controller Synthesis incorporating Multiple Convex Constraints
We study the problem of multivariable anti-windup controller synthesis that incorporates trade-offs between unconstrained linear performance and constrained anti-windup performance. In our previous work [MuKo02], we presented a framework for simultaneous design of a linear output feedback controller and a static anti-windup compensator using Linear Matrix Inequalities (LMIs). This framework deviated from the conventional two step paradigm of anti-windup by proposing a one-step synthesis while still maintaining the anti-windup structure. However, the work in [MuKo02] did not provide conclusive demonstration of the efficacy of this formulation through realistic practical examples. In this paper, we further study the framework in our previous work and explicitly demonstrate its capability in providing effective constrained anti-windup controller performance for two benchmark problems, viz. a vibration isolation table and a multivariable model for the longitudinal dynamics of an F8 aircraft.