Action selection

From Wikipedia, the free encyclopedia

Artificial intelligence Portal

Action selection is a way of characterizing the most basic problem of intelligent systems: what to do next. In artificial intelligence and computational cognitive science, "the action selection problem" is typically associated with intelligent agents and animats—artificial systems that exhibit complex behaviour in an agent environment. The term is also sometimes used in ethology or animal behavior.

One problem for understanding action selection is determining the level of abstraction used for specifying an "act". At the most basic level of abstraction, an atomic act could be anything from contracting a muscle cell to provoking a war. Typically for any one action-selection mechanism, the set of possible actions is predefined and fixed.

Most researchers working in this field place high demands on their agents:

  • The acting agent typically must select its action in dynamic and unpredictable environments.
  • The agents typically act in real time; therefore they must make decisions in a timely fashion.
  • The agents are normally created to perform several different tasks. These tasks may conflict for resource allocation (e.g. can the agent put out a fire and deliver a cup of coffee at the same time?)
  • The environment the agents operate in may include humans, who may make things more difficult for the agent (either intentionally or by attempting to assist.)
  • The agents themselves are often intended to model animals and/or humans, and animal/human behaviour is quite complicated.

For these reasons action selection is not trivial and attracts a good deal of research.

Contents

[edit] Characteristics of the action selection problem

The main problem for action selection is complexity. Since all computation takes both time and space (in memory), agents cannot possibly consider every option available to them at every instant in time. Consequently, they must be biased, and constrain their search in some way. For AI, the question of action selection is what is the best way to constrain this search? For biology and ethology, the question is how do various types of animals constrain their search? Do all animals use the same approaches? Why do they use the ones they do?

One fundamental question about action selection is whether it is really a problem at all for an agent, or whether it is just a description of an emergent property of an intelligent agent's behaviour. However, if we consider how we are going to build an intelligent agent, then it becomes apparent there must be some mechanism for action selection. This mechanism may be highly distributed (as in the case of distributed organisms such as social insect colonies or slime mold) or it may be a special-purpose module.

The action selection mechanism (ASM) determines not only the agent’s actions in terms of impact on the world, but also directs its perceptual attention, and updates its memory. These egocentric sorts of actions may in turn result in modifying the agents basic behavioural capacities, particularly in that updating memory implies some form of learning is possible. Ideally, action selection itself should also be able to learn and adapt, but there are many problems of combinatorial complexity and computational tractability that may require restricting the search space for learning.

In AI, an ASM is also sometimes either referred to as an agent architecture or thought of as a substantial part of one.

[edit] AI Mechanisms of action selection

Generally, artificial action selection mechanisms can be divided into several categories: symbol-based systems sometimes known as classical planning, distributed solutions, and reactive or dynamic planning. Some approaches do not fall neatly into any one of these categories. Others are really more about providing scientific models than practical AI control, these last are described further in the next section.

[edit] Symbolic approaches

Main article: Automated planning and scheduling

Early in the history of artificial intelligence, it was assumed that the best way for an agent to choose what to do next would be to compute a provably optimal plan, and then execute that plan. This led to the physical symbol system hypothesis, that a physical agent that can manipulate symbols is necessary and sufficient for intelligence. Many software agents still use this approach for action selection. It normally requires describing all sensor readings, the world, all of ones actions and all of one's goals in some form of predicate logic. Critics of this approach complain that it is too slow real-time planning and that, despite the proofs, it is still unlikely to produce optimal plans because reducing descriptions of reality to logic is a process prone to errors.

Satisficing is a decision-making strategy which attempts to meet criteria for adequacy, rather than identify an optimal solution. A satisficing strategy may often, in fact, be (near) optimal if the costs of the decision-making process itself, such as the cost of obtaining complete information, are considered in the outcome calculus.

  • Goal driven architectures In these symbolic architectures, agent's behaviour is typically described by a set of goals. Each goal can be achieved by a process or an activity, which is described by a prescripted plan. The agent must just decide which process to carry on to accomplish a given goal. The plan can expand to subgoals, which makes the process slightly recursive. Technically, more or less, the plans exploits condition-rules. These architectures are reactive or hybrid. Classical examples of goal driven architectures are implementable refinements of Belief-Desire-Intention architecture like JAM or IVE.
  • Excalibur is a research project led by Alexander Nareyek featuring any-time planning agents for computer games. The architecture is based on structural constraint satisfaction, which is an advanced artificial intelligence technique. Recently, the project is discontinued.

[edit] Distributed approaches

In contrast to the symbolic approach, distributed systems of action selection actually have no one "box" in the agent which decides the next action. At least in their idealized form, distributed systems have many modules running in parallel and determining the best action based on local expertise. In these idealized systems, overall coherence is expected to emerge somehow, possibly through careful design of the interacting components. This approach is often inspired by neural networks research. In practice, there is almost always some centralised system determining which module is "the most active" or has the most salience. There is evidence real brains also have such executive decision systems which evaluate which of competing systems deserves the most attention, or more properly, has its desired actions disinhibited.

[edit] Dynamic planning approaches

Because purely distributed systems are difficult to construct, many researchers have turned to using explicit hard-coded plans to determine the priorities of their system.

Dynamic or reactive planning methods compute just one next action in every instant based on the current context and pre-scripted plans. In contrast to classical planning methods, reactive or dynamic approaches do not suffer combinatorial explosion. On the other hand, they are sometimes seen as too rigid to be considered strong AI, since the plans are coded in advance. At the same time, natural intelligence can be rigid in some contexts although it is fluid and able to adapt in others.

Example dynamic planning mechanisms include:

  • Finite-state machines These are reactive architectures used mostly for computer game agents, in particular for first-person shooters bots, or for virtual movie actors. Typically, the state-machines are hierarchical. For concrete game examples, see Halo 2 bots paper by Damian Isla (2005) or Quake III bots disertation of Jean Paul van Waveren (2001). For movie example, see Softimage.
  • Other structured reactive plans tend to look a little more like conventional plans, often with ways to represent hierarchical and sequential structure. Some, such as PRS's 'acts', have support for partial plans. Many agent architectures from the mid 1990s included such plans as a "middle layer" that provided organization for low-level behavior modules while being directed by a higher level real-time planner. Despite this supposed interoperability with automated planners, most structured reactive plans are hand coded (Bryson 2001, ch. 3).

Examples of structured reactive plans include James Firby's RAP System and the Nils Nilsson's Teleo-reactive plans. PRS, RAPs & TRP are no longer developed or supported. One still-active (as of 2006) descendent of this approach is the Parallel-rooted Ordered Slip-stack Hierarchical (or POSH) action selection system, which is a part of Joanna Bryson's Behaviour Oriented Design.

Sometimes to attempt to address the perceived inflexibility of dynamic planning, hybrid techniques are used. In these, a more conventional AI planning system searches for new plans when the agent has spare time, and updates the dynamic plan library when it finds good solutions. The important aspect of any such system is that when the agent needs to select an action, some solution exists that can be used immediately (see further anytime algorithm).

[edit] Others

  • Soar is a symbolic cognitive architecture. It is based on condition-action rules known as productions. Programmers can use the Soar development toolkit for building both reactive and planning agents, or any compromise between these two extremes.
  • ACT-R is similar to Soar. It is less powerful as a programming language, but simpler to get working. It includes a Bayesian learning system to help prioritize the productions.
  • ABL/Hap
  • Fuzzy architectures The Fuzzy approach in action selection produces more smooth behaviour than can be produced by architectures exploiting boolean condition-action rules (like Soar or POSH). These architectures are mostly reactive and symbolic. See the work of Alex Champandard.

[edit] Theories of action selection in nature

Many dynamic models of artificial action selection were originally inspired by research in ethology. In particular, Konrad Lorenz and Nikolaas Tinbergen provided the idea of an innate releasing mechanism to explain instinctive behaviors (fixed action patterns). Influenced by the ideas of William McDougall, Lorenz developed this into a "psychohydraulic" model of the motivation of behavior. In ethology, these ideas were influential in the 1960s, but they are now regarded as outdated because of their use of an energy flow metaphor; the nervous system and the control of behavior are now normally treated as involving information transmission rather than energy flow. Dynamic plans and neural networks are more similar to information transmission, while spreading activation is more similar to the diffuse control of emotional / hormonal systems.

Stan Franklin has proposed that action selection is the right perspective to take in understanding the role and evolution of mind. See his page on the action selection paradigm.

[edit] AI models of neural action selection

Some researchers create elaborate models of neural action selection. See for example:

[edit] See also

[edit] External links

This theme issue focusses on a particular strategy for finding scientific explanations - computer modelling. The contributions employ state-of-the-art modelling techniques ranging from large networks of simulated brain cells, through to models of individuals (people or animals) viewed as agents operating in simulated worlds.

[edit] References

  • Bratman, N.: Intention, plans, and practical reason. Cambridge, Mass: Harvard University Press (1987)
  • Brom, C., Lukavský, J., Šerý, O., Poch, T., Šafrata, P.: Affordances and level-of-detail AI for virtual humans. In: Proceedings of Game Set and Match 2, Delft (2006)
  • Bryson, J.: Intelligence by Design: Principles of Modularity and Coordination for Engineering Complex Adaptive Agents. PhD thesis, Massachusetts Institute of Technology (2001)
  • Champandard, A. J.: AI Game Development: Synthetic Creatures with learning and Reactive Behaviors. New Riders, USA (2003)
  • Grand, S., Cliff, D., Malhotra, A.: Creatures: Artificial life autonomous software-agents for home entertainment. In: Johnson, W. L. (eds.): Proceedings of the First International Conference on Autonomous Agents. ACM press (1997) 22-29
  • Huber, M. J.: JAM: A BDI-theoretic mobile agent architecture. In: Proceedings of the Third International Conference on Autonomous Agents (Agents'99). Seatle (1999) 236-243
  • Isla, D.: Handling complexity in Halo 2. In: Gamastura online, 03/11 (2005)
  • Maes, P.: The agent network architecture (ANA). In: SIGART Bulletin, 2 (4), pages 115–120 (1991)
  • Nareyek, A. Excalibur project
  • Reynolds, C. W. Flocks, Herds, and Schools: A Distributed Behavioral Model. In: Computer Graphics, 21(4) (SIGGRAPH '87 Conference Proceedings) (1987) 25-34.
  • de Sevin, E. Thalmann, D.:A motivational Model of Action Selection for Virtual Humans. In: Computer Graphics International (CGI), IEEE Computer SocietyPress, New York (2005)
  • Tyrrell, T.: Computational Mechanisms for Action Selection. Ph.D. Dissertation. Centre for Cognitive Science, University of Edinburgh (1993)
  • van Waveren, J. M. P.: The Quake III Arena Bot. Master thesis. Faculty ITS, University of Technology Delft (2001)
  • Wooldridge, M. An Introduction to MultiAgent Systems. John Wiley & Sons (2002)