A Hierarchical Modular Architecture for Embodied Cognition
In: Multisensory ResearchSearch for other papers by Dana H. Ballard in
Current site
Google Scholar
Search for other papers by Dmitry Kit in
Current site
Google Scholar
Search for other papers by Constantin A. Rothkopf in
Current site
Google Scholar
Search for other papers by Brian Sullivan in
Current site
Google Scholar
Cognition can appear complex owing to the fact that the brain is capable of an enormous repertoire of behaviors. However, this complexity can be greatly reduced when constraints of time and space are taken into account. The brain is constrained by the body to limit its goal-directed behaviors to just a few independent tasks over the scale of 1–2 min, and can pursue only a very small number of independent agendas. These limitations have been characterized from a number of different vantage points such as attention, working memory and dual task performance. It may be possible that the disparate perspectives of all these methodologies can be unified if behaviors can be seen as modular and hierarchically organized. From this vantage point, cognition can be seen as having a central problem of scheduling behaviors to achieve short term goals. Thus dual-task paradigms can be seen as studying the concurrent management of simultaneous, competing agendas. Attention can be seen as focusing on the decision as to whether to interrupt the current agenda or persevere. Working memory can be seen as the bookkeeping necessary to manage the state of the current active agenda items.
Purchase
Buy instant access (PDF download and unlimited online access):
Institutional Login
Log in with Open Athens, Shibboleth, or your institutional credentials
Personal login
Log in with your brill.com account
-
Adams F. (2010). Embodied cognition, Phenomenol. Cogn. 9, 619–628.
-
Anderson J. (1983). The Architecture of Cognition. Harvard University Press, New Haven, CT, USA.
-
Arbib M. A. (1988). The Handbook of Brain Theory and Neural Networks, pp. 830–834. MIT Press, Cambridge, MA, USA.
-
Arkin R. (1998). Behavior Based Robotics. MIT Press, Cambridge, MA, USA.
-
Baddeley A. (1992). Working memory, Science 255, 556–559.
-
Ballard D. H., Hayhoe M. M., Pook P., Rao R. (1997a). Deictic codes for the embodiment of cognition, Behav. Brain Sci. 20, 723–767.
-
Ballard D. H., Hayhoe M. M., Pook P. K., Rao R. P. N. (1997b). Deictic codes for the embodiment of cognition, Behav. Brain Sci. 20, 723–742.
-
Barto A. G., Mahadevan S. (2003). Recent advances in hierarchical reinforcement learning, Discrete Event Dyn. S. 13, 41–77.
-
Bonasso E. P., Firby R. J., Gat E., Kortenkamp D., Miller D. P., Slack M. G. (1997). Experiences with an architecture for intelligent reactive agents, J. Exp. Theor. Artif. Intell. 9, 237–256.
-
Bowman H., Wyble B. (2007). The simultaneous type, serial token model of temporal attention and working memory, Psychol. Rev. 114, 38–70.
-
Brooks R. (1986). A robust layered control system for a mobile robot, robotics and automation, IEEE J. 2, 14–23.
-
Bryson J. J., Stein L. A. (2001). Modularity and design in reactive intelligence, in: Internat. Joint Conf. on Artificial Intelligence, Seattle, Washington, USA.
-
Clark A. (1999). An embodied model of cognitive science? Trends Cogn. Sci. 3, 345–351.
-
Craig Boutilier R. D., Goldszmidt M. (2000). Stochastic dynamic programming with factored representations, Artifi. Intell. 121, 49–107.
-
Daw N. D., O’Doherty J. P., Dayan P., Seymour B., Dolan R. J. (2006). Cortical substrates for exploratory decisions in humans, Nature 44, 876–879.
-
Dayan P., Hinton G. E. (1992). Feudal reinforcement learning, Neural Information Processing Systems 5, 271.
-
Doya K., Samejima K., Katagiri K. I., Kawato M. (2002). Multiple model-based reinforcement learning, Neural Comput. 14, 1347–1369.
-
Droll J., Hayhoe M., Triesch J., Sullivan B. (2005). Task demands control acquisition and storage of visual information, J. Exp. Psychol. Human 31, 1416–1438.
-
Fan J., McCandliss B. D., Fossella J., Flombaum J. I., Posner M. I. (2005). The activation of attentional networks, NeuroImage 26, 471–479.
-
Firby R. J., Kahn R. E., Prokopowicz P. N., Swain M. J. (1995). An architecture for vision and action, in: Int. Joint Conf. on Artificial Intelligence, pp. 72–79.
-
Gibson J. J. (1979). The Ecological Approach to Visual Perception. Houghton Mifflin, Boston, USA.
-
Guestrin C. E., Koller D., Parr R., Venkataraman S. (2003). Efficient solution algorithms for factored MDPs, J. Artif. Intell. Res. 19, 399–468.
-
Hayhoe M. M., Shrivastava A., Mruczek R., Pelz J. (2003). Visual memory and motor planning in a natural task, J. Vision 3, 49–63.
-
Hikosaka O., Bromberg-Martin E., Hong S., Matsumoto M. (2008). New insights on the subcortical representation of reward, Curr. Opin. Neurobiol. 18, 203–208.
-
Humphrys M. (1996). Action selection methods using reinforcement learning, in: From Animals to Animats 4: Proc. 4th Internat. Conf. Simulation of Adaptive Behavior, P. Maes, M. Mataric, J.-A. Meyer, J. Pollack and S. W. Wilson (Eds), MIT Press, Bradford Books, Cambridge, MA, USA, pp. 135–144.
-
Hurley S. (2008). The shared circuits model (scm): how control, mirroring, and simulation can enable imitation, deliberation, and mindreading, Behav. Brain Sci. 31, 1–22.
-
Itti L. (2005). Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes, Vis. Cogn. 12, 1093–1123.
-
Itti L., Koch C. (2000). A saliency based search mechanism for overt and covert shifts of attention, Vision Research 40, 1489–1506.
-
Karlsson J. (1997). Learning to solve multiple goals, PhD thesis, University of Rochester, NY, USA.
-
Laird J. E., Newell A., Rosenblum P. S. (1987). Soar: an architecture for general intelligence, Artif. Intell. 33, 1–64.
-
Langley P., Choi D. (2006). Learning recursive control programs from problem solving, J. Mach. Learn. Res. 7, 493–518.
-
Luck S. J., Vogel E. K. (1997). The capacity of visual working memory for features and conjunctions, Nature 390, 279–281.
-
Meuleau N., Hauskrecht M., Kim K.-E., Peshkin L., Kaelbling L., Dean T., Boutilier C. (1998). Solving very large weakly coupled Markov decision processes, in: Proc. 15th Natl/10th Conf. Madison, WI, USA, AAAI/IAAI, pp. 165–172.
-
Navalpakkam V., Koch C., Rangel A., Perona P. (2010). Optimal reward harvesting in complex perceptual environments, Proc. Nat. Acad. Sci. USA 107, 5232–5237.
-
Neisser U. (1967). Cognitive Psychology. Appleton-Century-Crofts, New York.
-
Noe A. (2005). Action in Perception. MIT Press, Cambridge, MA, USA.
-
Nordfang M., Dyrholm M., Bundesen C. (2012). Identifying bottom-up and top-down components of attentional weight by experimental analysis and computational modelling, J. Exp. Psychol. Gen., DOI:10.1037/a0029631.
-
O’Regan J. K., Noe A. (2001). A sensorimotor approach to vision and visual consciousness, Behav. Brain Sci. 24, 939–973.
-
Parr R., Russell S. (1997). Reinforcement learning with hierarchies of machines, in: Advances in Neural Information Processing Systems, Jordan M. I., Kearns M. J., Solla S. A. (Eds). MIT Press, Cambridge, MA, USA.
-
Pfeifer R., Scheier C. (1999). Understanding Intelligence. Bradford Books, Cambridge, MA, USA.
-
Posner M. I., Rothbart M. K. (2007). Research on attention networks as a model for the integration of psychological science, Ann. Rev. Psychol. 58, 1–23.
-
Rao R. P. N., Zelinsky G. J., Hayhoe M. M., Ballard D. H. (2002). Eye movements in iconic visual search, Vision Research 42, 14447–14463.
-
Ritter S., Anderson J. R., Cytrynowicz M., Medvedeva O. (1998). Authoring content in the pat algebra tutor, J. Interact. Media Educ. 98, 1–30.
-
Roelfsema P. R., Khayat P. S., Spekreijse H. (2003). Sub-task sequencing in the primary visual cortex, Proc. Nat. Acad. Sci. USA 100, 5467–5472.
-
Rothkopf C. A., Ballard D. H. (2009). Image statistics at the point of gaze during human navigation, Vis. Neurosci. 26, 81–92.
-
Rothkopf C. A., Ballard D. H. (2010). Credit assignment in multiple goal embodied visuomotor behaviour, Front. Psychol. 1, 1–13, online.
-
Rothkopf C. A., Ballard D. H., Hayhoe M. M. (2007). Task and context determine where you look, J. Vision 7, 1–20.
-
Roy D. K., Pentland A. P. (2002). Learning words from sights and sounds: a computational model, Cogn. Sci. 26, 113–146.
-
Rummery G. A., Niranjan M. (1994). Online Q-learning using connectionist systems, Technical Report CUED/FINFENG/TR 166, Cambridge University Engineering Department, UK.
-
Russell S., Zimdars A. (2003). Q-decomposition for reinforcement learning agents, in: Proc. Int. Conf. Machine Learning.
-
Ruthruff E., Pashler H. E., Hazeltine E. (2003). Dual-task interference with equal task emphasis: graded capacity-sharing or central postponement? Atten. Percept. Psycho. 65, 801–816.
-
Sallans B., Hinton G. E. (2004). Reinforcement learning with factored states and actions, J. Mach. Learn. Res. 5, 1063–1088.
-
Samejima K., Doya K., Kawato M. (2003). Inter-module credit assignment in modular reinforcement learning, Neural Networks 16, 985–994.
-
Schultz W. (2000). Multiple reward signals in the brain, Nat. Rev. Neurosci. 1, 199–207.
-
Schultz W., Dayan P., Montague P. R. (1997). A neural substrate of prediction and reward, Science 275, 1593–1599.
-
Shapiro L. (2011). Embodied Cognition. Routledge, New York, USA.
-
Singh S., Cohn D. (1998). How to dynamically merge Markov decision processes, in: Neural Information Processing Systems Conf., Denver, CO, USA, 1997, Vol. 10, pp. 1057–1063.
-
Sprague N., Ballard D. (2003). Multiple-goal reinforcement learning with modular sarsa(0), in: Internat. Joint Conf. Artificial Intelligence, Acapulco, USA.
-
Sprague N., Ballard D., Robinson A. (2007). Modeling embodied visual behaviors, ACM Trans. Appl. Percept. 4, 11.
-
Stewart J., Gapenne O., Di Paolo E. (Eds) (2010). En-action: Toward a New Paradigm for Cognitive Science. MIT Press, Cambridge, MA, USA.
-
Sun R. (2006). Cognition and Multi-Agent Interaction, Ch. 4, pp. 79–99. Cambridge University Press, UK.
-
Sutton R. S., Barto A. G. (1998). Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, USA.
-
Sutton R. S., Precup D., Singh S. P. (1999). Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning, Artif. Intell. 112, 181–211.
-
Tolman E. C. (1948). Cognitive maps in rats and men, Psycholog. Rev. 55, 189–208.
-
Torralba A., Oliva A., Castelhano M., Henderson J. M. (2006). Contextual guidance of attention in natural scenes: the role of global features on object search, Psychol. Rev. 113, 766–786.
-
Treisman A. M. (1980). A feature-integration theory of attention, Cogn. Psychol. 12, 97–136.
-
Trick L. M., Pylyshyn Z. W. (1994). Why are small and large numbers enumerated differently? A limited-capacity preattentive stage in vision, Psychol. Rev. 101, 80–102.
-
Ullman S. (1985). Visual routines, Cognition 18, 97–159.
-
Vareala F. J., Thompson E., Rosch E. (1991). The Embodied Mind: Cognitive Science and Human Experience. MIT Press, Cambridge, MA, USA.
-
Vigorito C. M., Barto A. G. (2010). Intrinsically motivated hierarchical skill learning in structured environments, IEEE Trans. Auton. Mental Dev. 2, 132–143.
-
Watkins C. J. C. H. (1989). Learning from delayed rewards, PhD thesis, University of Cambridge, UK.
-
Yu C., Ballard D. (2004). A multimodal learning interface for grounding spoken language in sensorimotor experience, ACM Trans. Appl. Percept. 1, 57–80.
| All Time | Past Year | Past 30 Days | |
|---|---|---|---|
| Abstract Views | 625 | 122 | 11 |
| Full Text Views | 110 | 19 | 3 |
| PDF Views & Downloads | 66 | 24 | 1 |
A Hierarchical Modular Architecture for Embodied Cognition
In: Multisensory ResearchSearch for other papers by Dana H. Ballard in
Current site
Google Scholar
PubMed
Search for other papers by Dmitry Kit in
Current site
Google Scholar
PubMed
Search for other papers by Constantin A. Rothkopf in
Current site
Google Scholar
PubMed
Search for other papers by Brian Sullivan in
Current site
Google Scholar
PubMed
Cognition can appear complex owing to the fact that the brain is capable of an enormous repertoire of behaviors. However, this complexity can be greatly reduced when constraints of time and space are taken into account. The brain is constrained by the body to limit its goal-directed behaviors to just a few independent tasks over the scale of 1–2 min, and can pursue only a very small number of independent agendas. These limitations have been characterized from a number of different vantage points such as attention, working memory and dual task performance. It may be possible that the disparate perspectives of all these methodologies can be unified if behaviors can be seen as modular and hierarchically organized. From this vantage point, cognition can be seen as having a central problem of scheduling behaviors to achieve short term goals. Thus dual-task paradigms can be seen as studying the concurrent management of simultaneous, competing agendas. Attention can be seen as focusing on the decision as to whether to interrupt the current agenda or persevere. Working memory can be seen as the bookkeeping necessary to manage the state of the current active agenda items.
| All Time | Past Year | Past 30 Days | |
|---|---|---|---|
| Abstract Views | 625 | 122 | 11 |
| Full Text Views | 110 | 19 | 3 |
| PDF Views & Downloads | 66 | 24 | 1 |