Three brief statements on research strategy to understand the mammalian brain


Included here are:

(1) “Adaptive and Intelligent Systems” – description of a topic for funding within the field of engineering, which is crucial to building up, enhancing and disseminating the kind of mathematics we need in order to really understand how the brain works;

(2) “Cognitive Optimization and Prediction” – description of a new crossdisciplinary topic for funding now in place at the National Science Foundation; and

(3) Reverse-engineering the brain – a response to this grand challenge, recently listed as one of the 14 grand challenges in engineering by the National Academy of Engineering.


Item (2) was copied directly from the official web site of a major NSF funding opportunity. Items 1 and 3 are my personal views, and not anything official. Item 3 was recently published in the newsletter Autonomous Mental Development (AMD), the newsletter of the AMD task force of the IEEE Computational Intelligence Society.




Adaptive and Intelligent Systems (AIS)


Examples of Subtopics: computational intelligence, neural networks, approximate dynamic programming, machine learning, adaptive signal processing, pattern recognition, data mining, on-chip learning, biologically inspired computing, quantum learning, applications to testbeds in vehicles, robots, infrastructure, energy, manufacturing and other.


Scope: Many research disciplines work to develop or use general-purpose nonlinear algorithms for discovering patterns, making predictions or making inferences from a stream of sensor data (or databases), and for using such information to make decisions or to control engineering systems or other plants. The Adaptive and Intelligent Systems (AIS) area supports research in any of these disciplines, or combinations of disciplines, which helps us make greater use of adaptation or learning, or more distributed processing, in order to achieve greater generality and less domain-dependence in such algorithms or architectures. The ultimate goal is to develop the principles, subsystems and unifications which will someday let us build an integrated all-purpose system which can “do it all” – connect a huge set of diverse sensors to a huge set of actuators, using a massively parallel set of simple underlying processing units, able to learn how to manage complex systems in general in a near-optimal fashion without major external help. More precisely, it should be able to replicate at least that level of general-purpose higher intelligence that is found in the brains of even the smallest mammals.

          AIS does not expect any three-year research project to reach this long-term goal, but it will give priority to proposals based on their importance in bringing it closer to reality. Much AIS research involves general adaptation or inference methods which can be applied to a wide range of nonlinear function approximators, general enough to include neural networks as an important special case. Some even develops new linear methods as part of a progression well-planned to provide new general-purpose nonlinear capabilities or to enhance the use of learning. About half addresses “value-free” tasks like prediction, system identification, pattern recognition or state estimation. About half addresses control or decision or optimization or reinforcement learning; see for some examples. Proposals with titles beginning with “AIS:” will be initially assigned to this area.  



1.   Cognitive Optimization and Prediction: From Neural Systems to Neurotechnology (COPN)

2.   Resilient and Sustainable Infrastructures (RESIN)

Preliminary Proposal Due Date(s) (required):

October 26, 2007

Full Proposal Deadline(s) (due by 5 p.m. proposer's local time):

April 30, 2008

Cognitive Optimization and Prediction: From Neural Systems to Neurotechnology (COPN) -This EFRI topic provides partnership opportunities for engineers and neuroscientists to address two goals.The first goal is to understand how massively parallel circuits in brains address complex tasks in adaptive optimal decision-making and prediction, and to understand the system identification circuits in the brain which help make it possible.The second goal is to use this understanding to develop new general-purpose designs or algorithms for optimal decision-making over time, or prediction, or both, powerful enough to work existing benchmark challenges in simulation or in physical testbeds taken from engineering. This EFRI topic does not include all aspects of cognitive science or cognitive engineering.The focus is on subsymbolic intelligence, because it is a difficult and important prerequisite to a deeper understanding of human intelligence. Engineers have unique qualifications to address optimal decision-making and prediction over time under uncertainty.A grand challenge is to develop new concepts of anticipatory optimization that can cope with spatial complexity in time and nonconvexity as required to improve probability of survival in nonlinear, stochastic environments. Transformative benefits expected are: (1) to put science firmly on the path to a truly functional, unified mathematical and systems understanding of intelligence in the brain – an objective as important as the search for unified models in physics; (2) new designs for optimal decision-making which can handle complexity beyond the capacity of today’s methods, as required for truly optimal rational management of complex engineered systems; (3) improved performance in specified simulation testbeds; and (4) development of new and more general ways to harness the potential power of massively parallel “supercomputers on a chip."



1.     Cognitive Optimization and Prediction: From Neural Systems to Neurotechnology (COPN)

Vertebrate brains, from fish to the smallest rodents, show an amazing ability to learn how to maximize probability of survival under all kinds of diverse, complex and unexpected circumstances. The first goal of this topic is to mobilize systems engineers (as defined below) to lead partnerships which help us understand how massively parallel circuits in vertebrate brains can learn to address such complex tasks through adaptive optimal decision-making, and through the prediction or system identification circuits in the brain which help make it possible. The goal is to achieve the kind of understanding which can be represented as new general-purpose designs or algorithms for optimal decision-making over time, or prediction, or both, powerful enough to work on existing benchmark challenges or testbeds taken from engineering. Prediction in the brain includes and unifies capabilities such as pattern recognition, state estimation and memory.

This topic does not include all aspects of cognitive science or cognitive engineering. For example, it does not address how mirror neurons, empathy and symbolic reasoning give human brains additional hard-wired capabilities beyond the part which we inherit from the mouse. It does not address phenomena like social or collective intelligence. This topic focuses on subsymbolic intelligence and learning, because it is a difficult and important prerequisite to a deeper understanding of human intelligence, and because engineers have unique qualifications to address optimal decision-making and prediction over time under uncertainty. Many engineers have focused on a narrow, deterministic or even linear concept of optimization, which cannot address challenges like maximizing a probability of survival in a nonlinear environment, where it is impossible to prove a theorem that survival can be guaranteed. The second goal of this topic is to stimulate and enlarge the important emerging communities of engineers who work on truly stochastic optimization and prediction, who use quality of service concepts to translate reliability issues into an optimization problem, and who are finding ways to use new massively parallel chips with teraflops on a chip.

This topic will fund new crossdisciplinary teams, led by systems engineers, to reverse-engineer the brain’s general-purpose domain-independent ability to converge towards optimal decisions when confronted with spatial complexity, temporal complexity, or challenges which require a high degree of creativity (nonconvex optimization). All teams must include at least three types of expertise, led by a systems engineer: (1) systems engineering, such as control theory, artificial neural networks, signal processing, nonlinear dynamical systems or operations research, with capabilities in stochastic optimization tasks requiring foresight; (2) relevant neuroscience, such as cognitive systems, computational neuroscience, or behavioral neuroscience; (3) device or systems technology for studying the brain or systems of neurons (e.g., novel neural recording technology, technology for probing systems of neurons on a chip or intracellular sensing) or massively parallel neuromorphic chips like Cellular Neural Network (CNN) chips.

All proposals must have the potential to make significant contribution in both of two areas:

A.   Progress towards understanding of learning in the vertebrate brain.

The long-term goal is to move closer to a truly functional, unified mathematical and systems understanding of learning in the brain – a transformation as important in its way as the Newtonian revolution was to physics. Proposals will not be expected to accomplish this revolution in four years, but will be judged by how much they really bring us closer to it. Thus all proposals must address empirical data from neuroscience areas, and include co-PIs who have participated in projects collecting such data.

Proposals need not include the development of new devices for recording from neural systems. If they do, however, they will be judged on the basis of the new knowledge which may result from using these devices as part of the project; device development for its own sake will not be funded. No projects will be funded which include development or use of neural implants to stimulate living whole brains.

B.   Progress towards handling spatial or temporal complexity in technology, or replicating brain-like creativity.

The long-term goal is to develop general-purpose engineering designs capable of learning to perform optimization or prediction, using massively parallel computing hardware, in the face of complexity, nonconvexity and nonlinearity so severe that absolute guarantees of stability are impossible. Projects will be judged on the basis of whether they effectively address the challenge of getting us to that goal as soon as possible. Thus all projects must include the use of a difficult benchmark challenge for technology. This could be a challenge in simulation or in real hardware, but the benchmarking and the level of difficulty are what really test the relevance of technology. Even such testbeds as the game of Go would be acceptable, so long as the models to be tested are both general and linked to biological testing.

There are many new and emerging opportunities to bring engineering and neuroscience together in the bold way that is called for by this topic; here are just a few examples:

·         Reverse-engineering the circuits which learn to predict in the thalamo-cortical systems. This would build on work on the barrel system of rats, for which it has been shown that certain thalamic cells predict other cells and relearn to predict when the initial circuit is disrupted. Millisecond-level multielectrode data might show waves of information during calculation and adaptation similar to those in artificial nonlinear estimation systems, drawing on systems concepts such as time-lagged recurrent networks, extended Kalman filtering, real-time least squares and seasonal adjustment, and gating based on encoder/decoder designs. (Seasonal adjustment addresses the presence of multiple lags, such as 100 millisecond delays induced by the alpha rhythm pacemaker cells, and local codes based on the strengths of bursts at discrete time intervals.)

·         Reverse-engineering cultures/circuits of multiple types of neurons on a chip, where two-way chemical and electrical stimulation and real-time imaging or sending is possible. A key question is how to “persuade” such cultures to learn to address a menu of optimization or prediction tasks provided by an engineer. Controlled experiments in this venue may allow rapid progress in understanding the capabilities of various types of neurons. Progress in replicating capabilities similar to those of an entire brain may help us understand what kinds of cells, conditions and connections are necessary to achieve (and enhance) such capabilities.

·         Development of a new system which uses new biologically-grounded models of learning to train a CNN chip to learn the non-Euclidean image transformations essential to improved performance on face-recognition tasks. Research on biological face recognition by humans, other mammals or birds could play a crucial role here, and help elucidate the role of symmetry properties and attention to objects in coping with spatial complexity.

·         Understanding how phenomena like chemically-encoded memory can interact with larger-scale prediction circuits, in order to make the multilevel system perform so well in difficult prediction tasks.  Just as the equations of the Kalman filter are a general-purpose system for estimation and prediction of linear, stochastic systems, the cerebro-thalamic system appears to include a general-purpose ability to learn to predict the entire “input vector” of sensory information registered in the thalamus, “the movie screen of the upper brain."

Please pay special attention to the additional review criteria in Section VI.A.

Reverse Engineering the Brain: A Proposed New Thrust


On February 19, 2008, the National Academy of Engineering listed fourteen grand challenges for the coming century, including: “Reverse-engineering the brain, to determine how it performs it magic, should offer the dual benefits of helping treat diseases while providing clues for new approaches to computerized artificial intelligence.”


(1) Specific Goals for Engineering

1a. New methods for more accurate adaptive, anticipatory optimization in controlling systems of ever

greater complexity in space and in time, with additional “creativity” mechanisms to anticipate

future possibilities and escape from local minima. Such adaptive systems may start from management systems designed by humans for the specific systems being controlled; however, in this topic, the challenge is to build adaptive systems which can improve upon such initial starting points, and adapt to new circumstances which the human designer did not specifically anticipate.

1b. New methods for designing general-purpose domain-independent systems which learn to predict more

and more accurately, in the face of  nonlinearity, random disturbance, unobserved variables, and ever greater complexity in space and in time.

1c.  A new paradigm of resilient control, where one minimizes the probability of undesired events, under

conditions so complex and challenging that guaranteed survival is impossible under realistic assumptions.

1d. Methods, algorithms and architectures which fit with the most important constraint which computations

in the brain must live with: massively parallel computation across a large number of (mostly) sparsely connected processing units. This can vastly increase our ability to harness the potential

benefits of massively parallel “supercomputers on a chip,” such as Cellular Neural Network chips. The use of general-purpose learning to better harness such new capabilities is part of this topic.

1e. Contribution to NSF’s mission in basic science. Arguably the most important single goal here is to

demonstrate what engineers can contribute to enhancing our understanding of the brain and of the mind, above and beyond what other disciplines are now able to do. New engineering technology for studying the brain is part of this, although neuroscientists already look for new technology. New systems level models and methods, and true reverse engineering of the mammalian brain, is the biggest unmet opportunity of importance to humanity.

1f. Better performance in some selected testbeds used for this effort, such as control of electric power

infrastructures, face recognition, video processing, and others.


(2) Role of Engineering in the Broader Context of NSF Adaptive Systems Theory

          People have talked about understanding how intelligence emerges in the brain, and building artificial intelligence, for decades and decades. For example, in the 1960’s, one of the two leading centers on artificial intelligence promised NASA that they would deliver a full-fledged artificial human brain, based on linguistics and symbolic reasoning, in a robot ready to be sent to Mars in the 1980’s. Likewise, in the past, many neuroscientists suggested that we could reach an understanding of intelligence and learning in the brain by first doing exhaustive probes of simple organisms like snails (aplysia), and extracting the mathematical principles; however, current research on such organisms suggests that much of their behavior is indeed genetically programmed, and is not so helpful in understanding how we can learn new things above and beyond what is already coded into our genes. Yet another stream of research has stressed that the human mind has an almost magical ability to understand things beyond the scope of conventional computing – and then suggested that we follow up by building “computers to compute the noncomputable.”

          Engineering today has made enough progress to allow us to construct a fourth pathway of research, which can allow us to bypass the obstacles which have limited the accomplishments of the old paths. The old paths still have much to contribute, but something new is needed, to achieve a real functional understanding of intelligence in the brain. In order to meet this new opportunity,  we will need to focus on the unique new directions in engineering which make it possible, and to focus on ways to exploit these new directions. It is necessary to build strong partnerships with empirical neuroscience (especially systems neuroscience) and psychology, without falling into the limitations of traditional research in those fields.

          The first key goal on this new path is to focus on trying to understand learning-based general intelligence, as it exists in brains ranging from the crudest fish to the smallest mammal. While others focus on snails or try to build an “artificial Einstein,” the new opportunity lies at this middle level. Symbolic reasoning and empathy (“mirror neurons”) have given great power to the human brain, not shared by these lower organisms. Yet before we can really understand the “deep structure” which underlies language and semantics, we must first understand the functional wiring and learning which human brains and mouse brains share. The six-layer neocortex of the human brain is almost indistinguishable from that of the mouse. Different topographic regions in the human brain usually contain different knowledge from what mouse brains contain, but human brains differ from each other, and can change with time; the famous “edge detectors” hard-wired into certain regions are easily relearned in other parts of the brain, when the system is damaged, if the wiring from thalamus to neocortex is not cut.

          The next key element on this new path is to focus on cognitive optimization and prediction. General intelligence is not the ability to do well in a narrow specific task, like playing a good game of chess. Computer scientists have shown how that kind of narrow focus is a dead-end, when it comes to learning about intelligence in general. On other hand, the task of adaptive, anticipatory optimization in the face of a large, unknown, nonlinear stochastic environment is of general importance. It is a task where engineers have led the development of new types of mathematical design, which can handle far more complexity than traditional forms of reinforcement learning which have been popularized in computer science. The most important research questions here are: (1) how can we deepen our understanding of the mathematical principles of adaptive, anticipatory optimization and of prediction, so as to cope with the kind of complexity in time and in space which the brain can handle?; (2) how can we insert more brain-like approaches into engineering technology?; (3) how can we unravel the circuits and methods which the brain uses to learn to predict or to optimize better, in a way which transcends specific testbeds for prediction or optimization? How can the smallest mouse learn to maximize its probability of survival?

          As an example, Nicolelis and Chapin showed in the 1990’s that certain cells in the thalamus predict the sensor inputs that are received by other cells, and relearn to predict well after damage to the system. How does this finding apply across all sensory inputs, and how does the brain learn this?

          In general, the goal here is not to predict  brains in a “behaviorist” or “positivist” way, as if they were like a passive crystal of ice; rather it is to understand the general functional capabilities of brains, so that we can better understand, work with and enhance the intelligence possessed even by the smallest mouse.  How do brains themselves learn to predict better and better with time?


(3) Some Specific Themes.

          Many, many disciplines have approached these questions from different perspectives. A one or two page statement of goals cannot do justice to the important details of ideas and strategies and mathematical principles which will be essential to success in this venture. The EFRI announcement on Cognitive Optimization and Prediction fills in some of these details, as do a number of other sources of the web (such as workshop reports or papers for workshops). Certainly the improvement of dialogue between disciplines will be a crucial factor in determining how much we actually accomplish.