Annotated List of Selected Papers on Neural Nets or Mind

The annotated list of downloadable papers here is very far from complete. It does not include the first papers I published on many topics.

But they are all papers easily available in electronic form, with some special value added. This is in addition to the other papers posted on my main “What is Mind?” web page.

Older Papers Combining Sensitivity Analysis, Backwards Differentiation and Use in Training Intelligent Neural Network Systems

P.Werbos, Applications of advances in nonlinear sensitivity analysis, in R.Drenick & F. Kozin (eds), System Modeling and Optimization: Proc. IFIP Conf. (1981),Springer 1982; reprinted in P. Werbos, The Roots of Backpropagatoin, Wiley, 1994.

P.Werbos, Minimum Cost Differentiation Methods and their Uses in Understanding, Designing and Optimizing Complex Systems, an EIA/DOE “Validation Report”, 1982. This was the last version of the longer report cited in the IFIP paper. These two papers – especially this longer one – were the two papers discussed with Charles Smith, mentioned in the historical account in Anderson, J., and E.Rosenfeld, eds, Talking Nets, MIT Press, 1998.

Papers Connecting Control Theory and Neural Networks, addressing stability and basic “vector to vector” designs:

P.Werbos, Optimization methods for brain-like intelligent control, Proc. IEEE Conf. CDC, IEEE, 1995

P. Werbos, Neural Networks for Control: Research Opportunities and Recent Developments, Proc. IEEE Conf. CDC, IEEE, 2002

P. Werbos, New Designs for Universal Stability in Classical Adaptive Control and Reinforcement Learning. Proc. Int’l Joint Conf. Neural Networks (IJCNN92), IEEE, 1999.

This provides a more straightforward summary and explanation of the results of my longer 1998 paper.

Papers Focused On Practical Applications and Methods of Neurocontrol – Especially Robotics and Saving Damaged Aircraft:

P. Werbos, Neurocontrollers, in J.Webster, ed, Encyclopedia of Electrical and Electronics Engineering, Wiley, 1999. Relatively easy to read, but with crucial techniques, and examples from Hirzinger’s seminal work in robotics, as well as other areas I have discussed more often.

P.Werbos, Neural networks and flight control: Overview of capabilities and emerging applications, Proc. Of American Control Conference (ACC), 2001.

P. Werbos, Neural Networks for Flight Control: A Strategic and Scientific Assessment. In C. Jorgensen, ed., Proc. of Neural Nets for Aero Control Symp., NASA Ames Research Center, Moffett Field, CA, 16 pp., Aug. 1994.

P.Werbos, Elastic fuzzy logic: a better fit to neurocontrol and true intelligence, J. Intelligent & Fuzzy Systems, Vol. 1, No.4, 1993. This essentially gives my version of how to get “white box” rules out of complex neural-net type training systems. Yager and Fukuda applied equivalent (if less direct) notions of adaptive fuzzy logic in later years. There is also a US patent, rooted in 1992 papers and filing. As a practical matter, however, much of the rhetoric about people wanting “white box” rules may constitute an excuse rather than somehting people are really looking for. Furthermore, truly new or creative solutions learned by an unconstrained artificial intelligence could, in principle, be as hard to explain as the ideas in the mind of an Einstein.

For a more general discussion of learning, robotics and the needs of space solar power, see the NSF/NASA year 2000 workshop report and presentations.

Papers Focused on More Complex (Still Subsymbolic) Intelligent Designs

P.Werbos, Supervised learning: can it escape its local minimum?, WCNN93 Proceedings, Erlbaum, 1993. Reprinted in V. Roychowdhury et al (eds), Theoretical Advances in Neural Computation and Learning, Kluwer, 1994. This paper still addreses vector-to-vector learning systems, but it discusses important themes like syncretism and ways to get closer to some of the thoughts of Karl Pribram about real neurons.

P.Werbos & X.Z.Pang, Generalized maze navigation: SRN critics solve what feedforward or Hebbian nets cannot.Proc. Conf. Systems, Man and Cybernetics (SMC) (Beijing), IEEE, 1996. (An earlier version appeared in WCNN96 and Yale96[57].) This is the brief paper where I report the extension of my earlier work with Pang, to the case of a system which is trained on six easy mazes, and generalizes well to six hard test mazes it was not trained on. There was later important work speeding this up with chip designs (by Chua) and by uisng faster training procedures (ways of uisng BTT derivatives), by Ilin, Kozma and myself. Robert Kozma at the University of Memphis has more details, incluidng MatLab code..

X.Z.Pang & P.Werbos, Neural network design for J function approximation in dynamic programming, Math. Modelling and Scientific Computing (a Principia Scientia journal), Vol. 5, NO.2/3, 1996 (physically 1998, work done in 1994). The paper is long, but provides a relatively straightforward introduction to some key issues in training recurrent networks.

P. Werbos, Brain-Like Intelligent Control: From Neural Nets to True Brain-Like Intelligence. In Proc. World Automation Conference 1998.

P.Werbos, Multiple Models for Approximate Dynamic Programming and True Intelligent Control: Why and How. In K. Narendra, ed., Proc. 10th Yale Conf. on Learning and Adaptive Systems. New Haven: K.Narendra, EE Dept., Yale U., 1998. This is essentially a brief explanation of the more complicated ideas regarding miultiple time scales – and a method which is substantialy more powerful than an earlier idea by Sutton – as described in more detail in the next paper.

P.Werbos, A Brain-Like Design To Learn Optimal Decision Strategies in Complex Environments, in M.Karny, K.Warwick and V.Kurkova, eds, Dealing with Complexity: A Neural Networks Approach. Springer, London, 1998. Also in S.Amari and N.Kasabov, Brain-Like Computing and Intelligent Information Systems. Springer,1998. This is a long and very complex paper, from which I later pulled out the smaller pieces on Object Nets, Brain-Like Stochastic Search, and so on – pieces which have great potential value in advancing the ANN field one step ahead, with important new practical capabilities, even though they don’t go in one step all the way to the mouse-brain level.

Papers Addressing Links to Neuroscience and Consciousness

P.Werbos, The brain as a neurocontroller: New hypotheses and experimental possibilities. In K.Pribram, ed., Origins: Brain and Self-Organization, Erlbaum, 1994, p.680-706

(I regret that I do not have electronic images of the original figures – though in most cases I have newer and better versions inserted.)

P. Werbos and A. Pellionisz, Neurocontrol and Neurobiology: New Developments and New Connections. In Proc. Of the Internatoinal Joint Conference on Neural Networks (IJCNN92), IEEE, 1992.

P.Werbos, Optimization: A Foundation for understanding consciousness. In D.Levine & W. Elsberry (eds) Optimality in Biological and Artificial Networks?, Erlbaum, 1997

Conversations on the path from mouse-level intelligence to the human level, excerpted from the seminar on “The Evolution of Human Intelligence.”

The full seminar report is available from the Foundation for the Future.