Perceptrons (book)
Encyclopedia
Perceptrons: an introduction to computational geometry is a book authored by Marvin Minsky
and Seymour Papert
, published in 1969. An edition with handwritten corrections and additions was released in the early 1970s. An expanded edition was further released in 1987, containing a chapter dedicated to counter the criticisms made in the 1980s towards it.
The main subject of the book is the perceptron
, an important kind of artificial neural network
developed in the late 1950s and early 1960s. The main researcher on perceptrons was Frank Rosenblatt
, author of the book Principles of Neurodynamics. Rosenblatt and Minsky knew each other since adolescence, having studied with a one year difference at the Bronx High School of Science
. They became at one point central figures of a debate inside the AI research community, and are known to have promoted loud discussions in conferences. Despite the dispute, the corrected version of the book released after Rosenblatt's death contains a dedication to him.
This book is the center of a long-standing controversy in the study of artificial intelligence
. It is claimed that pessimistic predictions made by the authors were responsible for an erroneous change in the direction of research in AI, concentrating efforts on so-called "symbolic" systems, and contributing to the so-called AI winter
. This decision, supposedly, proved to be unfortunate in the 1980s, when new discoveries showed that the prognostics in the book were wrong.
The book contains a number of mathematical proofs regarding perceptrons, and while it highlights some of perceptrons' strengths, it also shows some previously unknown limitations. The most important one is related to the computation of some predicates, as the XOR function, and also the important connectedness predicate. The problem of connectedness is illustrated at the awkwardly colored cover of the book, intended to show how humans themselves have difficulties in computing this predicate.
There are many mistakes in this story. Although a single neuron can in fact compute only a small number of logical predicates, it was widely known that networks of such elements can compute any possible boolean function. This was known by Warren McCulloch and Walter Pitts
, who even proposed how to create a Turing Machine
with their formal neurons, is mentioned in Rosenblatt's book, and is even mentioned in the book Perceptrons. Minsky also extensively uses formal neurons to create simple theoretical computers in his book Computation: Finite and Infinite Machines.
What the book does prove is that in three-layered feed-forward perceptrons (with a so-called "hidden" or "intermediary" layer), it is not possible to compute some predicates unless at least one of the neurons in the first layer of neurons (the "intermediary" layer) is connected with a non-null weight to each and every input. This was contrary to a hope held by some researchers in relying mostly on networks with a few layers of "local" neurons, each one connected only to a small number of inputs. A feed-forward machine with "local" neurons is much easier to build and use than a larger and recurrent neural network, so researchers at the time concentrated on these instead of on more complicated models.
in H. P. Lovecraft
's tales, a book known to many, but only read by fewhttp://www.ucs.louisiana.edu/~isb9112/dept/phil341/histconn.html. The authors talk in the expanded edition about the criticism of the book that started in the 1980s, with a new wave of research symbolized by the PDP book.
How Perceptrons was explored first by one group of scientists to drive research in AI in one direction, and then later by a new group in another direction, has been the subject of a peer-reviewed sociological study of scientific developmenthttp://www.jstor.org/pss/285702.
Marvin Minsky
Marvin Lee Minsky is an American cognitive scientist in the field of artificial intelligence , co-founder of Massachusetts Institute of Technology's AI laboratory, and author of several texts on AI and philosophy.-Biography:...
and Seymour Papert
Seymour Papert
Seymour Papert is an MIT mathematician, computer scientist, and educator. He is one of the pioneers of artificial intelligence, as well as an inventor of the Logo programming language....
, published in 1969. An edition with handwritten corrections and additions was released in the early 1970s. An expanded edition was further released in 1987, containing a chapter dedicated to counter the criticisms made in the 1980s towards it.
The main subject of the book is the perceptron
Perceptron
The perceptron is a type of artificial neural network invented in 1957 at the Cornell Aeronautical Laboratory by Frank Rosenblatt. It can be seen as the simplest kind of feedforward neural network: a linear classifier.- Definition :...
, an important kind of artificial neural network
Artificial neural network
An artificial neural network , usually called neural network , is a mathematical model or computational model that is inspired by the structure and/or functional aspects of biological neural networks. A neural network consists of an interconnected group of artificial neurons, and it processes...
developed in the late 1950s and early 1960s. The main researcher on perceptrons was Frank Rosenblatt
Frank Rosenblatt
Frank Rosenblatt was a New York City born computer scientist who completed the Perceptron, or MARK 1, computer at Cornell University in 1960...
, author of the book Principles of Neurodynamics. Rosenblatt and Minsky knew each other since adolescence, having studied with a one year difference at the Bronx High School of Science
Bronx High School of Science
The Bronx High School of Science is a specialized New York City public high school often considered the premier science magnet school in the United States. Founded in 1938, it is now located in the Bedford Park section of the Bronx...
. They became at one point central figures of a debate inside the AI research community, and are known to have promoted loud discussions in conferences. Despite the dispute, the corrected version of the book released after Rosenblatt's death contains a dedication to him.
This book is the center of a long-standing controversy in the study of artificial intelligence
Artificial intelligence
Artificial intelligence is the intelligence of machines and the branch of computer science that aims to create it. AI textbooks define the field as "the study and design of intelligent agents" where an intelligent agent is a system that perceives its environment and takes actions that maximize its...
. It is claimed that pessimistic predictions made by the authors were responsible for an erroneous change in the direction of research in AI, concentrating efforts on so-called "symbolic" systems, and contributing to the so-called AI winter
AI winter
In the history of artificial intelligence, an AI winter is a period of reduced funding and interest in artificial intelligence research. The process of hype, disappointment and funding cuts are common in many emerging technologies , but the problem has been particularly acute for AI...
. This decision, supposedly, proved to be unfortunate in the 1980s, when new discoveries showed that the prognostics in the book were wrong.
The book contains a number of mathematical proofs regarding perceptrons, and while it highlights some of perceptrons' strengths, it also shows some previously unknown limitations. The most important one is related to the computation of some predicates, as the XOR function, and also the important connectedness predicate. The problem of connectedness is illustrated at the awkwardly colored cover of the book, intended to show how humans themselves have difficulties in computing this predicate.
The XOR affair
Critics of the book state that the authors imply that, since a single artificial neuron is incapable of implementing some functions such as the XOR logical function, larger networks also have similar limitations, and therefore should be dropped. Later research on three-layered perceptrons showed how to implement such functions, therefore saving the technique from obliteration.There are many mistakes in this story. Although a single neuron can in fact compute only a small number of logical predicates, it was widely known that networks of such elements can compute any possible boolean function. This was known by Warren McCulloch and Walter Pitts
Walter Pitts
Walter Harry Pitts, Jr. was a logician who worked in the field of cognitive psychology.He proposed landmark theoretical formulations of neural activity and emergent processes that influenced diverse fields such as cognitive sciences and psychology, philosophy, neurosciences, computer science,...
, who even proposed how to create a Turing Machine
Turing machine
A Turing machine is a theoretical device that manipulates symbols on a strip of tape according to a table of rules. Despite its simplicity, a Turing machine can be adapted to simulate the logic of any computer algorithm, and is particularly useful in explaining the functions of a CPU inside a...
with their formal neurons, is mentioned in Rosenblatt's book, and is even mentioned in the book Perceptrons. Minsky also extensively uses formal neurons to create simple theoretical computers in his book Computation: Finite and Infinite Machines.
What the book does prove is that in three-layered feed-forward perceptrons (with a so-called "hidden" or "intermediary" layer), it is not possible to compute some predicates unless at least one of the neurons in the first layer of neurons (the "intermediary" layer) is connected with a non-null weight to each and every input. This was contrary to a hope held by some researchers in relying mostly on networks with a few layers of "local" neurons, each one connected only to a small number of inputs. A feed-forward machine with "local" neurons is much easier to build and use than a larger and recurrent neural network, so researchers at the time concentrated on these instead of on more complicated models.
Analysis of the controversy
Although it is a widely available book, many scientists talk about Perceptrons only echoing what others have said, which helps to spread misconceptions about it. Minsky has even compared the book to the fictional book NecronomiconNecronomicon
The Necronomicon is a fictional grimoire appearing in the stories by horror writer H. P. Lovecraft and his followers. It was first mentioned in Lovecraft's 1924 short story "The Hound", written in 1922, though its purported author, the "Mad Arab" Abdul Alhazred, had been quoted a year earlier in...
in H. P. Lovecraft
H. P. Lovecraft
Howard Phillips Lovecraft --often credited as H.P. Lovecraft — was an American author of horror, fantasy and science fiction, especially the subgenre known as weird fiction....
's tales, a book known to many, but only read by fewhttp://www.ucs.louisiana.edu/~isb9112/dept/phil341/histconn.html. The authors talk in the expanded edition about the criticism of the book that started in the 1980s, with a new wave of research symbolized by the PDP book.
How Perceptrons was explored first by one group of scientists to drive research in AI in one direction, and then later by a new group in another direction, has been the subject of a peer-reviewed sociological study of scientific developmenthttp://www.jstor.org/pss/285702.