The Mind's Eye is a video analysis research project using artificial intelligence

Artificial intelligence

Artificial intelligence is the intelligence of machines and the branch of computer science that aims to create it. AI textbooks define the field as "the study and design of intelligent agents" where an intelligent agent is a system that perceives its environment and takes actions that maximize its...

. It is funded by the Defense Advanced Research Projects Agency

Defense Advanced Research Projects Agency

The Defense Advanced Research Projects Agency is an agency of the United States Department of Defense responsible for the development of new technology for use by the military...

.

Twelve research teams have been contracted by DARPA for the Mind's Eye: Carnegie Mellon University, Co57 Systems, Inc., Colorado State University, Jet Propulsion Laboratory/CALTECH, Massachusetts Institute of Technology, Purdue University, SRI International, State University of New York at Buffalo, TNO (Netherlands), University of Arizona, University of California Berkeley and the University of Southern California.

Mission

"The Mind's Eye program seeks to develop in machines a capability that exists only in animals: visual intelligence. This program pursues the capability to learn generally applicable and generative representations of action between objects in a scene directly from visual inputs, and then reason over those learned representations. A key distinction between this research and the state of the art in machine vision is that the latter has made continual progress in recognizing a wide range of objects and their properties - what might be thought of as the nouns in the description of a scene. The focus of Mind's Eye is to add the perceptual and cognitive underpinnings for recognizing and reasoning about the verbs in those scenes, enabling a more complete narrative of action in the visual experience."