Visual routine
Encyclopedia
A visual routine is a means of extracting information from a visual scene. In his studies on human visual cognition
, Shimon Ullman
proposed that the human visual system
's task of perceiving shape properties and spatial relations is split into two successive stages: an early "bottom-up
" state during which base representations are generated from the visual input, and a later "top-down" stage during which high-level primitives dubbed "visual routines" extract the desired information from the base representations. In humans, the base representations generated during the bottom-up stage correspond to retinotopic maps (more than 15 of which exist in the cortex) for properties like color, edge orientation, speed of motion, and direction of motion. These base representations rely on fixed operations performed uniformly over the entire field of visual input, and do not make use of object-specific knowledge, task-specific knowledge, or other higher-level information.
The visual routines proposed by Ullman are high-level primitives which parse the structure of a scene, extracting spatial information from the base representations. These visual routines are composed of a sequence of elementary visual operators specific to the task at hand. Visual routines differ from the fixed operations of the base representations in that they are not applied uniformly over the entire visual field
--- rather, they are only applied to objects or areas specified by the routines. Ullman lists the following as examples of visual operators: shifting the processing focus, indexing a salient item for further processing, spreading activation over an area delimited by boundaries, tracing boundaries, and marking a location or object for future reference. When combined into visual routines, these elementary operators can be used to perform relatively sophisticated spatial tasks such as counting the number of objects satisfying a certain property, or recognizing a complex shape.
A number of researchers have implemented visual routines for processing camera images, to perform tasks like determining the object a human in the camera image is pointing at. Researchers have also applied the visual routines approach to artificial map representations, for playing real-time 2D video games. In those cases, however, the map of the video game was provided directly, alleviating the need to deal with real-world perceptual tasks like object recognition
and occlusion
compensation.
Cognition
In science, cognition refers to mental processes. These processes include attention, remembering, producing and understanding language, solving problems, and making decisions. Cognition is studied in various disciplines such as psychology, philosophy, linguistics, and computer science...
, Shimon Ullman
Shimon Ullman
Shimon Ullman is a professor of computer science at the Weizmann Institute of Science, Israel. Ullman's main research area is the study of vision processing by both humans and machines. Specifically, he focuses on object and facial recognition, and has made a number of key insights in this field...
proposed that the human visual system
Visual system
The visual system is the part of the central nervous system which enables organisms to process visual detail, as well as enabling several non-image forming photoresponse functions. It interprets information from visible light to build a representation of the surrounding world...
's task of perceiving shape properties and spatial relations is split into two successive stages: an early "bottom-up
Bottom-up
Bottom-up may refer to:* In business development, a bottom-up approach means that the adviser takes the needs and wishes of the would-be entrepreneur as the starting point, rather than a market opportunity ....
" state during which base representations are generated from the visual input, and a later "top-down" stage during which high-level primitives dubbed "visual routines" extract the desired information from the base representations. In humans, the base representations generated during the bottom-up stage correspond to retinotopic maps (more than 15 of which exist in the cortex) for properties like color, edge orientation, speed of motion, and direction of motion. These base representations rely on fixed operations performed uniformly over the entire field of visual input, and do not make use of object-specific knowledge, task-specific knowledge, or other higher-level information.
The visual routines proposed by Ullman are high-level primitives which parse the structure of a scene, extracting spatial information from the base representations. These visual routines are composed of a sequence of elementary visual operators specific to the task at hand. Visual routines differ from the fixed operations of the base representations in that they are not applied uniformly over the entire visual field
Visual field
The term visual field is sometimes used as a synonym to field of view, though they do not designate the same thing. The visual field is the "spatial array of visual sensations available to observation in introspectionist psychological experiments", while 'field of view' "refers to the physical...
--- rather, they are only applied to objects or areas specified by the routines. Ullman lists the following as examples of visual operators: shifting the processing focus, indexing a salient item for further processing, spreading activation over an area delimited by boundaries, tracing boundaries, and marking a location or object for future reference. When combined into visual routines, these elementary operators can be used to perform relatively sophisticated spatial tasks such as counting the number of objects satisfying a certain property, or recognizing a complex shape.
A number of researchers have implemented visual routines for processing camera images, to perform tasks like determining the object a human in the camera image is pointing at. Researchers have also applied the visual routines approach to artificial map representations, for playing real-time 2D video games. In those cases, however, the map of the video game was provided directly, alleviating the need to deal with real-world perceptual tasks like object recognition
Object recognition
Object recognition in computer vision is the task of finding a given object in an image or video sequence. Humans recognize a multitude of objects in images with little effort, despite the fact that the image of the objects may vary somewhat in different view points, in many different sizes / scale...
and occlusion
Occultation
An occultation is an event that occurs when one object is hidden by another object that passes between it and the observer. The word is used in astronomy . It can also refer to any situation wherein an object in the foreground blocks from view an object in the background...
compensation.
Resources
- Automated creation of visual routines using genetic programming(M.P. Johnson)
- Ullman's Visual Routines, and Tekkotsu Sketches(David S. Touretzky, Ethan Tira-Thompson)
- Visual routines for eye location using learning and evolution(J. Huang, H. Wechsler)
- Visual routines for real-time monitoring of vehicle behavior(Marco Aste, Massimo Rossi, Roldano Cattoni, Bruno Caprile)
- Visual routines(Garbis Salgian)