In computing

Computing

Computing is usually defined as the activity of using and improving computer hardware and software. It is the computer-specific part of information technology...

, a pointing device gesture or mouse gesture is a way of combining pointing device

Pointing device

A pointing device is an input interface that allows a user to input spatial data to a computer...

 movements and click

Point-and-click

Point-and-click is the action of a computer user moving a cursor to a certain location on a screen and then pressing a mouse button, usually the left button , or other pointing device...

s which the software recognizes as a specific command. Pointing device gestures can provide quick access to common functions of a program. They can also be useful for people who have difficulties typing on a keyboard

Computer keyboard

In computing, a keyboard is a typewriter-style keyboard, which uses an arrangement of buttons or keys, to act as mechanical levers or electronic switches...

. For example, in a web browser

Web browser

A web browser is a software application for retrieving, presenting, and traversing information resources on the World Wide Web. An information resource is identified by a Uniform Resource Identifier and may be a web page, image, video, or other piece of content...

, the user could navigate to the previously viewed page by pressing the right pointing device button, moving the pointing device briefly to the left, then releasing the button.

History

The first pointing device gesture, the "drag," was introduced by Apple to replace a dedicated "move" button on mice shipped with its Macintosh and Lisa computers. Dragging involves holding down a pointing device button while moving the pointing device; the software interprets this as an action distinct from separate clicking and moving behaviors. Unlike most pointing device gestures, it does not involve the tracing of any particular shape. Although the "drag" behavior has been adopted in a huge variety of software packages, few other gestures have been as successful.

Current use

, most programs do not support gestures other than the drag operation. Each program that recognizes pointing device gestures does so in its own way, sometimes allowing for very short mouse movement distances to be recognized as gestures, and sometimes requiring very precise emulation of a certain movement pattern (e.g. circle). Some implementations allow users to customize these factors.

Some video games have used gestures. For example, in the Myth

Myth (computer game)

Myth is a series of real-time tactics computer games. The games are:*Myth: The Fallen Lords*Myth II: Soulblighter*Myth III: The Wolf Age...

real-time tactics

Real-time tactics

Real-time tactics or RTT is a subgenre of tactical wargames played in real-time simulating the considerations and circumstances of operational warfare and military tactics...

 series, originally created by Bungie Software, players use them to order battlefield units to face in a desired direction. Another game using gestures is Lionhead

Lionhead Studios

Lionhead Studios is a British computer game development company led by industry veteran Peter Molyneux, and acquired by Microsoft Game Studios in April 2006. Lionhead started as a breakaway from the developer Bullfrog, which was also founded by Molyneux. Lionhead's first game was Black & White, a...

's Black & White. The game Arx Fatalis

Arx Fatalis

Arx Fatalis is a partially open source first-person role-playing video game for the Xbox and PC, released on November 2002 by Arkane Studios, a video game developer based in Lyon, France....

uses mouse gestures for drawing runes in the air to cast spells. Several Nintendo

Nintendo

is a multinational corporation located in Kyoto, Japan. Founded on September 23, 1889 by Fusajiro Yamauchi, it produced handmade hanafuda cards. By 1963, the company had tried several small niche businesses, such as a cab company and a love hotel....

 Wii

Wii

The Wii is a home video game console released by Nintendo on November 19, 2006. As a seventh-generation console, the Wii primarily competes with Microsoft's Xbox 360 and Sony's PlayStation 3. Nintendo states that its console targets a broader demographic than that of the two others...

 games take advantage of such a system. Ōkami

Okami

is an action-adventure video game developed by Clover Studio and published by Capcom. It was released for Sony's PlayStation 2 video game console in 2006 in Japan and North America, and 2007 in Europe and Australia...

uses a system similar to mouse gestures; the player can enter a drawing mode in which the shape they create (circle, lightning bolt, line, etc.) performs a function in the game such as creating a bomb or changing the time from night to day.
Another example is Silver where basic mouse gestures actually map attack moves and such in real-time combat.

The Opera web browser

Opera (web browser)

Opera is a web browser and Internet suite developed by Opera Software with over 200 million users worldwide. The browser handles common Internet-related tasks such as displaying web sites, sending and receiving e-mail messages, managing contacts, chatting on IRC, downloading files via BitTorrent,...

 has recognized gestures since version 5.10 (April 2001) but this feature was disabled by default. Opera browser also supports mouse chording

Mouse chording

Mouse chording is the capability of performing actions when multiple mouse buttons are held down, much like a chorded keyboard. Like mouse gestures, chorded actions may lack feedback and affordance and would therefore offer no way for users to discover possible chords without reference...

 which serves a similar function but doesn't necessitate mouse movement. First browser that used advanced mouse gestures (in 2002) was Maxthon

Maxthon

Maxthon is a free web browser for Microsoft Windows. The latest release, Maxthon 3, supports both the Trident and the WebKit rendering engines....

 where highly customizable interface allowed to assign almost every action to one of 52 mouse gestures and few mouse chords. Several mouse gesture extensions are also available for the Mozilla Firefox

Mozilla Firefox

Mozilla Firefox is a free and open source web browser descended from the Mozilla Application Suite and managed by Mozilla Corporation. , Firefox is the second most widely used browser, with approximately 25% of worldwide usage share of web browsers...

 browser. These extensions use almost identical gestures as Opera.

Some tools provide mouse gestures support in any application for Microsoft Windows

Microsoft Windows

Microsoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...

. KDE

KDE

KDE is an international free software community producing an integrated set of cross-platform applications designed to run on Linux, FreeBSD, Microsoft Windows, Solaris and Mac OS X systems...

 includes universal mouse gesture support since version 3.2.

Drawbacks

A major drawback of current gesture interaction solutions is the lack of support for two necessary user interface design principles, feedback and visibility (or affordance

Affordance

An affordance is a quality of an object, or an environment, which allows an individual to perform an action. For example, a knob affords twisting, and perhaps pushing, while a cord affords pulling...

). Feedback notification is required to indicate whether the gesture has been entered correctly by indicating the gesture recognized and the corresponding command activated, although Sensiva does approach this to some extent in providing voice notification. The other principle is visibility of gestures, providing the user some means of learning the necessary gestures and the contexts they can be used in. Both Mouse Gestures for Internet Explorer and ALToolbar Mouse Gestures display colored tracers that indicate the current motion that the user is taking to facilitate visual clues for the user. Also pie menu

Pie menu

In computer interface design, a pie menu is a circular context menu where selection depends on direction. A pie menu is made of several "pie slices" around an inactive center and works best with stylus input, and well with a mouse...

s and marking menus have been proposed as solutions for both problems, since they support learning of the available options but can also be used with quick gestures. The current version of Opera uses a pie menu to simply and non-obstructively display which mouse gestures are available and how to activate them.

One limitation with gesture interaction is the scope context in which the gestures can be used. For example each gesture has only one corresponding command for each application window.

Holding down buttons while moving the mouse can be awkward and requires some practice, since the downwards action increases friction for the horizontal motion. An optical mouse would be less susceptible to changes in behavior than a ball mouse with increased friction because the sensor does not rely on mechanical contact to sense movement; a touchpad provides no added friction with all its buttons held down with a thumb. However, it was also argued that muscular tension resulting from holding down buttons could be exploited in user interface design

User interface design

User interface design or user interface engineering is the design of computers, appliances, machines, mobile communication devices, software applications, and websites with the focus on the user's experience and interaction...

 as it gives constant feedback that the user is in a temporary state, or mode

Mode (computer interface)

In user interface design, a mode is a distinct setting within a computer program or any physical machine interface, in which the same user input will produce perceived different results than it would in other settings....

 (Buxton, 1995).

See also

• Mouse chording
  Mouse chording
  Mouse chording is the capability of performing actions when multiple mouse buttons are held down, much like a chorded keyboard. Like mouse gestures, chorded actions may lack feedback and affordance and would therefore offer no way for users to discover possible chords without reference...
  
  
• Computer accessibility
  Computer accessibility
  In human-computer interaction, computer accessibility refers to the accessibility of a computer system to all people, regardless of disability or severity of impairment...
  
  
• Drag-and-drop
  Drag-and-drop
  In computer graphical user interfaces, drag-and-drop is the action of selecting a virtual object by "grabbing" it and dragging it to a different location or onto another virtual object...
  
  
• Pie menu
  Pie menu
  In computer interface design, a pie menu is a circular context menu where selection depends on direction. A pie menu is made of several "pie slices" around an inactive center and works best with stylus input, and well with a mouse...
  
  
• Pen computing
  Pen computing
  Pen computing refers to a computer user-interface using a pen and tablet, rather than devices such as a keyboard, joysticks or a mouse....
  
  

External links to applications

Some programs and applications of mouse gestures and similar features:

Windows

• Just Gestures (freeware)
• Sensiva (shareware, has disappeared; last version (3.14) can still be found around the Internet)
• MicePlus (shareware)
• Mouse Gesture Desktop Tools (shareware)
• Brass via Mime plugin (shareware)
• Gesture Magic (free/open source application launcher)
• gMote (freeware)
• Mojo Sidekick (shareware)
• StrokeIt (free for non-profit use)
• PowerPro (Freeware - Has a gesture plugin)
• High Sign (Freeware/Open Source - Currently in development, Supports plugins)
• LiveEdge (freeware)

Mac OS X

• CocoaGestures
• FlyGesture, a freeware application
• Mojo Sidekick
• Quicksilver
  Quicksilver (software)
  Quicksilver is a computer utility software program for Mac OS X, originally developed by Blacktree Software and distributed freely. It is essentially a graphical shell for the Mac OS X operating system, allowing users to use the keyboard to rapidly perform tasks such as launching...
  
   has a feature for recognizing mouse gestures
• Sapiens
• xGestures

X11

• Easystroke has packages for Ubuntu
  Ubuntu (operating system)
  Ubuntu is a computer operating system based on the Debian Linux distribution and distributed as free and open source software. It is named after the Southern African philosophy of Ubuntu...
  
   and can be compiled on other Linux distributions
• Gestikk
• Freehand Gestures for KDE
• Configuring native strokes in FVWM
  FVWM
  The F Virtual Window Manager is a virtual window manager for the X Window System. Originally a twm derivative, FVWM has evolved into a powerful and highly configurable environment for Unix-like systems.- History:...
  
  
• KHotkeys provides native gestures support in KDE
  KDE
  KDE is an international free software community producing an integrated set of cross-platform applications designed to run on Linux, FreeBSD, Microsoft Windows, Solaris and Mac OS X systems...
  
  
• wayV
• xgestures
• xstroke

SkyOS

• SkyOS Mouse Gestures SkyOS has native support of mouse gestures

Applications

These apps support gestures on their own:

• Mouse Gesture Task Switcher for Windows. Use mouse gestures to switch between application windows. By Encsoft.
• Mouse Gesture Application Launcher for Windows. Use mouse gestures to launch applications, files.
• Mouse Gesture Window Controller for Windows. Use mouse gestures to close, maximize, minimize the active window directly.
• Documentation of mouse gestures in Maxthon
  Maxthon
  Maxthon is a free web browser for Microsoft Windows. The latest release, Maxthon 3, supports both the Trident and the WebKit rendering engines....
  
  .
• Documentation of mouse gestures in Opera browser
  Opera (web browser)
  Opera is a web browser and Internet suite developed by Opera Software with over 200 million users worldwide. The browser handles common Internet-related tasks such as displaying web sites, sending and receiving e-mail messages, managing contacts, chatting on IRC, downloading files via BitTorrent,...
  
  .
• Avant Browser
  Avant Browser
  Avant Browser is a freeware web browser from a Chinese programmer named Anderson Che, which unites the Trident layout engine built into Windows with an interface intended to be more feature-rich, flexible and ergonomic than Microsoft's Internet Explorer . It runs on Windows 2000 and above,...
  
  
• Mouse Gestures for Internet Explorer
• Mouse Gestures for Internet Explorer 7 by IE7pro
  IE7Pro
  IE7Pro is an add-on for Internet Explorer 6, 7, 8, and 9 that aims to enhance the feature set provided by the browser. IE7Pro adds features such as tab enhancement, an ad blocker and flash blocker, mouse gestures, inline search, privacy enhancements, online bookmark service, Greasemonkey-like user...
  
  
• Mouse Gestures in ALToolbar for Internet Explorer.
• Reference of gestures in Motion
  Apple Motion
  Motion is a software application produced by Apple Inc. for their Mac OS X operating system. It is used to create and edit motion graphics, titling for video production and film production, and 2D and 3D compositing for visual effects.-History:...
  
  .
• Mouse gestures in raster editor of applications based on RealWorld Designer framework.
• There are many add-ons which add the ability to Firefox
  Mozilla Firefox
  Mozilla Firefox is a free and open source web browser descended from the Mozilla Application Suite and managed by Mozilla Corporation. , Firefox is the second most widely used browser, with approximately 25% of worldwide usage share of web browsers...
  
  

Software libraries

• iGesture Open Source Java framework for pen and mouse-based gesture recognition.
• Java Swing Mouse Gestures Open source pure Java library for recognition and processing mouse gestures.
• Lipi Toolkit Open source toolkit that supports recognition of arbitrary pen and mouse-based gestures as well as handwritten characters.
• LibStroke is a stroke translation library in C/Java
• Mouse Gestures for .NET Open source .NET component for mouse gestures recognition