VirtualGL
Encyclopedia
VirtualGL is an open source
program which redirects the 3D rendering commands from Unix
and Linux
OpenGL
applications to 3D accelerator hardware in a dedicated server and displays the rendered output interactively to a thin client
located elsewhere on the network.
and other thin client environments for Unix and Linux either do not support running OpenGL applications at all or force the OpenGL applications to be rendered without the benefit of OpenGL hardware acceleration. Remotely displaying 3D applications with hardware acceleration has traditionally required the use of "indirect rendering." Indirect rendering uses the GLX
extension to the X Window System ("X11" or "X") to encapsulate the OpenGL commands inside of the X11 protocol stream
and ship them from an application to an X display. Traditionally, the application runs on a remotely-located application server, and the X display runs on the user's desktop. In this scenario, all of the OpenGL commands are executed by the user's desktop machine, so that machine must have a fast 3D graphics accelerator. This limits the type of machine that can remotely display a 3D application using this method.
Indirect rendering can be shown to perform well if the network is sufficiently fast (Gigabit Ethernet
, for instance), if the application does not dynamically modify the geometry of the object being rendered, if the application uses display list
s, and if the application does not use a great deal of texture mapping
. Many OpenGL applications, however, do not meet these criteria. To further complicate matters, some OpenGL extensions do not work in an indirect rendering environment. Some of these extensions require the ability to directly access the 3D graphics hardware and thus can never be made to work indirectly. In other cases, the user's X display may not provide explicit support for a needed OpenGL extension, or the extension may rely on a specific hardware configuration that is not present on the user's desktop machine.
Performing OpenGL rendering on the application server circumvents the issues introduced by indirect rendering, since the application now has a fast and direct path to the 3D rendering hardware. If the 3D rendering occurs on the application server, then only the resulting 2D images must be sent to the user's desktop. Images can be delivered at the same frame rate regardless of how big the 3D data was that was used to generate them, so performing 3D rendering on the application server effectively converts the 3D performance problem into a 2D performance problem. The problem then becomes how to stream 1-2 megapixels of image data over a network at interactive frame rates, but commodity technologies (HDTV, to name one) already address this problem.
Internally, VirtualGL's interposer engine also maintains a map of windows to Pbuffers, matches visual attributes between the destination X display and the X display on which the 3D rendering will occur, and performs a variety of other hashing functions to assure that the GLX redirection is seamless. But essentially, once the OpenGL context is established on the application server's X display, VirtualGL gets out of the way and allows all subsequent OpenGL commands to pass through unimpeded to the application server's 3D hardware. Thus, the application can automatically use whatever OpenGL features and extensions are provided by the application server's hardware and drivers.
Apart from marshaling GLX commands and managing Pbuffers, VirtualGL also reads back the rendered pixels at the appropriate time (usually by monitoring
Using VirtualGL in concert with VNC or another X proxy allows multiple users to simultaneously run 3D applications on a single application server and multiple clients to share each session. However, VNC and its ilk are tuned to handle 2D applications with large areas of solid color, few colors, and few inter-frame differences. 3D applications, on the other hand, generate images with fine-grained, complex color patterns and much less correlation between subsequent frames. The workload generated by drawing rendered images from an OpenGL application into an X window is essentially the same workload as a video player, and off-the-shelf thin client software typically lacks sufficiently fast image codec
s to be able to handle this workload with interactive frame rates.
VirtualGL works around this problem in two ways:
which accelerates the Tight and JPEG encoding paths of the latter, in part by taking advantage of libjpeg-turbo, a SIMD-accelerated version of libjpeg
. On 100 Megabit Ethernet
networks, TurboVNC is capable of displaying full-screen (1280x1024-pixel) images with perceptually lossless image quality at greater than 20 frames/second. TurboVNC includes further optimizations which allow it to display full-screen images at 7-10 frames/second over broadband, with noticeably less but usable image quality. TurboVNC also extends TightVNC to include client-side double buffering
and other features targeted at 3D applications, as well as full support for Solaris
platforms. TurboVNC and VirtualGL are used by the Texas Academic Computing Center at UT
to allow users of the TeraGrid
to remotely access the 3D rendering capabilities of the Longhorn Visualization Cluster.
When using the VGL Image Transport, VirtualGL compresses the rendered 3D images in process using the same optimized JPEG codec that TurboVNC uses. VirtualGL then sends the compressed images over a dedicated TCP socket to a VirtualGL Client application running on the client machine. The VirtualGL Client is responsible for decompressing the images and drawing the pixels into the appropriate X window. Meanwhile, the non-OpenGL elements of the application's display are sent over the network using the standard remote X11 protocol and rendered on the client machine.
This approach requires that an X display be present on the client machine, and the reliance upon the remote X11 protocol for performing 2D rendering means that many applications will perform poorly when using the VGL Image Transport on high-latency networks. Additionally, the VGL Image Transport does not inherently support collaboration (multiple clients per session), since the images are being pushed to the users' machines rather than being pulled. But the use of the VGL Image Transport does provide a completely seamless application experience, whereby every application window corresponds to a single desktop window. The VGL Image Transport also reduces the server CPU load, since the 2D rendering is occurring on the client, and the VGL Image Transport allows advanced OpenGL features, such as quad-buffered stereo
, to be used.
The developers of VirtualGL envision the primary users of the VGL Image Transport to be laptop users with an 802.11g wireless or a fast Ethernet connection to the application server.
product from Sun Microsystems
, which was discontinued in April of 2009. The two open source packages were combined with a closed source plugin that allowed VirtualGL to send compressed images to Sun Ray
thin clients and another closed source package that integrated VirtualGL with Sun Grid Engine
, providing resource management and scheduling for remote 3D jobs. The combination of these packages, dubbed "Sun Shared Visualization", was available as a free download (Sun charged for support.)
v2.1 of the Scalable Visualization Array software from HP also includes components which integrate with VirtualGL and TurboVNC, allowing 3D jobs to be scheduled on and remotely displayed from a visualization cluster.
v3.0.0 of ThinLinc
is designed to work in conjunction with VirtualGL.
v2010 of EnginFrame Views supports VirtualGL as one of the remote protocol options.
Open source
The term open source describes practices in production and development that promote access to the end product's source materials. Some consider open source a philosophy, others consider it a pragmatic methodology...
program which redirects the 3D rendering commands from Unix
Unix
Unix is a multitasking, multi-user computer operating system originally developed in 1969 by a group of AT&T employees at Bell Labs, including Ken Thompson, Dennis Ritchie, Brian Kernighan, Douglas McIlroy, and Joe Ossanna...
and Linux
Linux
Linux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
OpenGL
OpenGL
OpenGL is a standard specification defining a cross-language, cross-platform API for writing applications that produce 2D and 3D computer graphics. The interface consists of over 250 different function calls which can be used to draw complex three-dimensional scenes from simple primitives. OpenGL...
applications to 3D accelerator hardware in a dedicated server and displays the rendered output interactively to a thin client
Thin client
A thin client is a computer or a computer program which depends heavily on some other computer to fulfill its traditional computational roles. This stands in contrast to the traditional fat client, a computer designed to take on these roles by itself...
located elsewhere on the network.
The problem
Normally, VNCVirtual Network Computing
In computing, Virtual Network Computing is a graphical desktop sharing system that uses the RFB protocol to remotely control another computer...
and other thin client environments for Unix and Linux either do not support running OpenGL applications at all or force the OpenGL applications to be rendered without the benefit of OpenGL hardware acceleration. Remotely displaying 3D applications with hardware acceleration has traditionally required the use of "indirect rendering." Indirect rendering uses the GLX
GLX
GLX provides the interface connecting OpenGL and the X Window System: it enables programs wishing to use OpenGL to do so within a window provided by the X Window System.-History:...
extension to the X Window System ("X11" or "X") to encapsulate the OpenGL commands inside of the X11 protocol stream
X Window System protocols and architecture
In computing, the X Window System is a network-transparent windowing system for bitmap displays. This article details the protocols and technical structure of X11.-Client–server model and network transparency:...
and ship them from an application to an X display. Traditionally, the application runs on a remotely-located application server, and the X display runs on the user's desktop. In this scenario, all of the OpenGL commands are executed by the user's desktop machine, so that machine must have a fast 3D graphics accelerator. This limits the type of machine that can remotely display a 3D application using this method.
Indirect rendering can be shown to perform well if the network is sufficiently fast (Gigabit Ethernet
Gigabit Ethernet
Gigabit Ethernet is a term describing various technologies for transmitting Ethernet frames at a rate of a gigabit per second , as defined by the IEEE 802.3-2008 standard. It came into use beginning in 1999, gradually supplanting Fast Ethernet in wired local networks where it performed...
, for instance), if the application does not dynamically modify the geometry of the object being rendered, if the application uses display list
Display list
A display list is a series of graphics commands that define an output image. The image is created by executing the commands....
s, and if the application does not use a great deal of texture mapping
Texture mapping
Texture mapping is a method for adding detail, surface texture , or color to a computer-generated graphic or 3D model. Its application to 3D graphics was pioneered by Dr Edwin Catmull in his Ph.D. thesis of 1974.-Texture mapping:...
. Many OpenGL applications, however, do not meet these criteria. To further complicate matters, some OpenGL extensions do not work in an indirect rendering environment. Some of these extensions require the ability to directly access the 3D graphics hardware and thus can never be made to work indirectly. In other cases, the user's X display may not provide explicit support for a needed OpenGL extension, or the extension may rely on a specific hardware configuration that is not present on the user's desktop machine.
Performing OpenGL rendering on the application server circumvents the issues introduced by indirect rendering, since the application now has a fast and direct path to the 3D rendering hardware. If the 3D rendering occurs on the application server, then only the resulting 2D images must be sent to the user's desktop. Images can be delivered at the same frame rate regardless of how big the 3D data was that was used to generate them, so performing 3D rendering on the application server effectively converts the 3D performance problem into a 2D performance problem. The problem then becomes how to stream 1-2 megapixels of image data over a network at interactive frame rates, but commodity technologies (HDTV, to name one) already address this problem.
VirtualGL's solution
VirtualGL uses "GLX forking" to perform OpenGL rendering on the application server. Unix and Linux OpenGL applications normally send both GLX commands and ordinary X11 commands to the same X display. The GLX commands are used to bind OpenGL rendering contexts to a particular X window, obtain a list of pixel formats that the X display supports, etc. VirtualGL takes advantage of a feature in Unix and Linux that allows one to "preload" a library into an application, effectively intercepting (AKA "interposing") certain function calls that the application would normally make to shared libraries with which it is linked. Once VirtualGL is preloaded into a Unix or Linux OpenGL application, it intercepts the GLX function calls from the application and rewrites them such that the corresponding GLX commands are sent to the application server's X display, which presumably has a 3D hardware accelerator attached. Thus, VirtualGL prevents GLX commands from being sent over the network to the user's X display or to a virtual X display ("X proxy"), such as VNC, that does not support GLX. In the process of rewriting the GLX calls, VirtualGL also redirects the OpenGL rendering into off-screen pixel buffers ("Pbuffers.") Meanwhile, the rest of the function calls from the application, including the ordinary X11 commands used to draw the application's user interface, are allowed to pass through VirtualGL without modification.Internally, VirtualGL's interposer engine also maintains a map of windows to Pbuffers, matches visual attributes between the destination X display and the X display on which the 3D rendering will occur, and performs a variety of other hashing functions to assure that the GLX redirection is seamless. But essentially, once the OpenGL context is established on the application server's X display, VirtualGL gets out of the way and allows all subsequent OpenGL commands to pass through unimpeded to the application server's 3D hardware. Thus, the application can automatically use whatever OpenGL features and extensions are provided by the application server's hardware and drivers.
Apart from marshaling GLX commands and managing Pbuffers, VirtualGL also reads back the rendered pixels at the appropriate time (usually by monitoring
glXSwapBuffers or glFinish) and then draws those pixels into application's X window using standard X image drawing commands. Since VirtualGL is redirecting the GLX commands away from the destination X display, it can be used to add accelerated 3D support to X proxies (such as VNC) as well as to prevent indirect OpenGL rendering from occurring when using a remote X display.Codec
A codec is a device or computer program capable of encoding or decoding a digital data stream or signal. The word codec is a portmanteau of "compressor-decompressor" or, more commonly, "coder-decoder"...
s to be able to handle this workload with interactive frame rates.
VirtualGL works around this problem in two ways:
- TurboVNC
- The VGL Image Transport
TurboVNC
TurboVNC is an offshoot of TightVNCTightVNC
TightVNC is a cross-platform open source remote desktop software application that uses and extends VNC's RFB protocol to control another computer's screen remotely. It was created by Constantin Kaplinsky...
which accelerates the Tight and JPEG encoding paths of the latter, in part by taking advantage of libjpeg-turbo, a SIMD-accelerated version of libjpeg
Libjpeg
libjpeg is a library written entirely in C which contains a widely-used implementation of a JPEG decoder, JPEG encoder and other JPEG utilities...
. On 100 Megabit Ethernet
Fast Ethernet
In computer networking, Fast Ethernet is a collective term for a number of Ethernet standards that carry traffic at the nominal rate of 100 Mbit/s, against the original Ethernet speed of 10 Mbit/s. Of the fast Ethernet standards 100BASE-TX is by far the most common and is supported by the...
networks, TurboVNC is capable of displaying full-screen (1280x1024-pixel) images with perceptually lossless image quality at greater than 20 frames/second. TurboVNC includes further optimizations which allow it to display full-screen images at 7-10 frames/second over broadband, with noticeably less but usable image quality. TurboVNC also extends TightVNC to include client-side double buffering
Double buffering
In computer science, multiple buffering is the use of more than one buffer to hold a block of data, so that a "reader" will see a complete version of the data, rather than a partially-updated version of the data being created by a "writer"...
and other features targeted at 3D applications, as well as full support for Solaris
Solaris Operating System
Solaris is a Unix operating system originally developed by Sun Microsystems. It superseded their earlier SunOS in 1993. Oracle Solaris, as it is now known, has been owned by Oracle Corporation since Oracle's acquisition of Sun in January 2010....
platforms. TurboVNC and VirtualGL are used by the Texas Academic Computing Center at UT
University of Texas at Austin
The University of Texas at Austin is a state research university located in Austin, Texas, USA, and is the flagship institution of the The University of Texas System. Founded in 1883, its campus is located approximately from the Texas State Capitol in Austin...
to allow users of the TeraGrid
TeraGrid
TeraGrid is an e-Science grid computing infrastructure combining resources at eleven partner sites. The project started in 2001 and operated from 2004 through 2011....
to remotely access the 3D rendering capabilities of the Longhorn Visualization Cluster.
The VGL Image Transport (formerly ("Direct Mode"))
This approach requires that an X display be present on the client machine, and the reliance upon the remote X11 protocol for performing 2D rendering means that many applications will perform poorly when using the VGL Image Transport on high-latency networks. Additionally, the VGL Image Transport does not inherently support collaboration (multiple clients per session), since the images are being pushed to the users' machines rather than being pulled. But the use of the VGL Image Transport does provide a completely seamless application experience, whereby every application window corresponds to a single desktop window. The VGL Image Transport also reduces the server CPU load, since the 2D rendering is occurring on the client, and the VGL Image Transport allows advanced OpenGL features, such as quad-buffered stereo
Quad buffering
Quad buffering is a technology for implementing stereoscopic frame rendering in computer graphics. For stereoscopic rendering, each eye must receive a separate image...
, to be used.
The developers of VirtualGL envision the primary users of the VGL Image Transport to be laptop users with an 802.11g wireless or a fast Ethernet connection to the application server.
Commercial solutions using VirtualGL
VirtualGL and TurboVNC were core components of the Sun Visualization SystemSun Visualization System
Sun Visualization System was a sharable visualization solution introduced by Sun Microsystems in January of 2007. It used other Sun technologies, including Sun servers, Solaris, Sun Ray Ultra-Thin Clients, and Sun Grid Engine...
product from Sun Microsystems
Sun Microsystems
Sun Microsystems, Inc. was a company that sold :computers, computer components, :computer software, and :information technology services. Sun was founded on February 24, 1982...
, which was discontinued in April of 2009. The two open source packages were combined with a closed source plugin that allowed VirtualGL to send compressed images to Sun Ray
Sun Ray
The Sun Ray from Oracle is a stateless thin client solution aimed at corporate environments, originally introduced by Sun Microsystems in September 1999...
thin clients and another closed source package that integrated VirtualGL with Sun Grid Engine
Sun Grid Engine
Oracle Grid Engine, previously known as Sun Grid Engine , previously known as CODINE or GRD , is an open source batch-queuing system, developed and supported by Sun Microsystems...
, providing resource management and scheduling for remote 3D jobs. The combination of these packages, dubbed "Sun Shared Visualization", was available as a free download (Sun charged for support.)
v2.1 of the Scalable Visualization Array software from HP also includes components which integrate with VirtualGL and TurboVNC, allowing 3D jobs to be scheduled on and remotely displayed from a visualization cluster.
v3.0.0 of ThinLinc
ThinLinc
ThinLinc is a cross-platform thin client solution developed by Cendio AB. The server software runs on Linux or Solaris. Applications and desktops from other platforms can be published using different virtualization techniques: Windows applications can be executed from a server with Remote Desktop...
is designed to work in conjunction with VirtualGL.
v2010 of EnginFrame Views supports VirtualGL as one of the remote protocol options.
See also
- X Window SystemX Window SystemThe X window system is a computer software system and network protocol that provides a basis for graphical user interfaces and rich input device capability for networked computers...
- XglXglXgl was an X server architecture designed to take advantage of modern graphics cards via their OpenGL drivers, layered on top of OpenGL via glitz. It supported hardware acceleration of all X, OpenGL and XVideo applications and graphical effects by a compositing window manager such as Compiz or...
- AIGLXAIGLXAccelerated Indirect GLX is an open source project founded by Red Hat and the Fedora community, led by Kristian Høgsberg, to allow accelerated indirect GLX rendering capabilities to the X.Org Server and DRI drivers...
- OpenGLOpenGLOpenGL is a standard specification defining a cross-language, cross-platform API for writing applications that produce 2D and 3D computer graphics. The interface consists of over 250 different function calls which can be used to draw complex three-dimensional scenes from simple primitives. OpenGL...
- VNC