VirtualGL
From Wikipedia, the free encyclopedia
VirtualGL is an open source program which redirects the 3D rendering commands from Unix and Linux OpenGL applications to 3D accelerator hardware in a dedicated server and displays the rendered output interactively to a thin client located elsewhere on the network.
Contents |
[edit] The Problem
Normally, VNC and other thin client environments for Unix and Linux either do not support running OpenGL applications at all or force the OpenGL applications to be rendered without the benefit of OpenGL hardware acceleration. Remotely displaying 3D applications with hardware acceleration has traditionally required the use of "indirect rendering." Indirect rendering uses the GLX extension to X Windows to encapsulate the OpenGL commands inside of the X Windows protocol stream and ship them over the network from the application server to an X Windows server running on the user's desktop. All of the OpenGL commands are executed on the user's desktop, so the client machine must have a fair amount of CPU, memory, and graphics rendering horsepower.
Indirect rendering can be shown to perform well if the network is sufficiently fast (gigabit ethernet, for instance), if the application does not dynamically modify the geometry of the object being rendered, if the application uses display lists, and if the application does not use a great deal of texture mapping. But many OpenGL applications do not meet these criteria. To further complicate matters, it is often the case that OpenGL extensions, such as vertex/fragment programs, vertex arrays, etc. do not work in an indirect rendering environment. Some of these extensions require the ability to directly access the 3D graphics hardware and thus can never be made to work indirectly. In other cases, either the server's OpenGL library or the client's X server do not provide explicit support for the needed OpenGL extension, or the extension relies on a specific hardware configuration that is not present on the client machine.
Performing OpenGL rendering on the server circumvents the issues introduced by indirect rendering, since the application now has a fast and direct path to the 3D rendering hardware. If the rendering occurs on the server, then only the resulting 2D images must be sent to the client. Images can be delivered at the same frame rate regardless of how big the 3D data was that was used to generate them, so performing 3D rendering on the server effectively converts the 3D performance problem into a 2D performance problem. The problem then becomes how to stream 1-2 megapixels of image data over a network at interactive frame rates. But commodity technologies (HDTV, to name one) already exist to address this problem.
[edit] VirtualGL's Solution
VirtualGL uses "GLX forking" to implement server-side OpenGL rendering. Unix and Linux OpenGL applications normally send GLX commands to the X server to bind OpenGL rendering contexts to a particular X window, obtain a list of pixel formats that the X server supports, etc. But if the X server is in fact a thin client proxy application (such as VNC) that doesn't support the GLX X Windows extension, then these applications will fail to execute.
VirtualGL takes advantage of a feature in Unix and Linux that allows one to "preload" a library into an application, effectively intercepting (AKA "interposing") certain function calls that the application would normally make to shared libraries with which it is linked. Once VirtualGL is preloaded into a Unix or Linux OpenGL application, it intercepts the GLX function calls from the application and redirects them onto the server's X display, which presumably has a 3D hardware accelerator attached. In the process of redirecting the GLX calls, VirtualGL also modifies the calls in order to force OpenGL rendering to occur in off-screen pixel buffers ("Pbuffers") rather than in X windows. Meanwhile, the rest of the function calls from the application, including non-OpenGL X Windows calls, are allowed to pass through without modification.
Internally, VirtualGL's interposer engine also maintains a map of windows to Pbuffers, matches visual attributes between the destination X server and the X server on which the 3D rendering will occur, and performs a variety of other hashing functions to assure that the GLX redirection is seamless. But essentially, once the OpenGL context is established on the server's display, VirtualGL gets out of the way and allows all subsequent OpenGL commands to pass through unimpeded to the server's 3D hardware. Thus, the application can automatically use whatever OpenGL features and extensions are provided by the server's hardware and drivers.
Apart from marshalling GLX commands and managing Pbuffers, VirtualGL also reads back the rendered pixels at the appropriate time (usually by monitoring glXSwapBuffers() or glFinish()) and then draws those pixels into the appropriate X window using standard X image drawing commands. Since VirtualGL is redirecting the GLX commands away from the destination X server, it can be used both to add 3D support to X proxies (such as VNC) which lack 3D support as well as to prevent indirect OpenGL rendering from occurring in a remote X display scenario.
Using VirtualGL in concert with VNC or another X proxy allows multiple users to simultaneously run 3D applications on a single server and multiple clients to share each session. However, VNC and its ilk are tuned to handle 2D applications with large areas of solid color, few colors, and few inter-frame differences. 3D applications, on the other hand, generate images with fine-grained, complex color patterns and much less correlation between subsequent frames. The workload generated by drawing rendered images from an OpenGL application into an X window is essentially the same workload as a video player, and off-the-shelf thin client software typically lacks sufficiently fast image codecs to be able to handle this workload with interactive frame rates.
VirtualGL works around this problem in two ways:
- TurboVNC
- Direct Mode
[edit] TurboVNC
TurboVNC is an offshoot of TightVNC which accelerates the JPEG encoding paths of the latter by taking advantage of hand-tuned multimedia primitives from Intel and Sun Microsystems. When used with VirtualGL, TurboVNC is capable of streaming the output of a full-screen (1280x1024) 3D application at greater than 15 frames/second with perceptually lossless image quality on 100 Megabit ethernet. TurboVNC includes further optimizations to allow it to achieve 7-10 frames/second over broadband, with noticeable but usable image quality loss. TurboVNC also extends TightVNC to include client-side double buffering and optimized binaries for Solaris. TurboVNC and VirtualGL are used by the Texas Academic Computing Center at UT to allow users of the TeraGrid to remotely access the 3D rendering capabilities of the Maverick terascale visualization supercomputer.
[edit] Direct Mode
In Direct Mode, VirtualGL compresses the rendered 3D images in process using the same optimized JPEG codec that TurboVNC uses. VirtualGL then sends the compressed images directly to a VirtualGL Client application running on the client machine. The VirtualGL Client is responsible for decompressing the images and drawing the pixels into the appropriate X window. Meanwhile, the non-OpenGL elements of the application's display are sent over the network using the standard remote X Windows protocol and rendered on the client machine.
This approach requires that an X server be present on the client machine, and the reliance upon the remote X Windows protocol means that Direct Mode performs poorly on high-latency networks. Additionally, Direct Mode does not inherently support collaboration (multiple clients per session), since the images are being pushed to the client rather than pulled from the server. But Direct Mode does provide a completely seamless application experience, whereby every application window corresponds to a single desktop window. Direct Mode also provides better performance than TurboVNC on local area networks and reduces the server CPU load, since the 2D rendering is now occurring on the client.
The developers of VirtualGL envision the primary users of Direct Mode to be laptop users connected to the 3D server with 802.11g wireless or fast ethernet.
