Microsoft Talisman

From Wikipedia, the free encyclopedia

Talisman was a Microsoft project to build a new 3D graphics architecture based on quickly compositing 2D "sub-images" onto the screen, an adaptation of tiled rendering. In theory, this approach would dramatically reduce the amount of memory bandwidth required for 3D games and thereby lead to lower-cost graphics accelerators. The project took place during the introduction of the first high-performance 3D accelerators, and these quickly surpassed Talisman in both performance and price. No Talisman-based systems were ever released commercially, and the project was eventually cancelled in the late 1990s.

1 Background
2 Talisman
3 Escalante
4 The problem
5 End of Talisman
6 References

[edit] Background

Creating a 3D image for display consists of a series of steps. First, the objects to be displayed are loaded up into memory from individual "models", along with their "textures", graphics that will be applied to the outside of them in a fashion similar to painting. The display system then applies mathematical functions to transform the objects into a common coordinate system, the "world view". A compositing system then applies the textures to the objects, and then combines the results, along with various special effects, into the display.

Typical 3D accelerators of the mid-90s helped only with one step of this process, compositing the image. The cards had memory buffers for the textures that were loaded up as required, and were then instructed to composite those textures onto geometry being handed to it by the CPU. The CPU was still responsible for all of the other work, mostly the conversion of the model objects coordinates into the world coordinates, and then sending the resulting models to the card as a series of polygons. By offloading the "easy part" of the operation to the card, the CPU could spend more time working on the models, allowing the complexity of the scenes to improve. Nevertheless, the required bandwidth even for the models was fairly high.

In this mode of operation every frame was created though what was essentially brute force. As soon as one frame had completed drawing, the CPU would start the process over, sending in a new set of polygons and asking the card to composite and render them. There was no memory of the already rendered objects, so even objects that did not move in the display were nevertheless re-rendered from scratch for every frame.

A typical gaming display of the era using 640 x 480 resolution with 24-bit color, with basic 3D compositing with trilinear filtering and no anti-aliasing, would require 1,900 MB/s of memory bandwidth. The same rendering settings on a 1024 x 768 display would require 14,200 MB/s, and even basic anti-aliasing would be expected to roughly double that figure.^[1] For reference, SGI's then-current Reality Engine II machines featured a then-high memory bandwidth of about 10,000 MB/s, which was the reason these machines were widely used in 3D graphics. A typical PC of the era using AGP 2X could offer only 508 MB/s.

[edit] Talisman

The Talisman effort was Microsoft's attempt to commercialize concepts that had been experimented on for some time. In particular, the PixelFlow system developed at a Hewlett-Packard research lab at the University of North Carolina at Chapel Hill can be considered Talisman's direct "parent".^[2]

PixelFlow, and Talisman, attempted to reduce the memory bandwidth requirements primarily by leaving unchanged post-rendered objects in memory to be re-composited without change. The system used a series of memory buffers and parallel compositing engines to quickly "assemble" a display that was made up of a number of 2D images, or "tiles". This technique is immediately suitable for 2D images, as any possible change in the display mapping can be made by individual mappings on the tiles. The same is not true from 3D, however, where rotations of the user's viewpoint requires objects in the display to rotate in 3D, something that can only be simulated by modifying a 2D image.

To produce the illusion of 3D, each of the individual tiles were manipulated through an affine transform before compositing, allowing the image to appear to "move" in space, at least in terms of perspective changes. The host CPU would only be asked to re-render a particular tile when the changes in perspective were great enough that the transform could no longer result in an artifact-free image. In that case the CPU would run its transformations again, handing the card a new model to be rendered back into the buffer. Microsoft calculated that each tile could be re-used for about four frames on average, thereby reducing load on the CPU by about four times.

The Talisman image buffers were broken down into 32 x 32 pixel "chunks" that were individually rendered using the 3D objects and textures provided by the CPU. The card would determine which objects overlapped which chunks, transforming those objects into 2D objects describing the chunk, which was then compressed and stored. Pointers to the chunks were then stored in a z-ordered (front to back) list for every 32 scan-lines on the display.

To render an image, the card would pick one of the 32 scan-line sets, and retrieve any chunks that were visible given the current camera location. In the typical case many of the chunks would be obscured by other chunks, and could be ignored during compositing, saving more time. This is the reason for the z-sorting of the chunks, which allows them to be efficiently retrieved in "visibility order". The chunks were then sent into the compositing engine, which applied textures, transforms to render minor changes in perspective, then optionally filtered to add anti-aliasing and other effects, and finally displayed directly on the screen. Talisman had no analog of a framebuffer, rendering chunks on demand directly to the screen as the monitor's scan line progressed down the screen.

One concern is that the chunks cannot be cleanly "stitched together", a problem that has sometimes been visible in various videogames using software rendering. To avoid this, Talisman also stored a separate "edge buffer" for every chunk that stored an "overflow" area that would cover gaps in the mapping.

[edit] Escalante

When Talisman was first made widely public at the 1996 SIGGRAPH meeting, they promised a dramatic reduction in the cost of implementing a graphics subsystem.^[3] They planned on working with vendors to sell the concept of Talisman for inclusion into other company's display systems. That is, Talisman was hoped to be a part of a larger media chip, as opposed to an entire 3D system that would stand alone in a system. Their basic system would support 20-30,000 polygons on a 1024 x 768 display at 32 bit/pixel, with a 40 Mpixel/s polygon rendering rate and 320 Mpixel/s image layer compositing rate.

At the time, Microsoft was working with several vendors in order to develop a reference implementation known as Escalante. Samsung and 3DO were working together to design a single-chip DSP-like "Media Signal Processor" (MSP), combining Talisman functionality with additional media functionality. Cirrus Logic would provide a VLSI chip that would retrieve data placed in memory by the MSP, apply effects, and send it off for display. Known as the "Polygon Object Processor" (POP), this chip was periodically polled by another Cirrus Logic chip, the "Image Layer Compositor" (ILC), which was tied to the video circuitry. Additionally, Escalante intended to feature 4 MB of RDRAM on two 600 MHz 8-bit channels, offering 1.2 GB/s throughput. ^[4] Later Philips entered the fray with a planned new version of their TriMedia processor, which implemented most of Talisman in a single CPU, and Trident Microsystems, with similar plans.

[edit] The problem

It was in the midst of the Talisman project that the first person shooter genre started to come to the fore in gaming. This created market demand for accelerators that could be used with existing games with minimal changes. By the time the Escalante reference design was ready for production, the market forces had already resulted in a series of newer card designs with such improved performance that the Talisman cards simply couldn't compete. Cards with large amounts of RAM arranged to allow for extremely high speeds solved the bandwidth issue, simply brute forcing the problem instead of attempting to solve it though clever implementation.

Additionally, the Talisman concept required tight integration between the display system and the software using it. Unlike the new 3D cards coming to market at the time, Talisman systems would have to be able to ask the CPU to re-render portions of the image in order to update their chunks. This required the games to have a specific organization in memory in order to respond to these requests. In order to aid developers in this task, Direct3D was changed to more closely match the Talisman needs. However, for any game that had already been written, or those that didn't want to be tied to Talisman, this made the D3D system slower and considerably less interesting.

As a result of these changes, Talisman never became a commercial product. Cirrus Logic and Samsung both gave up on the system some time in 1997, leading Microsoft to abandon plans to release Escalante in 1997, and to external observers it appeared the entire project was dead.^[5]

[edit] End of Talisman

There was a brief rebirth soon after, however, when Fujitsu claimed to be working on a single-chip implementation that would be available in 1998, with rumors of similar projects at S3 Graphics and ATI Technologies.^[6] None of these systems ever shipped and Talisman was quietly killed. This was much to the delight of the 3rd party graphics accelerator vendors, as well as the people within Microsoft that supported them in the market with DirectX.

Nevertheless, several of the ideas pioneered in the Talisman system have since become common in most accelerators. In particular, texture compression is now widely used. On more recent cards, compression has also been used on the z-buffers to reduce memory demands while sorting the display. The idea of using "chunks" to sort the display has also been used in a small number of cards, referred to as tile based rendering, but like Talisman in general these have never become competitive in the desktop space due to the rapid changes in the market. However, many recent graphics processors specifically designed for mobile devices (such as cell phones) employ a tile-based approach. Only the one key idea of Talisman, asking for updates to geometry only "when needed", has not been attempted since.