Sword of Damocles

The Sword of Damocles is the nickname for the ceiling-suspended mechanical head-tracking arm of the experimental head-mounted display system built by Ivan Sutherland and his student Bob Sproull in 1968 at Harvard University. The system as a whole is widely regarded as the first virtual reality and augmented reality head-mounted display, and the name is often applied to the entire apparatus. In its original and narrower sense, recorded by Sutherland and by later historians, the phrase refers only to the large jointed metal arm that hung from the laboratory ceiling to measure the wearer's head position, which happened to look like a giant overhead cross or a blade poised above the user.

The device showed simple white wireframe graphics through see-through optics, so a viewer saw computer-drawn lines overlaid on the real room. It is the earliest example of an optical see-through stereoscopic display and a direct ancestor of modern VR and AR goggles.

The companion article Ivan Sutherland's head-mounted 3D display covers the full rendering pipeline, the perspective mathematics, and the optics of the headset in more depth. This article focuses on the device itself, its tracking hardware, and the origin and meaning of the nickname.

Name and meaning

The phrase "sword of Damocles" comes from a moral story retold by the Roman writer Cicero. Damocles was a flatterer at the court of Dionysius the Elder, ruler of Syracuse in the 4th century BCE. When Damocles praised the king's good fortune, Dionysius offered to let him sit on the throne, then had a sharp sword hung above the seat by a single horsehair so that Damocles would feel the constant danger that comes with power. In modern English the phrase stands for a looming threat or a sense of impending peril.

The link to Sutherland's machine is visual rather than literary. The mechanical tracking arm was a tall, jointed metal structure bolted to the ceiling directly above the wearer's head, and to people in the laboratory it resembled a weapon hanging overhead, much like the sword in the story. Sutherland has described the name as a joke for that overhead mechanical assembly, which looked like a giant cross suspended above the user. Stephen LaValle, in his virtual reality textbook, notes that the nickname properly belongs to the mechanical head-position sensor alone, and that many later references mistakenly extend it to the whole system.

Background

Sutherland had set out the concept years before he built the hardware. His 1965 essay "The Ultimate Display" imagined a room in which a computer could control the existence of matter, a chair a person could sit in and a bullet that could be fatal, framing the display as a window into a mathematical world. He carried out early three-dimensional display experiments in late 1966 and early 1967 at the MIT Lincoln Laboratory, using the TX-2 computer and a crude optical system that fed an image to only one eye. That early rig had no clipping, so any line that ran partly off the edge of the screen vanished entirely, yet observers reported that the three-dimensional illusion was already convincing.

The working binocular system was then built at Harvard University, where Sutherland was a faculty member before moving to the University of Utah. He described the project in the paper "A head-mounted three dimensional display," presented at the Fall Joint Computer Conference in 1968. The paper's title note states plainly that the work was performed at Harvard, funded in part by the Advanced Research Projects Agency and the Office of Naval Research, and that the earlier Lincoln Laboratory work was also ARPA supported. Photographs of the apparatus, used by Sutherland in his talks, were taken in the summer of 1968 and show the project member H. Quintin Foster Jr. wearing the display.

Several people built parts of the system. In the paper's acknowledgments Sutherland credits Robert (Bob) Sproull, then a Harvard student, with designing and building most of the clipping divider; a Harvard staff member with the matrix multiplier; Ted Lee and Dan Cohen with the demonstration software; Charles Seitz and Stylianos Pezaris with the ultrasonic sensor built at Lincoln Laboratory; and Quintin Foster, who supervised construction and debugging as project engineer.

The display

The fundamental idea, in Sutherland's words, is to present the user with a perspective image which changes as he moves, so that a flat picture on each eye produces the same shifting view a real solid object would. The headset was a pair of special spectacles holding two tiny cathode ray tube displays, one for each eye, driven by an analog line generator. Each CRT produced a glowing picture about half an inch square. The optics magnified that picture into a virtual image roughly eighteen inches in front of each eye, and the user saw about a 40 degree field of view of the computer-drawn graphics.

What made the device see-through, and therefore an early form of augmented reality, was a set of half-silvered mirrors in prisms in front of the eyes. These let the wearer see the images from the cathode ray tubes and the objects in the real room at the same time. Sutherland noted that the displayed lines could be made to hang in empty space or to coincide with real surfaces such as desk tops, walls, or the keys of a typewriter. Because no general-purpose computer of the period was fast enough to draw the moving picture directly, the system used purpose-built digital hardware, including a matrix multiplier and a clipping divider, to compute and refresh the view; it could draw about 3000 lines at 30 frames per second.

The graphics were transparent wireframe figures rather than solid shaded objects. Demonstrations included a single cube floating in front of the user, a "room" with four walls labeled N, S, E and W along with a ceiling and a floor that the wearer could appear to stand inside and look around by turning the head, and the wire model of a cyclohexane molecule. The first display application was the floating cube.

Head tracking

For the perspective image to update correctly, the computer had to know where the wearer's head was and which way it pointed. Sutherland built two different head-position sensors for this, and the contrast between them is the source of the famous nickname.

The first was the mechanical arm, the literal Sword of Damocles. It hung from the laboratory ceiling and was free to swing about a vertical pivot in its ceiling mount. The arm carried a universal joint at the top and another at the bottom, plus a sliding center section, which together provided the six independent motions, three of position and three of orientation, needed to track a head fully. A digital shaft encoder at each joint reported its angle to the computer. The mechanical arm was, in Sutherland's description, rather heavy and uncomfortable to wear, but it gave a sure and reliable measurement of head position, so it was the sensor the demonstrations depended on.

The second sensor was a continuous-wave ultrasonic system, which LaValle records was nicknamed the "shower stall" for the frame of hardware around the user. Three ultrasonic transmitters mounted on the headset emitted at 37, 38.6 and 40.2 kHz, and four receivers in a square array in the ceiling picked up the sound, giving twelve sound paths whose phase shifts the computer read to work out head position. Sutherland chose continuous-wave ultrasound rather than the pulsed approach of the earlier Lincoln Wand because narrow-band transducers were inexpensive and avoided confusion with the pulsed noise that office equipment such as typewriters produced. The trade-off was an ambiguity in the measurement: because the wavelength of 40 kHz sound in air is roughly a third of an inch, the readings repeated every third of an inch and the system had to track continuous changes in phase to stay locked on. He reported some encouraging results but wrote that a full account of the ultrasonic sensor was not yet possible, and the early Lincoln Laboratory version had worked only for a few minutes before cumulative errors became objectionable. In practice the mechanical arm was the dependable tracker and the ultrasonic unit remained experimental.

The working volume allowed by the equipment was about six feet across and three feet high. Within it the user could move freely, turn completely around, and tilt the head up or down by roughly forty degrees before the sensor lost track.

Limitations

The system was primitive by later standards. The pictures were thin wireframe outlines with no hidden-line removal in the head-mounted demonstrations, no surfaces, no color and no texture, so a transparent shape could be read more than one way. Sutherland recounts a cube placed above a larger cube that looked like a chimney on a house, but seen from certain angles appeared to sit inside the building rather than on its roof, because the wireframe gave no cue to which face was nearer. The optics introduced a pin-cushion distortion of about three percent across the field. The mechanical arm was heavy and constrained the wearer to a small area directly beneath it, and the ultrasonic alternative was not yet trustworthy. The whole rig had to be tethered to room-sized racks of special-purpose hardware, since the graphics could not be generated by an ordinary computer of the day.

Significance

Despite its crudeness, the Sword of Damocles system is generally treated as the first virtual reality and augmented reality head-mounted display, and the first to be driven by interactive computer graphics with head tracking. Because the optics were see-through and the wireframe lines were overlaid on the real room, the experience sits closer to augmented reality than to the fully enclosed virtual reality of later headsets, and the device is frequently cited as the first functional optical see-through AR system. Earlier stereoscopic head-worn viewers existed, notably Morton Heilig's Telesphere Mask patented in 1960, but those presented fixed or filmed images and did not respond to the wearer's movements with computer-generated graphics.

The ideas it introduced, a head-tracked stereoscopic display that redraws the scene as the viewer moves, are the foundation of every modern VR and AR headset. The phrase "Sword of Damocles" has since become shorthand in the field for this pioneering 1968 apparatus, even though Sutherland coined it only for the overhead mechanical arm.