Ivan Sutherland
More actions
Ivan Sutherland (Ivan Edward Sutherland, born 16 May 1938 in Hastings, Nebraska) is an American computer scientist and inventor who created the computer-driven head-mounted display and pioneered direct interaction with graphics on a computer screen.[1] His 1963 MIT doctoral program Sketchpad let a user draw and edit pictures directly on a display with a light pen, and his 1965 essay "The Ultimate Display" described a computer display that could serve "as many senses as possible".[2][3]
He created Sketchpad, which is a drawing program using direct interaction.[4]
His research project for the HMD is titled, "A Head-Mounted Three-Dimensional Display".[5] The work was performed at Harvard University with student Bob Sproull and others, and the paper was presented at the 1968 Fall Joint Computer Conference, where it received a prize paper award.[5][6]
The HMD displays graphics generated from a computer in real time, instead of a camera feed. This is the point of novelty: speaking about the project in 2018, Sutherland said his contribution was to recognize that "instead of a camera, we could substitute a computer."[7]
Sutherland later co-founded the graphics company Evans & Sutherland with David C. Evans, led the computer science department at Caltech, co-founded the consulting firm Sutherland, Sproull and Associates (which became the core of Sun Microsystems Laboratories), and with his wife Marly Roncken established the Asynchronous Research Center at Portland State University.[6] He received the 1988 ACM Turing Award "for his pioneering and visionary contributions to computer graphics, starting with Sketchpad, and continuing after" and the 2012 Kyoto Prize in Advanced Technology, whose citation credits his head-mounted display with "opening up the new field of virtual reality."[1][8]
Early life and education
Sutherland was born in Hastings, Nebraska. His father was a civil engineer from New Zealand who had earned a PhD in London, and his mother was a teacher; his elder brother, Bert Sutherland, also became a computer scientist.[1][9] As a boy he programmed SIMON, a small relay computer lent to the family by its designer Edmund Berkeley; his program made the machine divide, which required adding a conditional stop to its instruction set.[1][9]
After graduating from Scarsdale High School in New York in 1955, he studied electrical engineering at the Carnegie Institute of Technology on a George Westinghouse scholarship (B.S., 1959), then at the California Institute of Technology (M.S., 1960) and the Massachusetts Institute of Technology (Ph.D., 1963).[1][6] His thesis supervisor was Claude Shannon, and his doctoral committee also included Marvin Minsky and Steven Coons.[1][9] He later received honorary degrees from Harvard University (M.A., 1966), the University of North Carolina (D.Sc., 1986), the University of Utah (2000) and Carnegie Mellon University (2003).[6]
Career
Sketchpad
In 1963, Ivan Sutherland's PhD dissertation project at MIT involved Sketchpad, which was the first real-time CAD system, which used a pen on a screen.[10]
Sketchpad's full title is "Sketchpad: A Man-Machine Graphical Communication System"; the dissertation was submitted to MIT in January 1963.[2]
Sketchpad does not enable drawing something in three dimensions simultaneously.[11]
The program ran on the TX-2, an experimental computer built by Wesley Clark at MIT Lincoln Laboratory, which had a nine-inch CRT and a light pen. The light pen supplied coordinates while keyboard commands told the program what to draw; earlier drawn objects could be recalled, rotated, scaled and moved, and drawings could be stored on magnetic tape.[1] The 2003 electronic edition of the thesis notes that executable versions were limited to that customized Lincoln Laboratory machine, so Sketchpad spread through its ideas, a film of the program in use and a 1963 conference paper; it describes Sketchpad as one of the first graphical user interfaces and a precedent for clicking and dragging objects on screen.[2] Writing in IEEE Spectrum in 2023, David C. Brock quoted Sutherland's own summary: "The Sketchpad system makes it possible for a man and a computer to converse rapidly through the medium of line drawings."[12]
Military service and ARPA
After MIT, Sutherland served in the U.S. Army in 1963 as a technical operations officer with Project MICHIGAN at the University of Michigan, then worked as an electrical engineer at the National Security Agency from 1963 to 1964.[6] In 1964, at age 26, he succeeded J. C. R. Licklider as head of the Information Processing Techniques Office (IPTO) of the Advanced Research Projects Agency, a post he held until 1966.[1][6] According to his CV, IPTO contracts during his tenure included Project MAC at MIT, the Illiac IV project and programs led by Alan Perlis at Carnegie Mellon, David Evans at the University of California, Berkeley and Wesley Clark at Washington University in St. Louis.[6]
"The Ultimate Display"
While at ARPA, Sutherland presented "The Ultimate Display" at the IFIP Congress in New York in May 1965.[3][6] The short paper calls a computer display "a looking glass into a mathematical wonderland" and argues that such a display "should serve as many senses as possible." It proposes a "kinesthetic display" in which the force needed to move a joystick is computer controlled, a form of force feedback now discussed as haptic feedback. It predicts that machines to sense and interpret eye motion (eye tracking) "can and will be built" and suggests, as an experiment, making "the display presentation depend on where we look." Its final paragraph begins: "The ultimate display would, of course, be a room within which the computer can control the existence of matter. A chair displayed in such a room would be good enough to sit in."[3]
Sutherland's motivation was to build an "ultimate display" that would allow people to interact with computers by being inside 3D computer graphics and to use direct interaction.[3][13]
Head mounted display
Sutherland was influenced to build a head-mounted display by a display he saw as part of a helicopter. He saw a head-mounted display connected to a camera system in a helicopter developed by Bell Helicopter Company.[14][15] In his 2017 Computer History Museum oral history he said he visited Bell Helicopter while at DARPA, where the company had mounted an infrared camera under a helicopter to help the pilot land at night, and that he went to Harvard knowing he wanted to build a head-mounted display.[9] Sutherland received a copy of the display system and invented a head-mounted computer-driven display system: he recalled that the head-mounted optics had been designed by Perkin-Elmer and built for Bell Helicopter's pilot experiments, and that his group bought another copy using money from a standing Bell Labs agreement with Harvard.[9] He also recalled a research contract from the CIA of about $80,000 among the project's grants; the 1968 paper credits support from ARPA, the Office of Naval Research and Bell Telephone Laboratories.[9][5]
The display system Sutherland invented used a mounting system called the Sword of Damocles, which is a position measuring device, so that the computer knows where the user is facing so as to generate a picture of a virtual scene at the correct perspective.[15] It was designed at Harvard. This HMD was part of an augmented reality system which displayed a 3D wire-frame cube that appeared to hang in the room.[10][5] The ACM biography says the nickname came from the apparatus being suspended from the ceiling above the user's head, while the BBVA Foundation attributes it to the device's weight.[1][16]
Sutherland did work at Harvard using a PDP-1.[14] He said the machine came from the Air Force and was installed in Harvard's Cruft Hall, arranged by Harvey Brooks rather than by Sutherland himself.[9]
The 1968 paper, written after Sutherland had moved to the University of Utah, sets out how the system worked.[5]
- Preliminary tests: in late 1966 and early 1967 Sutherland ran experiments at MIT Lincoln Laboratory with a crude optical system for one eye, an ultrasonic tracking head sensor and perspective computed in software on the TX-2. Even then, he wrote, "the three dimensional illusion was real."
- Optics: the Harvard headset held two miniature cathode ray tubes whose pictures were magnified into virtual images about 18 inches in front of each eye, with a 40 degree field of view. Half-silvered mirrors let the user see the room at the same time, so graphics could be made to "hang disembodied in space or to coincide with maps, desk tops, walls, or the keys of a typewriter." This makes it an optical see-through head-mounted display.
- Head tracking: two sensors were tried. A mechanical arm hanging from the ceiling, with two universal joints, a sliding center section and digital shaft encoders, measured all six motions of the head; the paper calls it "rather heavy and uncomfortable to use." A continuous-wave ultrasonic sensor used three transmitters on the headset (37, 38.6 and 40.2 kHz) and four ceiling receivers, measuring phase over twelve paths. The working volume was about six feet in diameter and three feet high, with head tilt of about forty degrees.
- Graphics hardware: a special-purpose digital matrix multiplier (designed by Charles Seitz) and a "clipping divider" (simulated, mostly designed, partly built and debugged by Bob Sproull) computed the perspective image, because no general-purpose computer of the time was fast enough. The system could show 3,000 lines at 30 frames per second, as transparent wire-frame drawings without hidden-line removal.
- Results: test scenes included a room with walls marked N, S, E and W and a small cube in the center of the operating area. The paper names "the favorable response of users to good stereo" as the project's biggest surprise; a mechanical adjustment accommodated different pupil separations, and observers "uniformly remark on the realism of the resulting images."
The paper also thanks Stylianos Pezaris, Ted Lee, Dan Cohen, Quintin Foster and Stewart Ogden. In August 2018, at a SIGGRAPH panel marking the system's 50th anniversary, Sutherland appeared with original team members Charles Seitz, Robert Sproull and Quintin Foster. He said there: "I didn't give the name 'virtual reality' to this phenomenon. Jaron Lanier named it some years later."[7] The full system is described in Ivan Sutherland's head-mounted 3D display.
At Harvard, work with student Danny Cohen also produced the Cohen-Sutherland line clipping algorithm, which removes the parts of lines that fall outside the viewing region.[1]
University of Utah and Evans & Sutherland
In 1968 Sutherland moved to the University of Utah, where David C. Evans, whom he had met at the University of California, Berkeley, was building a computer science department focused on 3D computer graphics, and the two founded the Evans and Sutherland Computer Corporation in Salt Lake City.[1][12][17] He was the company's vice president and chief scientist from 1968 to 1974 and stayed on its board for 35 years, until he reached age 65; his CV states that the firm grew to a US$200 million per year business making high-performance computer graphics equipment.[6] Utah students during this period included Jim Clark, who designed a virtual reality system before founding Silicon Graphics, and Henri Gouraud, who devised Gouraud shading; Sutherland also sat on the PhD committees of Alan Kay and Edwin Catmull.[1]
Publications and patents from the Utah years include a 1969 Applied Optics paper with R. Land on a real-time color stereo computer display, the 1974 "Twinkle Box" paper on three-dimensional computer input devices (with Robert Burton), the Sutherland-Hodgman reentrant polygon clipping algorithm (1974), and "A Characterization of Ten Hidden-Surface Algorithms" (1974) with Sproull and Robert Schumacker. He shared a 1973 patent with Evans for an "Incremental Position-Indicating System".[6]
RAND, Caltech and walking machines
Sutherland co-founded the Picture Design Group in Santa Monica in 1974 and was a senior technical staff member at the RAND Corporation in 1975 and 1976.[6] From 1976 to 1980 he was the Fletcher Jones Professor of Computer Science and founding head of the computer science department at Caltech, where the group's work helped academia teach integrated circuit design.[6][17] (The ACM biography gives 1974 to 1978 for the Caltech appointment.[1]) With Carver Mead he wrote "Microelectronics and Computer Science" for Scientific American in 1977.[6]
From 1980 to 1984 he was a visiting scientist at the Robotics Institute of Carnegie Mellon University, where he built a six-legged hydraulic walking machine that appeared on the cover of the January 1983 Scientific American; the issue carried "Machines That Walk" by Marc Raibert and Sutherland.[6][1] He was a visiting professor at Imperial College London from 1985 to 1987.[6]
Sutherland, Sproull and Associates and Sun Microsystems
In 1980 Sutherland and Bob Sproull incorporated the consulting firm Sutherland, Sproull and Associates; the same year, with other partners, he started the venture capital firm Advanced Technology Ventures.[6] Sun Microsystems bought Sutherland, Sproull and Associates in 1990 to form the nucleus of its research laboratory, and Sutherland was a vice president and Fellow at Sun Microsystems Laboratories from 1991 to 2009.[6][1]
His later research centered on asynchronous (self-timed) digital circuits. His Turing Award lecture, "Micropipelines", was published in Communications of the ACM in June 1989, and he co-wrote the 1999 book Logical Effort with Sproull and David Harris.[6] One of his patents from this period, shared with B. Reichlen (U.S. 6,351,261, 2002), covers a virtual reality system with a frame buffer that stores several viewpoints for the user to select.[6]
Portland State University
Sutherland was a visiting scientist at the University of California, Berkeley from 2005 to 2007.[6] He married Marly Roncken in 2006, and in 2008 the couple established the Asynchronous Research Center at Portland State University, which works on self-timed computers that do without a central clock.[1][6] His October 2024 CV lists him as a visiting scientist at Portland State, a part-time consultant to Oracle Laboratory and to the U.S. government, and co-author of 2023 papers in IEEE Transactions on Applied Superconductivity on test circuits for superconducting (RSFQ) logic. The CV quotes him: "I'm planning not to retire."[6] He has two children.[1]
Role in virtual and augmented reality
Award committees and museums treat the Harvard head-mounted display as a starting point for the field. The Kyoto Prize citation calls the Sword of Damocles "the world's first head-mounted 3D display system, thus opening up the new field of virtual reality."[8] The BBVA Foundation's 2019 announcement of his Frontiers of Knowledge Award says its citation describes the headset as a precursor of "today's virtual and augmented reality systems such as HoloLens, Rift and Vive."[16] The National Inventors Hall of Fame, which inducted him in 2016, credits him with developing the first virtual reality head-mounted display system in 1967, with student Robert Sproull, while he was an associate professor at Harvard.[18] The Computer History Museum's fellow citation says he introduced concepts including 3D computer modeling, visual simulations, computer-aided design and virtual reality.[17]
The 1968 system combined head-coupled perspective rendering, tracking of the head in six degrees of freedom, stereo images with an adjustable eye separation, and see-through optics that mixed graphics with the real room.[5] At the 2018 SIGGRAPH panel Sutherland said, "I never thought much about the future. I couldn't imagine what people would do with this thing."[7]
Awards and honors
| Year | Honor |
|---|---|
| 1968 | Prize Paper, Fall Joint Computer Conference[6] |
| 1972 | First Zworykin Award, National Academy of Engineering[6] |
| 1973 | Member, National Academy of Engineering[6] |
| 1975 | Award for Outstanding Accomplishment, IEEE Systems, Man and Cybernetics Society[6] |
| 1978 | Member, National Academy of Sciences[6] |
| 1983 | First Steven Anson Coons Award, ACM SIGGRAPH[6] |
| 1986 | IEEE Emanuel R. Piore Award[6] |
| 1988 | ACM A.M. Turing Award[1] |
| 1993 or 1994 | ACM Software System Award for Sketchpad (ACM lists 1993; his CV lists 1994)[1][6] |
| 1994 | Pioneer Award, Electronic Frontier Foundation; ACM Fellow[1] |
| 1996 | Price Waterhouse Information Technology Leadership Award for Lifetime Achievement (Smithsonian); Certificate of Merit, Franklin Institute[6] |
| 1998 | IEEE John von Neumann Medal[6][17] |
| 2005 | Fellow, Computer History Museum[17] |
| 2012 | Kyoto Prize in Advanced Technology (Information Science)[8] |
| 2015 | Founders Award, Proto Awards[6] |
| 2016 | National Inventors Hall of Fame[18] |
| 2018 | BBVA Foundation Frontiers of Knowledge Award in Information and Communication Technologies (11th edition, announced February 2019)[6][16] |
See also
References
- ↑ 1.00 1.01 1.02 1.03 1.04 1.05 1.06 1.07 1.08 1.09 1.10 1.11 1.12 1.13 1.14 1.15 1.16 1.17 1.18 1.19 Robert Burton. "Ivan Sutherland". A.M. Turing Award. Association for Computing Machinery. https://amturing.acm.org/award_winners/sutherland_3467412.cfm. Retrieved 2026-10-06.
- ↑ 2.0 2.1 2.2 Ivan Edward Sutherland (2003-09). "Sketchpad: A man-machine graphical communication system". University of Cambridge Computer Laboratory, Technical Report UCAM-CL-TR-574. https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-574.pdf. Retrieved 2026-10-06.
- ↑ 3.0 3.1 3.2 3.3 Ivan E. Sutherland (1965-05). "The Ultimate Display". Proceedings of the IFIP Congress 1965, vol. 2, pp. 506-508. https://worrydream.com/refs/Sutherland_1965_-_The_Ultimate_Display.pdf. Retrieved 2026-10-06.
- ↑ "Sketchpad". YouTube. 2024-03-06. https://www.youtube.com/watch?v=eV8wSXF4NFA. Retrieved 2024-08-28.
- ↑ 5.0 5.1 5.2 5.3 5.4 5.5 Ivan E. Sutherland (1968). "A head-mounted three dimensional display". Proceedings of the AFIPS Fall Joint Computer Conference, vol. 33, part I, pp. 757-764. Thompson Books. doi:10.1145/1476589.1476686. https://www.cise.ufl.edu/~lok/teaching/ve-s07/papers/sutherland-headmount.pdf. Retrieved 2026-10-06.
- ↑ 6.00 6.01 6.02 6.03 6.04 6.05 6.06 6.07 6.08 6.09 6.10 6.11 6.12 6.13 6.14 6.15 6.16 6.17 6.18 6.19 6.20 6.21 6.22 6.23 6.24 6.25 6.26 6.27 6.28 6.29 6.30 6.31 6.32 6.33 "Ivan Edward Sutherland (curriculum vitae, October 2024)". Asynchronous Research Center, Portland State University. 2024-10. https://arc.cecs.pdx.edu/wp-content/uploads/2024/10/CV_IvanSutherland.pdf. Retrieved 2026-10-06.
- ↑ 7.0 7.1 7.2 "VR @ 50: Ivan Sutherland's 1968 Head-Mounted 3D Display System". ACM SIGGRAPH Blog. 2018-08-14. https://blog.siggraph.org/2018/08/vr-at-50-celebrating-ivan-sutherland.html/. Retrieved 2026-10-06.
- ↑ 8.0 8.1 8.2 "Ivan Edward Sutherland". Kyoto Prize. Inamori Foundation. https://www.kyotoprize.org/en/laureates/ivan-edward-sutherland/. Retrieved 2026-10-06.
- ↑ 9.0 9.1 9.2 9.3 9.4 9.5 9.6 Robert Sproull, David C. Brock (interviewers) (2017-02-03). "Oral History of Ivan Sutherland: Part 1 of 2". Computer History Museum. https://archive.computerhistory.org/resources/access/text/2023/02/102738195-05-01-acc.pdf. Retrieved 2026-10-06.
- ↑ 10.0 10.1 Denis Laurendeau, A. Branzan-Albu, E. Boivin, R. Drouin, J-M. Schwartz (2023-07-27). "(PDF) Survey of the State-of-the-Art on Synthetic Environments, Sensori-Motor Activities in Synthetic Environments, Simulation Frameworks and Real-World Abstraction Models". ResearchGate. p. 21. https://www.researchgate.net/publication/266216435_Survey_of_the_State-of-the-Art_on_Synthetic_Environments_Sensori-Motor_Activities_in_Synthetic_Environments_Simulation_Frameworks_and_Real-World_Abstraction_Models. Retrieved 2024-05-24.
- ↑ 11.0 11.1 "Sketchpad (1963) 3 of 3". YouTube. 2024-03-06. https://www.youtube.com/watch?v=t3ZsiBMnGSg. Retrieved 2024-08-26.
- ↑ 12.0 12.1 David C. Brock (2023-04-16). "The Tremendous VR and CG Systems, of the 1960s". IEEE Spectrum. https://spectrum.ieee.org/sketchpad. Retrieved 2026-10-06.
- ↑ Ben Delaney, Sex, Drugs and Tessellation: The truth about Virtual Reality, as revealed in the pages of CyberEdge Journal (2014), ISBN 978-1500893293, p. 8.
- ↑ 14.0 14.1 "Oral History of Ivan Sutherland, part 1 of 2 (2:27:35)". YouTube. 2024-03-06. https://www.youtube.com/watch?v=Bjr7qLeyvlw. Retrieved 2024-08-26.
- ↑ 15.0 15.1 "Odysseys in Technology: Research and Fun, lecture by Ivan Sutherland (16:55)". YouTube. 2024-03-06. https://www.youtube.com/watch?v=FIMaf4RemOU. Retrieved 2024-08-26.
- ↑ 16.0 16.1 16.2 "The BBVA Foundation recognizes Ivan Sutherland for revolutionizing human-machine interaction through computer graphics and virtual reality". BBVA Foundation Frontiers of Knowledge Awards. 2019-02-19. https://www.frontiersofknowledgeawards-fbbva.es/noticias/the-bbva-foundation-recognizes-ivan-sutherland-for-revolutionizing-human-machine-interaction-through-computer-graphics-and-virtual-reality/. Retrieved 2026-10-06.
- ↑ 17.0 17.1 17.2 17.3 17.4 "Ivan Sutherland: 2005 Fellow". Computer History Museum. https://computerhistory.org/profile/ivan-sutherland/. Retrieved 2026-10-06.
- ↑ 18.0 18.1 "Ivan E. Sutherland". National Inventors Hall of Fame. https://www.invent.org/inductees/ivan-e-sutherland. Retrieved 2026-10-06.