No-code History: Sketchpad - A man-machine graphical communication system (1963)
Below are all slightly edited quotes from the material listed in the Resources section, emphasis mine.
Introduction (2003)
Ivan Sutherland’s Sketchpad is one of the most influential computer programs ever written by an individual, as recognized in his citation for the Turing award in 1988.
Executable versions were limited to a customized machine at the MIT Lincoln Laboratory — so its influence has been via the ideas that it introduced rather than in its execution.
After 40 years, ideas introduced in Sketchpad still influence how every computer user thinks about computing. It made fundamental contributions in the area of human–computer interaction, being one of the first graphical user interfaces. It exploited the light-pen, predecessor of the mouse, allowing the user to point at and interact with objects displayed on the screen.
Introduction (1963)
The Sketchpad system uses drawing as a novel communication medium for a computer. The system contains input, output, and computation programs which enable it to interpret information drawn directly on a computer display.
It has been used to draw electrical, mechanical, scientific, mathematical, and animated drawings; it is a general purpose system.
A Sketchpad user sketches directly on a computer display with a “light pen”.
The light pen is used both to position parts of the drawing on the display and to point to them to change them. A set of push buttons controls the changes to be made such as ”erase”, or “move”.
Except for legends, no written language is used.
The Sketchpad system makes it possible for a man and a computer to converse rapidly through the medium of line drawings. Heretofore, most interaction between men and computers has been slowed down by the need to reduce all communication to written statements that can be typed; in the past, we have been writing letters to rather than conferring with our computers.
The Sketchpad system, by eliminating typed statements (except for legends) in favor of line drawings, opens up a new area of man-machine communication.
Influence
Smith’s Pygmalion, heavily influenced by Sketchpad, made a more explicit argument for the cognitive benefits of this kind of direct interaction and feedback, coining the term “icon”, and making it clear that graphical images could represent abstract entities of a programming language.
Sketchpad influenced Star’s user interface as a whole as well as its graphics applications
Sketchpad’s implementation of class and instance-based inheritance (though not called objects) predated Simula by several years.
Alan Kay’s seminal Dynabook project, which led both to the Xerox Star and to the explosion of interest in object oriented programming through his language Smalltalk, was directly influenced by Sketchpad.
Kay has written of the fact that the genesis of Smalltalk lay in the coincidental appearance on his desk of both a distribution tape of Simula and a copy of Sutherland’s Sketchpad thesis.
Motivation
Sutherland’s original aim was to make computers accessible to new classes of user (artists and draughtsmen among others), while retaining the powers of abstraction that are critical to programmers.
In contrast, direct manipulation interfaces have since succeeded by reducing the levels of abstraction exposed to the user. Ongoing research in end-user programming continues to struggle with the question of how to reduce the cognitive challenges of abstract manipulation.
Sutherland’s attempt to remove the division between users and programmers was not the only system that, in failing to do so, provided the imaginative leap to a new programming paradigm.
Design
The decision actually to implement a drawing system reflected our feeling that knowledge of the facilities which would prove useful could only be obtained by actually trying them.
Had a working system not been developed, our thinking would have been too strongly influenced by a lifetime of drawing on paper to discover many of the useful services that the computer can provide.
Early in December 1961 Professor Shannon visited TX-2 to see the work I had been doing. As a result of that visit the entire effort took new form.
As a result of including circles into the Sketchpad system a richness of display experience has been obtained without which the research might have been rather dry.
As a result of trying to improve upon conventional drafting tools the full new capability of the computer-aided drafting system has come into being.
In making the second generation drawing program, explicit representation of constraints and automatic constraint satisfaction were to be included.
The second generation drawing program included for the first time the recursive instance expansion which made possible instances within instances.
It was possible for me, armed with photographs of the latest developments, to approach a great many people in an effort to get new ideas to carry the work on to a successful conclusion.
Out of these discussions came the notions of copying definitions and of recursive merging which are, to me, the most important contributions of the Sketchpad system.
Addition of new types of things to the Sketchpad system’s vocabulary of picture parts requires only the construction of a new generic block and the writing of appropriate subroutines for that thing.
The subroutines might be easy to write, as they usually are for new constraints, or difficult to write, as for adding ellipse capability, but at least a finite, well-defined task faces one to add a new ability to the system.
Before the generic structure was clarified, it was almost impossible to add the instructions required to handle a new type of element.
In the process of making the Sketchpad system operate, a few very general functions were developed which make no reference at all to the specific types of entities on which they operate. These general functions give the Sketchpad system the ability to operate on a wide range of problems.
The rewards that come from implementing general functions are so great that the author has become reluctant to write any programs for specific jobs.
The power obtained from the small set of generalized functions in Sketchpad is one of the most important results of the research.
In order of historical development, the recursive functions in use in the Sketchpad system are:
Expansion of instances, making it possible to have subpictures within subpictures to as many levels as desired.
Recursive deletion, whereby removal of certain picture parts will remove other picture parts in order to maintain consistency in the ring structure.
Recursive merging, whereby combination of two similar picture parts forces combination of similarly related other picture parts, making possible application of complex definitions to an object picture.
Recursive moving, wherein moving certain picture parts causes the display of appropriately related picture parts to be regenerated automatically.
The major feature which distinguishes a Sketchpad drawing from a paper and pencil drawing is the user’s ability to specify to Sketchpad mathematical conditions on already drawn parts of his drawing which will be automatically satisfied by the computer to make the drawing take the exact shape desired.
For example, to draw a square, any quadralateral is created by sloppy light pen manipulation, closure being assured by the pseudo light pen position and merging of points.
The sides of this quadralateral may then be specified to be equal in length and any angle may be required to be a right angle.
Given these conditions, the computer will complete a square. Given an additional specification, say the length of one side, the computer will create a square of the desired size.
The process of fixing up a drawing to meet new conditions applied to it after it is already partially complete is very much like the process a designer goes through in turning a basic idea into a finished design.
As new requirements on the various parts of the design are thought of, small changes are made to the size or other properties of parts to meet the new conditions.
By making Sketchpad able to find new values for variables which satisfy the conditions imposed it is hoped that designers can be relieved of the need of much mathematical detail.
Arbitrary symbols may be defined from any collection of line segments, circle arcs, and previously defined symbols. A user may define and use as many symbols as he wishes. Any change in the definition of a symbol is at once seen wherever that symbol appears.
It is easy to add entirely new types of conditions to Sketchpad’s vocabulary.
Since the conditions can involve anything computable, Sketchpad can be used for a very wide range of problems.
How it Works
If we point the light pen at the display system and press a button called “draw”, the computer will construct a straight line segment which stretches like a rubber band from the initial to the present location of the pen.
Additional presses of the button will produce additional lines until we have made six, enough for a single hexagon. To close the figure we return the light pen to near the end of the first line drawn where it will “lock on” to the end exactly. A sudden flick of the pen terminates drawing.
To make the hexagon regular, we can inscribe it in a circle. To draw the circle we place the light pen where the center is to be and press the button “circle center”, leaving behind a center point. Now, choosing a point on the circle (which fixes the radius,) we press the button “draw” again, this time getting a circle arc.
Next we move the hexagon into the circle by pointing to a corner of the hexagon and pressing the button “move” so that the corner followsk the light pen, stretching two rubber band line segments behind it. By pointing to the circle and giving the termination flick we indicate that the corner is to lie on the circle.
If we also insist that the sides of the hexagon be of equal length, a regular hexagon will be constructed. This we can do by pointing to one side and pressing the “copy” button, and then to another side and giving the termination flick.
We now file away the basic hexagon and begin work on a fresh “sheet of paper” by changing a switch setting. On the new sheet we assemble, by pressing a button to create each hexagon as a subpicture, six hexagons around a central seventh in approximate position.
An entire group of hexagons, once assembled, can be treated as a symbol. The entire group can be called up on another “sheet of paper” as a subpicture and assembled with other groups or with single hexagons to make a very large pattern.
Information about how the drawing is tied together is stored in the computer as well as the information which gives the drawing its particular appearance. Since the drawing is tied together, it will keep a useful appearance even when parts of it are moved.
Again, since we indicated that the corners of the hexagon were to lie on the circle they remained on the circle throughout our further manipulations. It is this ability to store information relating the parts of a drawing to each other that makes Sketchpad most useful.
If the master hexagon is changed, the entire appearance of the hexagonal pattern will be changed.
It took about one half hour to generate the 900 hexagons, including the time taken to figure out how to do it. Plotting them takes about 25 minutes. The drafting department estimated it would take them two days to produce a similar pattern.
By far the most interesting application of Sketchpad so far has been drawing and moving linkages.
The ability to draw and then move linkages opens up a new field of graphical manipulation that has never before been available.
One of the largest untapped fields for application of Sketchpad is as an input program for other computation programs.
The ability to place lines and circles graphically, when coupled with the ability to get accurately computed results pictorially displayed, should bring about a revolution in computer application.
With Sketchpad we have a powerful graphical input tool. It happened that the relaxation analysis built into Sketchpad is exactly the kind of analysis used for many engineering problems. By using Sketchpad’s relaxation procedure we were able to demonstrate analysis of the force distribution in the members of a pin connected truss.
A graphical input coupled to some kind of computation which is in turn coupled to graphical output is a truly powerful tool for education and design.
To draw this figure, one bay of the truss (shown below the bridge) was first drawn with enough constraints to make it geometrically accurate. These constraints were then deleted and each member was made to behave like a bridge beam.
Applying a load where desired and attaching supports, one can observe the forces in the various members. It takes about 30 seconds for new force values to be computed.
Having drawn a basic bridge shape, one can experiment with various loading conditions and supports to see what the effect of making minor modifications is.
Since Sketchpad is able to accept topological information from a human being in a picture language perfectly natural to the human, it can be used as an input program for computation programs which require topological data, e.g., circuit simulators.
Sketchpad itself is able to move parts of the drawing around to meet new conditions which the user may apply to them. The user indicates conditions with the light pen and push buttons. For example, to make two lines parallel,
The conditions themselves are displayed on the drawing so that they may be erased or changed with the light pen language. Any combination of conditions can be defined as a composite condition and applied in one step.
Hardware
Lincoln Laboratory provided not only advice but also technical support including to date about 600 hours of time on the TX-2.
Whatever success the Sketchpad effort has had can in no small measure be traced to the use of TX-2. TX-2’s 70,000 word memory, 64 index registers, flexible input-output control and liberal supply of manual intervention facilities such as toggle switches, shaft encoder knobs, and push buttons all contributed to the speed with which ideas could be tried and accepted or rejected.
Moreover, being an experimental machine it was possible to make minor modifications to TX-2 to match it better to the problem. For example, a push button register was installed at my request.
Summary of Vital Statistics — TX-2 — December 1962
- Word Length
-
36 bits, plus parity bit, plus debugging tag bit
- Memory
-
256 × 256 core 65,536 words 6.0 µsec cycle time
64 × 64 core 4,096 words 4.4 µsec cycle time
Toggle switch 16 words
Plugboard 32 words
- Auxiliary Memory
-
Magnetic Tape 2+ million words, 70+ million bits per unit (2 units in use, total of 10 planned)
- Tape Speeds
-
selectable 60-300 inches/sec, search at 1000 inches/sec (i.e. about 1600 to 8000 36 bit words/sec)
Input
Paper Tape Reader: 400-2000 6 bit lines/sec
2 keyboards — Lincoln writer 6 bit codes
Random number generator — average 57.6 µsec per 9 bit number
IBM Magnetic Tape (Model 729 M6)
Miscellaneous pulse inputs — 9 channels — push buttons or other source
Analog input — Epsco Datrac — nominal 11 bit sample, 27 kilocycle max. rate
2 light pens — work with either scope or both on one
Special memory registers
Real time clock
4 shaft encoder knobs, 9 bits each
592 toggle switches (16 registers)
37 push buttons — any or all can be pushed at once
Output
Paper tape punch — 300 6 bit lines/sec
2 typewriters — 10 characters per second
IBM Magnetic Tape (729 M6)
Miscellaneous pulse/light/relay contacts — 9 channels (low rates)
Xerox printer — 1300 char. sec
2 display scopes — 7 × 7 inch usable area, 1024 × 1024 raster
Large board pen and ink plotter — 29”×29” plotting area. 15 in/sec slew speed. Off line paper tape control as well as direct computer control.
Lessons Learned
Had I to do the work again, I could start afresh with the sure knowledge that generic structure, separation of subroutines into general purpose ones applying to all types of picture parts and ones specific to particular types of picture parts, and unlimited applicability of functions (e.g. anything should be moveable) would more than recompense the effort involved in achieving them.
I have great admiration for those people who were able to tell me these things all along, but I, personally, had to follow the stumbling trail described in this chapter to become convinced myself.
Conclusion
We conclude from these examples that Sketchpad drawings can bring invaluable understanding to a user. For drawings where motion of the drawing, or analysis of a drawn problem is of value to the user, Sketchpad excells.
For highly repetitive drawings or drawings where accuracy is required, Sketchpad is sufficiently faster than conventional techniques to be worthwhile.
For drawings which merely communicate with shops, it is probably better to use conventional paper and pencil.
Trivia
Claude E. Shannon was the thesis supervisor.
Marvin Minsky gave advise during development.
To initially establish pen tracking the Sketchpad user must inform the computer of an initial pen location. This has come to be known as “inking-up” and is done by “touching” any existing line or spot on the display whereupon the tracking cross appears. If no picture has yet been drawn, the letters INK are always displayed for this purpose.