No-code History: Lotus Improv - Spreadsheets Done Right (1991)
Below are all slightly edited quotes from the material listed in the Resources section, emphasis mine. My notes prefixed with 💭
Introduction
One of the hardest things to do with a computerized spreadsheet like Lotus' 1-2-3 is to lay out the initial modeler.
What should go in the rows? What should go in the columns?
Fundamentally, even the most basic spreadsheet program will let the user put a number or a formula in any cell desired. It's a lot of power --- a lot more than you need for most applications.
Enter Pito Salas, a bright developer in the Advanced Technology Group. Pito's project was to look at a variety of complicated models that had been built with conventional spreadsheets and see what they had in common.
Pito looked at financial statements, revenue forecasts, and future tax estimates, to name a few. He discovered that most spreadsheets have many patterns in them. If a spreadsheet program could be taught to understand those patterns, he realized, he could make it easier to build and use complicated models.
Within a few months, Pito had come up with the fundamental idea at the core of Improv: that the raw data in a spreadsheet, the way that the user views the data, and the formulas used to perform calculations can all be separated from each other.
The formulas should be general, so that the user can type something like PROFIT = PRICE - COST and have the spreadsheet calculate every PROFIT cell from its corresponding PRICE and COST cells.
The user should be able to rearrange the views to highlight the information and relations that she is trying to convey. And the data itself should be put into a multi-dimensional database. A slick interface should sit on top to make it easy to get information in and out.
The result was Improv, a multi-dimensional spreadsheet product with natural language formulas and dynamic views.
History
In 1986, Pito Salas joined the Advanced Technology Group at Lotus to think about a totally new kind of spreadsheet.
The decision was made in September 1988 to go ahead with the "Back Bay" project.
After experimenting with interfaces and a database engine under DOS and the Macintosh operating system, the group decided that the product would be based on OS/2 and Microsoft's Presentation Manager. They even picked a mascot --- Fluffy Bunny --- and started up an underground newsletter, "Fluffy Bunny Goes to Back Bay".
In October 1988, Steve Jobs came to Lotus to show off his new computer. After the talk, Lotus' top management did a private show-and-tell for Steve of their most interesting products that were under development.
Pito showed Steve a clunky, character-based, primitive spreadsheet, but all of the elements of the future were there: there were formulas at the bottom of the spreadsheet, rather than integrated in the cells; it was multi-dimensional; and the user could instantly call up different views of the same data set.
Immediately, Jobs wanted Back Bay for the NeXT.
Another reason that Lotus decided to go with NeXT, says Jeff Anderholm, was that the NeXT didn't run 1-2-3, Lotus' cash cow. "We didn't have to worry about any [marketing] conflict with 1-2-3".
By January 1989, Improv didn't have category tiles. Instead, all of the view rearrangement was done with menu commands.
Then the group hit upon the idea to use icons. "We realized that if we represented these things as icons, all these manipulations could be represented by moving icons from one place to another".
But where should the icons for the categories go? After trying a lot of different ideas, the developers decided to create a special icon window.
"And then Steve Jobs came", says Paul, remembering Job's visit in April 1989.
Jobs then said that the category manipulation had to be more direct. "You have to be able to touch the categories and move them around. Having them off in a separate window is too removed", Paul remembers.
"He didn't even want to have the tiles; he wanted to just move them around. He's really a fanatic for direct manipulation, and it really shows".
Jobs didn't have an answer, says Paul, but "one of the benefits of that [meeting] was we junked the idea of the extra panel", and put the category tiles on the worksheet itself.
The product was released for the NeXT brand computers in 1989 as Improv.
Lotus chose not to include this functionality in their flagship Lotus 1-2-3 product. Instead, Lotus Improv for Windows came out in 1991.
How it Works
Column letters, row numbers, and cell formulas, the main source of frustration for spreadsheet users, are all gone.
In their place are rows and columns labeled in plain English, and an independent list of formulas that use those labels, making Improv, in essence, a relational spreadsheet.
For example, suppose you have rows labeled
UNITS BOUGHT
UNITS SOLD
WHOLESALE PRICE
RETAIL PRICE
EXPENSES
GROSS SALES
NET PROFITS
In the formula panel, you would enter the following formulas:
EXPENSES = UNITS BOUGHT * WHOLESALE PRICE
RETAIL PRICE = 1.4 * WHOLESALE PRICE
GROSS SALES = UNITS SOLD * RETAIL PRICE
NET PROFIT = GROSS SALES - EXPENSES
Now, no matter how those rows grow or shrink in size, and no matter where they appear in your spreadsheet, the appropriate calculations always take place.
Other innovative features abound. You can easily rearrange a spreadsheet by moving around small tokens that represent row or column categories.
You may open multiple windows to a spreadsheet, allowing you to view (and change) data in several different ways simultaneously.
Rather than use letters and numbers to describe data, it lets you use real words, like "Tons" and "Dollar Value." Or anything you are comfortable with.
The benefit of this is that now your formulas read like English. Instead of seeing something like =BD2*BD3, you see Dollar Value = Tons * 5.75
And Improv lists all your formulas in one place, as opposed to hiding them in individual cells.
So when you revisit a complicated spreadsheet months later, it's sure to make sense. Likewise if you're looking at a spreadsheet that's been designed by someone else.
All you do is use the mouse to click one of the category "tiles" located along the edges of the spreadsheet - such as "Region" or "Material"- and drag it to a new location.
Improv allows you to move your column and row headings from one part of the spreadsheet to another, even interchange them - and without the slightest hesitation, the spreadsheet will automatically rearrange itself.
💭 For more usage examples check:
Programmability
Improv comes with LotusScript, which lets you build custom applications and front ends for any Lotus application.
If you have a basic programming experience, LotusScript makes understanding the process of creating custom applications relatively simple.
You can attach a script to a model or save it independently and use it with other models.
LotusScript and Lotus Dialog Editor give Improv all the tools necessary to create a custom front end or complete custom application.
Why It Failed
Lotus marketed Improv more as "spreadsheets done right" (referring to the separation of data and formulas, and the more rigid structure of an Improv model) rather than it's OLAP capabilities, which unfortunately had the effect of confusing those customers who now had to choose between 1-2-3 and Improv, and instead chose Microsoft Excel instead.
From an architecture viewpoint, Improv was also limited by the fact that its cubes ran in memory rather than being paged to disk, so it was always limited in what it could hold, especially with the typical amount of memory a PC had then.
💭 A comment from Jeff Anderholm in 1991 that points to the need to train users
We have to figure out what we can add to the product to help people learn it, because you have to unlearn what you know about conventional spreadsheets like Excel and 1-2-3. Our challenge is to convince people that the benefits of this new spreadsheet are worth the cost of switching.
💭 First person here is Salas
It’s hard to argue that one of the keys is the maleability of the spreadsheet as a medium. The fact that a spreadsheet can grow organically, be modified and grown in a kind of an Improvisational manner. But when spreadsheets get complicated they get messy and error-prone and this is what Improv set out to address.
In the end it didn’t go anywhere, probably because in setting out to improve on spreadsheets, Improv lost the essence of a spreadsheet and in doing so lost the market.
Innovators Dilemma.
I am not sure it applies, but one could argue a parallel here with Improv. In particular this would lead you to the conclusion that the key strategy mistake was to try to market Improv to the existing spreadsheet market. Instead, if the product were marketed to a segment where the more structured model was a ‘feature’ not a ‘bug’ would have given Lotus the time to learn and improve and refine the model to a point where it would have satisfied the larger market as well.
Lotus was positioning Improv as a spreadsheet replacement, rather than a specialized tool to better perform an important subset of tasks currently performed with a spreadsheet.
Lotus was in the throes of a heroic battle for survival against Microsoft’s Excel – causing undue pressure on the company to make its product portfolio clean and understandable, and to organize all resources behind the flagship product. Introducing new technology costs in a scenario such as that.
