William H. Gates III (Also sent to Steven Ballmer - head of Windows Development at this time)

Microsoft Corporation
16011 NE 36th Way
Box 97017
Redmond, WA 98073-9717

Re: FYI, Learning Sets, Algorithms, Convergent Thinking

Dear Mr. Gates:

Motive

This letter is in regards to OS/2 and your new Windows PORTHOLE product. THIS IS NOT A JOB APPLICATION. I was part of the test team for Yesler, an OS/2 version of a graphics application. There were many problems with this project that interfered with development, costing The Company a lot. These problems were directly related the way people were thinking about OS/2. I would like to share my experience with you so you will understand why The Company lost money working with OS/2.

I am genuinely concerned about the problem I am going to present (and how it will effect Porthole). Because I have a computer engineer education and a background in liberal arts, I know the power of book technology. Few engineers read anything except technical books and journals. This is a problem because they don't understand human relationships. I don't want to sound a pompous, but someone has to take the risk.

We all have our abilities and disabilities. I have no talent for writing and am a terrible speller. I love computers because they help me get around these problems. Communicating new ideas is very difficult, especially in writing. Being misunderstood goes with the territory. On the other hand, I do have an unusual talent for integration and logic. Please be patient while I try to make all the right connection here. Try not to judge this paper by its grammar. Please read it and think about it, then decide.

Prolog

Enclosed is an article on problem solving and creativity taken from Psychology by Tavris and Wade. It includes a definition for learning set (or as I like to call it, cognitive bias) and functional fixedness. It also describes the creative personality. Please read this article before you continue with this letter.

At the end of this letter is a list of book supporting the following thesis: Learning sets (especially those associated with OS/2) are costing Microsoft and other software developers millions every year from product release delays, poor product quality, and a decrease in user perceived quality value - even after improvements.

Example:

At The Company I tested the GPI graphics interface, graphics import and export, and PostScript driver for Yesler. To do this, I had to learn most of the Dos/Windows graphic programs: Designer, Scandal (scanners), Corel Draw, Clip 3D, Freelance, Excel, etc. To verify portability, I make test files for all graphic file formats: Tiff, HPGL, PIC, WFN, DRW, MFD, PCX, EPS, etc. The ones we weren't translating were imported into a graphics package like Designer, then translated to a format that we used and tested for portability. I became very familiar with many Window application and their bugs. After testing graphics import/export, I tested the printer interpretation from the PostScript Driver on the most laser and ink jet printers (including color): QMS, Linotronic, Apple, HP, IMB, etc. (some of which where beta product).

Yesler's performance was compared to The Windows Application (it product spec). To be honest, there just wasn't any comparison between Yesler and Windows. OS/2 runs much faster and allows more flexibility. Even the PostScript Driver run faster than the one for Windows (and was cleaner and better designed). In fact, some of Yesler's problems resulted from the increase in performance speed. The user needs the ability to suspend the redraw thread, and use a Tiff format for complex draw objects (they did this in other places in the program) during document editing (like greeking fonts). They also needed timing delays in places to allow the input to stabilize before taking action (this is often necessary in real-time programming like robotics).

I've been a professional artist and this was the first time saw a PC graphics application that I wanted to buy. I fell in love with this product. I want to go into business for myself again, combining my art and other talents with my engineering background. This motivated me to fight for Yesler and OS/2, which didn't make me very popular...

Having done graphics in 'C' and programming in other multi-tasking systems, I felt I was seeing valid errors. Eventually, I was able to prove these errors. I was the only engineer on the testing team and used traditional trouble shooting techniques used in system engineering. This was alien to the non-technically oriented testers and software engineers. I had a very difficult time getting isolated bugs into the database. They kept telling me the bugs belonged to OS/2, Presentation Manager or the PostScript Driver. This was due to their OS/2 learning set. They were still clinging to the same assumptions regarding OS/2's effect on Yesler, long after OS/2 was fixed. This blinded them to breakthroughs that lead to solutions.

Incident Example:

We received a new CSD for OS/2 to correct some of our bugs. It was immediately installed on someone's machine without any configuration control, then tested using old PUB files generated with pre beta software (and most likely corrupt). In less than a hour they were running around proclaimed, "Its toast! Its toast!"

I backed-up my system, then removed all of OS/2. I went through a complete reinstallation of OS/2, followed with the CSD. I tested it by making a new PUB with the same data in it. The CSD wasn't "toast" at all. Similar situation went on with the PostScript driver as well. Many of the testers didn't understand configuration control.

If incorrect data is gathered from testing.

And engineers use that data to fix the product

Then the problem will be incorrectly fixed.

???

(At least there is a very high probability that it will not get fixed right.)

Comment:

These trouble shooting techniques are often alien to both software and hardware engineers because they work on a detailed level; system engineers work on the integrated level. The methods I use have been in the computer industry for a long time. Some people think they take too much time, but they don't. Over a long period they save time and energy because they provide correct information regarding problems and enable the product to get fixed right the first time.

I've seen the effect of this learning sets at four companies. Software engineers (including myself when developing software) acquire a different learning set from work with linear, algorithm based technology. We tend to see the code and its algorithms in our head; that's what we test. This interferes with our ability to get a real integrated view of the program, making it difficult to see all the relationships, use heuristics, divergent thinking, and deductive logic. What's needed is integrated testing at a technical level using a creative approach base on the rules of logic (like system trouble shooting)!!!

At The Company, I ran standard deductive tests against the engineers' hypothesis for the problems (like I would on any integrated computer system). I was able to get results that proved many of the bugs being attributed to OS/2 PM and the PostScript Driver really existed in Yesler.

Being from a different computer discipline, I couldn't communicate what I was doing or its validity with my co-workers. They thought this type of analysis was something that only applied to hardware. Yet another learning set to overcome; this one is called functional fixedness. I wasn't totally alone in my observations. Another engineer, formerly from HP, was having the same problem in her group. We both had a hard time believing that they would alter such a complex integrate program without verifying their assumptions. Their trial and error approach was really generating stress and chaos for everyone.

Comment:

Computer are based on deductive logic. It makes very little difference if that logic is generated with hardware or software when you're testing at the integrated level. Because integrated systems (including PC 286/386, with all their drivers and peripherals) are a combination of software and hardware, they can be tested at the data flow level. This require education in problem solving techniques and common pit falls. To model the program mentally requires a high degree of creativity, good spatial perception, integration, and imagination.

I decided to take some risk to see if Yesler could be saved (The Company did need the revenue). They were in the process of releasing it with a number of bad crashers, and some graphic and printing problems that were unbelievable. These problems included: scaling attributes that were wrong, graphics objects printing off the page, printing backwards on the screen display, reversal of graphic colors, two child processes trying to access the same data, etc. As an OS/2 evaluation application, I was afraid it would give the public the wrong impression of OS/2. I went over the head of the testing manager, taking my case directly to engineering with some of the information I had for months. This was a "show stopper!"

I managed to get the product release delayed at least three times. It was eventually released so that the CEO could win a bet he had with Ventura Publishing. All products have bugs, but when they get functionally fixed to the wrong product it hurts everyone. There are still engineers at The Company that are convinced OS/2 in "Toast".

I didn't gain popularity from this. Some people thought I was a perfectionist. Because Yesler was as fast as Frame on the Sun; I thought it had a lot of potential for generating new revenue for The Company. The engineers and testers were so sick the chaos they didn't even want to work with OS/2 again. They nicknamed Microsoft, "Dribblesoft", because of the CSDs.

The other testers got very hostile; with good reason considering the trouble we were in. They were scared; afraid of loosing their jobs to more technical people. They tried to sabotage my efforts, putting an old version of the PostScript driver on my machine after I left work, corrupting my system configuration. On another occasion they went so far as to change an old beta version of Yesler, making the title screen read the most recent version, then gave it to me to test. Fortunately I caught on quickly but still wasted valuable company time (the person who did this isn't in testing any more). After this, I checked the other computers and found four different versions of the PostScript Driver. We didn't even know what we were testing.

All in all, many of the bad bugs did get fixed. Another thing positive did result. After this project, The Company made testing a part of engineering. Hopefully the engineers will know more about what's going on. When my contract ended, I decided they just don't pay enough to except another (and put up with this stuff).

Discussion

Many of the software companies working on OS/2 apps, including engineers in apps at Microsoft, are comparing notes. There's a gossip mill because of the beta testing going on between different companies (most of this gossip is totally fallacious). This is reinforcing their learning set for OS/2 and interfering with the development of products. I don't know who owns the bugs, but I know that testing and trouble shooting techniques based on inductive logic alone are dangerous. A hypothesis developed from induction needs to be truth tested with deduction before you change the code. Otherwise you get chaos.

Logic Example:

Induction:

All Black people have curly hair.
That person has curly hair.
Therefore he is black.

??? Can not prove, but it is possible.

Deduction:

All Black people have curly hair.
That person is black.
Therefore he has curly hair.

??? Can prove.

Induction:

All human beings die.

It is dead.

Therefore it is a human being.

??? Can not prove.

Deduction:

All human beings die.
I am a human being.
Therefore I will die.

??? Can prove.

OS/2 logic:

Scaling attribute are read from the PostScript Driver.
This bug is from a scaling error.
Therefore it is caused by the PostScript Driver.

??? Can not prove, but is possible.

This is inductive logic! Other things cause scaling error: import filters, the screen driver, mathematical error, round off errors, etc. The above logic construct was used repeatedly at The Company. Knowing that it was based on inductive logic, I truth tested it and frequently found it invalid.

Problem

Many engineers change code literally on a hunch or do what they consider a work-around to an operating system or hardware bug without knowing if this is true. Under stress, they often forget what they did. When they find that the problem still exist or see another problem caused by their work-around, they may not remember to change it back. I could give you many examples. This creates the chaos!

What was going on between The Company and Microsoft engineers regarding the PostScript driver was unbelievable. Being under stress, Microsoft Engineers were changing things on the word of engineers at The Company. The Company was supposed to be testing the driver, but they were depending on non-technical people. Some of the scaling attributes were used between the screen and printer driver were a mess (especially the EGA). If only they had use the logic based systematic approach.

Example:

A Company engineer changed a global variable to compensate for an earlier Driver bug. Then when the bug got repaired, he didn't think to change the code back, so a whole set of new bugs appeared that were now supposed to be the Drivers fault. When he did remember to change things back, three other seemingly unrelated bugs went away at the same time.

Solution

The PC 286 and 386 with their present speed, their I/O channel, and all the available devices and drivers for peripherals are approaching the complexity mini or mainframe computer system. I've tested OS/2 using trouble shooting techniques I would use on a mini or a mainframe, with some modifications and they do work. By developing a standard trouble shooting techniques for OS/2, based on a creative approach and the traditional methods that implement: configuration control, hypotheses testing, verification techniques; you can reduce time loss, saving lots of money, as well as reducing the amount of stress on people. Offering classes to your testing personnel and software companies interested in doing applications can help overcome this Learning Set problem through education, that teaches formal logic and problem solving, and the pit falls that interfere with correct results. It has to be modeled. Here is how this technique works:

Example:

At Applied Microsystem, while testing the user interface on the EL3200, an emulator for the 68020/30, on the Sun, the following incident was encountered:

Incident:

While scrolling in the Memory Window the program broke up and some windows would close while others open. Sometimes the program would completely go away! It was totally out of user control and sometimes took several minutes to stop. The user had to reset all the Windows and often lost valuable test data and time.

Hypothesis:

The Sun's keyboard buffer became too full, losing a character or two and causing the cursor inputs (which were escape sequences of two char) to get out of sync.

Because the escape key was used to close the active Window, once a sequence was broken it produce several single escape characters causing the windows to close and others to open.

This was considered an operating system bug. The engineers didn't know how to fix it. I didn't trust this without a proof.

Hypothesis test:

If the Sun's key board buffer is the problem then scrolling in other windows should produce the same results.

I opened all the other windows that had a scrolling option, and repeated the same thing I did on the Memory Window. I couldn't repeat the problem. I was literally hitting the keys as fast as humanly possible and the key board buffer did just fine.

The hypothesis failed!

I went back to the Memory Window to play with it, testing to see how fast I had to go to insight an incident. While playing with the window a pattern became clear. I notice that the menu bar was flickering a lot. I made some more observations and counted at least 5 flicker for every cursor move.

Second Hypothesis:

The flickering menu bar is caused by executing the code to refresh the window 5 times for each cursor movement. This is keeping the input interpreter from running fast enough to process the inputs from the cursor keys, resulting in a keyboard buffer overflow (previously mentioned).

Second hypothesis test:

To test this, I made the engineer aware or the menu bar's flicker (he hadn't noticed it before) and I presented my new hypothesis. Then suggested that they find out where the window refresh function was being called and remove all the redundant code to test this hypothesis.

Result:

The removed of redundant calls fixed this problem (which wasn't even a reported bug) and one other reported bug (that they had been working on without success).

Comment:

The second bug fixed had also been discounted as related to low memory. I had managed to disprove that assumption and got them to except it. There are some things that hit me as illogical because of my understanding of systems as integrated wholes and my strength in deductive logic (resulting to a difference in computer training); I just see it .

This creative approach based on the systematic use of deductive logic is much more cost effective than hit and miss. I spent little more than an hour testing the problem to get accurate results. This approach also takes advantage of what is called problem incubation (a well documented technique used by highly creative individuals). Its actually desirable to have people working less. This gets them away from problem analysis and allows other cognitive process in the brain to attack the problem (running in background mode). They get the same work done (sometimes more).

In every company whose products I've tested, I've kept track of the time loss due to the nonsystematic testing. (The labor costs are around a third higher in development and technical support). The loses in revenue due to poor product performance in the market place can only be imagine. The health risks due to the stress resulting from chaos is also a factor (it contribute to burnout).

This approach is a supportive testing situation, but sometime it takes a little time for engineers to adjust to it. When I work with engineers, I use my experience as a programmer and my understanding of cognitive science and human psychology to break their learning set. I will show them seemingly unrelated bugs that I have picked up a similar behavior pattern in, and ask if they are accessing the same functions, global variables, etc.; anything I can think of based on my own experience. I call this cueing. This gets them thinking in different ways (stimulates divergent thinking) so they look at different parts of the program. I started this approach while teaching. I have found that engineers want their program to be the best and usually appreciate the added insight once they understand that you are not attacking them (many creative people have difficulty separating themselves from their work).

The complexity of software today demands a more creative approach. From experience I know that this old way works (with a few creative modifications). It was the rediscovery of deductive logic (originally used by the Greeks) that brought about the Renaissance. Is it time for a Technology Renaissance? Yes!

I've done research in cognitive science since the mid-70s, primarily because my own cognitive differences put me into less than .001% of the population. My first encounter with the effect of Learning Sets in the computer industry was when I taught undergraduate software engineers. I noticed the software engineers got so caught up in testing code algorithms, that they didn't pick up on relational pattern between different parts of their program. This would also happen to me. Yet everyone would come to me to debug their programs because they thought I had this sixth sense about it. It never occurred to me not to use the same methods from systems engineering on software systems.

I also taught formal trouble shooting to students using deductive logic. That made me a real believer. I put bugs in their systems, then instructed them to follow their procedures. But, they wanted to do hit and miss testing (was concerned that they couldn't read). It never failed. There was always a group or two who injected several bugs into their systems use hit and miss techniques. I would fix one bug after the other, using the systematic approach to prove it works.

Trial and error (hit and miss) approach can also be used to play with the system and acquire more data, but only with controls, verification and documentation. There are a set of rules to follow: Never change more than on variable or component in the system at a time. If the change didn't fix the problem, change it back before doing anything else. While testing under OS/2, this required that only one program was changed and tested at a time to verify the actual results. IT was common for tester to install an new CSD, PostScript Driver, and The Application on their machines at the same time. This prevented them from identifying which product was responsible for any changes (either new bugs or fixes). Without following these rules the results can be are tragic and expensive (you loose valuable information).

I have studied Industrial and Organizational Psychology, and am interested in motivational theory. My primary interest in Microsoft is its culture. Written commentary is too limited. I would like to discuss my research and observations since entering the computer industry with you.

Concern

Please do not waste this letter by giving to your Human Resource Department; I am not applying for a job (I will not talk to them). My motivation is for other reasons. I want to see computers get into the hands of the people that can use them the most. Having worked with ordinary people (for about 25 years), I know what their limitations are and what it will take to sell computers to them: an operation system with a graphic interface, speed and multi-tasking. The interface must be intuitively correct and compatible with the 4 million years of Human physical and mental evolution towards tool use. Computer have to be more humane (not designed for people who can think in algorithms).

Recently, I was approached about testing Porthole. During that interview, I listened and hear the same learning set regarding OS/2 that I heard at The Company; This is Microsoft! I could see that this learning set might interfere with the development of Porthole the way it did with Yesler.

When I found out This Guy was working on the Porthole progect, I remembered him from working at Microsoft in Another Group Testing . The people in had a learning set that made them think a lot of valid bugs were hardware problems. Being a systems engineer, I was trained to discriminate hardware from software and knew the different. I wasn't the only person in the group concerned about this. Other testers would just shrug their shoulders and walk away.

Example

While trying to generate an error message in the format command, I found that a 360 disk in a 1.2M drive would cause the program to go on forever, requiring you to turn off the computer to get it to stop. This Guy classed this bug as a hardware problem (?) even though it didn't do it in DOS 3.3. I let it pass.

When I uncovered a repeatable bug in the mode command I got more concerned. While trying to generate an error message in one of the foreign versions the keyboard locked up. I went to use someone else's US version to see if I could reproduce it, but I couldn't. When I came back the data segment was writing to the screen, then it crashed. Because I use strict controls and log my inputs, I was able to repeat it. I wanted to show it to an engineer (someone might not have nulled an string or something). It hadn't been caught by the system either (When the PC runs away it usually causes a core dump when it encounters a fence address, but I didn't know if I was still running OS/2). Again This Guy told me it was a hardware error. I knew from experience that without the code you couldn't make that kind of a judgment, and thought it should be left to an engineer. I also knew that the mode command was used to program the hardware through BIOS (some of the hardware components can only be set once after turning on the computer). A wrong address or something like that could totally trash the systems hardware. This is one of the hard things to trace. They didn't understand what I was talking about.

(At The Company the mode command was one of OS/2's known crashers. There was a pattern to it. I think you had to follow it with a DIR command to crash, but am not sure. It was frustrating because when our system crashed we would go into DOS to check the Swapper size to make sure you hadn't run out of disk space, The Application's configuration file, and save our temp file. Hopefully it or IBM's BIOS has been fixed by now. The Mode and Format command were on our CSD for OS/2.)

Because Another Group test language translation, they are testing the error handling of products at a more detailed level that the US groups (who have more pressing issues to be concerned with). We were isolating a lot of bug in the US versions. The manager of International Testing harbored a lot of resentment toward the US groups. They had not treated her well when Another Group first started. I can still remember the speech she gave about how Another Group wasn't really a part of Microsoft. She didn't want to do anything that would assist the US groups. This really seem nuts when it come to OS/2. It is a shame Another Group holds a grudge. They have valuable information. It should be passed on.

Please understand. This is only for your information. I am genuinely concerned about the computer industry. I would appreciate some confidentiality here. Sometimes its easier to identify a problem from a distance. I'm not attempting to discredit anyone. But some of these people just don't have enough training to break their learning set, or have the technical comprehension to decide what should go into the bugbase. These people can be trained, however.

I experienced the same kind of problem at Microsoft from the testers as I did at The Company . They pulled pranks on me. I've been a nerd all my life, and sometimes I forget that some people don't have the same lust for knowledge that I do or learn at the same fast speed. This sometimes offends or frightens them . I've never known quit what to do about it (Creative people are often considered a threat because other people don't understand that creative people have a different type of motivation. This is why I only do consulting and contract work now).

I tried to reason with people who had less technical education and experience; to convince them to work with the engineers in the US groups. It was a big mistake! I got fired from Microsoft. I originally was fired for having auditory dyslexia (I don't discriminate sounds used in speech very well which is why I have problems writing. This is also why my other cognitive skills like integratetion and deductive logic are so excelled). When I challenged this, I was given many other reasons. I also knew too much about computers, fixed things that were broken, and made people uncomfortable.

Example

MIS was having difficulty getting an Apple PostScript printer to communicate with OS/2 via the serial port. When I found out they'd been trying for a month, I fixed it. Rick Wood and Thijs Gates were very grateful. I also fixed some of the printers that the people had broken (so we could print labels for Gold Masters, prevented a delay in product release). I tried to explain the importance of configuration control. Was concerned about people who were trying to test after messing up their machines I/O channel (breaking things) because they didn't have training to understand the documentation. I got laughed at for suggesting that static sticker on devices where there for a reason (Do you know what a soft cells is in a computer and how much havoc it can cause in software testing ?).

They thought I didn't know what I was talking about because I didn't have a Computer Science degree (another learning set), yet I had studied compilers, operating systems, computer languages, data structures, AI, data processing, etc. (like computer science people), and done development in C for a word processor application and a Lisp interpreter. I tried to get other people in the company interested in this problem, but they totally discounted me. Yet, management has picked me for interviews four times. They even flew me here and put me up in a motel for two days once. Does the right hand know what the left is doing??? This is very confusing.

Hopefully, with my thesis, you will identify this learning set problem and do more research this subject. An engineer can't fix something if he doesn't know its broken. Porthole sounds like a really great product. I am concerned that the Learning Set problem, both at Microsoft and in other software companies, may interfere with product development for OS/2.

I gave up trying to explain this before, because dealing with lower level management in Microsoft is not in my best interest. That's why I'm giving you this information. I will (want to) buy a new 386 and OS/2 if you can get Porthole to work (I think other people will too!), but this kind of thinking can interfering.

Please take my letter seriously. We computer users need the Porthole product. We need a multi-tasking operating system (OS/2) for business computers (OS/2 sure spoiled me).

This isn't the first time I've encountered cognitive learning sets. I came across it while managing several small business and operating businesses of my own. It destroys productivity! Its really counter productive and expensive. It destroys creativity and burns people out. Here is a list of books that will add support to this thesis:

Management; Krienter
Psychology and Industry Today; iiSchultz & Schultz
Psychology; Wade and Tavris
Lessons of Experience; McCall, Lombardo, Morris
Mythical Man-Month; Frederick Brooks
Software Engineering Concepts; Fairley
Software Engineering Economics; Barry Boehm
Engineering Economic Analysis; Newnan
The Complete Guide to Software Testing; Hetzle
What is Quality Control; Kaoru Ishikawa
Commit to Quality; Townsend
The Ropes to Skip and the Ropes to Know; Ritti, Funkhouser
Artificial Intelligence; Charniak, McDermot
Who's Afraid of Big Blue; McKenna
Ceativity in Business; Ray and Myers
The Origin of Consciousness in the Breakdown of the Bicameral Mind; Jayne
The Drama of the Gifted Child; Miller

I would like to wish you the best of luck with OS/2 and Porthole.

Sincerely,

J. L. Brewer
16421 NE 92nd #59
Redmond, WA 98052
Phone: (206) 869-8702