Role of Cache in the PC



In early PCs, the various components had one thing in common: 
they were all really slow :^). The processor was running at 
8 MHz or less, and taking many clock cycles to get anything 
done. It wasn't very often that the processor would be held up 
waiting for the system memory, because even though the memory 
was slow, the processor wasn't a speed demon either. In fact, on 
some machines the memory was faster than the processor.

In the 15 or so years since the invention of the PC, every 
component has increased in speed a great deal. However, some 
have increased far faster than others. Memory, and memory 
subsystems, are now much faster than they were, by a factor of 
10 or more. However a current top of the line processor has 
performance over 1,000 times that of the original IBM PC!

This disparity in speed growth has left us with processors that 
run much faster than everything else in the computer. This means 
that one of the key goals in modern system design is to ensure 
that to whatever extent possible, the processor is not slowed 
down by the storage devices it works with. Slowdowns mean wasted 
processor cycles, where the CPU can't do anything because it is 
sitting and waiting for information it needs. We want it so that 
when the processor needs something from memory, it gets it as 
soon as possible.

The best way to keep the processor from having to wait is to 
make everything that it uses as fast as it is. Wouldn't it be 
best just to have memory, system buses, hard disks and CD-ROM 
drives that just went as fast as the processor? Of course it 
would, but there's this little problem called "technology" that 
gets in the way. :^)

Actually, it's technology and cost; a modern 2 GB hard disk costs 
less than $200 and has a latency (access time) of about 10 
milliseconds. You could implement a 2 GB hard disk in such a way 
that it would access information many times faster; but it would 
cost thousands, if not tens of thousands of dollars. Similarly, 
the highest speed SRAM available is much closer to the speed of 
the processor than the DRAM we use for system memory, but it is 
cost prohibitive in most cases to put 32 or 64 MB of it in a PC.

There is a good compromise to this however. Instead of trying to 
make the whole 64 MB out of this faster, expensive memory, you 
make a smaller piece, say 256 KB. Then you find a smart algorithm 
(process) that allows you to use this 256 KB in such a way that 
you get almost as much benefit from it as you would if the whole 
64 MB was made from the faster memory. How do you do this? The 
short answer is by using this small cache of 256 KB to hold the 
information most recently used by the processor. Computer science 
shows that in general, a processor is much more likely to need 
again information it has recently used, compared to a random 
piece of information in memory. This is the principle 
behind caching.
