"Layers" of Cache



There are in fact many layers of cache in a modern PC. This does 
not even include looking at caches included on some peripherals, 
such as hard disks. Each layer is closer to the processor and 
faster than the layer below it. Each layer also caches the 
layers below it, due to its increased speed relative to the 
lower levels:

Level	Devices Cached	
Level 1 Cache	Level 2 Cache, System RAM, Hard Disk / CD-ROM	
Level 2 Cache	System RAM, Hard Disk / CD-ROM	
System RAM	Hard Disk / CD-ROM	
Hard Disk / CD-ROM	     --

What happens in general terms is this. The processor requests a 
piece of information. The first place it looks is in the level 1 
cache, since it is the fastest. If it finds it there (called a 
hit on the cache), great; it uses it with no performance delay. 
If not, it's a miss and the level 2 cache is searched. If it 
finds it there (level 2 "hit"), it is able to carry on with 
relatively little delay. Otherwise, it must issue a request to 
read it from the system RAM. The system RAM may in turn either 
have the information available or have to get it from the still 
slower hard disk or CD-ROM. The mechanics of how the processor 
(really the chipset controlling the cache and memory) "look" for 
the information in these various places is discussed here .

It is important to realize just how slow some of these devices 
are compared to the processor. Even the fastest hard disks have 
an access time measuring around 10 milliseconds. If it has to 
wait 10 milliseconds, a 200 MHz processor will waste 2 million 
clock cycles! And CD-ROMs are generally at least 10 times slower. 
This is why using caches to avoid accesses to these slow devices 
is so crucial.

Caching actually goes even beyond the level of the hardware. For 
example, your web browser uses caching itself, in fact, two 
levels of caching! Since loading a web page over the Internet is 
very slow for most people, the browser will hold 
recently-accessed pages to save it having to re-access them. It 
checks first in its memory cache and then in its disk cache to 
see if it already has a copy of the page you want. Only if it 
does not find the page will it actually go to the Internet to 
retrieve it.


Level 1 (Primary) Cache

Level 1 or primary cache is the fastest memory on the PC. It is 
in fact, built directly into the processor itself. This cache is 
very small, generally from 8 KB to 64 KB, but it is extremely 
fast; it runs at the same speed as the processor. If the 
processor requests information and can find it in the level 1 
cache, that is the best case, because the information is there 
immediately and the system does not have to wait. The level 1 
cache is discussed in more detail here , in the section 
on processors.

 Note: Level 1 cache is also sometimes called "internal" cache 
since it resides within the processor.


Level 2 (Secondary) Cache

The level 2 cache is a secondary cache to the level 1 cache, and 
is larger and slightly slower. It is used to catch recent 
accesses that are not caught by the level 1 cache, and is usually 
64 KB to 2 MB in size. Level 2 cache is usually found either on 
the motherboard or a daughterboard that inserts into the 
motherboard. Pentium Pro processors actually have the level 2 
cache in the same package as the processor itself (though it 
isn't in the same circuit where the processor and level 1 cache 
are) which means it runs much faster than level 2 cache that is 
separate and resides on the motherboard. Pentium II processors 
are in the middle; their cache runs at half the speed of the CPU.

 Note: Level 2 cache is also sometimes called "external" cache 
since it resides outside the processor. (Even on Pentium Pros... 
it is on a separate chip in the same package as the processor.)


Disk Cache

A disk cache is a portion of system memory used to cache reads 
and writes to the hard disk. In some ways this is the most 
important type of cache on the PC, because the greatest 
differential in speed between the layers mentioned here is 
between the system RAM and the hard disk. While the system RAM 
is slightly slower than the level 1 or level 2 cache, the hard 
disk is much slower than the system RAM.

Unlike the level 1 and level 2 cache memory, which are entirely 
devoted to caching, system RAM is used partially for caching but 
of course for other purposes as well. Disk caches are usually 
implemented using software (like DOS's SmartDrive). They are 
discussed in more detail in the section on hard disk performance.


Peripheral Cache

Much like the hard disk, other devices can be cached using the 
system RAM as well. CD-ROMs are the most common device cached 
other than hard disks, particularly due to their very slow 
initial access time , measured in the tens to hundreds of 
milliseconds (which is an eternity to a computer). In fact, in 
some cases CD-ROM drives are cached to the hard disk, since the 
hard disk, despite its slow speed, is still much faster than a 
CD-ROM drive is.
