Performance

A Performance Comparison of UNIX Operating Systems on the Pentium

Kevin Lai and Mary Baker, Stanford University.

Kevin Lai's presentation was motivated by his need too choose a Pentium
based system to use for operating system research. His main candidates
were LINUX, FreeBSD and Solaris. The features he was mainly interested
in were:
	Performance
	Reliability
	Availability of kernel source
	Technical support and availability of drivers
	Application support
	Large User base

His presentation concentrated on the performance of his three candidate
OS's. His benchmarks were based on John Ousterhout's microbenchmarks and
the Modified Andrew Benchmark. His conclusions were that FreeBSD performed 
reasonably on all benchmarks, Linux and Solaris ran well on some but on 
others. Linux performed well system calls, context switching and pipe
bandwidth, but particularly poorly on the networking benchmarks, although 
it has improved significantly since the tests were performed. Solaris 
had performed poorly on the system calls, context switching and pipe 
benchmarks, but well on the large file file benchmarks. His eventual 
choice was Linux for reasons other than performance (kernel source 
availability and support).

	Q: Why choose Linux when it has poor network performance ?
	A: Not particularly important for his research (MosquitoNet)
	Q: Why choose a Linux NFS server for the NFS client benchmarks ?
	A: It was better than the Sun they originally used.


lmbench: Portable Tools for performance analysis

Larry McVoy, Silicon Graphics, and Carl Staelin, Hewlett-Packard
Laboratories

Larry McVoy presented the latest implementation of his benchmarking
toolkit lmbench. He illustrated his talk with a mass of data that he has
gathered using this suite of tools. lmbench has evolved from Larry
McVoy's experience and measure the performance of the total system
(operating system and hardware). He mentioned that Intel used lmbench
when designing the P6, and that it can be used to predict the critical
path in a system. 

The main results he presented were memory system latency and context
switching performance of several systems using various hardware 
(including P6, UltraSparc, MIPS R10000, DEC ALPHA) and software 
(including Linux, Solaris, IRIX).

The memory latency benchmark was based on walking a linked list (using 
p = *p) and varying the stride and size of the list. He pointed out that
cache size and latency were obvious from the results produced. He
concluded this section by saying that Intel's P6 was catching up with the
workstation vendors and that HP's memory system was very good.

The context switching benchmark was based on using a ring of processes
connected by pipes. The variables were the number of processes and the
footprint size of the processes. This benchmark was very system
dependent stressing both the OS and the hardware.

He quickly commented about file system benchmarks saying that "Most
either cheat or suck".

	Q: this is not a strict apples-to-apples comparison 
	A: true, but can show differences
	Q: Can this explain application performance ?
	A: not completely, but it can highlight bottlenecks


Process-labelled kernel profiling: a new facility to profile system activities

Shingo Nishioka, Atsou Kawaguchi, and Hiroshi Motoda, Advanced Research
Laboratory, Hitachi Ltd.

Shingo Nishioka presented Pkprof - a solution to the kernel profiling problem 
of being unable to determine which part of kernel execution time is related
to which specific user process.

Pkprof implements a profile buffer for each user process. This allows
the profiler to know to which user process to attribute kernel time.
Interrupts present a problem since it is not immediately apparent which
process an interrupt relates to. Pkprof solves this by using the buf or
mbuf structure as an identifier for such an asynchronous event and
storing this in a temporary profile buffer. When control returns to the
top half of a driver the receiver process can be identified and the
profile modified accordingly.

The processing of asynchronous events involves a high overhead. 
The overhead of pkprof is 36% of kernel time compared with kprof 
which takes 30% of kernel time.