                                The Art
                              of lossless
                           image compression
                                vol. 15

Here are the results of 10 tests performed in November-December 1999
to compare lossless compression of True Color (24-bit) images
by HP Labs' software implementation of JPEG-LS,
by popular PkZip, 7Zip, Boa, BMF, UHIC,
by archivers with _special_ algorithms for 24-bit images:
Eri32,ArHanGeL,Ufa32,777,ESP,Rkive,RKim,RK,UHarc,Imp,RAR,WinRAR,
and results on same images of the famous PNG algorithm and format
created by PngCrush, Display 1.89a, SEA 1.3, Image Alchemy 1.10

See Archive Comparison Test by Jeff Gilchrist for more details:
                        http://act.by.net/

If anybody wants to start or continue such tests,
or can suggest some other sets of images, or other compression programs,
     (not sources or algorithm descriptions, programs for DOS or Windows only)
or knows we have missed something important,
     (some new fantastic technology, an algorithm or even a program capable
      of lossless compression of up to 1000:1 and lossy compression of
      up to 10000:1 , we hear about it from time to time since our XTs)
please let us know immediately: ratush@srsc-gw.sscc.ru   Thank you!


[[1]] COMPRESSION QUALITY
=========================
             (see also
             [[2]] Time
             [[3]] Details
             [[4]] Comments)

The last eleventh line shows results for the sum of all ten tests (303 images)

Original  BMF    RKim     RK     UHIC  ArHanGeL   NK     Eri     RAR     Imp
275.95%   100%  101.71  120.20  100.55  110.52  112.65  117.81  135.03  154.65
289.11%   100%  124.97  127.28  104.88  114.61  135.54  122.36  140.58  152.88
245.22%   100%  101.61  123.16  107.93  112.92  111.48  123.50  124.40  144.52
238.64%   100%  113.35  124.20  104.20  110.53  120.45  121.23  134.57  146.59
448.61%   100%  138.97  130.88  108.64  110.21  117.94  122.84  155.31  154.64
237.91%   100%  101.95  120.45  105.20  110.49  106.65  121.87  128.47  144.18
283.29%   100%  104.03  122.94  104.06  110.26  110.86  120.60  137.53  147.66
309.74%   100%  102.03  127.03  106.25  111.90  112.06  120.65  139.49  154.26
225.19%   100%  104.61  118.81  104.00  108.82  110.47  118.20  123.11  141.08
547.50%   100%  104.05  131.46  104.84  107.76  127.75  115.26  174.60  157.56
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
288.02%   100%  103.94  123.65  105.03  110.73  111.84  120.60  136.45  148.95

JPEG-LS  {             Display              }    SEA   PngCrush  7Zip    PkZip
149.81% 154.30  158.67  157.93  156.06  153.89  150.94  150.53  159.16  169.14
156.41% 153.58  186.20  177.21  169.62  164.86  163.71  151.56  205.21  210.53
119.20% 145.60  151.16  140.98  141.56  139.53  146.28  129.55  193.86  196.15
127.45% 142.67  148.25  146.42  145.46  144.00  147.31  134.10  182.24  185.48
169.22% 157.12  156.38  183.73  153.70  153.19  157.52  141.51  182.29  185.19
114.03% 140.20  136.69  131.69  135.10  131.44  139.37  122.31  173.38  176.24
121.09% 147.00  144.58  142.62  142.07  140.13  146.61  131.22  176.27  180.97
121.86% 153.19  150.82  144.91  143.49  142.10  152.03  132.68  199.74  204.38
117.84% 137.05  140.75  133.78  135.20  132.58  139.93  121.89  179.50  181.40
151.17% 159.42  153.92  164.21  150.26  148.74  156.87  141.66  186.28  195.49
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
123.39% 147.22  146.18  142.74  141.93  139.57  146.85  130.50  182.92  187.28


[[2]] Time
==========
A SET of 230 images totalling 271,331,460 bytes
from ftp://ipl.rpi.edu/pub/image/still/KodakImages/color/
was used for this test and an AMD-Pr-350 machine with 64M RAM running Windows98
Well-known 23 Kodak images copied ten times (see icopy.bat in artest15.zip)

 Programs,options   Overall Average Compress Extract Compressed
                     score,  Users'   time,   time,    size,
                    seconds  score   seconds seconds   bytes

-777    a -m4        63352   48757   16216   10465   132018518
 7zip   a -mx        45029   43778    1390     169   156492462
 7zip   a            44443   43761     757     169   156662222
+arhangel a -mm -mz  33079   31471    1786    1107   108669974
+arhangel a -mm      33059   31469    1766    1107   108669974
+arhangel a -mm1     46796   44770    2251    1747   154074824
+arhangel a -mm3     41337   39555    1980    1317   136944664
 bmf   -f  -s  -q9   37166   30332    7593    2261    98323280
 bmf   -f  -s        32252   29800    2724    2216    98323280
 bmf   -f  -q9       32534   29731    3114     268   104947560
 bmf   -f            30243   29584     732     281   105229600
-boa  -m15           60564   48628   13259   13872   120353098
+eri   a -3ma8       31705   30076    1809     424   106102020
+eri   a -3ma8 -m3   31253   30040    1347     411   106182410
+eri   a -3ma8 -m2   31094   30253     934     333   107379300
+eri   a -3ma8 -m1   31959   31272     763     282   111292130
-esp  /m0 /mm2 /ml7  36738   35441    1438     132   126599845
-esp  /m1 /mm2 /ml7  36572   35474    1218     131   126800674
-esp  /m2 /mm2 /ml7  36304   35488     906     129   126968539
-esp  /m3 /mm2 /ml7  38469   38033     483     133   136268163
+imp98   a -2        37890   37360     588     231   133457225
+imp98   a -2  -s4   38003   37458     605     238   133779384
+imp98   a -mm       43094   42369     805     134   151758584
+imp98   a -mm -s4   43062   42305     841     132   151521710
+imp98   a -mm -m3   43294   42174    1244     146   150856308
+imp98   a -mm -m1   46104   45545     621     137   163246816
+imp-dos a -mm -m3   43339   42202    1263     172   150855765
+Nk                  35441   31720    4134     539   110767370
 pkzip  -exx         46766   46407     398     173   166302252
 pkzip  -a           46687   46399     318     171   166309392
 pkzip  -es          55425   55221     225     164   198128322
+WinRar  a -mm -mde  37679   37190     543     283   132671570
+WinRar  a -mm -m5   37723   37217     562     282   132765230
+WinRar  a -mm -s-   37702   37219     536     284   132777620
+WinRar  a -mm -s    37660   37170     544     279   132614830
+WinRar  a -mmf      37022   36609     458     300   130552530
+rar32   a -mm -mde  37803   37244     621     329   132671570
+rar32   a -mm -m5   37844   37272     635     330   132765230
+rar32   a -mm -s-   37828   37278     611     335   132777620
+rar32   a -mm -s    37776   37225     612     327   132614830
+rar32   a -mmf      37161   36669     546     351   130552530
 rkim  c             33476   31271    2450    2400   103056300
 rkim  cx            35322   30297    5583    1958   100013340
-rkive  -mt3         35073   33938    1260    1282   117108857
-rkive  -mt1         35050   33930    1243    1276   117108868
-rkive  -mf3         34960   33921    1153    1276   117108724
-rkive  -mf1         34957   33915    1157    1270   117108780
-ufa    a -m5 -mu32  47658   42322    5927    6186   127961498
-ufa    a -m5 -mu4   47906   43617    4764    5087   136996108
-ufa    a -m4 -mx    41496   36117    5975    1279   123269848
-ufa    a -m4 -mq    36969   35707    1400    1383   123064358
-uharc  a -m3        39266   33496    6411     796   115412754
-uharc  a -m3 -md64  36807   33259    3941     799   115439014
-uharc  a -m2        37269   33551    4131     814   116367752
-uharc  a -m1        36161   33512    2942     825   116616698
 uhic  m             34331   31276    3393    3473    98870920
 uhic  e             31230   29686    1713    1701   100132460
 Display -i --same -bpng
   filter=5,level=9  43891   42400    1654     203   151317180
   filter=5,level=1  44264   43849     460     211   156934080
   filter=1,level=9  43935   42492    1602     190   151714120
   filter=1,level=5  43832   43290     601     194   154930330
   filter=1,level=1  44634   44336     330     203   158763080
 SEA /png:100        42141   41458     757     157   148414950
+pngcrush -q         47048   42442    5117     541   149004140
+pngcrush -brute    119504   49565   77710     680   148011370
 Alchemy ---n        46293   44162    2365     610   155940960
         ---n 3      46345   44637    1896     737   157361408
         ---n 2      46073   44714    1510     515   158572993
         ---n 1      44997   43647    1497     510   154758381
 locoe -N -c1        41580   41296     314     347   147305210
 locoe -N -c2        41823   41445     420     375   147701630
-PhotoJazz           ?????   ?????     ???     ???   111673670

+ indicates new lines in current volume of ARTest
- indicates oldest lines, that will not be present in next volume.

Overall score is calculated by adding compress_time, extract_time, and
time it would take to transfer the compressed file over a 28,800bps network:
(compressed_size)/3600 , because 28800 bits_per_second is 3600 bytes_per_second

Average Users' score is calculated by adding (compress_time/10) + extract_time+
time it would take to transfer the compressed file over a 28,800bps network.
Compression time is devided by 10 here, because more than 90% of people would
never compress anything during their life (with compression programs), but they
use compressed data almost _every_ time they use computers and/or internet.
That's why compression time is not so actual for them.


[[3]] Details
=============
are no longer put to this main text
(816 lines reporting 5757 results on 303 files in 10 sets),
but can be found in FULL version with DETAILS.DAT and *.BAT
at http://geocities.com/SiliconValley/Bay/1995/artest15.zip
or http://artest1.tripod.com/artest15.zip

1,2,3,4,9 image sets were taken from ftp://ipl.rpi.edu/pub/image/ and /image2/
converted from Sun Raster format to Windows Bitmap,
(this conversion changes image headers only, 32 bytes
are replaced by 54 bytes, but image data is not modified)

This site ipl.rpi.edu is described in comp.copression FAQ by Jean-loup Gailly,
in 29 lines of "Subject: [55] Where can I find Lenna and other images?"

Eight images in the 5th set are from the famous Waterloo Color Set,
         ( losslessly compressed with Eri32 4.5fre to 2,894,045 bytes,
           it can be found at http://geocities.com/eri32 )
6,7,8th image sets are from  ftp://download.intel.com/pressroom/images/
10th set was downloaded from ftp://ftp.kodak.com/www/images/dental/
They were converted from TIFF format to Windows Bitmap.

Some images were slightly renamed. See DETAILS.DAT from artest15.zip for more
details. Some new images (from download.intel.com ) were not added yet.


[[4]] Comments
==============
Links to download programs:
~~~~~~~~~~~~~~~~~~~~~~~~~~~
ArHanGeL 1.40:a http://geocities.com/SiliconValley/Lab/6606/arh140a2.zip          50K
ERI32  4.5fre:e http://geocities.com/eri32/eri45fre.zip                           90K
RK     1.02a1:W http://members.tripod.com/~malcolmt/downloads/rk102a01.exe       247K
RKim   1.03  :e http://members.tripod.com/~malcolmt/downloads/rkim103.zip         84K
Rkive  1.92b1:e http://members.tripod.com/~malcolmt/downloads/rkv192b1.zip       129K
Imp-dos 1.1b5:e http://www.technelysium.com.au/imp110b5d.zip                     267K
Imp98  1.1b5 :W http://www.technelysium.com.au/imp110b5.exe                      119K
PkZip  2.50  :a ftp://ftp.simtel.net/pub/simtelnet/msdos/arcers/pk250dos.exe     202K
BMF    1.1   :e ftp://ftp.simtel.net/pub/simtelnet/win95/graphics/bmf_1_10.zip   132K
Ufa32  0.04b1:W ftp://ftp.simtel.net/pub/simtelnet/win95/compress/ufa004b1.zip    64K
777    0.04b1:W ftp://ftp.elf.stuba.sk/pub/pc/pack/777004b1.zip                   72K
7Zip   2.01  :W ftp://ftp.elf.stuba.sk/pub/pc/pack/7zip201.zip                    81K
Boa    0.58b :e ftp://ftp.elf.stuba.sk/pub/pc/pack/boa058.zip                     74K
RAR32  2.60  :e ftp://ftp.elf.stuba.sk/pub/pc/pack/rarx260.exe                   255K
WinRAR 2.60  :W ftp://ftp.elf.stuba.sk/pub/pc/pack/wrar260.exe                   529K
ESP    1.92  :a ftp://ftp.elf.stuba.sk/pub/pc/pack/esp_v192.exe                   68K
UHarc  0.2   :e ftp://ftp.elf.stuba.sk/pub/pc/pack/uharc02.zip                    99K
UHIC   1.0   :e ftp://ftp.elf.stuba.sk/pub/pc/pack/uhic10.zip                     52K
NK     99nov :W ftp://tu.tusur.ru/pub/ImageTools/compress.zip                     81K
JPEG-LS 0.90 :W http://www.hpl.hp.com/loco/locoX900.zip                          375K
PngCrush1.1.8:e ftp://ftp.cdrom.com/.1/png/src/pngcrush-1.1.8-dos.zip            134K

:a - DOS programs, will run under pure DOS or in a Windows DOS box
:e - DOS programs using DOS extenders like DOS/4GW or CWSDPMI
:W - Windows95/98/NT programs

If direct link doesn't work-most probably newer version of the program appeared
at the same site: visit web page, or read the whole directory from ftp server
(i.e. try the same URL, but without filename).


Homepages:
~~~~~~~~~~
Arhangel     : http://geocities.com/SiliconValley/Lab/6606
Eri32        : http://geocities.com/eri32
      mirror : http://artest1.tripod.com
RK, RKim     : http://malcolmt.tripod.com
Imp,WinImp   : http://www.technelysium.com.au
PkZip        : http://www.pkware.com
Ufa,777,7Zip : http://compression.hypermart.net
RAR,WinRAR   : http://www.rarsoft.com
ESP          : http://esp-team.scene.hu
      mirror : http://banki1.banki.hu/~arpi
JPEG-LS      : http://www.hpl.hp.com/loco
PngCrush     : http://www.xnot.com/pmt/pngcrush


What's new:
~~~~~~~~~~~
9 new programs were tested: ArHanGeL 1.40a2, Eri32 4.5fre, NK 99nov, RK 1.02a1,
PngCrush 1.1.8, Imp 1.10b5 (both DOS and Windows95/98/NT versions),
WinRAR 2.60, RAR32 for Dos and OS/2 2.60 .

NK is new image compressor with good compression ratio and time.
RK 1.02a1 with -mx1   was only 0.42% worser than   RK 1.02a1 with -mx2
(304355092/303087960) and only 0.0006% worser than RK 1.01a1 with -mx1
(304355092/304353168).
PngCrush worked about 3 hours with -q switch , and about 64 hours with -brute .
WinRAR and RAR32 were compressing files to exactly same sizes,
but RAR32 was slower, as a rule.

33 new images appeared at ftp://download.intel.com/pressroom/images/ ,
they were already downloded,  converted to .BMP,  converted to .BMF,
and written to CDROM, but will be added to 6th, 7th and 8th sets later.

Three lossless JPEG programs mentioned in comp.compression FAQ -
>     [15] Where can I get image compression programs?
>   ftp://havefun.stanford.edu/pub/jpeg/JPEGv1.2.1.tar.Z   (has lossless mode)
>   ftp://ftp.cs.cornell.edu/pub/multimed/ljpg.tar.Z (lossless jpeg)
>   http://idt.net/~dclunie/jpegls.html  (lossless jpeg by David Clunie)
were not tested at all: no executables were found in these packages:
C sources only, and other interesting texts
(readme files, copyright and portability notices, test images, etc.)

The third link reports that
>  It only handles one component (ie. grayscale images) for now.
while the forth desirable link from that section [15]
>   ppm2pz: (lossless 24-bit image compression)
>       http://www.jyu.fi/~kuru/compression.html
seems to be not valid (since 1997, by the way).


Some WARNINGS again:
~~~~~~~~~~~~~~~~~~~~
  ESP 1.92 has some bugs and was unable to correctly uncompress
  some test images: they are (re)named in our tests as
                AYMAR,GELSNGR,NACHTSH,OTELLIN,VADASZ,YASSO,YU1,
                DP08189,DP2179D,L217C-3,CONCEP3,PS09157,V42798C.
  ESP doesn't even report something like "CRC fails"...as if everything is OK.

   Rkive 1.92b1 failed to losslessly uncompress most images
   from 6th, 7th and 8th tests, and also Clegg, Frymire, Serrano from 5th.
   It reports
   ERROR: File fails CRC check!.
   Archive failed integrity check.

 RKim 1.03 failed to compress 3 images (AIRPLANE,GIRL3,SPLASH), and to extract
 some images compressed with "cx" (CBARRETT,CHOU,GELSING2,DP011199,SP011199,
 ED50799B,ED50799E,CONCEPT4,VC42798C,LIDOCAN1,LIDOCARP,RAPIDACC,COULSON).

  RK 1.02a1 fails to extract set of two exactly-same bitmap files:
  For example, "copy kodak01.bmp kodak01a.bmp",
  and try "RK -mf1 Z kodak01*.bmp" , "RK -i Z"
  The same happens if you try RK with any of -mx1, -mx2, -mx3, -mf1, -mf2, -mf3
  ERROR 303: CRC check failed.


The LATEST RELEASE, and twelve previous versions of these tests can be found at
http://geocities.com/SiliconValley/Bay/1995/
http://artest1.tripod.com/


The FINAL PART
==============
>     [[5]] PLEASE read THIS before replying to this article
was removed from this text, but can be easily found at
http://geocities.com/SiliconValley/Bay/1995/artest10.html
http://artest1.tripod.com/artest10.html

Send your suggestions, comments to ratush@srsc-gw.sscc.ru
With best kind regards,
RAO Inc.