                                The Art
                              of lossless
                           image compression
                                vol. 16

Here are the results of 10 tests performed in June 2000 to compare
lossless compression of True Color (24-bit) images by all known programs
developed for such purpose, including BMF, JPEG-LS and PNG software,
7-Zip, Pkzip, and archivers with special algorithms for 24-bit images.

See Archive Comparison Test by Jeff Gilchrist for more details: http://ACT.i.am

If anybody wants to start or continue such tests,
or can suggest some other sets of images, or other compression programs,
 (not sources or algorithm descriptions, programs for DOS or Windows only)
or knows we have missed something important,
 (some new fantastic technology, an algorithm or even a program capable
 of lossless compression of up to 1000:1 and lossy compression of
 up to 10000:1, we hear about it from time to time since our XTs)
please let us know immediately: ratush@srsc-gw.sscc.ru   Thank you!


[[1]] COMPRESSION QUALITY
=========================
             (see also
             [[2]] Time
             [[3]] Details
             [[4]] Comments)

The last eleventh line shows results for the sum of all ten tests (336 images)

Original  BMF    RKim     RK     UHIC  ArHanGeL   NK     Eri     RAR     Imp
275.95%  100%   101.59  120.20  100.55  110.52  112.65  117.81  133.15  154.65
289.11%  100%   124.50  127.28  104.88  114.61  135.54  122.36  140.60  152.88
245.22%  100%   101.38  123.16  107.93  112.92  111.48  123.50  124.25  144.52
238.64%  100%   112.66  124.20  104.20  110.53  120.45  121.23  132.94  146.59
457.28% 101.93   100%   133.41  110.74  112.34  120.22  125.22  146.11  157.63
234.93%  100%   101.72  119.42  107.07  110.08  106.85  121.92  125.63  143.10
278.95%  100%   103.17  123.14  103.40  109.60  110.16  119.57  135.88  146.37
332.49%  100%   102.12  126.63  105.92  111.58  112.18  120.18  139.49  153.56
225.19%  100%   104.47  118.81  104.00  108.82  110.47  118.20  122.22  141.08
547.50%  100%   104.25  131.46  104.84  107.76  127.75  115.26  170.38  157.56
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
286.32%  100%   102.98  122.95  105.47  110.35  111.15  120.43  133.89  147.68

JPEG-LS  SPIHT   BTPC  {       Display        }  SEA   PngCrush  7-Zip   PkZip
149.81% 115.50  164.36  157.93  156.06  153.89  150.94  150.33  159.16  169.90
156.41% 136.11  170.42  177.21  169.62  164.86  163.71  151.73  205.21  211.56
119.20% 111.39  129.21  140.98  141.56  139.53  146.28  129.83  193.86  196.55
127.45% 122.73  137.49  146.42  145.46  144.00  147.31  134.68  182.24  185.85
172.49% 224.13  201.52  187.28  156.67  156.16  160.57  145.35  185.82  188.99
112.56% 111.61  119.89  129.58  132.73  129.30  137.85  120.59  170.09  172.74
118.69% 119.12  131.29  140.90  139.04  137.13  143.73  129.64  171.53  175.85
123.33% 117.86  136.63  145.92  143.78  142.44  151.77  133.46  197.76  202.49
117.84% 109.95  122.53  133.78  135.20  132.58  139.93  121.86  179.50  181.44
151.17% 132.31  176.94  164.21  150.26  148.74  156.87  142.19  186.28  195.55
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
121.48% 117.50  132.78  140.67  139.78  137.42  144.92  128.95  179.40  183.54


[[2]] Time
==========
A set of 230 images totalling 271,331,460 bytes
from ftp://ipl.rpi.edu/pub/image/still/KodakImages/color/
was used for this test and an AMD-K6-400 machine with 64M RAM and Windows98.
Well-known 23 Kodak images copied ten times (see icopy.bat in artest16.zip)

 Programs,options   Overall Average Compress Extract Compressed
                     score,  Users'   time,   time,    size,
                    seconds  score   seconds seconds   bytes

+7-zip  a            44471   43736     817     137   156662222
+7-zip  a -mx        44960   43743    1352     138   156492462
+arhangel a -mm -mz  34295   31998    2552    1557   108669974
+arhangel a -mm      34260   31989    2523    1551   108669974
+arhangel a -mm1     48908   45779    3476    2633   154074824
+arhangel a -mm3     42870   40214    2951    1879   136944664
 bmf   -f  -s  -q9   38628   30978    8500    2816    98323280
 bmf   -f  -s        33477   30467    3345    2820    98323280
 bmf   -f  -q9       33681   29829    4280     249   104947560
 bmf   -f            30324   29572     836     258   105229600
 eri   a -3ma8       31885   30118    1963     449   106102020
 eri   a -3ma8 -m3   31413   30078    1483     435   106182410
 eri   a -3ma8 -m2   31255   30290    1072     355   107379300
 eri   a -3ma8 -m1   32073   31299     860     299   111292130
+imp98   a -2        38040   37366     749     222   133449566
+imp98   a -2  -s4   38123   37453     744     218   133779179
+imp98   a -mm       43242   42363     977     110   151758474
+imp98   a -mm -s4   43204   42299    1006     109   151521544
+imp98   a -mm -m3   43528   42174    1505     119   150856163
+imp98   a -mm -m1   46105   45522     648     111   163246850
+imp-dos a -mm -m3   43565   42194    1524     137   150855710
 Nk                  35338   31666    4080     489   110767370
 pkzip  -exx         46871   46394     530     146   166302252
 pkzip  -a           46708   46380     365     146   166309392
 pkzip  -es          55426   55211     238     152   198128322
+pkzip32 -add -max   47059   46592     519     138   167047382
+pkzip32 -add        47018   46662     396     136   167349362
+WinRar  a -mm -mde  37138   36685     504     259   130951080
+WinRar  a -mm -m5   37149   36678     523     259   130921330
+WinRar  a -mm -s-   37137   36685     503     259   130951080
+WinRar  a -mm -s    37124   36648     529     258   130813550
+WinRar  a -mmf      36740   36324     462     267   129638820
+rar32   a -mm -mde  37254   36731     582     297   130951080
+rar32   a -mm -m5   37263   36724     599     297   130921330
+rar32   a -mm -s-   37253   36730     581     297   130951080
+rar32   a -mm -s    37236   36692     605     294   130813550
+rar32   a -mmf      36860   36371     543     306   129638820
+rk    -mx2          27554   16839   11906   12305    12035652
+rk    -mx1          46827   37353   10527   11174    90452920
+rk    -mf3           7011    5469    1713    1191    14784112
+rk    -mf2           7863    6571    1435    1176    18906828
+rk    -mf1          39048   37937    1235     887   132934712
+rkim  cx            35399   30769    5144    2506    99896080
+rkim  c             32861   31261    1778    2178   104059250
 uhic  m             36260   32373    4319    4477    98870920
 uhic  e             32812   30565    2496    2501   100132460
 Alchemy ---n        46843   44143    3000     526   155940956
         ---n 3      46602   44584    2242     648   157361419
         ---n 2      46338   44670    1853     437   158572986
         ---n 1      45301   43610    1879     434   154758370
 SEA /png:100        42487   41480    1119     142   148414950
 Display -i --same -bpng
+  filter=5,level=9  44739   42446    2547     159   151317180
+  filter=5,level=1  44203   43805     442     168   156934080
+  filter=1,level=9  44739   42537    2447     149   151714120
+  filter=1,level=5  43955   43263     769     150   154930330
+  filter=1,level=1  44605   44292     348     156   158763080
+pngcrush -q         49026   42265    7512     455   147813040
+pngcrush -brute    165325   53632  124103     460   146744530
+btpc  100           45618   45209     454     273   161606980
+lsp   -g            42530   42119     456     375   150115060
+lsp                 32507   32034     526     434   113570440
+locoe -N -c1        41544   41271     303     323   147305210
+locoe -N -c2        41732   41406     362     342   147701630
-PhotoJazz           ?????   ?????     ???     ???   111673670

+ indicates new lines in current volume of ARTest
- indicates oldest lines, that will not be present in next volume.

Overall score is calculated by adding compression time, extraction time, and
time it would take to transfer the compressed file over a 28,800bps network:
(compressed_size)/3600 , because 28800 bits_per_second is 3600 bytes_per_second

Average Users' score is calculated by adding (compress_time/10)+ extract_time +
time it would take to transfer the compressed file over a 28,800bps network.
Compression time is divided by 10 here, because more than 90% of people would
never compress anything during their life (with compression programs), but they
use compressed data almost _every_ time they use computers and/or Internet.
That's why compression time is not so actual for them.


[[3]] Details
=============
are no longer put to this main text
(380 lines reporting 10080 results on 336 files in 10 sets),
but can be found in FULL version with DETAILS.DAT and *.BAT
at http://geocities.com/SiliconValley/Bay/1995/artest16.zip
or http://artest1.tripod.com/artest16.zip

1,2,3,4,9 image sets were taken from ftp://ipl.rpi.edu/pub/image/ and /image2/
converted from Sun Raster format to Windows Bitmap,
(this conversion changes image headers only, 32 bytes
are replaced by 54 bytes, but image data is not modified)

This site ipl.rpi.edu is described in comp.compression FAQ by Jean-loup Gailly,
in 29 lines of "Subject: [55] Where can I find Lenna and other images?"

Eight images in the 5th set are from the famous Waterloo Color Set,
         ( losslessly compressed with Eri32 4.5fre to 2,894,045 bytes,
           it can be found at http://geocities.com/eri32 )
6,7,8th image sets are from  ftp://download.intel.com/pressroom/images/
10th set was downloaded from ftp://ftp.kodak.com/www/images/dental/
They were converted from TIFF format to Windows Bitmap.

Some images were slightly renamed. See DETAILS.DAT from artest16.zip for more
details. Some new images (from download.intel.com) were not added yet.


[[4]] Comments
==============
Links to download programs:
~~~~~~~~~~~~~~~~~~~~~~~~~~~
7-Zip  2.11   :W http://www.7-zip.com/dl/7zip211.exe                              493K
ArHanGeL 1.40 :a http://geocities.com/SiliconValley/Lab/6606/arh140.zip            50K
BMF    1.1    :e ftp://ftp.simtel.net/pub/simtelnet/win95/graphics/bmf_1_10.zip   132K
ERI32  4.5fre :e http://geocities.com/eri32/eri45fre.zip                           90K
Imp-dos 1.1   :e http://www.winimp.com/imp110d.zip                                266K
Imp    1.12   :W http://www.winimp.com/imp112.exe                                 122K
PkZip   2.50  :a ftp://ftp.simtel.net/pub/simtelnet/msdos/arcers/pk250dos.exe     202K
PkZip32 2.50  :W ftp://ftp.simtel.net/pub/simtelnet/win95/compress/pk250c32.exe   300K
RK     1.02a5 :W http://malcolmt.tripod.com/downloads/rk102a05.exe                191K
RKim   1.06   :W http://malcolmt.tripod.com/downloads/rkim106.zip                  57K
RAR32  2.71   :e ftp://ftp.netlab.sk/public/rarsoft/rar/rarx271.exe               257K
WinRAR 2.71   :W ftp://ftp.netlab.sk/public/rarsoft/rar/wrar271.exe               588K
UHIC   1.0    :e ftp://ftp.elf.stuba.sk/pub/pc/pack/uhic10.zip                     52K
NK     Nov'99 :W ftp://tu.tusur.ru/pub/ImageTools/compress.zip                     81K
JPEG-LS 1.00  :W http://www.hpl.hp.com/loco/jlsrefV100.zip                        325K
PngCrush 1.4.4:e http://download.sourceforge.net/pmt/pngcrush-1.4.4-dos.zip       123K
SPIHT  Nov'96 :W http://www.cipr.rpi.edu/research/SPIHT/EW_Code/lsp.exe            72K
BTPC   4.1    :W http://www.engr.mun.ca/~john/btpc4.1.zip                         715K
Display 1.90b5:e http://www.freenet.edmonton.ab.ca/~crnelson/disp.exe             811K
Alchemy 1.10  :a ftp://ftp.elf.stuba.sk/pub/pc/graph/alch110.zip                 1462K
SEA 1.3       :e ftp://ftp.elf.stuba.sk/pub/pc/graph/sea13.zip                    705K

:a - DOS programs, will run under pure DOS or in a Windows DOS box
:e - DOS programs using DOS extenders like DOS/4GW or CWSDPMI
:W - Windows95/98/NT programs

If direct link doesn't work-most probably newer version of the program appeared
at the same site: visit web page, or read the whole directory from ftp server
(i.e. try the same URL, but without filename).


Homepages:
~~~~~~~~~~
Arhangel     : http://geocities.com/SiliconValley/Lab/6606
Eri32        : http://geocities.com/eri32
      mirror : http://artest1.tripod.com
RK, RKim     : http://malcolmt.tripod.com
Imp,WinImp   : http://www.technelysium.com.au
      mirror : http://www.winimp.com
PkZip        : http://www.pkware.com
Ufa,777,7-Zip: http://www.7-zip.com
RAR,WinRAR   : http://www.rarsoft.com
JPEG-LS      : http://www.hpl.hp.com/loco
PngCrush     : http://pmt.sourceforge.net/pngcrush
SPIHT        : http://www.ipl.rpi.edu/research/SPIHT
BTPC         : http://www.engr.mun.ca/~john/btpc.html
Display      : http://fn2.freenet.edmonton.ab.ca/~crnelson/display.html
Alchemy      : http://www.handmadesw.com/Products/Image_Alchemy.htm
SEA          : http://www.photodex.com/products/dos

What's new:
~~~~~~~~~~~
14 new programs were tested: 7-Zip 2.11, ArHanGeL 1.40, JPEG-LS 1.00, RK 1.02a5,
RKim 1.06, PkZip32 2.5, Imp 1.1 for DOS, Imp 1.12 for Windows95/98/NT, BTPC 4.1,
PngCrush 1.4.4, WinRAR 2.70, RAR32 for DOS and OS/2 2.70, SPIHT, Display 1.90b5.

33 new images from ftp://download.intel.com/pressroom/images were added to
6th, 7th and 8th sets. Two of them turned out to be 8-bit grayscale.
UHIC was the only program unable to compress grayscale images, and sizes of
uncompressed files were added, as if it could store them without compression.


WARNINGS:
~~~~~~~~~
  ESP 1.92 has some bugs and was unable to correctly uncompress
  some test images: they are (re)named in our tests as
                AYMAR,GELSNGR,NACHTSH,OTELLIN,VADASZ,YASSO,YU1,
                DP08189,DP2179D,L217C-3,CONCEP3,PS09157,V42798C.
  ESP doesn't even report something like "CRC fails"...as if everything is OK.

   Rkive 1.92b1 fails to losslessly uncompress most images
   from 6th, 7th and 8th tests, and also Clegg, Frymire, Serrano from 5th.
   It reports
   ERROR: File fails CRC check!.
   Archive failed integrity check.

 ESP, Rkive and some other programs are not tested any more,
 their results and links can be found in previous versions:
 ESP   - artest3...artest12
 Rkive - 3...12
 UFA   - 3...12
 777   - 13...14
 BOA   - 7...14
 UHARC - 5...10
 CompuShow 2000! - 3...11

The LATEST RELEASE, and thirteen previous versions of these tests can be found
at http://geocities.com/SiliconValley/Bay/1995/ and http://artest1.tripod.com/


The FINAL PART
==============
>     [[5]] PLEASE read THIS before replying to this article
was removed from this text, but can be easily found at
http://geocities.com/SiliconValley/Bay/1995/artest10.html
http://artest1.tripod.com/artest10.html

Send your suggestions, comments to ratush@srsc-gw.sscc.ru
With best kind regards,
RAO Inc.
