What program is best at JPEG compression?

On Aug 1, 6:26 pm, Bill Tuthill <ccree....RemoveThis@yahoo.com> wrote:
> In rec.photo.digital Martin Brown <|||newspam...@nezumi.demon.co.uk> wrote:
>
> > If you want to send me a set of tiny 16x16 images compressed at each
> > of the 12 different levels of the current Photoshop release I will
> > compute what their approximate IJG equivalent is. The public utility
> > DUMPJPEG doesn't give very accurate answers on coarse quantised
> > heavily compressed images.
>
> Deal. Thanks very much. Do you want the red/blue squares 16x16,
> or something else?

Hi Bill,

Anything will do - although you may find it amusing to try the red/
blue pattern out on levels 6,7.
White cat in snowstorm or black cat in coal cellar will be marginally
smaller. Just dump them in a directory and post the URL if that is
more convenient for you.
>
> Gordon Richardson's articlehttp://photo.net/learn/jpeg/
> contains this similar information, but note 10 is missing:
> <QUOTE>
> Photoshop Jpeg's do not use the IJG tables, so their
> equivalent quality can only be estimated:
>
> 48X32_12: Approximate quality 98 horizontal sampling 1
> 48X32_11: Approximate quality 94 horizontal sampling 1
> 48X32_09: Approximate quality 91 horizontal sampling 1
> 48X32_08: Approximate quality 88 horizontal sampling 1
> 48X32_07: Approximate quality 83 horizontal sampling 1
> 48X32_06: Approximate quality 86 horizontal sampling 2
> 48X32_05: Approximate quality 82 horizontal sampling 2
> 48X32_04: Approximate quality 77 horizontal sampling 2
> 48X32_03: Approximate quality 73 horizontal sampling 2
> 48X32_02: Approximate quality 62 horizontal sampling 2
> 48X32_01: Approximate quality 52 horizontal sampling 2
> </QUOTE>

They look very believable. I have the figures for the old 10 level
Photoshop somewhere, but I would like to see if the new version is
doing anything different. Thanks for your help. Obviously there are
more levles now.

BTW Certain versions of Photoshop appear not to recognise their own
embedded JPEG thumbnails - something odd about the NASA workflow nests
multiple identical thumbnails in some of their published images.

Regards,
Martin Brown
 >> Stay informed about: What program is best at JPEG compression? 

On Aug 2, 6:28 am, John Turco <j... RemoveThis @concentric.net> wrote:
> Martin Brown wrote:
>
> > On Jul 31, 12:56 pm, "HEMI-Powered" <n... RemoveThis @none.en> wrote:
>
> <edited, for brevity>
>
> > > That is what mine is. I understand that 350D is the name outside
> > > the U.S. and it is Rebel XT here. Is that right?
>
> > I think that is the case, but maybe there are some other differences.
> > US domestic "pet" names for cameras are something of a mystery to me.
> > I can't see what is wrong with calling it Canon 350D.
>
> Hello, Martin:
>
> The "Rebel" label dates to Canon's 35mm film cameras, in the 1980's.
> There was an advertising campaign, based on it, which featured tennis
> star Andre Agassi (who is considered something of a "rebel," himself,
> apparently).

Hi John,

BOGGLE!
Thanks for the explanation... I'd never have guessed that!

Cheers,
Martin Brown
 >> Stay informed about: What program is best at JPEG compression? 

In rec.photo.digital Martin Brown <|||newspam|||@nezumi.demon.co.uk> wrote:
>>
>> > If you want to send me a set of tiny 16x16 images compressed at each
>> > of the 12 different levels of the current Photoshop release I will
>> > compute what their approximate IJG equivalent is.
>>
>> Deal. Thanks very much.
>
> Anything will do - although you may find it amusing to try the red/
> blue pattern out on levels 6,7.

I scaled down a square portion of the Macbeth chart, leftmost colors,
but Photoshop CS2 refuses to save such a small (16x16) file as JPEG,
or perhaps Photoshop thinks it is a diagram.

Suggestions?
 >> Stay informed about: What program is best at JPEG compression? 

On Aug 3, 5:23 am, Bill Tuthill <ccree... DeleteThis @yahoo.com> wrote:
> In rec.photo.digital Martin Brown <|||newspam...@nezumi.demon.co.uk> wrote:
>
>
>
> >> > If you want to send me a set of tiny 16x16 images compressed at each
> >> > of the 12 different levels of the current Photoshop release I will
> >> > compute what their approximate IJG equivalent is.
>
> >> Deal. Thanks very much.
>
> > Anything will do - although you may find it amusing to try the red/
> > blue pattern out on levels 6,7.
>
> I scaled down a square portion of the Macbeth chart, leftmost colors,
> but Photoshop CS2 refuses to save such a small (16x16) file as JPEG,
> or perhaps Photoshop thinks it is a diagram.

That makes sense - since even in compact "save for web" mode Photoshop
adds a small essay to the header of every JPEG it saves. 32bit BMP
files could prossibly be smaller for anything < 32x32. And all bets
are off it is adds a 160x120 thumbnail in as well!
>
> Suggestions?

Just pick a smallish size that it will let you save. I am only really
interested in the header details so the image itself size or content
doesn't matter. I suggested the tiny size only to save bandwidth.

Regards,
Martin Brown
 >> Stay informed about: What program is best at JPEG compression? 

In rec.photo.digital Martin Brown <|||newspam|||@nezumi.demon.co.uk> wrote:
>>
>> I scaled down a square portion of the Macbeth chart, leftmost colors,
>> but Photoshop CS2 refuses to save such a small (16x16) file as JPEG,
>> or perhaps Photoshop thinks it is a diagram.
>
> That makes sense - since even in compact "save for web" mode Photoshop
> adds a small essay to the header of every JPEG it saves. 32bit BMP
> files could prossibly be smaller for anything < 32x32. And all bets
> are off it is adds a 160x120 thumbnail in as well!

Ha ha! Funny. The problem was that even in PNG it was Indexed color
so after switching Photoshop to RGB colorspace, it could save JPEG.

Here they are, Photoshop quality 0 to 12. It's interesting that, for
this image, PS quality=0 looks better than quality=2.

http://www.cacreeks.com/psp

Also, as previously observed, Photoshop quality < 7 is fairly worthless.
Quality 6 is only a bit smaller than 7 but looks much worse. I have
already run jpegdump so I know 7-6 is where Photoshop switches from
1x1 chroma to 2x2 chroma subsampling. TIA.
 >> Stay informed about: What program is best at JPEG compression? 

On Aug 4, 3:17 am, Bill Tuthill <ccree... RemoveThis @yahoo.com> wrote:
> In rec.photo.digital Martin Brown <|||newspam...@nezumi.demon.co.uk> wrote:

> > That makes sense - since even in compact "save for web" mode Photoshop
> > adds a small essay to the header of everyJPEGit saves. 32bit BMP
> > files could prossibly be smaller for anything < 32x32. And all bets
> > are off it is adds a 160x120 thumbnail in as well!
>
> Ha ha! Funny. The problem was that even in PNG it was Indexed color

Modern PhotoShop JPEGs are more than a bit voluminous. Each one
contains 2 identical thumnails in the header. And the highest quality
image Q=12 has a main JPEG stream 542 bytes long in a file totalling
over 22500 bytes. A back of the envelope calculation says it should
take about 1140 bytes for a genuine fully encoded legal JFIF file.

> Here they are, Photoshop quality 0 to 12. It's interesting that, for
> this image, PS quality=0 looks better than quality=2.

Sometimes happens. Some line art can become exact for lucky
combinations of custom quality table.
>
> http://www.cacreeks.com/psp
>
> Also, as previously observed, Photoshop quality < 7 is fairly worthless.
> Quality 6 is only a bit smaller than 7 but looks much worse. I have
> already run jpegdump so I know 7-6 is where Photoshop switches from
> 1x1 chroma to 2x2 chroma subsampling. TIA.

Thanks very much for the files.

OK here are the results (they match the old quality levels of
PhotoShop 5 + 2). Two extra compressed for web use new levels 0 and 1
have been added. The data below are based on least squares fitting of
the IJG scale factor to the actual PhotoShop quantisation tables with
the total residual as a measure of goodness (or badness) of fit.

I have done Y and C using IJG quality (100=best) separately for
reasons that should be obvious:

Level Y resid C resid
0 35 38k 81 16k
1 38 29k 82 13k
2 47 14k 84 7k
3 60 5k 86 4k
4 67 2250 87 2187
5 74 1003 89 930
6 82 254 91 299
* changes to 1x1 sampling
7 77 708 87 3200
8 85 140 89 1070
9 92 83 90 304
10 95 64 93 32
11 97 39 96 0
12 99 5 99 49

The highest quality 10,11,12 are a pretty good match to IJG. PhotoShop
has finer high frequency quantisation than the IJG tables at the lower
quality settings with a large triangular chunk of the matrix set to
12. The IJG values there are much coarser and explain the huge
residuals for low quality settings. The differences are so huge that
least squares may not be an appropriate fitting criterion - minimum 1
norm might be better.

I reckon PhotoShop tries a bit too hard at accurate quantisation of
the colour by using very high quality settings even at its lowest
quality. The luminence quantisation doesn't catch up until level 8.

Regards,
Martin Brown
 >> Stay informed about: What program is best at JPEG compression? 

In rec.photo.digital Martin Brown <|||newspam|||@nezumi.demon.co.uk> wrote:
>
> Modern PhotoShop JPEGs are more than a bit voluminous. Each one
> contains 2 identical thumnails in the header. And the highest quality
> image Q=12 has a main JPEG stream 542 bytes long in a file totalling
> over 22500 bytes. A back of the envelope calculation says it should
> take about 1140 bytes for a genuine fully encoded legal JFIF file.

Two identical thumbnails, whaaaaat, why?
It seems one can disable thumbnails by clicking off the Preview checkbox.

This supports earlier assertions on this thread not to use Photoshop JPEG,
although now I'm interested in investigating Photoshop SaveForWeb. It
might have been improved since it first came out, but at that time, it
was not able to set 1x1 chroma, only 2x2, which seemed fairly idiotic
at quality levels above IJG 85.

> OK here are the results (they match the old quality levels of
> PhotoShop 5 + 2). Two extra compressed for web use new levels 0 and 1
> have been added. The data below are based on least squares fitting of
> the IJG scale factor to the actual PhotoShop quantisation tables with
> the total residual as a measure of goodness (or badness) of fit.

By "Photoshop 5 + 2" you mean thru version 7? I thought PS version 6
had quality 1, but not 0.

> I have done Y and C using IJG quality (100=best) separately for
> reasons that should be obvious:
> Level Y resid C resid
> 0 35 38k 81 16k
> 1 38 29k 82 13k
> 2 47 14k 84 7k
> 3 60 5k 86 4k
> 4 67 2250 87 2187
> 5 74 1003 89 930
> 6 82 254 91 299
> * changes to 1x1 sampling
> 7 77 708 87 3200
> 8 85 140 89 1070
> 9 92 83 90 304
> 10 95 64 93 32
> 11 97 39 96 0
> 12 99 5 99 49

Thanks a lot, this is a valuable reference for the future.
Presumably Y=luminance, C=chroma. What does resid(ual) indicate?
Are jpegdump's qtable0 and qtable1 related to Y and C encodings?

> The highest quality 10,11,12 are a pretty good match to IJG. PhotoShop
> has finer high frequency quantisation than the IJG tables at the lower
> quality settings with a large triangular chunk of the matrix set to
> 12. The IJG values there are much coarser and explain the huge
> residuals for low quality settings. The differences are so huge that
> least squares may not be an appropriate fitting criterion - minimum 1
> norm might be better.

Yes, the fitting gets worse as the quality levels decrease below 6.
Hmm... I infer resid(ual) indicates accuracy of least squares fitting.

> I reckon PhotoShop tries a bit too hard at accurate quantisation of
> the colour by using very high quality settings even at its lowest
> quality. The luminence quantisation doesn't catch up until level 8.

This might be why Photoshop JPEG looks better at lower quality levels
than IJG, but the lower quality levels look pretty bad anyhow, so
particularly in these days of higher bandwidth, they are irrelevant.
 >> Stay informed about: What program is best at JPEG compression? 

On Aug 6, 7:57 pm, Bill Tuthill <ccree....DeleteThis@yahoo.com> wrote:
> In rec.photo.digital Martin Brown <|||newspam...@nezumi.demon.co.uk> wrote:
>
> > Modern PhotoShop JPEGs are more than a bit voluminous. Each one
> > contains 2 identical thumnails in the header. And the highest quality
> > image Q=12 has a mainJPEGstream 542 bytes long in a file totalling
> > over 22500 bytes. A back of the envelope calculation says it should
> > take about 1140 bytes for a genuine fully encoded legal JFIF file.
>
> Two identical thumbnails, whaaaaat, why?

Who knows. If I were to make a cynical guess it is that one of their
older versions of the program crashes horribly if it doesn't find a
thumbnail labelled "Photoshop 3.0" in the header somewhere.

> It seems one can disable thumbnails by clicking off the Preview checkbox.
>
> > OK here are the results (they match the old quality levels of
> > PhotoShop 5 + 2). Two extra compressed for web use new levels 0 and 1
> > have been added. The data below are based on least squares fitting of
> > the IJG scale factor to the actual PhotoShop quantisation tables with
> > the total residual as a measure of goodness (or badness) of fit.
>
> By "Photoshop 5 + 2" you mean thru version 7? I thought PS version 6
> had quality 1, but not 0.

Dunno about PS6, but PS5 will definitely save at quality level 0 if
you type a zero in the text box. But the old Level 0 is now called 2
in the later versions of software with 12 levels.

> > I have done Y and C using IJG quality (100=best) separately for
> > reasons that should be obvious:
> > Level Y resid C resid
> > 0 35 38k 81 16k
> > 1 38 29k 82 13k
> > 2 47 14k 84 7k
> > 3 60 5k 86 4k
> > 4 67 2250 87 2187
> > 5 74 1003 89 930
> > 6 82 254 91 299
> > * changes to 1x1 sampling
> > 7 77 708 87 3200
> > 8 85 140 89 1070
> > 9 92 83 90 304
> > 10 95 64 93 32
> > 11 97 39 96 0
> > 12 99 5 99 49
>
> Thanks a lot, this is a valuable reference for the future.
> Presumably Y=luminance, C=chroma. What does resid(ual) indicate?
> Are jpegdump's qtable0 and qtable1 related to Y and C encodings?

Yes. There is a possible catch here that at the lower qualities the
quantisation matrix is so unlike the Photoshop one that the least
square solution may not be the optimum approximation. There are a
handful of optimisers about that will tweak the quantisation matrix to
match a given image - NASA holds a US patent on it. eg
http://vision.arc.nasa.gov/dctune/
>
> > The highest quality 10,11,12 are a pretty good match to IJG. PhotoShop
> > has finer high frequency quantisation than the IJG tables at the lower
> > quality settings with a large triangular chunk of the matrix set to
> > 12. The IJG values there are much coarser and explain the huge
> > residuals for low quality settings. The differences are so huge that
> > least squares may not be an appropriate fitting criterion - minimum 1
> > norm might be better.
>
> Yes, the fitting gets worse as the quality levels decrease below 6.
> Hmm... I infer resid(ual) indicates accuracy of least squares fitting.

Yes. Larger means a poor fit, zero means an exact fit. And because it
is the square of the residuals you can get a rough idea of how bad it
is by dividing that number by 64 and taking the square root to get an
average rms error. However, the errors are systematically distributed
so this is rather crude.

Residual 5 means one difference of 1 and one of 2. Residual 64 means
an average error of 1 on all 64 cells, but could be just one cell out
by 8. In reality it will be something inbetween these two extreme
cases.

The worst fit of level 0 luminence with residuals of 38000 translates
to an average error of 24 across every channel. In practice it is
actually due to some very large differences in the quantisation of the
higher spatial frequencies.

Regards,
Martin Brown
 >> Stay informed about: What program is best at JPEG compression?