From glennrp at gmail.com  Wed Jan 27 15:31:14 2010
From: glennrp at gmail.com (Glenn Randers-Pehrson)
Date: Wed, 27 Jan 2010 10:31:14 -0500
Subject: [Png-mng-security] vulnerability in png_decompress_chunk()
Message-ID: <e56ccc8f1001270731o5863b5c4p3d5a35941463c073@mail.gmail.com>

If you are wondering why I've pushed libpng-1.2.43beta01

The png_decompress_chunk() function bogs down when
reading iCCP, zTXt, or iTXt chunks that contain a zlib
datastream that expands considerably when decompressed.

A PNG has been found in the wild that has a small iCCP
chunk that expands to 60 MB.  This stalls Firefox for
about half an hour.

See http://www.simplesystems.org/users/glennrp/hanger
(best to view this without any valuable tabs open because
you'll lose them if you decide to kill the browser before
you regain control).

This means that a malevolent web page could contain
such a PNG with an iCCP chunk designed to stall the
browser for a predetermined period.

libpng-1.4.1beta05, libpng-1.0.53beta02, and libpng-1.2.43
address this by changing the way png_decompress_chunk()
acquires memory.  Previous versions add 8092 bytes at
a time, each time doing a memcpy of the entire expanded
chunk.  The new versions double the memory request
each time.

A more feeble way of avoiding the problem, somewhat, in
earlier libpng versions is to increase png_ptr->zbuf_size
with png_set_compression_buffer_size () so the increment
is larger.  That is what libpng-1.4.0 provides.

Libpng-1.4.1beta05 also provides png_set_chunk_malloc_limit()
which we will use in Firefox to limit the expanded iCCP chunk
to 4MB.  In earlier versions this can be accomplished by
using a replacement malloc_fn that has a (e.g., 4MB) limit.

With libpng-1.4.1beta05, the example page only stalls for
an imperceptible .08 sec.  With the 4MB memory limit
and expanding png_ptr->zbuf-size to 32kbytes, it stalls
for several seconds.

The relevant mozilla bug report is hidden behind a
security setting.  It was opened last June.

Glenn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osuosl.org/pipermail/png-mng-security-archive/attachments/20100127/f1e2273f/attachment.html>

From jbowler at frontiernet.net  Wed Jan 27 18:43:09 2010
From: jbowler at frontiernet.net (John Bowler)
Date: Wed, 27 Jan 2010 10:43:09 -0800
Subject: [Png-mng-security] vulnerability in png_decompress_chunk()
In-Reply-To: <e56ccc8f1001270731o5863b5c4p3d5a35941463c073@mail.gmail.com>
References: <e56ccc8f1001270731o5863b5c4p3d5a35941463c073@mail.gmail.com>
Message-ID: <001701ca9f80$93340290$b99c07b0$@net>

Can you make the width/height of hang20min.png something bigger than ?1? ? like ?100??  The issue is that it's impossible to see when the problem PNG appears.

I see sluggishness on IE8 too (before the first not-a-png appears), but I'm pretty sure it's not doing any ICC stuff so I suspect it may just be inter-web-site speed.

I'm not convinced that this is the right fix anyway.  It doesn't actually *fix* anything, it just moves the problem set out to a larger decompressed size.  The file can be displayed no problem in Word2007, and Word *does* do full ICC on PNG images.  Of course Word doesn't use libpng, but I'm pretty sure it doesn't use a massive buffer size increments either.

In any case if denial-of-service is possible on Firefox using *this* image I don't see why, with your fix, DoS won't be possible with a different, larger but still not massive, PNG (hang20min is 57.3Kbyte, but people routinely download JPEGs that are 200Kbyte.)

I guess you could justify a particular limit based on the Zlib maximum window size and the maximum code length permitted by LZ compression, but without that analysis I don't think we can be sure of the effectiveness of any change.

John Bowler <jbowler at acm.org>





From jbowler at frontiernet.net  Thu Jan 28 00:45:59 2010
From: jbowler at frontiernet.net (John Bowler)
Date: Wed, 27 Jan 2010 16:45:59 -0800
Subject: [Png-mng-security] vulnerability in png_decompress_chunk()
In-Reply-To: <001701ca9f80$93340290$b99c07b0$@net>
References: <e56ccc8f1001270731o5863b5c4p3d5a35941463c073@mail.gmail.com>
	<001701ca9f80$93340290$b99c07b0$@net>
Message-ID: <003301ca9fb3$42c130c0$c8439240$@net>

Executive summary: this is the wrong fix.

Details: I can reproduce the problem in Midori (running on gentoo).  It also shows in 'TweakPNG' (in the image viewer).  This problem is one instance of a general class of DoS attacks that libpng is vulnerable to.  The general case is as follows:

1) PNG allows compressed ancillary data in addition to the compressed image data.  The ancillary data is compressed using zlib (LZ77 'deflate' compression) and may therefore expand up to just under 258*4 times.  See the deflate definition in ftp://ftp.isi.edu/in-notes/rfc1951.txt; note that the length of a duplicated string is limited to 258 in section (2) on page 3 and that the minimum code size is 2 bits for each length/distance.)

2) PNG allows an unlimited number of such blocks of compressed ancillary data.

3) When libpng reads a PNG image it decompresses any compressed ancillary data that it recognizes on the fly (rather than on demand), consequently all compressed ancillary data is stored for a time in decompressed form in memory.  The best case performance that can be obtained with a worst-case PNG file is thus a memory requirement about 1000 times the file size.

The general case means that, with the currently libpng implementation, it is trivial to consume memory beyond the available RAM on typical systems.  For example a 4MByte PNG file could consume at least 4GByte of memory and a 250kbyte PNG file would expand to at least 250MByte of memory.  In the test case the actual expansion was from 58,339 bytes of compressed data to 60,000,000 bytes when uncompressed.  That's 4*257.11 : 1 (so very close to the theoretical no-overhead limit of 4*258:1).  I conclude that the test dataset is the worst case expansion.

All users of libpng are open to the memory based attack, because libpng unconditionally decompresses the compressed data and stores the result in memory.  There is no fix to the general case because the decompressed data is exposed in the ABI (in 1.2 and 1.4, although 1.4 deprecates access to it.)

The reported test case combines a small version of the memory DoS case with an additional implementation bug in libpng.  The implementation bug is as follows:

1) libpng decompresses all such ancillary data using an function png_decompress_chunk (see the head of pngrutil.c).  This function (pngrutil.c line 305) decompresses into a fixed size temporary buffer then copies the buffer into an output buffer.  The output buffer is reallocated for each new block of data from the temporary buffer by allocating a new output buffer, copying the old and freeing the old.  Thus the temporary memory requirement is twice the decompressed size (120MByte in the test case.)  At the same time memory is smashed by the repeated rapid malloc/free.

2) Because typical malloc implementations behave poorly with this access pattern the decompression takes a long time.  In other words it is the performance of the malloc implementation not the simple fact of the amount of memory used that causes the problem in the test case.

Here are some examples based on a 1GByte/1GHz single CPU x86 gentoo system based on the time to decompress the sample data using a very simple program based on zlib.  The decompression uses a 1024 byte input buffer:

0.6s: decompress using a 1024 byte (stack) temporary buffer, do nothing with the result
0.7s: decompress using a 60MByte (malloc) temporary buffer, do nothing with the result
1.3s: decompress using a 1024 byte stack buffer to find the size, allocate a full size buffer (60MByte) and decompress again to that
26s:  decompress to a 1024 byte buffer and write the result to a file using stdio (1024 bytes at time.)
26s:  find the size, allocate a 60MByte buffer, decompress to it and write the whole thing to a file (60Mbytes in one fwrite.)

I hope it's obvious that, while it doesn't fix the general case DoS the speed symptom should be fixed by the decompress twice approach (1.3s).

It is, perhaps, worth noting that it is almost invariably better to double the CPU time in order to replace multiple malloc/free or realloc calls with a single call to malloc.

John Bowler <jbowler at acm.org>






From jbowler at frontiernet.net  Thu Jan 28 18:07:42 2010
From: jbowler at frontiernet.net (John Bowler)
Date: Thu, 28 Jan 2010 10:07:42 -0800
Subject: [Png-mng-security] vulnerability in png_decompress_chunk()
In-Reply-To: <e56ccc8f1001272044g657074b6of7ccff95e01e8700@mail.gmail.com>
References: <e56ccc8f1001270731o5863b5c4p3d5a35941463c073@mail.gmail.com>	
	<001701ca9f80$93340290$b99c07b0$@net>	
	<003301ca9fb3$42c130c0$c8439240$@net>
	<e56ccc8f1001272044g657074b6of7ccff95e01e8700@mail.gmail.com>
Message-ID: <008701caa044$ca1b2f80$5e518e80$@net>

Ok; we have a good solution for the time issue.  When combined with the 4MByte *per-chunk* limit in 1.4 that causes the bogus chunk to be rejected with an imperceptible delay.

The general case is still present, but 1.4 includes limits on both the number of ancillary data chunks and the size of each of those chunks.  Applications have to invoke these limits to avoid vulnerability.

My "general case" remains in 1.2 and the majority of installed systems run a 1.2 shared library.  Since 1.2 is in maintenance mode we *can't* fix this with an API change - applications willing to make such a change should, instead, move to 1.4.

So the choices are either to ignore the problem in 1.2 or make an internal change in 1.2.43 that imposes appropriate limits.

I favor the latter, because it is simple, dropping the ancillary data silently doesn't harm image display and it is possible to come up with limits that are sufficiently large that it is very unlikely any real data will be dropped.

I suggest png_decompress_chunk in 1.2 impose a limit of the form <base + factor * compressed-size> where 'base' is 4MByte for profile data (which can't be repeated) and 10KByte for textual data (which can be repeated) and 'factor' is 16.  That means a PNG would have to have thousands of text chunks before it got to be a problem.  In addition I suggest an arbitrary fixed limit of 256 text chunks.

John Bowler <jbowler at acm.org>




From glennrp at gmail.com  Fri Jan 29 20:09:50 2010
From: glennrp at gmail.com (Glenn Randers-Pehrson)
Date: Fri, 29 Jan 2010 15:09:50 -0500
Subject: [Png-mng-security] vulnerability in png_decompress_chunk()
In-Reply-To: <008701caa044$ca1b2f80$5e518e80$@net>
References: <e56ccc8f1001270731o5863b5c4p3d5a35941463c073@mail.gmail.com>
	<001701ca9f80$93340290$b99c07b0$@net>
	<003301ca9fb3$42c130c0$c8439240$@net>
	<e56ccc8f1001272044g657074b6of7ccff95e01e8700@mail.gmail.com>
	<008701caa044$ca1b2f80$5e518e80$@net>
Message-ID: <e56ccc8f1001291209k13e030efgcb5fb8751ddafcc1@mail.gmail.com>

On Thu, Jan 28, 2010 at 1:07 PM, John Bowler <jbowler at frontiernet.net>wrote:

>
> So the choices are either to ignore the problem in 1.2 or make an internal
> change in 1.2.43 that imposes appropriate limits.
>
> I think we have always been reluctant to impose arbitrary (even if
appropriate) limits.  If we do, I think we must allow users to override
them.

This also involves adding member to the png_struct, to count the chunks and
to hold the memory limt.  If we do that, why not go whole hog and add the
set|get functions as well, for the memory limit and the cache limit, as
implemented in 1.4.1?

It has been said that even just *adding* a new exported function breaks
ABI compatibility, but I'm not sure that I really believe that.

I think maybe the best approach is to leave 1.2.42 alone and just tell
people it's time to upgrade if they want to avoid this newly publicized
vulnerabilty.  It will be most important to applications that accept
PNG files with unknown content.  Those are mainly browsers, and
I believe libpng-based browsers are going to be using libpng-1.4.1 very
shortly.

Glenn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osuosl.org/pipermail/png-mng-security-archive/attachments/20100129/9c1fcc52/attachment.html>

From jbowler at frontiernet.net  Fri Jan 29 20:36:53 2010
From: jbowler at frontiernet.net (John Bowler)
Date: Fri, 29 Jan 2010 12:36:53 -0800
Subject: [Png-mng-security] vulnerability in png_decompress_chunk()
In-Reply-To: <e56ccc8f1001291209k13e030efgcb5fb8751ddafcc1@mail.gmail.com>
References: <e56ccc8f1001270731o5863b5c4p3d5a35941463c073@mail.gmail.com>	
	<001701ca9f80$93340290$b99c07b0$@net>	
	<003301ca9fb3$42c130c0$c8439240$@net>	
	<e56ccc8f1001272044g657074b6of7ccff95e01e8700@mail.gmail.com>	
	<008701caa044$ca1b2f80$5e518e80$@net>
	<e56ccc8f1001291209k13e030efgcb5fb8751ddafcc1@mail.gmail.com>
Message-ID: <00b101caa122$cba83940$62f8abc0$@net>

From: Glenn Randers-Pehrson [mailto:glennrp at gmail.com] 
>So the choices are either to ignore the problem in 1.2 or
>make an internal change in 1.2.43 that imposes appropriate limits.

Right.

>I think we have always been reluctant to impose arbitrary (even if
>appropriate) limits.  If we do, I think we must allow users to
>override them.

That's not really an option because:

>It has been said that even just *adding* a new exported function
>breaks ABI compatibility, but I'm not sure that I really believe that.

Adding a new API that *must* be called in order for existing functionality to continue to function breaks the ABI - it's a semantic break rather than a syntactic one, but that's just as bad.

In this case if the call must be made to set the limit (as in 1.4) then either this is an ABI change or the change doesn't fix the vulnerability.  In effect 1.2.43 would be ABI compatible only so long as the (general) vulnerability was not fixed.  That makes it pointless to change 1.2 in this way.

>I think maybe the best approach is to leave 1.2.42 alone and just tell
>people it's time to upgrade if they want to avoid this newly publicized
>vulnerabilty.

A perhaps more correct statement is to say that applications concerned with the general vulnerability (DoS based on memory consumption) have to call new APIs and these APIs are only available in 1.4

This does mean that you will be changing the png_struct or png_info structure in 1.4 (as in the current 1.4.1beta06 - adding a member to a structure visible in the ABI; at extra 4 bytes at the end of png_struct).  IRC that happened several times in 1.2.

John Bowler <jbowler at acm.org>





From glennrp at gmail.com  Fri Jan 29 21:16:33 2010
From: glennrp at gmail.com (Glenn Randers-Pehrson)
Date: Fri, 29 Jan 2010 16:16:33 -0500
Subject: [Png-mng-security] vulnerability in png_decompress_chunk()
In-Reply-To: <00b101caa122$cba83940$62f8abc0$@net>
References: <e56ccc8f1001270731o5863b5c4p3d5a35941463c073@mail.gmail.com>
	<001701ca9f80$93340290$b99c07b0$@net>
	<003301ca9fb3$42c130c0$c8439240$@net>
	<e56ccc8f1001272044g657074b6of7ccff95e01e8700@mail.gmail.com>
	<008701caa044$ca1b2f80$5e518e80$@net>
	<e56ccc8f1001291209k13e030efgcb5fb8751ddafcc1@mail.gmail.com>
	<00b101caa122$cba83940$62f8abc0$@net>
Message-ID: <e56ccc8f1001291316p7d867c00h4f1f5ea1e0b51b84@mail.gmail.com>

On Fri, Jan 29, 2010 at 3:36 PM, John Bowler <jbowler at frontiernet.net>wrote:

> From: Glenn Randers-Pehrson [mailto:glennrp at gmail.com]
> >So the choices are either to ignore the problem in 1.2 or
> >make an internal change in 1.2.43 that imposes appropriate limits.
>
> Right.
>
> >I think we have always been reluctant to impose arbitrary (even if
> >appropriate) limits.  If we do, I think we must allow users to
> >override them.
>
> That's not really an option because:
>
> >It has been said that even just *adding* a new exported function
> >breaks ABI compatibility, but I'm not sure that I really believe that.
>
> Adding a new API that *must* be called in order for existing functionality
> to continue to function breaks the ABI - it's a semantic break rather than a
> syntactic one, but that's just as bad.
>
> In this case if the call must be made to set the limit (as in 1.4) then
> either this is an ABI change or the change doesn't fix the vulnerability.
>  In effect 1.2.43 would be ABI compatible only so long as the (general)
> vulnerability was not fixed.  That makes it pointless to change 1.2 in this
> way.
>


         I said  "If we do [impose limits], I think we must allow users to
         override them."  What I meant by that was that if we impose an
         arbitrary limit in 1.2.43, then we must allow people to override it
         with the set|get functions.   If they don't change their code then
         they get the new limits.

         But we can just as well say if they don't like the new limits,
upgrade
         to 1.4.1 or downgrade to 1.2.42.

 >I think maybe the best approach is to leave 1.2.42 alone and just tell
> >people it's time to upgrade if they want to avoid this newly publicized
> >vulnerabilty.
>
> A perhaps more correct statement is to say that applications concerned with
> the general vulnerability (DoS based on memory consumption) have to call new
> APIs and these APIs are only available in 1.4
>

        Yes.  If they want to avoid the newly publicized vulnerability and
        unnecessary memory consumption.

>
> This does mean that you will be changing the png_struct or png_info
> structure in 1.4 (as in the current 1.4.1beta06 - adding a member to a
> structure visible in the ABI; at extra 4 bytes at the end of png_struct).
>  IRC that happened several times in 1.2.
>

        It did happen, when I thought it was OK to do that.  I am now told
that
        there are platforms where you cannot even add new members at the
        end.

        I guess then, 1.4.1 should become 1.5.0 (and 1.5.0alpha01 in the GIT
        repository should become 1.6.0alpha01).

        People are already angry about having to "clean up after the libpng
        developers" for suddenly removing png_check_sig() [with only about
        13 years' notice!]

        Glenn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osuosl.org/pipermail/png-mng-security-archive/attachments/20100129/dcfbaa0f/attachment.html>

From bfriesen at simple.dallas.tx.us  Fri Jan 29 21:21:13 2010
From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn)
Date: Fri, 29 Jan 2010 15:21:13 -0600 (CST)
Subject: [Png-mng-security] vulnerability in png_decompress_chunk()
In-Reply-To: <e56ccc8f1001291209k13e030efgcb5fb8751ddafcc1@mail.gmail.com>
References: <e56ccc8f1001270731o5863b5c4p3d5a35941463c073@mail.gmail.com>
	<001701ca9f80$93340290$b99c07b0$@net>
	<003301ca9fb3$42c130c0$c8439240$@net>
	<e56ccc8f1001272044g657074b6of7ccff95e01e8700@mail.gmail.com>
	<008701caa044$ca1b2f80$5e518e80$@net>
	<e56ccc8f1001291209k13e030efgcb5fb8751ddafcc1@mail.gmail.com>
Message-ID: <alpine.GSO.2.01.1001291519490.17824@freddy.simplesystems.org>

On Fri, 29 Jan 2010, Glenn Randers-Pehrson wrote:
> 
> It has been said that even just *adding* a new exported function breaks
> ABI compatibility, but I'm not sure that I really believe that.

It certainly can.  For example, Cygwin only knows the major version 
('current') of the library and it is embedded in the library name.  If 
the library name was not changed, then it would probably work.

Bob
--
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/


From glennrp at gmail.com  Fri Jan 29 21:28:29 2010
From: glennrp at gmail.com (Glenn Randers-Pehrson)
Date: Fri, 29 Jan 2010 16:28:29 -0500
Subject: [Png-mng-security] vulnerability in png_decompress_chunk()
In-Reply-To: <alpine.GSO.2.01.1001291519490.17824@freddy.simplesystems.org>
References: <e56ccc8f1001270731o5863b5c4p3d5a35941463c073@mail.gmail.com>
	<001701ca9f80$93340290$b99c07b0$@net>
	<003301ca9fb3$42c130c0$c8439240$@net>
	<e56ccc8f1001272044g657074b6of7ccff95e01e8700@mail.gmail.com>
	<008701caa044$ca1b2f80$5e518e80$@net>
	<e56ccc8f1001291209k13e030efgcb5fb8751ddafcc1@mail.gmail.com>
	<alpine.GSO.2.01.1001291519490.17824@freddy.simplesystems.org>
Message-ID: <e56ccc8f1001291328g2cbc8bb4s73cca1521dcea078@mail.gmail.com>

On Fri, Jan 29, 2010 at 4:21 PM, Bob Friesenhahn <
bfriesen at simple.dallas.tx.us> wrote:

> On Fri, 29 Jan 2010, Glenn Randers-Pehrson wrote:
>
>>
>> It has been said that even just *adding* a new exported function breaks
>> ABI compatibility, but I'm not sure that I really believe that.
>>
>
> It certainly can.  For example, Cygwin only knows the major version
> ('current') of the library and it is embedded in the library name.  If the
> library name was not changed, then it would probably work.
>

Right, but the question is "does adding a new exported function break ABI
compatibility in e.g., Cygwin?"

1.4.1 is supposed to be ABI compatible with 1.4.0.  If it is not, then we
have
to go to 1.5.0 and libpng15.so.

One approach I explored in Firefox was to use a memory limit equal
to the zbuf_size times the cache_limit, both of which can be set by
function calls in libpng-1.4.0.

Changing the internal algorithm of png_compress_chunk() doesn't
break anything.

Glenn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osuosl.org/pipermail/png-mng-security-archive/attachments/20100129/334cd83b/attachment.html>

From jbowler at frontiernet.net  Fri Jan 29 22:52:45 2010
From: jbowler at frontiernet.net (John Bowler)
Date: Fri, 29 Jan 2010 14:52:45 -0800
Subject: [Png-mng-security] vulnerability in png_decompress_chunk()
In-Reply-To: <e56ccc8f1001291316p7d867c00h4f1f5ea1e0b51b84@mail.gmail.com>
References: <e56ccc8f1001270731o5863b5c4p3d5a35941463c073@mail.gmail.com>	
	<001701ca9f80$93340290$b99c07b0$@net>	
	<003301ca9fb3$42c130c0$c8439240$@net>	
	<e56ccc8f1001272044g657074b6of7ccff95e01e8700@mail.gmail.com>	
	<008701caa044$ca1b2f80$5e518e80$@net>	
	<e56ccc8f1001291209k13e030efgcb5fb8751ddafcc1@mail.gmail.com>	
	<00b101caa122$cba83940$62f8abc0$@net>
	<e56ccc8f1001291316p7d867c00h4f1f5ea1e0b51b84@mail.gmail.com>
Message-ID: <00be01caa135$c666b980$53342c80$@net>

From: Glenn Randers-Pehrson [mailto:glennrp at gmail.com] 
>        I guess then, 1.4.1 should become 1.5.0 (and 1.5.0alpha01 in the GIT
>        repository should become 1.6.0alpha01).

Definitely *NOT*.

This mess arises because the two C structs 'png_struct' and 'png_info' are declared in the public header file, png.h.  *Use* of these definitions beyond the jmp_buf member at the head has been deprecated for aeons.  Release 1.4 eliminates the need for access to the jmp_buf, therefore it eliminates the need to include png_struct in the header file (png_info hasn't been needed for a long time).

For certain release 1.5 (or 1.6, whatever) should no longer include either png_struct or png_info in *any* of the installed header files.

Perhaps 1.4.1 should have a define to remove everything that won't be in 1.5 just to make this easier.

>        People are already angry about having to "clean up after the libpng
>        developers" for suddenly removing png_check_sig() [with only about
>        13 years' notice!]

Software writers always complain if they have to maintain the software they've written.

John Bowler <jbowler at acm.org>