From ctruta at gmail.com Thu Feb 4 00:58:10 2010 From: ctruta at gmail.com (Cosmin Truta) Date: Wed, 3 Feb 2010 19:58:10 -0500 Subject: [Png-mng-security] vulnerability in png_decompress_chunk() In-Reply-To: <00be01caa135$c666b980$53342c80$@net> References: <001701ca9f80$93340290$b99c07b0$@net> <003301ca9fb3$42c130c0$c8439240$@net> <008701caa044$ca1b2f80$5e518e80$@net> <00b101caa122$cba83940$62f8abc0$@net> <00be01caa135$c666b980$53342c80$@net> Message-ID: <29d80c921002031658k22c9597cy3eeb828d8c9e63f2@mail.gmail.com> On Fri, Jan 29, 2010 at 5:52 PM, John Bowler wrote: > From: Glenn Randers-Pehrson [mailto:glennrp at gmail.com] >> ? ? ? ?I guess then, 1.4.1 should become 1.5.0 (and 1.5.0alpha01 in the GIT >> ? ? ? ?repository should become 1.6.0alpha01). > > Definitely *NOT*. John, why do you insist so much on breaking the ABI? Just bump the minor version number, recompile the software that, anyway, needs recompilation, and you're done. Version numbers are free... > This mess arises because the two C structs 'png_struct' and 'png_info' are declared in the public header file, png.h. ?*Use* of these definitions beyond the jmp_buf member at the head has been deprecated for aeons. The formal act of deprecation is not so relevant. It's the availability of reasonable alternatives that matter. And in this case, libpng is still lacking the alternatives. A few months ago I posted a message in which I was explaining that members of a structure should be hidden only when there is a complete API around it. In libpng, in particular, you can get the data, you can set the data, but you cannot clear the data. Most libpng developers might not care, but I need to clear the data in OptiPNG, and I must use direct access for that reason. > For certain release 1.5 (or 1.6, whatever) should no longer include either png_struct or png_info in *any* of the installed header files. So in my above-mentioned email I proposed the introduction of png_clear_XXX() functions in addition to png_get_XXX() and png_set_XXX(), to which Glenn replied something like "maybe later". That is good enough for me; but please, do not remove the direct access before the API is complete. Moreover, if png_clean_XXX() functions are added in libpng version x.y.z, do not remove the direct access until (at the very least) libpng version x.(y+1).0. >> ? ? ? ?People are already angry about having to "clean up after the libpng >> ? ? ? ?developers" for suddenly removing png_check_sig() [with only about >> ? ? ? ?13 years' notice!] > > Software writers always complain if they have to maintain the software they've written. I agree with your point in general, and PNG_DEPRECATED will probably help in the future. But this is not the issue here. Direct access is still needed. In addition, under-the-radar ABI break can be way worse than just removing a function. When the API breaks, compiler errors occur, and the programmer's intervention (i.e. the fix) is promptly required. But when the ABI breaks, bad things usually happen, without anyone knowing. Please try to avoid ABI breakage, as a general rule. One thing that is very nice in zlib, for instance, is that an obvious run-time error occurs across changes in the size of zlib's structures. Best regards, Cosmin From jbowler at frontiernet.net Thu Feb 4 02:06:21 2010 From: jbowler at frontiernet.net (John Bowler) Date: Wed, 3 Feb 2010 18:06:21 -0800 Subject: [Png-mng-security] vulnerability in png_decompress_chunk() In-Reply-To: <29d80c921002031658k22c9597cy3eeb828d8c9e63f2@mail.gmail.com> References: <001701ca9f80$93340290$b99c07b0$@net> <003301ca9fb3$42c130c0$c8439240$@net> <008701caa044$ca1b2f80$5e518e80$@net> <00b101caa122$cba83940$62f8abc0$@net> <00be01caa135$c666b980$53342c80$@net> <29d80c921002031658k22c9597cy3eeb828d8c9e63f2@mail.gmail.com> Message-ID: <013901caa53e$a63de4f0$f2b9aed0$@net> From: Cosmin Truta >John, why do you insist so much on breaking the ABI? You mean on *not* breaking the ABI. I.e. why am I so very much against an architecture where so many API additions end up breaking the ABI (because they need additions to png_struct or png_info and, because these structures are exposed, this changes the *ABI* even though the *API* is completely backwards compatible.) >Just bump the >minor version number, recompile the software that, anyway, needs >recompilation, and you're done. Version numbers are free... If the ABI changes the major version number must be changed, that's what "major" means in this context; that if it does not change the ABI is unchanged. We're living in a fantasy land. We assert that additions to *end* of png_struct or png_info aren't ABI changes because we effectively but quietly make it illegal for the user of libpng to declare a png_struct or png_info. We also deprecate direct access to png_struct or png_info other than the jmpbuf (and, in 1.4, including the jmpbuf). Despite this some very experienced libpng programmers do directly access the structures because they find the API inadequate: you and either Glenn or Bob (ImageMagick directly accesses 'ping_info', at least in 6.5.2.9). The rules need to be clearer. They need to be obeyed by, at the very least, everyone on this list and preferably everyone with voting rights on png-implement. The entire interface needs to be function based - no tweaking values out of memory. If we need a set of 'clear' functions in 1.5 that's fine by me. What I dislike is the idea that we don't make progress on this now. Better a 1.5.0 that breaks some apps than a 1.5.* that requires a 1.6 the next time we need to add a security related API enhancement. >In addition, under-the-radar ABI break can be way worse than just >removing a function. When the API breaks, compiler errors occur, and >the programmer's intervention (i.e. the fix) is promptly required. But >when the ABI breaks, bad things usually happen, without anyone >knowing. Please try to avoid ABI breakage, as a general rule. I certainly agree with all the above statements. John Bowler From ctruta at gmail.com Thu Feb 4 02:35:09 2010 From: ctruta at gmail.com (Cosmin Truta) Date: Wed, 3 Feb 2010 21:35:09 -0500 Subject: [Png-mng-security] vulnerability in png_decompress_chunk() In-Reply-To: <013901caa53e$a63de4f0$f2b9aed0$@net> References: <003301ca9fb3$42c130c0$c8439240$@net> <008701caa044$ca1b2f80$5e518e80$@net> <00b101caa122$cba83940$62f8abc0$@net> <00be01caa135$c666b980$53342c80$@net> <29d80c921002031658k22c9597cy3eeb828d8c9e63f2@mail.gmail.com> <013901caa53e$a63de4f0$f2b9aed0$@net> Message-ID: <29d80c921002031835t83af659wb7c3691a67c52ca1@mail.gmail.com> On Wed, Feb 3, 2010 at 9:06 PM, John Bowler wrote: > From: Cosmin Truta >>John, why do you insist so much on breaking the ABI? > > You mean on *not* breaking the ABI. ?I.e. why am I so very much against an architecture where so many API additions end up breaking the ABI (because they need additions to png_struct or png_info and, because these structures are exposed, this changes the *ABI* even though the *API* is completely backwards compatible.) Aha, ok, then I misread your previous discussion with Glenn (something that I started doing more frequently, recently, due to lack of time to read the whole discussion thoroughly, and for which I apologize...) I only care to have my programs still working reliably after a libpng upgrade in the system, that's all ;-) >>Just bump the >>minor version number, recompile the software that, anyway, needs >>recompilation, and you're done. Version numbers are free... > > If the ABI changes the major version number must be changed, that's what "major" means in this context; that if it does not change the ABI is unchanged. The upgrade from 1.2 to 1.4 worked, (almost) only with recompilation, and so did the upgrade from 1.0 to 1.2. I don't see why not continue this habit, and leave libpng-2.0.0 for a major overhaul (e.g. no more separation between png_struct and png_info). But if you insist on changing the major version number sooner, given that, perhaps, there will never be such a massive API change, I wouldn't be awfully opposed to it. > We're living in a fantasy land. ?We assert that additions to *end* of png_struct or png_info aren't ABI changes because we effectively but quietly make it illegal for the user of libpng to declare a png_struct or png_info. ?We also deprecate direct access to png_struct or png_info other than the jmpbuf (and, in 1.4, including the jmpbuf). ?Despite this some very experienced libpng programmers do directly access the structures because they find the API inadequate: you and either Glenn or Bob (ImageMagick directly accesses 'ping_info', at least in 6.5.2.9). Well, adding things at the end of png_struct/png_info, almost at every new libpng release, did not break anything for me yet, and I believe, neither for Glenn/Bob. But I don't deny a scenario in which things could break, and I would support removing the contents of the png structures, after the API is complete from the access perspective. > The entire interface needs to be function based - no tweaking values out of memory. I agree that this is a goal to reach, and I think it's reachable. > If we need a set of 'clear' functions in 1.5 that's fine by me. ?What I dislike is the idea that we don't make progress on this now. ?Better a 1.5.0 that breaks some apps than a 1.5.* that requires a 1.6 the next time we need to add a security related API enhancement. When Glenn postponed this project, he did so (I think) because he was busy getting libpng-1.4 out the door. Is there any other major libpng work pending? If not, this could be the next thing to do. I leave it up to you to decide on whether to do it for 1.4 or 1.5. Among other things, that depends on whether the introduction of a new module (e.g. pngclear.c) can or cannot be done in 1.4. Best regards, Cosmin From glennrp at comcast.net Thu Feb 4 12:52:18 2010 From: glennrp at comcast.net (glennrp at comcast.net) Date: Thu, 4 Feb 2010 12:52:18 +0000 (UTC) Subject: [Png-mng-security] vulnerability in png_decompress_chunk() In-Reply-To: <2117083195.3636541265287291859.JavaMail.root@sz0023a.westchester.pa.mail.comcast.net> Message-ID: <1398227657.3640071265287938891.JavaMail.root@sz0023a.westchester.pa.mail.comcast.net> ----- Original Message ----- From: "Cosmin Truta" To: png-mng-security at simple.dallas.tx.us Sent: Wednesday, February 3, 2010 9:35:09 PM GMT -05:00 US/Canada Eastern Subject: Re: [Png-mng-security] vulnerability in png_decompress_chunk() On Wed, Feb 3, 2010 at 9:06 PM, John Bowler wrote: > From: Cosmin Truta >>John, why do you insist so much on breaking the ABI? > >[...] >>Just bump the >>minor version number, recompile the software that, anyway, needs >>recompilation, and you're done. Version numbers are free... > > If the ABI changes the major version number must be changed, that's what "major" means in this context; that if it does not change the ABI is unchanged. Just to be precise, it is the minor version number that gets bumped when the ABI changes. 1.0.x, 1.2.x, 1.4.x are "minor versions". I guess the leading "1." is just a decoration for now, but when we make a libpng that looks really different from 1.x.x, we'll call it 2.0.0. I think making the png_struct opaque probably falls in that category, although we do seem to be sneaking up on that via the PNGDEPSTRUCT thing. > We're living in a fantasy land. ?We assert that additions to *end* of png_struct or png_info aren't ABI changes because we effectively but quietly make it illegal for the user of libpng to declare a png_struct or png_info. ?We also deprecate direct access to png_struct or png_info other than the jmpbuf (and, in 1.4, including the jmpbuf). ?Despite this some very experienced libpng programmers do directly access the structures because they find the API inadequate: you and either Glenn or Bob (ImageMagick directly accesses 'ping_info', at least in 6.5.2.9). I've been maintaining coders/png.c for both ImageMagick and GraphicsMagick, but I didn't design it. It does a lot of stuff that libpng could really be doing for it. Getting rid of those deprecated references has been on my low-priority "to do" list for maybe six years. It's a big, error-prone job since there are about 250 such references. And because of the GM/IM fork, the whole mess has really got to be fixed twice. Well, adding things at the end of png_struct/png_info, almost at every new libpng release, For the hundredth time, please don't exaggerate. We added new members five times in the 1.2.x series and once (proposed) so far in 1.4.x. did not break anything for me yet, and I believe, neither for Glenn/Bob. But I don't deny a scenario in which things could break, and I would support removing the contents of the png structures, after the API is complete from the access perspective. I still have not received a definitive answer to whether doing that requires a minor version bump (and a bump in the shared library name). > If we need a set of 'clear' functions in 1.5 that's fine by me. ?What I dislike is the idea that we don't make progress on this now. Is "png_set_invalid()" not sufficient? That's been in the library since 1.0.7. Maybe using it would cause memory leakage but that could be fixed without changing the ABI/API. Glenn From ctruta at gmail.com Fri Feb 5 02:29:41 2010 From: ctruta at gmail.com (Cosmin Truta) Date: Thu, 4 Feb 2010 21:29:41 -0500 Subject: [Png-mng-security] vulnerability in png_decompress_chunk() In-Reply-To: <1398227657.3640071265287938891.JavaMail.root@sz0023a.westchester.pa.mail.comcast.net> References: <2117083195.3636541265287291859.JavaMail.root@sz0023a.westchester.pa.mail.comcast.net> <1398227657.3640071265287938891.JavaMail.root@sz0023a.westchester.pa.mail.comcast.net> Message-ID: <29d80c921002041829t41f4667dr3ac79742661c1af0@mail.gmail.com> John wrote: >> If the ABI changes the major version number must be changed, that's what "major" means in this context; that if it does not change the ABI is unchanged. Glenn wrote: > ? ? Just to be precise, it is the minor version number that gets > ? ? bumped when the ABI changes. ?1.0.x, 1.2.x, 1.4.x are "minor > ? ? versions". ?I guess the leading "1." is just a decoration for > ? ? now, but when we make a libpng that looks really different > ? ? from 1.x.x, we'll call it 2.0.0. ?I think making the > ? ? png_struct opaque probably falls in that category, although > ? ? we do seem to be sneaking up on that via the PNGDEPSTRUCT > ? ? thing. I agree with this analysis. It's one thing when PNG_DEPSTRUCT nags you not to do it in libpng 1.x, but it's a different thing when you cannot do it in libpng 2.0. I wrote: >> Well, adding things at the end of png_struct/png_info, almost at every >> new libpng release, Glenn wrote: > ? ? For the hundredth time, please don't exaggerate. ?We added > ? ? new members five times in the 1.2.x series and once (proposed) > ? ? so far in 1.4.x. I apologize, please don't mind me... (and the same goes to John...) I tend to read fragments of discussions (as opposed to entire discussions) more often recently, although I should try harder to abstain from posting thoughts when I cannot read the full discussion thread. In my head, I'm hooked on the issue of having too many configuration macros, and because of that (to me) everything seems too complex in libpng, although there has been some really nice improvement from 1.2 to 1.4. Ok, I'll stop digressing. John wrote: >> If we need a set of 'clear' functions in 1.5 that's fine by me. ?What I dislike is the idea that we don't make progress on this now. Glenn wrote: > ? ? ? Is "png_set_invalid()" not sufficient? ?That's been in the > ? ? ? library > ? ? ? since 1.0.7. ?Maybe using it would cause memory leakage but > ? ? ? that could be fixed without changing the ABI/API. Whoa, I had no idea it ever existed, when in fact it has been under my nose all that time!! Ok, I'll go back to my optipng-0.7 alpha and see how I can use this old but new-for-me function. Best regards, Cosmin From bfriesen at simple.dallas.tx.us Fri Feb 5 03:25:32 2010 From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn) Date: Thu, 4 Feb 2010 21:25:32 -0600 (CST) Subject: [Png-mng-security] vulnerability in png_decompress_chunk() In-Reply-To: <29d80c921002041829t41f4667dr3ac79742661c1af0@mail.gmail.com> References: <2117083195.3636541265287291859.JavaMail.root@sz0023a.westchester.pa.mail.comcast.net> <1398227657.3640071265287938891.JavaMail.root@sz0023a.westchester.pa.mail.comcast.net> <29d80c921002041829t41f4667dr3ac79742661c1af0@mail.gmail.com> Message-ID: As a reminder, this is the png-mng-security list, which is supposed to only be used for discussion of important security issues. Many people on this list are only interested in security issues and fixes for their OS or application. It is not a list for libpng chatter. Try to use the normal PNG lists if you can. Never cross-post to this list, or mention it in public. Thanks, Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From glennrp at comcast.net Fri Feb 5 13:21:27 2010 From: glennrp at comcast.net (glennrp at comcast.net) Date: Fri, 5 Feb 2010 13:21:27 +0000 (UTC) Subject: [Png-mng-security] vulnerability in png_decompress_chunk() In-Reply-To: <1351258865.38701265375902282.JavaMail.root@sz0023a.westchester.pa.mail.comcast.net> Message-ID: <746967589.40291265376086952.JavaMail.root@sz0023a.westchester.pa.mail.comcast.net> ----- Original Message ----- From: "John Bowler" > This does mean that you will be changing the png_struct or png_info > structure in 1.4 (as in the current 1.4.1beta06 - adding a member to > a structure visible in the ABI; at extra 4 bytes at the end of > png_struct). IRC that happened several times in 1.2. Still waiting for a definitive answer to whether it's OK to do this any more without a minor version bump (i.e., 1.4.0 to 1.5.0 and libpng14.so to libpng15.so). For 1.4.1beta07 I am planning to implement essentially what's in beta06, but with the opportunity for the downstream libpng builder to impose default limits on the maximum size and number of compressed chunks via #define and for the application writer to override the limits via "png_set_*()" functions. The defaults set in our distribution will be 0 meaning unlimited. That includes the get|set functions and a new png_struct member to hold the malloc limit (1.4.0 already has one for the chunk counter). For 1.2.43 I am planning to implement the same but without the get|set functions and without a new member in png_struct. The memory limit will be a constant that can be defined by the downstream libpng builder, default 0, meaning unlimited. I don't see a way of imposing a limit on the number of compressed chunks because that requires a counter in the png_struct. Libpng-1.4.1 and 1.2.43 (and 1.0.53) will use the fast two-pass png_decompress_chunk() that measures the chunk before allocating memory for it. I have posted patches for Firefox/Mozilla to the bugzilla.mozilla.org bug #497056 along these lines, to replace the png_decompress_chunk() function with the two-pass function in the embedded library and a fixed 4-MB maximum on the size of decompressed iCCP chunks. If Firefox (or other gecko-based application) is built with the system libpng older than 1.4.1, then it uses a replacement malloc_fn() to impose the 4MB limit on the decompressed iCCP chunk size and increased the size of zbuf_size for iCCP chunk decompression to 32k, That's not an ideal solution but I don't have any other ideas. In libpng-1.4.0, 0x7fffffff means an unlimited number of chunks allowed, but "0" seems simpler to implement and to explain. It doesn't really make any practical difference so I think it's OK to make this change in 1.4.1. Glenn From glennrp at gmail.com Sat Feb 6 13:18:46 2010 From: glennrp at gmail.com (Glenn Randers-Pehrson) Date: Sat, 6 Feb 2010 08:18:46 -0500 Subject: [Png-mng-security] vulnerability in png_decompress_chunk() In-Reply-To: References: Message-ID: The existence of this bug has been inadvertently leaked to the public png-mng-implement list. Even though the message body doesn't say anything about the bug, the subject Subject: RE: [png-mng-implement] [Png-mng-security] vulnerability in png_decompress_chunk() seems to give enough information for a determined cracker to figure it out in short order. Glenn