[Png-mng-security] potential optimizing away of initialization loop

[John Bowler]

From: glennrp at comcast.net
>I seem to recall in the past having trouble with optimizing
>compilers removing loops that apparently don't really do
>anything.  In this bugfix we introduce the first loop in
>
>      for (i=0; i<n; i++)
>         element[i] = NULL;
>      for (i=0; i<n; i++)
>         element[i] = png_malloc(...);

The compiler cannot know that the pointer 'element' is not aliased, therefore element[i+1] might be accessed from within png_malloc while it still contains NULL.

What is more the compiler cannot know that png_malloc does not call longjmp, causing the loop to terminate before all element[i] are overwritten.  That applies to any function call unless the compiler knows about the function.

>I think the presence of the call to an external function
>i.e., png_malloc() in the second loop will prevent the first
>loop from being optimized away.  Is that correct, and
>therefore we should not worry about it?

Yes, that is correct, for the two reasons given above.  In it might be possible for the compilation system to work out that png_malloc does not leave 'element' accessible to other functions, but in that case the optimizer should also notice that png_malloc can all longjmp...

John Bowler <jbowler at acm.org>