Pasting DIBv5 image

[huyi7]

Coming from https://github.com/graphics32/graphics32/issues/257
Pasting an image with alpha from Delphi Graphics32, Paint.NET would have the image shifted. At least Affinity Photo can paste that image correctly. I wonder if it is a bug in Paint.NET. Test program is attached in the about github issue page. Thanks for your attention.

[Rick Brewster]

I can't use the sample app you posted in that issue. What you can do, however, is:

 

1) Copy the data to the clipboard

2) Using ClipSpy, save the raw CF_DIBV5 data to a file (File -> Save Selected Data)

3) ZIP it up and attach it here.

 

I've attached a ZIP with the ClipSpy executable. It's a very very old program you can find only in source code form here: https://www.codeproject.com/Articles/168/ClipSpy

ClipSpy.zip

 

I can then look at the binary data and try to figure out why it's not working correctly, and determine whether it's Paint.NET's fault or not.

[huyi7]

 

Thanks for your quick response! Attached is the content of the DIBV5. I've made the bitmap only 4x4 pixels, hopefully it's easily to trace. In the zip file, we have DIBV5content.bin and DIBV5content_modified.bin. The formal is what is originally written by Graphics32 by default. The latter is one written by Graphics32 after I made the change mentioned in the github post.

 

Pasting 'DIBV5content.bin' into Paint.NET, we get the shifted result:

Pasting 'DIBV5content_modified.bin' into Paint.NET, we get the correct result:

Please see if you can figure out what's wrong. Thank you!

DIBV5content.zip

[null54]

3 hours ago, huyi7 said:

Please see if you can figure out what's wrong. Thank you!

 

The issue appears to be that Graphics32 fails to write the 3 DWORD color masks after the header that are required for 32-bit bitmaps that use BI_BITFIELDS compression.

DIBV5content_modified.bin works because it has 12 bytes of padding after the header, matching the size of the 3 DWORD color masks that Paint.NET skips over when reading the image.

 

Quoting from https://learn.microsoft.com/en-us/windows/win32/api/wingdi/ns-wingdi-bitmapv4header:

 

Quote

If the bV4Compression member of the BITMAPV4HEADER is BI_BITFIELDS, the bmiColors member contains three DWORD color masks that specify the red, green, and blue components of each pixel. Each DWORD in the bitmap array represents a single pixel.

 

[AndersM]

19 hours ago, null54 said:

Quoting from https://learn.microsoft.com/en-us/windows/win32/api/wingdi/ns-wingdi-bitmapv4header:

Quote

If the bV4Compression member of the BITMAPV4HEADER is BI_BITFIELDS, the bmiColors member contains three DWORD color masks that specify the red, green, and blue components of each pixel. Each DWORD in the bitmap array represents a single pixel.

 

Hi, Anders Melander, the current Graphics32 maintainer here.

 

The GDI documentation is correct, albeit very poorly worded, but from my POW it's your interpretation of bmiColors that's the problem.

As I understand it, your assumption is that the bmiColors array should always follow the BITMAPV4HEADER structure. My claim is that it's part of the structure.

 

If you look at the documentation, bmiColors comes from the BITMAPINFO structure; The first member of the structure is BITMAPINFOHEADER and the second member is the bmiColors DWORD array.

 

Now if you look at all the different BMP header versions, they all extend on the previous version by adding fields to the end of it. We start with the v1 BITMAPINFOHEADER. The (undocumented) v2 header adds 3 DWORD RGB masks. The v3 header adds a DWORD alpha mask. the v4 adds gamma and color space stuff, etc. etc. The point here is that bmiColors really only applies when you are using a v1 header. All the later versions include bmiColors in their structure.

 

A clue is that the documentation first state that bmiColors specify the color masks:

Quote

the bmiColors member contains three DWORD color masks that specify the red, green, and blue components of each pixel.

and then later state (for BI_BITFIELDS):

Quote

The members bV4RedMask, bV4GreenMask, and bV4BlueMask specify the red, green, and blue components for each pixel. This is valid when used with 16- and 32-bpp bitmaps.

 

Ergo: bmiColors=bV4RedMask, bV4GreenMask, bV4BlueMask -> The 3 DWORDs are included in the header

 

Now, even though I claim my interpretation of the structures to be "the right one" it can also be argued that they are both correct. The documentation has been changed so many times with regard to this and it's still broken, some parts of Windows write one variant, other parts write another and it can usually read them both.

 

If possible, I suggest Paint.NET try to handle both variants (when reading). That's what I do at my end and it seems that that is also what Windows itself does.

[Rick Brewster]

Yeah I think you're right. I came to the same conclusion after reading some of your response ... then I read the rest of your response and it lined up exactly with what I had just deciphered from MSDN.

 

1 hour ago, AndersM said:

If possible, I suggest Paint.NET try to handle both variants (when reading). That's what I do at my end and it seems that that is also what Windows itself does.

 

So you look at the total size, and if it's got 12 extra bytes you assume it was has an "extra" "bmiColors" ?

 

My main anxiety here is that clipboard code has proven to be notoriously fragile. Even if I fix this, there's a high probability that it will break pasting from some other app -- not because of a bug in PDN or Graphics32, but because of the other app. Every app seems to have some kind of different bug with regard to all of this.

[Rick Brewster]

Also I think a lot of apps don't present issues here because using "PNG" instead of CF_DIBV5 for clipboard images has become more popular. It has no such issues, obviously, although I'm not a fan of the performance when copying to the clipboard. The other format I'm seeing is CF_DIBV5 + BI_RGB, I haven't found anything on my system that uses BI_BITFIELDS.

[Rick Brewster]

Okay this should be fixed in 5.0.8, barring any further complications

[AndersM]

1 hour ago, Rick Brewster said:

So you look at the total size, and if it's got 12 extra bytes you assume it was has an "extra" "bmiColors" ?

Yep.

Here's the code:

  // Work around BMPs with an extra color mask after the header:
  // If there's exactly 12 bytes too many then we assume that these are an extra
  // color mask and skip it.
  // For example the Windows resource compiler will add these 12 bytes to the header
  // of RT_BITMAP resources. This goes for both BI_RGB and BI_BITFIELDS.
  if (InfoHeaderVersion >= InfoHeaderVersion1) then
  begin
    ChunkSize := 3 * SizeOf(DWORD);
    if (Size = ChunkSize) then
      Stream.Seek(ChunkSize, soFromCurrent);
  end;

 

 

1 hour ago, Rick Brewster said:

Even if I fix this, there's a high probability that it will break pasting from some other app -- not because of a bug in PDN or Graphics32, but because of the other app. Every app seems to have some kind of different bug with regard to all of this.

Yes, with this it seems there's no "fixed". Just various degrees of broken.

 

2 hours ago, Rick Brewster said:

The other format I'm seeing is CF_DIBV5 + BI_RGB, I haven't found anything on my system that uses BI_BITFIELDS.

There's no point in using CF_DIBV5 unless you're also using BI_BITFIELDS as BI_BITFIELDS is the only way to specify that the bitmap contains alpha. Without BI_BITFIELDS you might as well just use CF_DIB.

 

Graphics32 originally used CF_DIB + BI_RGB for both clipboard and BMP files and while this worked if the target assumed that the bitmaps contained alpha, many applications rightly treated the bitmaps as 24-bit RGB. For example Windows Explorer; Only a v3+header with BI_BITFIELDS will make Windows Explorer treat a BMP as a 32-bit RGBA bitmap.

 

FWIW, the following comment from the Graphics32 bitmap/clipboard adapter code might provide some additional clues. The Windows clipboard isn't exactly makes things easier for us:

  (*
  ** We place the following data on the clipboard:
  **
  ** - CF_BITMAP
  **   This is the source bitmap rendered onto a white background.
  **   Transparency is not retained.
  **   For use by applications that doesn't support Alpha.
  **
  ** - CF_DIBV5
  **   A 32 bit DIB with alpha. This alone can be used to recreate the original
  **   32 bit bitmap, including alpha.
  **   This format provides round trip support.
  **
  ** Since Windows can synthesize between any of CF_DIB, CF_BITMAP and CF_DIBV5
  ** theoretically we could just supply the most capable format (CF_DIBV5) and
  ** let Windows supply the others. Unfortunately we need to supply both CF_DIBV5
  ** and CF_BITMAP/CF_DIB in order to work around various Windows bugs:
  **
  ** - When the clipboard synthesizes CF_DIBV5 from CF_DIB it uses BI_BITFIELDS.
  **   However, if the clipboard synthesizes CF_DIB from CF_DIBV5 with
  **   BI_BITFIELDS compression, the clipboard apparently forgets to take the
  **   extra 3 mask DWORDs into account which messes up the resulting DIB.
  **
  ** - When the clipboard synthesizes CF_DIB or CF_BITMAP from CF_DIBV5 it
  **   appears to require 68 extra bytes after the bitmap header.
  **   Inserting these 68 bytes would fix that but would also make the bitmap
  **   stream invalid for everything else.
  **   FWIW, 68 = SizeOf(BITMAPV4HEADER)-SizeOf(BITMAPINFOHEADER)...
  **
  ** As a bonus we get to control the background color of the CF_DIB/CF_BITMAP
  ** bitmap instead of the black one Windows would use.
  *)

 

[Rick Brewster]

5 minutes ago, AndersM said:

There's no point in using CF_DIBV5 unless you're also using BI_BITFIELDS as BI_BITFIELDS is the only way to specify that the bitmap contains alpha.

Do you have a source for this? From what I see, BI_BITFIELDS only talks about specifying the masks for red, green, blue. From https://learn.microsoft.com/en-us/windows/win32/api/wingdi/ns-wingdi-bitmapv5header: 

 

If this is indeed the case then I should switch PDN's code to use BI_BITFIELDS when copying to the clipboard. But, I can't find anything that says BI_BITFIELDS is required for using alpha.

[AndersM]

2 minutes ago, Rick Brewster said:

Do you have a source for this? From what I see, BI_BITFIELDS only talks about specifying the masks for red, green, blue.

I can't remember anymore and I'm afraid I don't have time to pursue it now.

 

You can deduce it from the structure format though. The bV5RedMask, bV5GreenMask, and bV5BlueMask fields are only used with BI_BITFIELDS. It follows that the bV5AlphaMask field, which is the only method there is to specify where to find the alpha value in the color DWORDs, is also only used with BI_BITFIELDS.

 

 

10 minutes ago, Rick Brewster said:

I can't find anything that says BI_BITFIELDS is required for using alpha.

I think you'll find that you can't find any mention of alpha at all in the BMP documentation.

There's nothing stopping you from putting alpha in a BI_RGB DWORD, but if the readers don't recognize it as alpha then what's the point? You can give it a try with Windows Explorer. The easiest test is probably just to place a 32-bit RGBA BMP on the desktop and see if it displays as opaque or transparent.

[Rick Brewster]

I was able to confirm this using Krita. Using BI_RGB, transparency is not preserved. Switching only that over to BI_BITFIELDS and it works fine.

[Rick Brewster]

14 hours ago, AndersM said:

I think you'll find that you can't find any mention of alpha at all in the BMP documentation.

 

And unfortunately the bitmap header provides no way to specify straight versus premultiplied alpha. Some apps use premultiplied alpha for some terrible reason, and I had to add code in PDN to detect that (if all R,G,B are <=A, then it's premultiplied) and then unpremultiply.

 

And I have to detect the case where a clipboard bitmap says it has alpha, but the alpha channel is incorrectly all 0's (it should've been set as BGRx32 instead of BGRA32, i.o.w.). In that case I rewrite the alpha channel to be all 255's.

[Rick Brewster]

14 hours ago, AndersM said:

You can deduce it from the structure format though. The bV5RedMask, bV5GreenMask, and bV5BlueMask fields are only used with BI_BITFIELDS. It follows that the bV5AlphaMask field, which is the only method there is to specify where to find the alpha value in the color DWORDs, is also only used with BI_BITFIELDS.

 

Yes although the documentation does not actually state this. In fact, it specifically says that the RGB masks are only for BI_BITFIELDS, but does not say that for the alpha mask.

 

In any event, it seems to be de facto the right way to do things, as evidenced by Krita's interpretation when pasting.

 

[AndersM]

10 minutes ago, Rick Brewster said:

Some apps use premultiplied alpha for some terrible reason, and I had to add code in PDN to detect that (if all R,G,B are <=A, then it's premultiplied) and then unpremultiply.

Your patience is admirable 🙂

I personally wouldn't try to handle broken input to that degree. Especially since it's impossible to guarantee that the fix wouldn't break valid input. For example, the following bitmap is valid and not premultiplied but all RGB <= A:

Oh... It seems I can't paste BMPs. Anyway, you get the picture - so to say.

[Rick Brewster]

2 hours ago, AndersM said:

I personally wouldn't try to handle broken input to that degree.

 

Normally I'd agree but these bitmaps were coming from Chrome and/or Firefox, IIRC.

 

But nowadays most apps seem to put `PNG` or `image/png` onto the clipboard, which sidesteps all these issues.

[Rick Brewster]

You can try the 5.0.8 beta and see what you think now: