ColorBgra.Blend loses data

[otuncelli]

I was working with pixels and I've encountered a problem.  Is there a bug in ColorBgra.Blend or am I doing something wrong?

 

Here's a simple program to reproduce:

class Program {
  static void Main() {
    // Generate original images for testing purposes
    Bitmap bmp1 = GetLoremIpsumWithAlphaBackground();
    Bitmap bmp2 = new Bitmap(bmp1);

    Rectangle rect = new Rectangle(0, 0, bmp1.Width, bmp1.Height);

    // Test 1
    BitmapData data1 = bmp1.LockBits(rect, ImageLockMode.ReadWrite, PixelFormat.Format32bppArgb);
    BlendOverWhite(data1.Scan0, data1.Width, data1.Height, data1.Stride);
    bmp1.UnlockBits(data1);
    bmp1.Save("im1out.png");

    // Test 2
    BitmapData data2 = bmp2.LockBits(rect, ImageLockMode.ReadWrite, PixelFormat.Format32bppArgb);
    BlendOverWhite2(data2.Scan0, data2.Width, data2.Height, data2.Stride);
    bmp2.UnlockBits(data2);
    bmp2.Save("im2out.png");
  }

  private static unsafe void BlendOverWhite(IntPtr scan0, int width, int height, int stride) {
    ColorBgra white = ColorBgra.White;
    ColorBgra * buf = (ColorBgra * ) scan0;
    for (var y = 0; y < height; y++) {
      ColorBgra * row = buf + y * width;
      for (var x = 0; x < width; x++) {
        ColorBgra * pixel = row + x;
        ColorBgra blended = ColorBgra.Blend(white, * pixel, pixel - > A);
        * pixel = blended;
      }
    }
  }

  [MethodImpl(MethodImplOptions.AggressiveInlining)]
  private static byte CustomBlend(int lhc, int rhc, int alpha, int alphaDiff) {
    return (byte)((rhc * alpha + lhc * alphaDiff) >> 8);
  }

  private static unsafe void BlendOverWhite2(IntPtr scan0, int width, int height, int stride) {
    ColorBgra white = ColorBgra.White;
    ColorBgra * buf = (ColorBgra * ) scan0;
    for (int y = 0; y < height; y++) {
      ColorBgra * row = buf + y * width;
      for (int x = 0; x < width; x++) {
        ColorBgra * pixel = row + x;
        int alpha = pixel - > A;
        int alphaDiff = byte.MaxValue - alpha;
        pixel - > B = CustomBlend(white.B, pixel - > B, alpha, alphaDiff);
        pixel - > G = CustomBlend(white.G, pixel - > G, alpha, alphaDiff);
        pixel - > R = CustomBlend(white.R, pixel - > R, alpha, alphaDiff);
        pixel - > A = white.A; // 255
      }
    }
  }

  private static Bitmap GetLoremIpsumWithAlphaBackground() {
    const string text = @ ""
    "Lorem ipsum dolor sit amet, consectetur adipiscing elit, 
    sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
    Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris
    nisi ut aliquip ex ea commodo consequat.Duis aute irure dolor in
      reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla
    pariatur.Excepteur sint occaecat cupidatat non proident, sunt in
      culpa qui officia deserunt mollit anim id est laborum.
    ""
    ";
    using(var font = new Font("Verdana", 12, FontStyle.Regular)) {
      SizeF sizef;
      using(var g = Graphics.FromHwnd(IntPtr.Zero)) {
        sizef = g.MeasureString(text, font);
      }

      Size size = Size.Round(sizef);

      var bitmap = new Bitmap(size.Width, size.Height, PixelFormat.Format32bppArgb);
      using(var g = Graphics.FromImage(bitmap)) {
        g.TextRenderingHint = TextRenderingHint.AntiAlias;

        g.DrawString(text, font, Brushes.Black, PointF.Empty);
      }
      bitmap.Save("im.png");
      return bitmap;
    }
  }
}

 

Outputs:

 

Original image (im.png) (text over transparent background):

 

BlendOverWhite (im1out.png) (ColorBgra.Blend):

 

BlendOverWhite2 (im2out.png) (CustomBlend): 

[Rick Brewster]

ColorBgra * row = buf + y * width;

This code is wrong. You must calculate the start of each row using the stride. You can’t just multiply y by width.

[otuncelli]

32 minutes ago, Rick Brewster said:

ColorBgra * row = buf + y * width;

This code is wrong. You must calculate the start of each row using the stride. You can’t just multiply y by width.

 

True, if it was a byte pointer. But its ColorBgra pointer. When I change width with stride it throws AccessViolationException.

 

Also this gives the same result:

ColorBgra* row = (ColorBgra*)((byte*)buf + y * stride);

 

[Rick Brewster]

It may give the same result when stride is equal to width*sizeof(ColorBgra), but stride is in no way guaranteed to equal width*sizeof(ColorBgra).

[Rick Brewster]

ColorBgra blended = ColorBgra.Blend(white, * pixel, pixel - > A);

Also, there is no bug in ColorBgra.Blend. You're effectively squaring the alpha value by passing it in for the 3rd parameter. Since alpha is essentially [0, 1] values mapped to a byte's [0, 255], you're getting a much lower alpha value than you're supposed to by doing this. Just use 255 for the cbAlpha parameter, which is combined with cb's existing alpha value. 

[otuncelli]

5 minutes ago, Rick Brewster said:

It may give the same result when stride is equal to width*sizeof(ColorBgra), but stride is in no way guaranteed to equal width*sizeof(ColorBgra).

 

Oh yes, I see.

 

2 minutes ago, Rick Brewster said:

ColorBgra blended = ColorBgra.Blend(white, * pixel, pixel - > A);

Also, there is no bug in ColorBgra.Blend. You're effective squaring the alpha value by passing it in for the 3rd parameter. Since alpha is essentially [0, 1] values mapped to a byte's [0, 255], you're getting a much lower alpha value than you're supposed to by doing this. Just use 255 for the cbAlpha parameter, which is combined with cb's existing alpha value. 

 

That fixed it. Thanks.

[_koh_]

Sorry for bumping but having the same problem.

 

I ran this code in CodeLab

 

void PreRender(Surface dst, Surface src) {
    ColorBgra a = ColorBgra.White;
    ColorBgra b = ColorBgra.Black.NewAlpha(128);
    Debug.WriteLine(ColorBgra.Blend(a, b, 0));
    Debug.WriteLine(ColorBgra.Blend(a, b, 255));
    Debug.WriteLine(ColorBgra.Blend(a, b, b.A));
}

 

and got this results.

 

B: 255, G: 255, R: 255, A: 255
B: 0, G: 0, R: 0, A: 128
B: 169, G: 169, R: 169, A: 191

 

No idea how ColorBgra.Blend() works.

[otuncelli]

Debug.WriteLine(ColorBgra.Blend(a, b, b.A));

This method's parameter name (cbAlpha) was also confused me. 

 

You should pass cb's original alpha value here (which is 255) but you're passing the new alpha value (which is 128).

 

void PreRender(Surface dst, Surface src) 
{
    ColorBgra a = ColorBgra.White;
    ColorBgra b = ColorBgra.Black;
    Debug.WriteLine(ColorBgra.Blend(a, b, 0));
    Debug.WriteLine(ColorBgra.Blend(a, b, 255));
    Debug.WriteLine(ColorBgra.Blend(a, b.NewAlpha(128), b.A));
}

 

Edited January 1, 2022 by otuncelli

[_koh_]

14 minutes ago, otuncelli said:

You should pass cb's original alpha value here (which is 255) but you're passing the new alpha value (which is 128).

 

That was my 2nd test

 

Debug.WriteLine(ColorBgra.Blend(a, b, 255));

 

and I got this for that.

 

B: 0, G: 0, R: 0, A: 128

 

So neither 128 nor 255 alpha gives me an expected result which is RGB(127, 127, 127) at least in my CodeLab.

 

But in your code, now ColorBgra.Blend() results matching your CustomBlend() results by using 255 for alpha, right?

[Rick Brewster]

@_koh_, what is your desired goal? RGB(127, 127, 127) with 255 alpha? Because what you're doing is a 50% blend of (black at 50% opacity) on top of (white at 100% opacity). You're not going to get (50% gray at 100% opacity) out of that.

 

Alpha blending won't necessarily produce intuitive numbers with you combine 2x BGRA colors with an additional alpha value for blending between them. It does produce the correct values, however. (fwiw, "blend" is sort of the wrong name for this function, as it's actually performing alpha compositing)

 

If cbAlpha is zero, the return value is always ca.

 

If cbAlpha is 255, the return value is always (cb OVER ca), and depends entirely on cb and ca's alpha values.

 

cbAlpha values in the range 1-254 produce a gradient between those two values, but it's not "linear."

 

If you just want a linear interpolation, use Lerp(). It won't necessarily produce good looking results but it may be more intuitive.

 

// NOTE: FastScale(a, b) => (a * b) / 255
//                     aka ((a / 255.0) * (b / 255.0)) * 255.0

/// <summary>
/// Smoothly blends between two colors.
/// </summary>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static ColorBgra Blend(ColorBgra ca, ColorBgra cb, byte cbAlpha)
{
    byte caA = ByteUtil.FastScale((byte)(255 - cbAlpha), ca.A);
    byte cbA = ByteUtil.FastScale(cbAlpha, cb.A);
    ushort cbAT = (ushort)(caA + cbA);

    // NOTE: We rely on the fast division function returning 0 if the denominator is 0

    uint r = UInt32Util.FastDivideByUInt16((uint)((ca.R * caA) + (cb.R * cbA)), cbAT);
    uint g = UInt32Util.FastDivideByUInt16((uint)((ca.G * caA) + (cb.G * cbA)), cbAT);
    uint b = UInt32Util.FastDivideByUInt16((uint)((ca.B * caA) + (cb.B * cbA)), cbAT);

    return ColorBgra.FromBgra((byte)b, (byte)g, (byte)r, (byte)cbAT);
}

 

[otuncelli]

15 minutes ago, _koh_ said:

But in your code, now ColorBgra.Blend() results matching your CustomBlend() results by using 255 for alpha, right?

 

Yes, it matches with my CustomBlend function.

[_koh_]

2 hours ago, Rick Brewster said:

@_koh_, what is your desired goal? RGB(127, 127, 127) with 255 alpha?

 

Yeah that was my goal.

I was trying to replace my normal layer blending replicating code with ColorBgra.Blend().

 

c.R = (b.R * b.A + a.R * (255 - b.A)) / 255;

 

And going by ColorBgra.Blend() code, seems like this is what I needed.

 

c = ColorBgra.Blend(a.NewAlpha(255), b.NewAlpha(255), b.A);

 

Thanks for answering folks and Happy New Year!

 

EDIT:

I should have realized this when I saw result being 169 (170 = 255 * 2 / 3). haha

Edited January 1, 2022 by _koh_
hindsight

[Rick Brewster]

You can also just

 

c = ColorBgra.Blend(ColorBgra.White, ColorBgra.Black, 128);

 

[_koh_]

Thanks.

What I'm actually doing is adding background to transparent images just like OP, so "How can I emulate layer blending with ColorBgra.Blend()?" was the appropriate question.
Code you posted gave me the answer and now it's working. Thank you!

[BoltBait]

Wouldn't it be faster to do a layer at a time instead of pixel at a time?

 

Something like this:

 

#region UICode
#endregion

// Working surface
Surface wrk = null;

// Setup for selected blending op
private BinaryPixelOp normalOp = LayerBlendModeUtil.CreateCompositionOp(LayerBlendMode.Normal);

protected override void OnDispose(bool disposing)
{
    if (disposing)
    {
        wrk?.Dispose(); wrk = null;
    }

    base.OnDispose(disposing);
}

// This single-threaded function is called after the UI changes and before the Render function is called
// The purpose is to prepare anything you'll need in the Render function
// For this example, I'm just filling the WRK surface with white.
void PreRender(Surface dst, Surface src)
{
    if (wrk == null)
    {
        wrk = new Surface(src.Size);
    }

    // Fill the wrk surface with White color
    wrk.Fill(ColorBgra.White);
}

// Here is the main multi-threaded render function
void Render(Surface dst, Surface src, Rectangle rect)
{
    // In this example, put your surface above a white surface and blend them.
    // Normal Blend the src surface and the wrk surface to the dst surface
    normalOp.Apply(dst, wrk, src, rect);
}

 

You may think doing this by pixel in a loop is the same speed... but, I can assure you that the internal functions (like BinaryPixelOp.Apply) is optimized for speed and will perform MUCH better than any loop you can write.

[_koh_]

Ahh, interesting.

 

My background surface is from clipboard, so I'm doing alignment like this,

 

ColorBgra b = aux.GetBilinearSample(x - sel.Left, y - sel.Top);

 

and blending is a preparation so can't get rid of nested loops anyway, but I overlooked built-in functions.

[otuncelli]

4 hours ago, BoltBait said:

You may think doing this by pixel in a loop is the same speed... but, I can assure you that the internal functions (like BinaryPixelOp.Apply) is optimized for speed and will perform MUCH better than any loop you can write.

 

public abstract ColorBgra Apply(ColorBgra lhs, ColorBgra rhs);

 

BinaryPixelOp.Apply calls a virtual method where JIT Inlining is not possible. I'm not sure about performance. It might be slower unless C++ or intrinsics are involved.

 

But it makes the code much cleaner and safer. I was not able to use it because my input was not a Surface.

Edited January 1, 2022 by otuncelli

[Rick Brewster]

If you use that overload of Apply then yes it's one virtual function call per pixel.

 

However, the other overloads (e.g. that take a Surface) will batch things up and are implemented with their own inlining. It's all generated code.

[otuncelli]

3 hours ago, Rick Brewster said:

If you use that overload of Apply then yes it's one virtual function call per pixel.

 

However, the other overloads (e.g. that take a Surface) will batch things up and are implemented with their own inlining. It's all generated code.

 

I'm not sure about this. There seems to be a deep inheritance going on there and all other overloads of Apply are end up calling this abstract Apply function which can not be resolved until runtime.

 

See:

https://sharplab.io/#v2:C4LghgzgtgPgAgBgARwIwDoBKBXAdsASygFN0BhAeygAcCAbYgJwGUmA3AgY2IgG4BYAFBC4AZhQAmFKgDsSAN5CkyyRMWCVmpABEmBNsQAmZOpAhIKAIwBWSALxJcxAO469B46YgQAFAEoBDS1lK2t0AEFqajoATx9UABokCSTRAKUVAF8hbOFBMUk3Rn0jEzMkECQAIUhiMu8FDOUCigNGYsNiJAJ8JEjouJ7gJDAkoaRLPybGoOC4OTAkAGoJwM1c3JFxMEsIYEYwTmG4KRqIOq9zdU0Cnb2Do+7e/tifcdGn4cm1lWmAbQAssRgAALCiGACSNDoPiBoPBUOiAHlqIQKLgIBEAOZYxg8CAlCG4Og9HpYvwAXWmBXGLxiADEKIxuESSU43r0PuNJvYAHwjZarJDTf5wsGQ6Gw4HixF0FFojHoAByFFZpNw5KpsxQ4lpUViKrV7PeY16PLs/MWK0svGFeRu4jgABY+vrBpzTV9PUgAPp+a7BV0DHyjb7TTR0xnM4hG4ghhJh7URt2G4k9OOh9LajZCIRAA=

Edited January 1, 2022 by otuncelli

[Rick Brewster]

I'm 100% sure about it.

 

The overload that's usually called is this one, which is also overridden by all of the composition ops. The overrides do not call Apply(ColorBgra, ColorBgra), its logic is inlined.

 

[otuncelli]

15 minutes ago, Rick Brewster said:

I'm 100% sure about it.

 

The overload that's usually called is this one, which is also overridden by all of the composition ops. The overrides do not call Apply(ColorBgra, ColorBgra), its logic is inlined.

 

 

Okay, I see. Implementations delegate to C++ to do the actual job.

 

Also there is PaintDotNet.Rendering.CompositionOps.{BlendMode}.Static property which seems to be marked as public. Is there a difference between this one and BinaryPixelOp? Which one should be used for plugin development?