How to Write an Effect Plugin (Part 8 - Alpha Manipulation)

[BoltBait]

How to Write an Effect Plugin (Part 8 - Alpha Manipulation)

 

Wouldn't it be cool if you could manipulate the alpha channel of an image independently of the colors on that layer?

 

Yeah, I thought it would be pretty cool too. So, I wrote some functions to help me achieve just that when using CodeLab!

 

Basically, we need to do 2 things: (1) extract the alpha channel of an image to a separate surface, and (2) apply the modified alpha back into our original image after we've modified it in some way.

 

 

Once the alpha channel is extracted to a new surface as a black and white image, we can use any built-in effects we want to modify it.

 

Extract Alpha Channel

 

OK, here's a function I wrote to extract the alpha channel from one surface and copy it as shades of gray to another surface:

unsafe void CopyMask(Surface dst, Surface src, Rectangle rect, bool aliased, bool invertMask = false)
{
    if (aliased)
    {
        byte shade;
        if (invertMask)
        {
            for (int y = rect.Top; y < rect.Bottom; y++)
            {
                if (IsCancelRequested) return;
                ColorBgra* srcPtr = src.GetPointAddressUnchecked(rect.Left, y);
                ColorBgra* dstPtr = dst.GetPointAddressUnchecked(rect.Left, y);
                for (int x = rect.Left; x < rect.Right; x++)
                {
                    shade = (byte)(255-(*srcPtr).A);
                    *dstPtr = ColorBgra.FromBgr(shade,shade,shade);
                    srcPtr++;
                    dstPtr++;
                }
            }
        }
        else
        {
            for (int y = rect.Top; y < rect.Bottom; y++)
            {
                if (IsCancelRequested) return;
                ColorBgra* srcPtr = src.GetPointAddressUnchecked(rect.Left, y);
                ColorBgra* dstPtr = dst.GetPointAddressUnchecked(rect.Left, y);
                for (int x = rect.Left; x < rect.Right; x++)
                {
                    shade = (*srcPtr).A;
                    *dstPtr = ColorBgra.FromBgr(shade,shade,shade);
                    srcPtr++;
                    dstPtr++;
                }
            }
        }
    }
    else
    {
        ColorBgra ObjectColor = ColorBgra.White;
        ColorBgra EmptyColor = ColorBgra.Black;
        if (invertMask)
        {
            ObjectColor = ColorBgra.Black;
            EmptyColor = ColorBgra.White;
        }
        for (int y = rect.Top; y < rect.Bottom; y++)
        {
            if (IsCancelRequested) return;
            ColorBgra* srcPtr = src.GetPointAddressUnchecked(rect.Left, y);
            ColorBgra* dstPtr = dst.GetPointAddressUnchecked(rect.Left, y);
            for (int x = rect.Left; x < rect.Right; x++)
            {
                if ((*srcPtr).A > 0)
                {
                    *dstPtr = ObjectColor;
                }
                else
                {
                    *dstPtr = EmptyColor;
                }
                srcPtr++;
                dstPtr++;
            }
        }
    }
}

void CopyMask(Surface dst, Surface src, bool aliased, bool invertMask = false)
{
    CopyMask(dst,src,src.Bounds,aliased,invertMask);
}

At first glance you might think the code is unnecessarily long with 3 loops in there.  However, I did several things to speed up the code:

 

1. By putting the "if (aliased)" check outside of the actual loop (and thus having 2 similar loops), I only execute the "if" command once per run instead of 480,000 times for an 800x600 canvas had I only had 1 loop with the "if" command inside.
2. Using pointers to access the surfaces saves a TON of time. For example, when you use code like "CurrentPixel = src[x,y];" which is included in the default CodeLab script, Paint.NET takes time to check to be sure that the "x" coordinate is contained within the bounds of the canvas.  It also does the same check for "y" before finally returning the pixel at that address.  By using pointers, like "CurrentPixel = *srcPtr;" Paint.NET doesn't spend any time verifying that the pixel you're requesting is located on the canvas. It simply gives you what's at that memory location directly. It is up to us to be sure that our loops and our pointers remain valid.  So, for our 800x600 src canvas, dst canvas, and aux canvas that saves 2,880,000 boundary checks--"if statements" (x and y, 2 variables, times 3 surfaces times 480,000 pixels).

 

Anyway, now that we have that code, we need a plan to create a plugin.

 

Modify Mask

 

For the purposes of this demo, once the alpha mask is extracted, we will use the Brightness/Contrast effect to lighten the mask making the resulting object slightly transparent.

 

Cool, now that we have a plan, let's start our plugin in CodeLab.  (CodeLab v6.0+ is required for this tutorial.)

 

Before we open CodeLab, let's create a sample image that we can view while we are working on the code.  Something like this should be fine:

 

 

It has an object on a transparent background. You can see some anti-aliased edges and some solid areas. Anything similar should work fine.

 

Now let's open Paint.NET, and open Effects > Advanced > CodeLab.  Inside of CodeLab, open File > New > Effect. This will open a screen that will allow us to design our effect.

 

Since CodeLab doesn't have built-in commands to extract and combine the alpha channel, we'll substitute the "Copy Surface" command instead.  The reason for that choice is that the Copy Surface commands are only a single line long so they'll be easy to replace later.

 

Enter the following items in the Pixel Flow area:

Looking at the arrows in the Pixel Flow list, you can kinda see the path the pixels take through your code.

 

Once you have everything entered properly (note the surfaces used at each step), click the "Generate Code" button.  This will fill CodeLab with the following code:

// Name:
// Submenu:
// Author:
// Title:
// Version:
// Desc:
// Keywords:
// URL:
// Help:
#region UICode
IntSliderControl Amount1 = 10; // [-100,100] Brightness/Contrast Brightness
IntSliderControl Amount2 = 10; // [-100,100] Brightness/Contrast Contrast
#endregion

// Aux surface
Surface aux = null;

// Setup for calling the Brightness and Contrast Adjustment function
BrightnessAndContrastAdjustment contrastEffect = new BrightnessAndContrastAdjustment();
PropertyCollection contrastProps;

protected override void OnDispose(bool disposing)
{
    if (disposing)
    {
        // Release any surfaces or effects you've created
        aux?.Dispose(); aux = null;
        contrastEffect?.Dispose(); contrastEffect = null;
    }

    base.OnDispose(disposing);
}

// This single-threaded function is called after the UI changes and before the Render function is called
// The purpose is to prepare anything you'll need in the Render function
void PreRender(Surface dst, Surface src)
{
    if (aux == null)
    {
        aux = new Surface(src.Size);
    }

    if (IsCancelRequested) return;
    // Copy the src surface to the aux surface
    aux.CopySurface(src);
    // Brightness/Contrast
    contrastEffect.EnvironmentParameters = EnvironmentParameters;
    contrastProps = contrastEffect.CreatePropertyCollection();
    PropertyBasedEffectConfigToken contrastParameters = new PropertyBasedEffectConfigToken(contrastProps);
    contrastParameters.SetPropertyValue(BrightnessAndContrastAdjustment.PropertyNames.Brightness, Amount1);
    contrastParameters.SetPropertyValue(BrightnessAndContrastAdjustment.PropertyNames.Contrast, Amount2);
    contrastEffect.SetRenderInfo(contrastParameters, new RenderArgs(aux), new RenderArgs(aux));
    if (IsCancelRequested) return;
    contrastEffect.Render(new Rectangle[1] {aux.Bounds},0,1);
}

// Here is the main multi-threaded render function
// The dst canvas is broken up into rectangles and
// your job is to write to each pixel of that rectangle
void Render(Surface dst, Surface src, Rectangle rect)
{
    // Copy the aux surface to the dst surface
    dst.CopySurface(aux,rect.Location,rect);


    // Step through each row of the current rectangle
    for (int y = rect.Top; y < rect.Bottom; y++)
    {
        if (IsCancelRequested) return;
        // Step through each pixel on the current row of the rectangle
        for (int x = rect.Left; x < rect.Right; x++)
        {
            ColorBgra SrcPixel = src[x,y];
            ColorBgra AuxPixel = aux[x,y];
            ColorBgra DstPixel = dst[x,y];

            ColorBgra CurrentPixel = DstPixel;

            // TODO: Add additional pixel processing code here


            dst[x,y] = CurrentPixel;
        }
    }
}

Now, go grab the code from above to extract the alpha channel and paste it into this script just above the PreRender function.

 

Next, find the line "aux.CopySurface(src);" in the PreRender function and change that line to read:

// Copy the src surface to the aux surface
CopyMask(aux,src,true,true);

Instead of copying the entire src surface to the aux surface, we're copying only the alpha mask to the aux surface.

 

Since we are working in the PreRender function, we're processing the entire canvas at once.  If we were in the multi-threaded Render function, we would also be specifying the ROI (rectangle of interest)... you'll see that a bit later, but no need to worry about it now.

 

Next, find the UI definition lines near the top and update them to have our planned defaults:

#region UICode
IntSliderControl Amount1 = 70; // [-100,100] Brightness/Contrast Brightness
IntSliderControl Amount2 = 0; // [-100,100] Brightness/Contrast Contrast
#endregion

At this point in the process, you should probably see the modified mask being output to the canvas by the effect. Now that we have the mask the way we want it, let's use it...

 

 

Apply Mask to Original Image

 

We're at the point now where we need some code to combine our alpha mask with the original image.  Here is a helper function I wrote to apply an alpha mask to a layer and store the results in a different layer:

unsafe void ApplyMask(Surface dst, Surface src, Surface mask, Rectangle rect, bool invertMask = false)
{
    if (invertMask)
    {
        for (int y = rect.Top; y < rect.Bottom; y++)
        {
            if (IsCancelRequested) return;
            ColorBgra* srcPtr = src.GetPointAddressUnchecked(rect.Left, y);
            ColorBgra* dstPtr = dst.GetPointAddressUnchecked(rect.Left, y);
            ColorBgra* maskPtr = mask.GetPointAddressUnchecked(rect.Left, y);
            for (int x = rect.Left; x < rect.Right; x++)
            {
                *dstPtr = (*srcPtr).NewAlpha((byte)(255-(*maskPtr).R));
                srcPtr++;
                dstPtr++;
                maskPtr++;
            }
        }
    }
    else
    {
        for (int y = rect.Top; y < rect.Bottom; y++)
        {
            if (IsCancelRequested) return;
            ColorBgra* srcPtr = src.GetPointAddressUnchecked(rect.Left, y);
            ColorBgra* dstPtr = dst.GetPointAddressUnchecked(rect.Left, y);
            ColorBgra* maskPtr = mask.GetPointAddressUnchecked(rect.Left, y);
            for (int x = rect.Left; x < rect.Right; x++)
            {
                *dstPtr = (*srcPtr).NewAlpha((*maskPtr).R);
                srcPtr++;
                dstPtr++;
                maskPtr++;
            }
        }
    }
}

void ApplyMask(Surface dst, Surface src, Surface mask, bool invertMask = false)
{
    ApplyMask(dst,src,mask,src.Bounds, invertMask);
}

void ApplyMask(Surface src, Surface mask, bool invertMask = false)
{
    ApplyMask(src, src, mask, src.Bounds, invertMask);
}

void ApplyMask(Surface src, Surface mask, Rectangle rect, bool invertMask = false)
{
    ApplyMask(src, src, mask, rect, invertMask);
}

 

Grab that code and paste it into your script just above the Render function.

 

 

See the Results

 

Find the following line in your Render function: dst.CopySurface(aux,rect.Location,rect);

 

Replace that line with:

ApplyMask(dst, src, aux, rect, true);

Instead of copying the aux surface to the dst surface, we will be applying the alpha mask on the aux surface to the object on the src surface and storing the results on the dst surface.

 

Since we are in the multi-threaded Render function, notice that we are also sending in the "rect"--that is our Rectangle of Interest (ROI).  This limits processing of only the portion of the layer our current thread is working on.

 

That should do it!

 

Here is a copy of the final assembled code:

// Name:
// Submenu:
// Author:
// Title:
// Version:
// Desc:
// Keywords:
// URL:
// Help:
#region UICode
IntSliderControl Amount1 = 70; // [-100,100] Brightness/Contrast Brightness
IntSliderControl Amount2 = 0; // [-100,100] Brightness/Contrast Contrast
#endregion

// Aux surface
Surface aux = null;

// Setup for calling the Brightness and Contrast Adjustment function
BrightnessAndContrastAdjustment contrastEffect = new BrightnessAndContrastAdjustment();
PropertyCollection contrastProps;

protected override void OnDispose(bool disposing)
{
    if (disposing)
    {
        // Release any surfaces or effects you've created
        aux?.Dispose(); aux = null;
        contrastEffect?.Dispose(); contrastEffect = null;
    }

    base.OnDispose(disposing);
}

unsafe void CopyMask(Surface dst, Surface src, Rectangle rect, bool aliased, bool invertMask = false)
{
    if (aliased)
    {
        byte shade;
        if (invertMask)
        {
            for (int y = rect.Top; y < rect.Bottom; y++)
            {
                if (IsCancelRequested) return;
                ColorBgra* srcPtr = src.GetPointAddressUnchecked(rect.Left, y);
                ColorBgra* dstPtr = dst.GetPointAddressUnchecked(rect.Left, y);
                for (int x = rect.Left; x < rect.Right; x++)
                {
                    shade = (byte)(255-(*srcPtr).A);
                    *dstPtr = ColorBgra.FromBgr(shade,shade,shade);
                    srcPtr++;
                    dstPtr++;
                }
            }
        }
        else
        {
            for (int y = rect.Top; y < rect.Bottom; y++)
            {
                if (IsCancelRequested) return;
                ColorBgra* srcPtr = src.GetPointAddressUnchecked(rect.Left, y);
                ColorBgra* dstPtr = dst.GetPointAddressUnchecked(rect.Left, y);
                for (int x = rect.Left; x < rect.Right; x++)
                {
                    shade = (*srcPtr).A;
                    *dstPtr = ColorBgra.FromBgr(shade,shade,shade);
                    srcPtr++;
                    dstPtr++;
                }
            }
        }
    }
    else
    {
        ColorBgra ObjectColor = ColorBgra.White;
        ColorBgra EmptyColor = ColorBgra.Black;
        if (invertMask)
        {
            ObjectColor = ColorBgra.Black;
            EmptyColor = ColorBgra.White;
        }
        for (int y = rect.Top; y < rect.Bottom; y++)
        {
            if (IsCancelRequested) return;
            ColorBgra* srcPtr = src.GetPointAddressUnchecked(rect.Left, y);
            ColorBgra* dstPtr = dst.GetPointAddressUnchecked(rect.Left, y);
            for (int x = rect.Left; x < rect.Right; x++)
            {
                if ((*srcPtr).A > 0)
                {
                    *dstPtr = ObjectColor;
                }
                else
                {
                    *dstPtr = EmptyColor;
                }
                srcPtr++;
                dstPtr++;
            }
        }
    }
}

void CopyMask(Surface dst, Surface src, bool aliased, bool invertMask = false)
{
    CopyMask(dst,src,src.Bounds,aliased,invertMask);
}

// This single-threaded function is called after the UI changes and before the Render function is called
// The purpose is to prepare anything you'll need in the Render function
void PreRender(Surface dst, Surface src)
{
    if (aux == null)
    {
        aux = new Surface(src.Size);
    }

    if (IsCancelRequested) return;
    // Copy the src surface to the aux surface
    CopyMask(aux,src,true,true);

    // Brightness/Contrast
    contrastEffect.EnvironmentParameters = EnvironmentParameters;
    contrastProps = contrastEffect.CreatePropertyCollection();
    PropertyBasedEffectConfigToken contrastParameters = new PropertyBasedEffectConfigToken(contrastProps);
    contrastParameters.SetPropertyValue(BrightnessAndContrastAdjustment.PropertyNames.Brightness, Amount1);
    contrastParameters.SetPropertyValue(BrightnessAndContrastAdjustment.PropertyNames.Contrast, Amount2);
    contrastEffect.SetRenderInfo(contrastParameters, new RenderArgs(aux), new RenderArgs(aux));
    if (IsCancelRequested) return;
    contrastEffect.Render(new Rectangle[1] {aux.Bounds},0,1);
}

unsafe void ApplyMask(Surface dst, Surface src, Surface mask, Rectangle rect, bool invertMask = false)
{
    if (invertMask)
    {
        for (int y = rect.Top; y < rect.Bottom; y++)
        {
            if (IsCancelRequested) return;
            ColorBgra* srcPtr = src.GetPointAddressUnchecked(rect.Left, y);
            ColorBgra* dstPtr = dst.GetPointAddressUnchecked(rect.Left, y);
            ColorBgra* maskPtr = mask.GetPointAddressUnchecked(rect.Left, y);
            for (int x = rect.Left; x < rect.Right; x++)
            {
                *dstPtr = (*srcPtr).NewAlpha((byte)(255-(*maskPtr).R));
                srcPtr++;
                dstPtr++;
                maskPtr++;
            }
        }
    }
    else
    {
        for (int y = rect.Top; y < rect.Bottom; y++)
        {
            if (IsCancelRequested) return;
            ColorBgra* srcPtr = src.GetPointAddressUnchecked(rect.Left, y);
            ColorBgra* dstPtr = dst.GetPointAddressUnchecked(rect.Left, y);
            ColorBgra* maskPtr = mask.GetPointAddressUnchecked(rect.Left, y);
            for (int x = rect.Left; x < rect.Right; x++)
            {
                *dstPtr = (*srcPtr).NewAlpha((*maskPtr).R);
                srcPtr++;
                dstPtr++;
                maskPtr++;
            }
        }
    }
}

void ApplyMask(Surface dst, Surface src, Surface mask, bool invertMask = false)
{
    ApplyMask(dst,src,mask,src.Bounds, invertMask);
}

void ApplyMask(Surface src, Surface mask, bool invertMask = false)
{
    ApplyMask(src, src, mask, src.Bounds, invertMask);
}

void ApplyMask(Surface src, Surface mask, Rectangle rect, bool invertMask = false)
{
    ApplyMask(src, src, mask, rect, invertMask);
}

// Here is the main multi-threaded render function
// The dst canvas is broken up into rectangles and
// your job is to write to each pixel of that rectangle
void Render(Surface dst, Surface src, Rectangle rect)
{
    // Copy the aux surface to the dst surface
    ApplyMask(dst,src,aux,rect, true);

    // Step through each row of the current rectangle
    for (int y = rect.Top; y < rect.Bottom; y++)
    {
        if (IsCancelRequested) return;
        // Step through each pixel on the current row of the rectangle
        for (int x = rect.Left; x < rect.Right; x++)
        {
            ColorBgra SrcPixel = src[x,y];
            ColorBgra AuxPixel = aux[x,y];
            ColorBgra DstPixel = dst[x,y];

            ColorBgra CurrentPixel = DstPixel;

            // TODO: Add additional pixel processing code here


            dst[x,y] = CurrentPixel;
        }
    }
}

Currently, the x and y loops inside the Render function aren't doing anything.  You can delete them.  Or, you can do some additional processing in there--your choice.

 

I will leave it up to you to investigate the alias and invertMask parameters on the CopyMask and ApplyMask functions.  

 

Final Thoughts

 

I hope this lesson sparks some creativity in you and you produce some wonderful plugins!  For example, you could probably modify the helper functions I wrote to extract other channels, like R, G, and/or B.  I'm sure you can come up with all sorts of interesting variations.

 

Thanks for reading!   

 

Big THANKS goes out to @toe_head2001 for reviewing my code for this tutorial.

[ReMake]

Perfectly!

[Pixey]

OMG @BoltBait you are such an inspiration   (Don't hold breath for moi trying this out.) 🤣

[toe_head2001]

Here are some potential improvements/optimizations:

 

In PreRender, you can constrain CopyMask() to the selection bounds. This way, you don't waste CPU cycles changing pixels that are out of scope.

Rectangle selBounds = EnvironmentParameters.SelectionBounds;
CopyMask(aux, src, selBounds, true, true);

Be aware, you may not always want to constrain to selection. It just depends on what your Effect plugin is doing.

For example, a Gaussian Blur can be influenced by pixels outside of the selection, so you wouldn't constrain to the selection bounds in that case.

 

 

 

The call to contrastEffect.Render() can be moved from PreRender to Render.

contrastEffect.Render(new Rectangle[] { rect }, 0, 1);

Again, this is something you may not always want to do, for mostly the same kind of reasons stated above.

[BoltBait]

1 hour ago, toe_head2001 said:

Here are some potential improvements/optimizations:

 

Those are some great optimizations!  Thanks for posting.

 

1 hour ago, toe_head2001 said:

The call to contrastEffect.Render() can be moved from PreRender to Render.

contrastEffect.Render(new Rectangle[] { rect }, 0, 1);

Again, this is something you may not always want to do, for mostly the same kind of reasons stated above.

 

CodeLab is pretty decent in deciding which of your selected effects should go in PreRender and which should go in Render. But, it is always best to examine those choices to make sure they are best.  When I wrote the code generator, I had to make it work in the general case.

 

In this case, you make a good point, the setup for calling the contrast effect should remain in PreRender and the actual call to contrast should be moved (and rewritten slightly) to the first line of the Render method.  The reason for this is that the PreRender method is single threaded and the Render method is multi-threaded.  So, you should do as much work as possible in Render and do as little as possible in PreRender.  (I felt that this optimization was outside of the scope of the tutorial, but I'm glad you brought it up so we can talk about it.)

 

Generally speaking, you should be preparing your AUX and WRK surfaces in PreRender and you should be using those surfaces in Render.  This is because you want your entire working canvas to be ready when the multi-threaded magic happens.  This is important if you'll be reading from multiple pixels from the working surface to apply to a pixel in your DST canvas (like using the Gaussian Blur effect) as there's really no way to stay within your ROI during that type of processing.

 

In the case of the contrast effect, it only looks at a single pixel when computing it's result. Due to that and the fact that our example is so simple, we could have actually moved EVERYTHING from PreRender to Render for better performance due to the multi-threaded pipeline and everything we're using can be limited to a single ROI.

 

So, that was an excellent suggestion!

 

Note: How to tell if something reads more than one pixel to do it's work?  Generally, things in the Adjustments menu work on individual pixels and things in the Effects menu GENERALLY read more than one pixel to calculate their results.  (Then, there are some special cases, like Floyd-Steinberg Dithering where the pixels you output can effect pixels to be calculated later outside of your ROI!)