Convolution Effect

[SeriousSam]

Convolution Effect Version 1.2 Out!

I think I am a better programmer than artist... so I decided to contribute with a convolution effect.

I kept the kernel at a 3x3 to be cpu friendly. Maybe in another release I will give an option to choose the size.

In my implementation I have included a quick drop down to set the kernel to some of the more common kernels including an identity, blur, sharpen, mean removal, edge detection. laplacian, and emboss. But with quick modifications it can come up with some very interesting effects. Have fun .

I have uploaded both the dll effect file and the VS2008 c# solution. The code was written in a very straight forward manor and should be easy to follow for anyone interested.

[EDIT] V1.1 release

Optimized code as much as could. I Measured speed improvements around 30% in the unoptimized debug version. In the release version my 4Ghz Duo Core will process a 1200x1600 image around 16ms. I think that is impressive without use of assembly and SIMD (Single instruction multiple data), never mind.

I left the timer code in the release version. To see how long it took to process the scene in milliseconds pressing a button called "get" after rendering is done that will retrieve the time.

[EDIT] V1.2 release

Added File IO Support to save and load kernels. Kernels can be added and removed with a click of a button. When the Convolution Dialog is opened it looks for "ConvolutionEffectKernels.txt" in the "*\Paint.NET\Effects" Directory. This file contains the kernels. If the text file exist it will be loaded or will be created with the preset kernels. Also, if there are any problems opening or creating the file the preset kernels can still be used, however the ability to Add, Remove, Update and Restore will be disabled. I know file io is a lot of work...

To add a kernel select "Add Current". A dialog will come up to give the kernel a name. The new kernel will be saved and added to the drop down list with the name assigned.

To remove the selected kernel from the drop down click "Remove Current".

To modify the selected kernel select "Update". This updates the selected kernel in the drop down with the text values.

"Restore", will overwrite the "ConvolutionEffectKernels.txt" file with only the preset kernels. Any kernels created will be lost! I did not add a warning so be careful not to press this unless desired.

ConvolutionEffect.zip

ConvolutionEffectSln.zip

[Blooper]

I have no idea what this does, but I'll check it out. Thanks!

[Edit] I don't really understand what is changed depending on what numbers I change. All I can tell so far is that the sharpening looks nice

[Edit] OH, so if I edit the bottom ones, the effect will move towards the bottom.

[davidtayhs]

Dear Serious Sam,

Thank you for all the effort in producing this plugin. But seriously Serious Sam, I think we would benefit from a little more documentation as to what the various controls do to which part of the edited photo, as well as info on some recommended settings for the control.

Would appreciate some explanations.

[pyrochild]

A convolution kernel applies a matrix to every pixel in the image.

The matrix is centered around the pixel of interest (POI). The middle value is the pixel itself. The top-left value is diagonally up 1 and 1 left from the POI, and so on. The final color of the POI is determined by multiplying the value in the matrix by that value's corresponding pixel, and adding these together, then dividing by the divisor.

For example, if the matrix was:

[[ 1, 1, 1]

[ 1, 0, 1]

[ 1, 1, 1]]

and the divisor is 8, then each pixel in the resulting image will be a blend/average of the surrounding pixels of the source image.

It's the average because we're adding together all of the surrounding values, and dividing by how many we added.

[[ 0, 0, 0]

[1, 0, 0]

[0, 0, 0]]

would move the image 1 pixel to the right.

[davidtayhs]

Thanks Pyrochild for the explanation. I will now have to pretend that I really understand it all! Lol!!! Thanks anyway!

[pyrochild]

I'll put together some example images.

Nevermind - This computer's .NET Framework is broken and I don't feel like messing with it tonight.

[SeriousSam]

It is exactly as pyrochild said, with the addition that the bias is added after the matrix is applied.

For anyone struggling try using the choosing some preset effects in the top right to load the kernel, divisor, and bias with preset values. In the screen shot it is set to "Identity". After loading one of the presets try adjusting the values to achieve a desired effect. As for documentation, I appologize but I did not think I needed to provide any. This is a very common technique with plenty of readily available information. And the values that have been used are well established.

But in short the algorithm does the following do every color in an image

How Kernel Is Mapped to every color in the Image

[Above Left, Above, Above Right ]

[Left, POI (point of interest), Right ]

[below Left, Below, Below Right ]

Values Set in Kernel

[A,B,C]

[D,E,F]

[G,H,I]

Color = A * (Color Above Left) + B * (Color Above) + C * (Color Above Right) + D * (Color Left) + E * (Color POI) + F * (Color Right) + G * (Color Below Left) + H * (Color Below) + I * (Color Below Right)

Color = Color / Divisor + Bias

Taking a look at the "Identity" kernel which is

[0,0,0]

[0,1,0]

[0,0,0]

Divisor = 1

Bias = 0;

Then

Color = 0 * (Color Above Left) + 0 * (Color Above) + 0 * (Color Above Right) + 0 * (Color Left) + 1 * (Color POI) + 0 * (Color Right) + 0 * (Color Below Left) + 0 * (Color Below) + 0 * (Color Below Right)

Color = Color / 1 + 0

It should be clear that the resulting color at the POI is the original color at the POI. I hope this cleared up some of the confusion.

[pyrochild]

It is exactly as pyrochild said, with the addition that the bias is added after the matrix is applied.

I didn't know what the bias did, so I conveniently left that out of my explanation...

I had actually thought about making a convolution filter a while back, and indeed it's on my list of projects, but now it's crossed off.

[Ego Eram Reputo]

So does this plugin allow for image transformations of this nature: http://www.graficaobscura.com/matrix/index.html ?

[SeriousSam]

Afraid not Ego Eram Reputo, this is not used to adjust the hue, brightness, and saturation as Paul had proposed in that article. This matrix is for blending the colors at a position with the neighbor colors to achieve different effects. The more common usages are for sharpening, blurring, edge detection, and emboss. That article did look interesting though.

[Ego Eram Reputo]

....The more common usages are for sharpening, blurring, edge detection, and emboss.

Aha - Sobel filters? Brilliant! Downloading in 3....2....1

[SeriousSam]

Yes it looks like it can do Sobel operation with a little manual work. From what I read Sobel uses two kernels. I got decent results with the following procedure.

Orig Image:

I Cloned the original image twice

Applied this Kernel to top layer and set to additive or lighten blending.

[1, 2, 1]

[0, 0, 0]

[-1,-2,-1] Divisor 1, Bias 0

Applied this Kernel to other cloned layer (one below)

[1, 0, -1]

[2, 0, -2]

[1, 0, -1] Divisor 1, Bias 0

And Flattening the two layers I got this result

Also interesting is blending the original image with the Sobel result.

With the original image as top layer.

Orig Image Layer Normal Blend opacity 107

Orig Image Layer Negative Blend opacity 167

Orig Image Layer Negative Blend opacity 255

I got the kernel values and image from http://en.wikipedia.org/wiki/Sobel_operator.

[M_Lyons4]

Wow, this looks pretty cool. It appears to be rather complicated though... I'll definitely play around with it some more here, but I'm not really sure what these values mean... :?

[Gamer_World14]

Just when I thought I could forget about matrices. I am also confused as M_Lyons4 was. But I will definitely download this and give it a try

Thanks!

[BoltBait]

This reminds me of the "Toon" filter. Except, Toon uses a 5x5 matrix.

Toon

I wonder how similar the code is...

[Rick Brewster]

My guess is that a Save/Load Kernel will be very useful on this plugin.

[MeowMeow]

I was looking for something like this... Thanks, bud.

[SeriousSam]

A toon filter with a 5x5, I've got matrix envy! A save/load would be nice. I would like to finish a different plugin first. It is something I have a personal interest in and am looking forward to getting done.

[Ego Eram Reputo]

A save/load would be nice.

Yes please!

I would actually prefer a load-all (into the dropdown list) from a specific area/folder/textfile. Thus all my matrices would be loaded and I could pick & choose from the dropdown list - rather than hunting and loading a specific one from (say) a folder.

[midora]

Would it be OK that I'm adapting this plugin?

The feature to load/save the kernels does not work on my machines because Paint.NET is not allowed to write to the Program files folder (best would be to store the file in the 'Paint.NET User Files' folder).

 

Or maybe a rewrite to allow 5x5.

 

BTW (after 4 years ;-)

 

So does this plugin allow for image transformations of this nature: http://www.graficaobscura.com/matrix/index.html ?

 

Did a 'Color Matrix' plugin which implements most what is described there. Just didn't publish it because there are so many Color plugins. But if there is interest, raise your hand.

[Ego Eram Reputo]

I think I found a workaround

[ArgusMagnus]

Has anyone adapted this now? I might do so otherwise.

 

 

 

How Kernel Is Mapped to every color in the Image
[Above Left, Above, Above Right ]
[Left, POI (point of interest), Right ]
[below Left, Below, Below Right ]

Values Set in Kernel
[A,B,C]
[D,E,F]
[G,H,I]

Color = A * (Color Above Left) + B * (Color Above) + C * (Color Above Right) + D * (Color Left) + E * (Color POI) + F * (Color Right) + G * (Color Below Left) + H * (Color Below) + I * (Color Below Right)
Color = Color / Divisor + Bias

 

I might be mistaken, but for mathematical correct convolution, the kernel would be flipped two times around the middle row and column, no? Might be even more confusing this way though. (And many often used kernels are symmetric any way)

Edited August 26, 2015 by ArgusMagnus

[MJW]

You're correct, ArgusMagnus. To be a true convolution, if the image contains a single pixel of intensity one, the convolution should be the convolution kernel centered at the pixel. If you consider a kernel which has the lower-left corner set and all the other pixels cleared, then the plugin's method would result in the upper-right corner set and the rest clear. I agree about the confusion factor; what the plugin does may be what the user wants to do. Ideally, the plugin could offer the option of reflecting or not.