Sign in to follow this  
SeriousSam

Convolution Effect

Recommended Posts

Convolution Effect Version 1.2 Out!

I think I am a better programmer than artist... so I decided to contribute with a convolution effect.

ConvolutionDialog.jpg

I kept the kernel at a 3x3 to be cpu friendly. Maybe in another release I will give an option to choose the size.

In my implementation I have included a quick drop down to set the kernel to some of the more common kernels including an identity, blur, sharpen, mean removal, edge detection. laplacian, and emboss. But with quick modifications it can come up with some very interesting effects. Have fun :D .

I have uploaded both the dll effect file and the VS2008 c# solution. The code was written in a very straight forward manor and should be easy to follow for anyone interested.

[EDIT] V1.1 release

Optimized code as much as could. I Measured speed improvements around 30% in the unoptimized debug version. In the release version my 4Ghz Duo Core will process a 1200x1600 image around 16ms. I think that is impressive without use of assembly and SIMD (Single instruction multiple data), never mind.

I left the timer code in the release version. To see how long it took to process the scene in milliseconds pressing a button called "get" after rendering is done that will retrieve the time.

[EDIT] V1.2 release

Added File IO Support to save and load kernels. Kernels can be added and removed with a click of a button. When the Convolution Dialog is opened it looks for "ConvolutionEffectKernels.txt" in the "*\Paint.NET\Effects" Directory. This file contains the kernels. If the text file exist it will be loaded or will be created with the preset kernels. Also, if there are any problems opening or creating the file the preset kernels can still be used, however the ability to Add, Remove, Update and Restore will be disabled. I know file io is a lot of work...

To add a kernel select "Add Current". A dialog will come up to give the kernel a name. The new kernel will be saved and added to the drop down list with the name assigned.

To remove the selected kernel from the drop down click "Remove Current".

To modify the selected kernel select "Update". This updates the selected kernel in the drop down with the text values.

"Restore", will overwrite the "ConvolutionEffectKernels.txt" file with only the preset kernels. Any kernels created will be lost! I did not add a warning so be careful not to press this unless desired.

ConvolutionEffect.zip

ConvolutionEffectSln.zip

Share this post


Link to post
Share on other sites

I have no idea what this does, but I'll check it out. Thanks!

[Edit] I don't really understand what is changed depending on what numbers I change. All I can tell so far is that the sharpening looks nice :D

[Edit] OH, so if I edit the bottom ones, the effect will move towards the bottom.

Share this post


Link to post
Share on other sites

Dear Serious Sam,

Thank you for all the effort in producing this plugin. But seriously Serious Sam, I think we would benefit from a little more documentation as to what the various controls do to which part of the edited photo, as well as info on some recommended settings for the control.

Would appreciate some explanations.

Share this post


Link to post
Share on other sites

A convolution kernel applies a matrix to every pixel in the image.

The matrix is centered around the pixel of interest (POI). The middle value is the pixel itself. The top-left value is diagonally up 1 and 1 left from the POI, and so on. The final color of the POI is determined by multiplying the value in the matrix by that value's corresponding pixel, and adding these together, then dividing by the divisor.

For example, if the matrix was:

[[ 1, 1, 1]

[ 1, 0, 1]

[ 1, 1, 1]]

and the divisor is 8, then each pixel in the resulting image will be a blend/average of the surrounding pixels of the source image.

It's the average because we're adding together all of the surrounding values, and dividing by how many we added.

[[ 0, 0, 0]

[1, 0, 0]

[0, 0, 0]]

would move the image 1 pixel to the right.

Share this post


Link to post
Share on other sites

I'll put together some example images.

Nevermind - This computer's .NET Framework is broken and I don't feel like messing with it tonight.

Share this post


Link to post
Share on other sites

It is exactly as pyrochild said, with the addition that the bias is added after the matrix is applied.

For anyone struggling try using the choosing some preset effects in the top right to load the kernel, divisor, and bias with preset values. In the screen shot it is set to "Identity". After loading one of the presets try adjusting the values to achieve a desired effect. As for documentation, I appologize but I did not think I needed to provide any. This is a very common technique with plenty of readily available information. And the values that have been used are well established.

But in short the algorithm does the following do every color in an image

How Kernel Is Mapped to every color in the Image

[Above Left, Above, Above Right ]

[Left, POI (point of interest), Right ]

[below Left, Below, Below Right ]

Values Set in Kernel

[A,B,C]

[D,E,F]

[G,H,I]

Color = A * (Color Above Left) + B * (Color Above) + C * (Color Above Right) + D * (Color Left) + E * (Color POI) + F * (Color Right) + G * (Color Below Left) + H * (Color Below) + I * (Color Below Right)

Color = Color / Divisor + Bias

Taking a look at the "Identity" kernel which is

[0,0,0]

[0,1,0]

[0,0,0]

Divisor = 1

Bias = 0;

Then

Color = 0 * (Color Above Left) + 0 * (Color Above) + 0 * (Color Above Right) + 0 * (Color Left) + 1 * (Color POI) + 0 * (Color Right) + 0 * (Color Below Left) + 0 * (Color Below) + 0 * (Color Below Right)

Color = Color / 1 + 0

It should be clear that the resulting color at the POI is the original color at the POI. I hope this cleared up some of the confusion.

Share this post


Link to post
Share on other sites
It is exactly as pyrochild said, with the addition that the bias is added after the matrix is applied.

I didn't know what the bias did, so I conveniently left that out of my explanation...

I had actually thought about making a convolution filter a while back, and indeed it's on my list of projects, but now it's crossed off. ;)

Share this post


Link to post
Share on other sites

Afraid not Ego Eram Reputo, this is not used to adjust the hue, brightness, and saturation as Paul had proposed in that article. This matrix is for blending the colors at a position with the neighbor colors to achieve different effects. The more common usages are for sharpening, blurring, edge detection, and emboss. That article did look interesting though.

Share this post


Link to post
Share on other sites
....The more common usages are for sharpening, blurring, edge detection, and emboss.

Aha - Sobel filters? Brilliant! Downloading in 3....2....1

Share this post


Link to post
Share on other sites

Yes it looks like it can do Sobel operation with a little manual work. From what I read Sobel uses two kernels. I got decent results with the following procedure.

Orig Image:

conv_orig.jpg

I Cloned the original image twice

Applied this Kernel to top layer and set to additive or lighten blending.

[1, 2, 1]

[0, 0, 0]

[-1,-2,-1] Divisor 1, Bias 0

Applied this Kernel to other cloned layer (one below)

[1, 0, -1]

[2, 0, -2]

[1, 0, -1] Divisor 1, Bias 0

And Flattening the two layers I got this result

conv_sobel.jpg

Also interesting is blending the original image with the Sobel result.

With the original image as top layer.

Orig Image Layer Normal Blend opacity 107

conv_norm.jpg

Orig Image Layer Negative Blend opacity 167

conv_diff.jpg

Orig Image Layer Negative Blend opacity 255

conv_diff_max.jpg

I got the kernel values and image from http://en.wikipedia.org/wiki/Sobel_operator.

Share this post


Link to post
Share on other sites

Wow, this looks pretty cool. It appears to be rather complicated though... I'll definitely play around with it some more here, but I'm not really sure what these values mean... :?

Share this post


Link to post
Share on other sites

Just when I thought I could forget about matrices. I am also confused as M_Lyons4 was. But I will definitely download this and give it a try ;)

Thanks!

Share this post


Link to post
Share on other sites

This reminds me of the "Toon" filter. Except, Toon uses a 5x5 matrix.

Toon

I wonder how similar the code is... :D

Share this post


Link to post
Share on other sites

A toon filter with a 5x5, I've got matrix envy! A save/load would be nice. I would like to finish a different plugin first. It is something I have a personal interest in and am looking forward to getting done.

Share this post


Link to post
Share on other sites
A save/load would be nice.

Yes please!

I would actually prefer a load-all (into the dropdown list) from a specific area/folder/textfile. Thus all my matrices would be loaded and I could pick & choose from the dropdown list - rather than hunting and loading a specific one from (say) a folder.

Share this post


Link to post
Share on other sites

Would it be OK that I'm adapting this plugin?

The feature to load/save the kernels does not work on my machines because Paint.NET is not allowed to write to the Program files folder (best would be to store the file in the 'Paint.NET User Files' folder).

 

Or maybe a rewrite to allow 5x5.

 

BTW (after 4 years ;-)

 

So does this plugin allow for image transformations of this nature: http://www.graficaobscura.com/matrix/index.html ?

 

Did a 'Color Matrix' plugin which implements most what is described there. Just didn't publish it because there are so many Color plugins. But if there is interest, raise your hand.

Share this post


Link to post
Share on other sites

Has anyone adapted this now? I might do so otherwise.

 

 

 

How Kernel Is Mapped to every color in the Image
[Above Left, Above, Above Right ]
[Left, POI (point of interest), Right ]
[below Left, Below, Below Right ]

Values Set in Kernel
[A,B,C]
[D,E,F]
[G,H,I]

Color = A * (Color Above Left) + B * (Color Above) + C * (Color Above Right) + D * (Color Left) + E * (Color POI) + F * (Color Right) + G * (Color Below Left) + H * (Color Below) + I * (Color Below Right)
Color = Color / Divisor + Bias

 

I might be mistaken, but for mathematical correct convolution, the kernel would be flipped two times around the middle row and column, no? Might be even more confusing this way though. (And many often used kernels are symmetric any way)

Edited by ArgusMagnus

Share this post


Link to post
Share on other sites

You're correct, ArgusMagnus. To be a true convolution, if the image contains a single pixel of intensity one, the convolution should be the convolution kernel centered at the pixel. If you consider a kernel which has the lower-left corner set and all the other pixels cleared, then the plugin's method would result in the upper-right corner set and the rest clear. I agree about the confusion factor; what the plugin does may be what the user wants to do. Ideally, the plugin could offer the option of reflecting or not.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this