2D Image Scaling Algorithms

[Hawkynt]

I finally packed my image filtering library into a Paint.NET Plugin which can be downloaded here : 2dImageFilter.

This library contains all of the standard algorithms used in various emulators (like VirtualGB, Mame, DOSBox, SNES9x, ZSNES to name some of them).

I adapted them to be used not only on pixel graphics by giving them the ability to identify pixels that are alike and not exactly the same color. They will interpolate on matches to give slightly better results than their original. Even though some scaler's seem to result in the same image, they're all slightly different and the best fit heavily depends on the source material. All Algorithms have been rewritten from scratch by looking at the original code, understanding and then adapting so these should be clean (schoolar) version of the algo's.

Thank's to all the people that made up their minds and invented these scalers. Without them I couldn't have coded this.

To demonstrate some of the filtering techniques, I will use this image:

Scale2x:

AdvInterp2x:

Scale3x:

AdvInterp3x:

Bilinear+(Original):

Bilinear+:

Eagle2x:

Eagle3x (I invented this one because it did not exists before):

Eagle3x(Variant B ):

Super Eagle:

Scale and Interpolate (alias SaI2x):

Super Scale and Interpolate (alias SuperSai):

EPXB:

EPXC:

EPX3:

HQ2x(Normal, Bold and Smart, HQ4x, HQ2x3, HQ2x4, etc.):

HQ3x(which is also available as normal, bold and smart version):

LQ2x(also LQ3x, LQ4x, Normal, Smart, Bold, LQ2x3, LQ2x4, etc.):

Also included in this package: Scanline Effects, RGB Effects, TV Effects, RGB- and YUV- Channel extraction and many more.

I recommend when using scaling algorithms to resize the image before applying them because I don't know how to do this from within a plugin or use "Render to Clipboard" and paste the result into a new image.

Hope you'll give me feedback about it.

Edited February 24, 2012 by Hawkynt

[nitenurse79]

Looks interesting so far, maybe a screenshot and some more info might be of use

[Ego Eram Reputo]

A whole lot more info please!

[nitenurse79]

When I run this plugin on an image it crops to a small portion only and not keeping the whole image as your examples show

[Hawkynt]

Jep, as I told: It does not resize the image but the filters are meant to work this way.

So the following happens:

- the filter takes your whole image and magnifies it by eg factor 2 in x and y (like Eagle does or Hq2x)

- now it tries to write back the information to paint

- paint is still having an image with the original size, so it only writes the upper left part of the resized version back

This leads to your problem.

There are two workarounds:

Either resize the image first and let your source image be in the upper left corner. (This is slower because white space is also filtered and you have to know the exact dimensions of the filtered image beforehand)

Or simply press "Render to Clipboard" instead of the "OK" button in the filter configuration and then make a new image with the size from the clipboard and simply paste. (CTRL+N, ENTER, CTRL+V should do the trick )

If someones knows how to programmatically tell paint that the image sizes was changed by the filter, tell me how and I will fix that. ^^

[nitenurse79]

There are two workarounds:

Either resize the image first and let your source image be in the upper left corner. (This is slower because white space is also filtered and you have to know the exact dimensions of the filtered image beforehand)

Or simply press "Render to Clipboard" instead of the "OK" button in the filter configuration and then make a new image with the size from the clipboard and simply paste. (CTRL+N, ENTER, CTRL+V should do the trick )

Do I select only part of my resized image to make my source be in the upper left corner (I'm confused now)

[Hawkynt]

OK try this:

-open an image you want to resize

-go to effect->Hawkynt->2dimagefilter

-choose for example hq4x from the dropdown

-press "render to clipboard"

-click on "cancel" or press "ESC"

-click File->New or press "CTRL+N"

-click OK or press "Enter"

-click Edit->Paste or press "CTRL+V"

Your resized image should now be there in full resolution.

BTW: I'd really like to make this more intuitive but according to this post it seems to be impossible.

Edited December 29, 2011 by Hawkynt

[nitenurse79]

OK try this:

-open an image you want to resize

-go to effect->Hawkynt->2dimagefilter

-choose for example hq4x from the dropdown

-press "render to clipboard"

-click on "cancel" or press "ESC"

-click File->New or press "CTRL+N"

-click OK or press "Enter"

-click Edit->Paste or press "CTRL+V"

Your resized image should now be there in full resolution.

Thank you Hawkynt works like a charm now

[Ego Eram Reputo]

Ok, now I get it. Looks similar to :

Apart from the obvious number of scaling options, how is this better/worse/different from that plugin? (I'm not trying to be difficult, just trying to get you to explain the points of difference between the two).

[Hawkynt]

As you mentioned, I already looked at the plugin you described and I also had a post there, encouraging the author to use my library. But sadly he did not answer, so I wrote my own one. You already noted that there are many more filtering algo's in my plugin than in his own and there are also differencies in how they exactly work. Pikalek (that's the author of the other plugin) tried to get the implementation as close to the original as possible while I tried to rewrite them and adapt them to a "far-near color model" similar to the one that's used in Hq filters.

To get more in detail:

The original scalers from Kreed and Co. compare pixel by color, and only apply their patterns if the pixels have the exact same colors: Something like

if(pixel1.Color==pixel2.Color) {;}

I changed that code everywhere and compared using YUV color space with thresholds to identify similar color values:

if(pixel1.Color.IsLike(pixel2.Color)) {;}

This also meant that I had to modify the pattern code to take into account that the source pixels are now slightly different under certain conditions.

So I took code like:

if(pixel1.Color==pixel2.Color) outputPixel1.Color=pixel1.Color

and changed it into

if(pixel1.Color==pixel2.Color) outputPixel1.Color=Interpolate(pixel1.Color,pixel2.Color)

This way the images got a much better filtering.

I did this years ago because I wanted to use these popular filtering techniques on images which are not that color perfect because of lossy compression and other degrading image manipulation. It was also interesting to see what happens when you apply these algo's to stuff that they werem't designed for: drawings, paintings and photos.

[nitenurse79]

Been playing with this on some old camphone images, I am very happy with the results, this and "remove noise" has cleaned up some pics I thought had gone forever

:star: :star:

[Hyllian]

Hi,

Is there any possibility to include my algorithm?

Here an explanation: http://board.byuu.org/viewtopic.php?f=10&t=2248

Some standalone app here: http://www.wayofthepixel.net/pixelation/index.php?topic=13508.0

and here: http://www.pixeljoint.com/forum/forum_posts.asp?TID=13667

Sources xBR in Paintwon game: http://paintown.svn.sourceforge.net/viewvc/paintown/trunk/src/util/sdl/xbr.cpp?revision=6823&view=markup

Original:

4xBR:

[Hawkynt]

yeah, i'd really enjoy including it. on the weekend i'll hopefully have enough time to study your rules. it's great to see that someone else is trying to improve image scaling algo's.

ok read it once now, having some questions:

- is it right that your filter does direct equal comparisons to pixel colors ?

- are there four variants of your algo: 2x, 3x, 4x and 5x (if so, is there code for 5x ?)

- I could not fully follow your specs, could you be more descriptive and send code with no preproc macros pls

- could you use the source image i provided here and scale it, so i could compare any of my implementations later

Thanks a lot in advance.

Edited January 26, 2012 by Hawkynt

[AhmedElyamani]

Cool Plugin

[Hyllian]

ok read it once now, having some questions:

- is it right that your filter does direct equal comparisons to pixel colors ?

Ok, Let's go:

I didn't understand what you mean. What are direct equal comparisons?

- are there four variants of your algo: 2x, 3x, 4x and 5x (if so, is there code for 5x ?)

Yes. But in C I only have 2x, 3x and 4x. The 5x is a shader implementation I made to run on emulators which run Cg files.

- I could not fully follow your specs, could you be more descriptive and send code with no preproc macros pls

Those macros are there to save me time in typing code. I don't have code without those macros. Does your code support macros?

- could you use the source image i provided here and scale it, so i could compare any of my implementations later

Thanks a lot in advance.

Yes, here they are:

2X

3X

4X

6X

9X

12X

You must be asking how I did those three last images (6x, 9x and 12x). As the 3X implementation doesn't blend pixel colors, it's possible to use the scaling recursively. I've used 2x over the 3x output to make a 6x. And so on.

[Hawkynt]

So does this mean that:

- your native scaler resolutions are 2x,3x,4x and 5x (because the others are only repeated scaling several times)

- you can not rewrite your C code to use functions declared as "inline", "_inline" or "__inline" to do the same (typing would be nearly equal )

- your 5x is only in cg

- the word group you asked for means comparing pixel colors for "equality" not "similarity", AFAIK your code does this, so I would rewrite it to support thresholds in comparisons because this would make your filters usable on aliased images

Are my assumptions correct ?

[Hyllian]

So does this mean that:

- your native scaler resolutions are 2x,3x,4x and 5x (because the others are only repeated scaling several times)

Yes.

- you can not rewrite your C code to use functions declared as "inline", "_inline" or "__inline" to do the same (typing would be nearly equal )

I'm used to macros. If you want, you can modify the code to use inline functions instead macros. I just don't have the (right now) to make this, because my first child was born last monday and I'm very very busy these days.

- your 5x is only in cg

Yes. It runs in some emulators. I don't have a C version of 5x.

- the word group you asked for means comparing pixel colors for "equality" not "similarity", AFAIK your code does this, so I would rewrite it to support thresholds in comparisons because this would make your filters usable on aliased images

Are my assumptions correct ?

My code supports anti-aliasing. See one of the arrows in the original image below and compare with that 4xBR filtered:

Original:

4xBR:

[Hyllian]

Please, get that standalone app I posted earlier and do some tests with your images to see how the filter looks.

[Corrupt Voyer]

hey this is exactly what ive been looking for, for quite awhile

is there a reason why all my Alpha channels go black? because that kinda prevents me using this plugin :S

[Hawkynt]

These filters were not invented to be used on alpha images. I even don't know how I should expand their algo's for a fourth channel. That's the reason why your alpha gets blacked out. Sorry

[null54]

These filters were not invented to be used on alpha images. I even don't know how I should expand their algo's for a fourth channel. That's the reason why your alpha gets blacked out. Sorry

You would not have to filter the alpha channel it can simply be copied from the source image.

For example:

Surface src = srcArgs.Surface;
Surface dst = dstArgs.Surface

for (int y = rect.Top; y < rect.Bottom; y++)
{
for (int x = rect.Left; x < rect.Right; x++)
{
	dst[x, y].A = src[x, y].A;
}
}

[Corrupt Voyer]

These filters were not invented to be used on alpha images. I even don't know how I should expand their algo's for a fourth channel. That's the reason why your alpha gets blacked out. Sorry

well i ended up using this (a commandline tool) with a batch script http://scale2x.sourceforge.net

and it was okay but you have some damn good filters that i wished would support alphas

Edited February 5, 2012 by Corrupt Voyer

[Hawkynt]

OK here you go. I figured out a way how the filters could implement the alpha channel without simply copying it from the source. So now there is support for alpha images, too. With releases starting from r20 on, the alpha channel will no longer being blacked out and alpha values will also be interpolated whenever a filter needs to interpolate points.

I think this way the alpha gets seamlessly integrated into existing filters.

Don't forget to remove the old plugin dll, because it has a different name.

Edited February 5, 2012 by Hawkynt

[Corrupt Voyer]

oh wow! such a fast patch

i will test it very soon (so many windows running atm XD)

[delpart]

Thanks for the work on this one mate. I've not had the need for it yet but the side link reads etc were very educational.