Jump to content


Photo

2D Image Scaling Algorithms


56 replies to this topic

#1 Hawkynt

Hawkynt
  • Members
  • 25 posts
  • Reputation:5

Posted 28 December 2011 - 10:29 AM

I finally packed my image filtering library into a Paint.NET Plugin which can be downloaded here : 2dImageFilter.

This library contains all of the standard algorithms used in various emulators (like VirtualGB, Mame, DOSBox, SNES9x, ZSNES to name some of them).
I adapted them to be used not only on pixel graphics by giving them the ability to identify pixels that are alike and not exactly the same color. They will interpolate on matches to give slightly better results than their original. Even though some scaler's seem to result in the same image, they're all slightly different and the best fit heavily depends on the source material. All Algorithms have been rewritten from scratch by looking at the original code, understanding and then adapting so these should be clean (schoolar) version of the algo's.
Thank's to all the people that made up their minds and invented these scalers. Without them I couldn't have coded this.


To demonstrate some of the filtering techniques, I will use this image:
Posted Image

Scale2x:
Posted Image

AdvInterp2x:
Posted Image

Scale3x:
Posted Image

AdvInterp3x:
Posted Image

Bilinear+(Original):
Posted Image

Bilinear+:
Posted Image

Eagle2x:
Posted Image

Eagle3x (I invented this one because it did not exists before):
Posted Image

Eagle3x(Variant B ):
Posted Image

Super Eagle:
Posted Image

Scale and Interpolate (alias SaI2x):
Posted Image

Super Scale and Interpolate (alias SuperSai):
Posted Image

EPXB:
Posted Image

EPXC:
Posted Image

EPX3:
Posted Image

HQ2x(Normal, Bold and Smart, HQ4x, HQ2x3, HQ2x4, etc.):
Posted ImagePosted ImagePosted Image

HQ3x(which is also available as normal, bold and smart version):
Posted Image

LQ2x(also LQ3x, LQ4x, Normal, Smart, Bold, LQ2x3, LQ2x4, etc.):
Posted Image


Also included in this package: Scanline Effects, RGB Effects, TV Effects, RGB- and YUV- Channel extraction and many more.

I recommend when using scaling algorithms to resize the image before applying them because I don't know how to do this from within a plugin or use "Render to Clipboard" and paste the result into a new image.

Hope you'll give me feedback about it.

Edited by Hawkynt, 24 February 2012 - 06:30 PM.

  • 1

#2 nitenurse79

nitenurse79
  • Members
  • 1,486 posts
  • LocationSweden
  • Reputation:284

Posted 28 December 2011 - 02:16 PM

Looks interesting so far, maybe a screenshot and some more info might be of use :)
  • 0

sig114-3D_zps0a21cdd5.png

Sig battles won - 18. Image battles won - 3.  2012 Signature of the year award winner.


#3 Ego Eram Reputo

Ego Eram Reputo

    2013 Proton Award Winner

  • Moderators
  • 7,849 posts
  • LocationNorth Canterbury, New Zealand
  • Reputation:619

Posted 29 December 2011 - 06:10 AM

A whole lot more info please!
  • 0

#4 nitenurse79

nitenurse79
  • Members
  • 1,486 posts
  • LocationSweden
  • Reputation:284

Posted 29 December 2011 - 06:58 PM

When I run this plugin on an image it crops to a small portion only and not keeping the whole image as your examples show :/
  • 0

sig114-3D_zps0a21cdd5.png

Sig battles won - 18. Image battles won - 3.  2012 Signature of the year award winner.


#5 Hawkynt

Hawkynt
  • Members
  • 25 posts
  • Reputation:5

Posted 29 December 2011 - 07:10 PM

Jep, as I told: It does not resize the image but the filters are meant to work this way.

So the following happens:
- the filter takes your whole image and magnifies it by eg factor 2 in x and y (like Eagle does or Hq2x)
- now it tries to write back the information to paint
- paint is still having an image with the original size, so it only writes the upper left part of the resized version back

This leads to your problem.

There are two workarounds:
Either resize the image first and let your source image be in the upper left corner. (This is slower because white space is also filtered and you have to know the exact dimensions of the filtered image beforehand)
Or simply press "Render to Clipboard" instead of the "OK" button in the filter configuration and then make a new image with the size from the clipboard and simply paste. (CTRL+N, ENTER, CTRL+V should do the trick :D )

If someones knows how to programmatically tell paint that the image sizes was changed by the filter, tell me how and I will fix that. ^^
  • 0

#6 nitenurse79

nitenurse79
  • Members
  • 1,486 posts
  • LocationSweden
  • Reputation:284

Posted 29 December 2011 - 07:18 PM

There are two workarounds:
Either resize the image first and let your source image be in the upper left corner. (This is slower because white space is also filtered and you have to know the exact dimensions of the filtered image beforehand)
Or simply press "Render to Clipboard" instead of the "OK" button in the filter configuration and then make a new image with the size from the clipboard and simply paste. (CTRL+N, ENTER, CTRL+V should do the trick :D )



Do I select only part of my resized image to make my source be in the upper left corner (I'm confused now) :/
  • 0

sig114-3D_zps0a21cdd5.png

Sig battles won - 18. Image battles won - 3.  2012 Signature of the year award winner.


#7 Hawkynt

Hawkynt
  • Members
  • 25 posts
  • Reputation:5

Posted 29 December 2011 - 07:27 PM

OK try this:
-open an image you want to resize
-go to effect->Hawkynt->2dimagefilter
-choose for example hq4x from the dropdown
-press "render to clipboard"
-click on "cancel" or press "ESC"
-click File->New or press "CTRL+N"
-click OK or press "Enter"
-click Edit->Paste or press "CTRL+V"

Your resized image should now be there in full resolution.

BTW: I'd really like to make this more intuitive but according to this post it seems to be impossible.

Edited by Hawkynt, 29 December 2011 - 07:32 PM.

  • 1

#8 nitenurse79

nitenurse79
  • Members
  • 1,486 posts
  • LocationSweden
  • Reputation:284

Posted 29 December 2011 - 07:30 PM

OK try this:
-open an image you want to resize
-go to effect->Hawkynt->2dimagefilter
-choose for example hq4x from the dropdown
-press "render to clipboard"
-click on "cancel" or press "ESC"
-click File->New or press "CTRL+N"
-click OK or press "Enter"
-click Edit->Paste or press "CTRL+V"

Your resized image should now be there in full resolution.


Thank you Hawkynt :) works like a charm now :D
  • 0

sig114-3D_zps0a21cdd5.png

Sig battles won - 18. Image battles won - 3.  2012 Signature of the year award winner.


#9 Ego Eram Reputo

Ego Eram Reputo

    2013 Proton Award Winner

  • Moderators
  • 7,849 posts
  • LocationNorth Canterbury, New Zealand
  • Reputation:619

Posted 30 December 2011 - 02:19 AM

Ok, now I get it. Looks similar to : http://forums.getpai...n-beta-release/

Apart from the obvious number of scaling options, how is this better/worse/different from that plugin? (I'm not trying to be difficult, just trying to get you to explain the points of difference between the two).
  • 0

#10 Hawkynt

Hawkynt
  • Members
  • 25 posts
  • Reputation:5

Posted 30 December 2011 - 09:00 AM

As you mentioned, I already looked at the plugin you described and I also had a post there, encouraging the author to use my library. But sadly he did not answer, so I wrote my own one. You already noted that there are many more filtering algo's in my plugin than in his own and there are also differencies in how they exactly work. Pikalek (that's the author of the other plugin) tried to get the implementation as close to the original as possible while I tried to rewrite them and adapt them to a "far-near color model" similar to the one that's used in Hq filters.

To get more in detail:
The original scalers from Kreed and Co. compare pixel by color, and only apply their patterns if the pixels have the exact same colors: Something like
if(pixel1.Color==pixel2.Color) {;}
I changed that code everywhere and compared using YUV color space with thresholds to identify similar color values:
if(pixel1.Color.IsLike(pixel2.Color)) {;}
This also meant that I had to modify the pattern code to take into account that the source pixels are now slightly different under certain conditions.
So I took code like:
if(pixel1.Color==pixel2.Color) outputPixel1.Color=pixel1.Color
and changed it into
if(pixel1.Color==pixel2.Color) outputPixel1.Color=Interpolate(pixel1.Color,pixel2.Color)
This way the images got a much better filtering.

I did this years ago because I wanted to use these popular filtering techniques on images which are not that color perfect because of lossy compression and other degrading image manipulation. It was also interesting to see what happens when you apply these algo's to stuff that they werem't designed for: drawings, paintings and photos.
  • 2

#11 nitenurse79

nitenurse79
  • Members
  • 1,486 posts
  • LocationSweden
  • Reputation:284

Posted 30 December 2011 - 09:46 PM

Been playing with this on some old camphone images, I am very happy with the results, this and "remove noise" has cleaned up some pics I thought had gone forever :)



:star: :star: :star: :star: :star:
  • 0

sig114-3D_zps0a21cdd5.png

Sig battles won - 18. Image battles won - 3.  2012 Signature of the year award winner.


#12 Hyllian

Hyllian
  • Members
  • 20 posts
  • Reputation:0

Posted 11 January 2012 - 01:57 PM

Hi,

Is there any possibility to include my algorithm?

Here an explanation: http://board.byuu.or...php?f=10&t=2248

Some standalone app here: http://www.wayofthep...p?topic=13508.0
and here: http://www.pixeljoin...s.asp?TID=13667

Sources xBR in Paintwon game: http://paintown.svn....823&view=markup

Original:
Posted Image


4xBR:
Posted Image
  • 0

#13 Hawkynt

Hawkynt
  • Members
  • 25 posts
  • Reputation:5

Posted 26 January 2012 - 04:37 PM

yeah, i'd really enjoy including it. on the weekend i'll hopefully have enough time to study your rules. it's great to see that someone else is trying to improve image scaling algo's.

ok read it once now, having some questions:

- is it right that your filter does direct equal comparisons to pixel colors ?
- are there four variants of your algo: 2x, 3x, 4x and 5x (if so, is there code for 5x ?)
- I could not fully follow your specs, could you be more descriptive and send code with no preproc macros pls
- could you use the source image i provided here and scale it, so i could compare any of my implementations later

Thanks a lot in advance.

Edited by Hawkynt, 26 January 2012 - 04:51 PM.

  • 0

#14 AhmedElyamani

AhmedElyamani
  • Members
  • 205 posts
  • LocationEgypt
  • Reputation:8

Posted 26 January 2012 - 04:54 PM

Cool Plugin Posted Image
  • 0

#15 Hyllian

Hyllian
  • Members
  • 20 posts
  • Reputation:0

Posted 26 January 2012 - 09:05 PM

ok read it once now, having some questions:

- is it right that your filter does direct equal comparisons to pixel colors ?

Ok, Let's go:

I didn't understand what you mean. What are direct equal comparisons?


- are there four variants of your algo: 2x, 3x, 4x and 5x (if so, is there code for 5x ?)

Yes. But in C I only have 2x, 3x and 4x. The 5x is a shader implementation I made to run on emulators which run Cg files.

- I could not fully follow your specs, could you be more descriptive and send code with no preproc macros pls

Those macros are there to save me time in typing code. I don't have code without those macros. Does your code support macros?


- could you use the source image i provided here and scale it, so i could compare any of my implementations later

Thanks a lot in advance.


Yes, here they are:

2X
Posted Image

3X
Posted Image

4X
Posted Image

6X
Posted Image

9X
Posted Image

12X
Posted Image


You must be asking how I did those three last images (6x, 9x and 12x). As the 3X implementation doesn't blend pixel colors, it's possible to use the scaling recursively. I've used 2x over the 3x output to make a 6x. And so on.
  • 0

#16 Hawkynt

Hawkynt
  • Members
  • 25 posts
  • Reputation:5

Posted 27 January 2012 - 12:14 PM

So does this mean that:
- your native scaler resolutions are 2x,3x,4x and 5x (because the others are only repeated scaling several times)
- you can not rewrite your C code to use functions declared as "inline", "_inline" or "__inline" to do the same (typing would be nearly equal :D )
- your 5x is only in cg
- the word group you asked for means comparing pixel colors for "equality" not "similarity", AFAIK your code does this, so I would rewrite it to support thresholds in comparisons because this would make your filters usable on aliased images

Are my assumptions correct ?
  • 0

#17 Hyllian

Hyllian
  • Members
  • 20 posts
  • Reputation:0

Posted 27 January 2012 - 05:09 PM

So does this mean that:
- your native scaler resolutions are 2x,3x,4x and 5x (because the others are only repeated scaling several times)

Yes.

- you can not rewrite your C code to use functions declared as "inline", "_inline" or "__inline" to do the same (typing would be nearly equal :D )

I'm used to macros. If you want, you can modify the code to use inline functions instead macros. I just don't have the (right now) to make this, because my first child was born last monday and I'm very very busy these days. :P

- your 5x is only in cg

Yes. It runs in some emulators. I don't have a C version of 5x.

- the word group you asked for means comparing pixel colors for "equality" not "similarity", AFAIK your code does this, so I would rewrite it to support thresholds in comparisons because this would make your filters usable on aliased images

Are my assumptions correct ?

My code supports anti-aliasing. See one of the arrows in the original image below and compare with that 4xBR filtered:

Original:
Posted Image

4xBR:
Posted Image
  • 0

#18 Hyllian

Hyllian
  • Members
  • 20 posts
  • Reputation:0

Posted 27 January 2012 - 05:12 PM

Please, get that standalone app I posted earlier and do some tests with your images to see how the filter looks.
  • 0

#19 Corrupt Voyer

Corrupt Voyer
  • Members
  • 14 posts
  • LocationAustralia
  • Reputation:0

Posted 01 February 2012 - 04:48 AM

hey this is exactly what ive been looking for, for quite awhile :P

is there a reason why all my Alpha channels go black? because that kinda prevents me using this plugin :S
  • 0
Posted Image


#20 Hawkynt

Hawkynt
  • Members
  • 25 posts
  • Reputation:5

Posted 03 February 2012 - 06:02 PM

These filters were not invented to be used on alpha images. I even don't know how I should expand their algo's for a fourth channel. That's the reason why your alpha gets blacked out. Sorry
  • 0