Sign in to follow this  
Simon Brown

NormalMapPlus v1.0

Recommended Posts

@MJW Normal maps are computed from a 3D mesh. This is an approximation method using edge detection. The purple shade is the neutral color for a normal map with RGB values 128, 128, 255 since the RGB channels correspond to XYZ offsets for the computed lighting. Sobel edge detection and blurring is a typical solution to faking a normal map.
https://en.wikipedia.org/wiki/Normal_mapping

  • Upvote 1

Share this post


Link to post
Share on other sites

Thanks, though I actually know what a normal map is. What I'd like to know from someone who uses the effect:

 

1) What's used as the effect input image? Is it it a grayscale image where brightness represents height, or do the RGB values actually represent 3D XYZ values? I can't see the sense in the second possibility, but I may misunderstand something.

 

 2) Why is there a lighting calculation? Normals are eventually used to do lighting, but I can't see the sense in "lighting" the image that's supposed to be the generated normals. I guess my question is: when computing a normal map, how is the lighting control used?

 

In looking at the code, I see some things that don't strike me as quite kosher, which doesn't give me a warm, fuzzy feeling that everything the code does makes sense.

 

Share this post


Link to post
Share on other sites

 

Hello AndrewDavid!

 

Glad null54 terrific update worked for you too! :)

 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 

Hello MJW!

 

"What this plugin does appears to be reasonably simple, but I'm not certain exactly what it does."

 

The advantage, for me, that NormalMapPlus has over other normal map plugins for Paint.net is that it's a very simple tool to create normals from the diffuse texture in one step. Simply using the plugin on a diffuse texture can produce a basic normal.

 

Though in practice you tend to add more detail by hand.

 

"What's used as the effect input image? Is it it a grayscale image where brightness represents height"

 

Darker coulours represent depth & lighter colours represent height. Black being deepest & white the highest.

 

"My question is: in actual use, what is the source image, and what is the desired output? "

 

I can explain my basic process of using NormalMapPlus.

 

Firstly I'd run NormalMapPlus on the main texture, also called the diffuse texture, as the basis for the normal map. Black & white or colour seems to make no difference.

 

46324093_1.jpg

 

Ending up with a basic normal map like:

 

46324121_2.jpg

 

Then I'd normally add more detail by making a new layer above the diffuse & hand drawing with white the highlights that are raised & with black for bits that are deeper, like:

 

46324132_3.jpg 46324141_4.jpg

 

Then I run NormalMapPlus on that & add it as a new Overlay layer to the first normal image created.

 

46324163_5.jpg

 

Multiple layers can be added to increase the height & depth as required.

 

I used to adjust the X,Y & Z sliders when using NormalMapPlus to increase the intensity of the heights created but I moved to adding extra overlay layers instead as a more controllable method. So I now normally just use the default slider settings.

 

The only thing needed after this normal is made in Paint.net is to add a detailed Alpha layer, since they cannot be added by Paint.net "in house".

The detailed aplha layer is added to normal maps for games like Oblivion, Fallout 3, New Vegas & Skyrim & controls specularity in game (shininess).

It's basically a B&W version of the main texture with black areas being dullest & white areas being shiniest & grey levels in between.

 

I use the excellent tiny program DXTBmp to combine the finished normal & a Bitmap B&W for its Alpha layer.

 

While it would be handy to be able to add the Alpha layer within Paint.net, using DXTBmp to do it is only a matter of seconds.

 

Resulting in game effect seen here:

 

46325311_drum-copy.jpg

 

The squarish bumps on the face of the drum appear raised, as they should, due to the normal map but in reality the mesh is completely flat there.

Saves on poly count of the model & is actually a better effect.

 

It should be noted that @null54 update seems to have fixed the issue with running with Net Framework 4.7.

 

Hope this gives you some insight!

 

 

Edited by Prensa
  • Upvote 2

Share this post


Link to post
Share on other sites

Thank you, Prensa. That was a very helpful description. I'll have think about what would be the best solution, though I have some ideas.


 

  • Upvote 1

Share this post


Link to post
Share on other sites

So there would be a working version, I pretty much directly ported NormalMapPlus to CodeLab: NormalMapPlus.zip

 

In order to understand the code better, I renamed some of the variables. I built the CodeLab version using the updated source. I may regret that if I made any mistakes during the renaming process. Beyond changing some variable names, the code is essentially unchanged. Let me know about any problems.

 

The source code:

Spoiler

// Name: NormalMapPlus
// Submenu:
// Author: harold, Simon Brown, MJW
// Title: NormalMapPlus
// Version: 1.2.*
// Desc: CodeLab port of NormalMapPlus
// Keywords: normal map
// URL: https://forums.getpaint.net/index.php?/topic/17010-normalmapplus-v10/
// Help:
#region UICode
DoubleSliderControl Amount1 = 0.3; // [0,1] X
DoubleSliderControl Amount2 = 0.5; // [0,1] Y
DoubleSliderControl Amount3 = 0.11; // [0,1] Z
#endregion

void Render(Surface dst, Surface src, Rectangle rect)
{
    // Delete any of these lines you don't need
    Rectangle selection = EnvironmentParameters.GetSelection(src.Bounds).GetBoundsInt();
    int top = rect.Top, bottom = rect.Bottom, left = rect.Left, right = rect.Right;
    float lightX = (float)Amount1;
    float lightY = (float)Amount2;
    float lightZ = (float)Amount3;
    Float4 lightDir = new Float4(lightX, lightY, lightZ, 0.0f);

    if (IsCancelRequested) return;
    for (int y = top; y < bottom; y++)
    {
        Float4 UL, UM, UR;
        Float4 ML, MM, MR;
        Float4 LL, LM, LR;
        if (y == 0)
        {
            UL = (Float4)src[0, y]; // (This, and similar statements are wrong. Should be something like src[left, y].)
            UM = UL;
            UR = (Float4)src[1, y];
            ML = UL;
            MM = ML;
            MR = UR;
            LL = (Float4)src[0, y + 1];
            LM = LL;
            LR = (Float4)src[1, y + 1];
            for (int x = left; x < right; x++)
            {
                Float4 dx = (UL - UR) + 2.0f * (ML - MR) + (LL - LR);
                Float4 dy = (UL - LL) + 2.0f * (UM - LM) + (UR - LR);
                float u = Float4.Dot(dx, lightDir);
                float v = Float4.Dot(dy, lightDir);
                Float4 normal = new Float4(u, v, 1.0f, 0.0f);
                normal.Normalize();
                normal = 0.5f * normal + 0.5f;
                normal.W = 1.0f;
                dst[x, y] = (ColorBgra)normal;
                if (x == (right - 1))
                {
                    break;
                }
                // (The normals are shifted a pixel right from where they should be.)
                UL = UM;
                UM = UR;
                UR = (Float4)src[x + 1, y];
                ML = MM;
                MM = MR;
                MR = UR;
                LL = LM;
                LM = LR;
                LR = (Float4)src[x + 1, y + 1];
            }
        }
        else if (y == (bottom - 1))
        {
            UL = (Float4)src[0, y - 1];
            UM = UL;
            UR = (Float4)src[1, y - 1];
            ML = (Float4)src[0, y];
            MM = ML;
            MR = (Float4)src[1, y];
            LL = ML;
            LM = LL;
            LR = MR;
            for (int x = left; x < right; x++)
            {
                Float4 dx = (UL - UR) + 2.0f * (ML - MR) + (LL - LR);
                Float4 dy = (UL - LL) + 2.0f * (UM - LM) + (UR - LR);
                float u = Float4.Dot(dx, lightDir);
                float v = Float4.Dot(dy, lightDir);
                Float4 normal = new Float4(u, v, 1.0f, 0.0f);
                normal.Normalize();
                normal = 0.5f * normal + 0.5f;
                normal.W = 1.0f;
                dst[x, y] = (ColorBgra)normal;
                if (x == (right - 1))
                {
                    break;
                }
                UL = UM;
                UM = UR;
                UR = (Float4)src[x + 1, y - 1];
                ML = MM;
                MM = MR;
                MR = (Float4)src[x + 1, y];
                LL = LM;
                LM = LR;
                LR = MR;
            }
        }
        else
        {
            UL = (Float4)src[0, y - 1];
            UM = UL;
            UR = (Float4)src[1, y - 1];
            ML = (Float4)src[0, y];
            MM = ML;
            MR = (Float4)src[1, y];
            LL = (Float4)src[0, y + 1];
            LM = LL;
            LR = (Float4)src[1, y + 1];
            for (int x = left; x < right; x++)
            {
                Float4 dx = (UL - UR) + 2.0f * (ML - MR) + (LL - LR);
                Float4 dy = (UL - LL) + 2.0f * (UM - LM) + (UR - LR);

                float u = Float4.Dot(dx, lightDir);
                float v = Float4.Dot(dy, lightDir);
                Float4 normal = new Float4(u, v, 1.0f, 0.0f);
                normal.Normalize();
                normal = 0.5f * normal + 0.5f;
                normal.W = 1.0f;
                dst[x, y] = (ColorBgra)normal;
                if (x == (right - 1))
                {
                    break;
                }
                UL = UM;
                UM = UR;
                UR = (Float4)src[x + 1, y - 1];
                ML = MM;
                MM = MR;
                MR = (Float4)src[x + 1, y];
                LL = LM;
                LM = LR;
                LR = (Float4)src[x + 1, y + 1];
            }
        }
    }
}

public struct Float4
{
    public float A;
    public float R;
    public float G;
    public float B;
    public float X
    {
        get
        {
            return this.R;
        }
        set
        {
            this.R = value;
        }
    }
    public float Y
    {
        get
        {
            return this.G;
        }
        set
        {
            this.G = value;
        }
    }
    public float Z
    {
        get
        {
            return this.B;
        }
        set
        {
            this.B = value;
        }
    }
    public float W
    {
        get
        {
            return this.A;
        }
        set
        {
            this.A = value;
        }
    }
    public static explicit operator ColorBgra(Float4 c)
    {
        return ColorBgra.FromBgra((byte)(255f * c.B), (byte)(255f * c.G), (byte)(255f * c.R), (byte)(255f * c.A));
    }

    public static explicit operator Float4(ColorBgra c)
    {
        return FromBgra(((float)c.B) / 255f, ((float)c.G) / 255f, ((float)c.R) / 255f, ((float)c.A) / 255f);
    }

    public static Float4 operator +(Float4 c, float s)
    {
        c.A += s;
        c.B += s;
        c.G += s;
        c.R += s;
        return c;
    }

    public static Float4 operator *(Float4 c, float s)
    {
        c.A *= s;
        c.R *= s;
        c.G *= s;
        c.B *= s;
        return c;
    }

    public static Float4 operator *(float s, Float4 c)
    {
        c.A *= s;
        c.R *= s;
        c.G *= s;
        c.B *= s;
        return c;
    }

    public static Float4 operator +(Float4 c0, Float4 c1)
    {
        c0.A += c1.A;
        c0.R += c1.R;
        c0.G += c1.G;
        c0.B += c1.B;
        return c0;
    }

    public static float Dot(Float4 c0, Float4 c1)
    {
        return ((((c0.A * c1.A) + (c0.B * c1.B)) + (c0.G * c1.G)) + (c0.R * c1.R));
    }

    public static Float4 operator -(Float4 c0, Float4 c1)
    {
        c0.A -= c1.A;
        c0.R -= c1.R;
        c0.G -= c1.G;
        c0.B -= c1.B;
        return c0;
    }

    public void Normalize()
    {
        float num = 1f / ((float)Math.Sqrt((double)((((this.A * this.A) + (this.R * this.R)) + (this.G * this.G)) + (this.B * this.B))));
        this = (Float4)(this * num);
    }

    public Float4(float x, float y, float z, float w)
    {
        this.R = x;
        this.G = y;
        this.B = z;
        this.A = w;
    }

    public static Float4 FromBgra(float b, float g, float r, float a)
    {
        Float4 num = new Float4();
        num.A = a;
        num.B = b;
        num.G = g;
        num.R = r;
        return num;
    }
}

 

 

  • Upvote 1

Share this post


Link to post
Share on other sites

Hey @MJW

 

Attempting to build the source code in codelab returns this error:

 

Quote

Unhandled Exception at line -9: 
System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at PaintDotNet.Effects.UserScript.Render(Surface dst, Surface src, Rectangle rect)
   at PaintDotNet.Effects.UserScript.Render(EffectConfigToken parameters, RenderArgs dstArgs, RenderArgs srcArgs, Rectangle[] rois, Int32 startIndex, Int32 length) in c:\Users\Andrew\AppData\Local\Temp\cvw1uui1.0.cs:line 47
   at PaintDotNet.Effects.CodeLab.Render(EffectConfigToken parameters, RenderArgs dstArgs, RenderArgs srcArgs, Rectangle[] rois, Int32 startIndex, Int32 length)

 This is a problem B)

Share this post


Link to post
Share on other sites

Build the code or run the code? I already built it, so building it shouldn't be necessary.

Share this post


Link to post
Share on other sites

Every chance I get to build the DLL myself allows me to keep them in my own sub folder in paint.

Not to mention the layman's attempt at trying to understand the coding.

Why post the source code then? The version @null54 works fine.

Share this post


Link to post
Share on other sites

I mostly post the source for those who want to see how it works, or want to use the algorithms for something else, but you're certainly welcome to build it yourself. It's a little difficult, though, to look at the error printout and figure out what caused it, since it seems build correctly for me. You'll have to provide a few more details on what you did, and where in the process of building you got the error. Also, what image and selection were active when you were in CodeLab? Perhaps that had something to do with it. Can you successfully run the pre-built version on the same image and selection?

Share this post


Link to post
Share on other sites
28 minutes ago, AndrewDavid said:

The version @null54 works fine.

 

I should have read the preceding posts more carefully, since I missed that there was a already working version. That does, however, solve your problem: use null54's version.

 

In any case, the time I spent looking at the plugin may not be wasted, since I may be able to improve it.

  • Upvote 1

Share this post


Link to post
Share on other sites

Your version generates the same errors I posted in this thread on page 1 after compiling and running it in paint..

The image I used was the gun magazine @Prensa provided earlier.

I thought you may have improved it. But yes - case closed.

Share this post


Link to post
Share on other sites
3 hours ago, MJW said:

In any case, the time I spent looking at the plugin may not be wasted, since I may be able to improve it.

 

The crash in the codelab script is caused by the same issue as the original plugin.

For some reason the Render method causes certain versions of the .NET JIT optimizer to generate code that produces an access violation.

Adding the following attribute to the Render method works around that problem.

[System.Runtime.CompilerServices.MethodImpl(System.Runtime.CompilerServices.MethodImplOptions.NoOptimization)]

As to improving the plugin the original thread states that it is based on the normal map GPU shader example from AMD's RenderMonkey toolkit (NormalMap Filter.rfx).

The following is the algorithm description from that file.

 

Quote

The Sobel filter extracts the first order derivates of the image,
that is, the slope. The slope in X and Y directon allows us to
given a heightmap evaluate the normal for each pixel. This is
the same this as ATI's NormalMapGenerator application does,
except this is in hardware.

These are the filter kernels:

  SobelX       SobelY
  1  0 -1      1  2  1
  2  0 -2      0  0  0
  1  0 -1     -1 -2 -1

  • Upvote 2

Share this post


Link to post
Share on other sites

Thanks for your efforts MJW, (though I will probably never use this plugin). I didn't even know it was possible to use structs in codelab!:roll::D

Share this post


Link to post
Share on other sites

The problem doesn't occur on my system, so I can't debug it. One thing I'd try if I could debug it is removing the line:

 

[StructLayout(LayoutKind.Sequential)]

 

from the Float4 definition.

 

The line should be harmless, and I believe it's the default, but it's also not necessary. The only reason I suspect it is because it's one of the few usual things I see.

Share this post


Link to post
Share on other sites
8 hours ago, null54 said:

As to improving the plugin the original thread states that it is based on the normal map GPU shader example from AMD's RenderMonkey toolkit (NormalMap Filter.rfx).

 

Null54, I'm not clear on what improvement is being suggested. That is how the plugin computes the derivative, and I think it's a pretty good method. Well, actually, the plugin stores the results one pixel to the right of where it should, since it saves the nine neighbor pixels in variables at the end of the loops, then increments x. That should be fixed.

Share this post


Link to post
Share on other sites
56 minutes ago, MJW said:

I'm not clear on what improvement is being suggested.

 

I was not suggesting any improvements.

Because the code in GitHub was based on Reflector output I thought that you may find the code the plugin was based on

helpful in reconstructing the original variable names.

Share this post


Link to post
Share on other sites
39 minutes ago, null54 said:

 

I was not suggesting any improvements.

Because the code in GitHub was based on Reflector output I thought that you may find the code the plugin was based on

helpful in reconstructing the original variable names.

 

Oh, I see. Thank you. I'll probably take a look at the code.

 

The fact that it's based on GPU code probably explains the rather odd way it treats each of the color components the same way, rather than just computing in some fashion a singe height from each color, then using those to calculate the derivatives.

Share this post


Link to post
Share on other sites
On 24/12/2012 at 11:25 AM, Vince said:

I'm surprised to find artifacts in the normal maps created by this plugin.  As no one has reported this before, I assumed it was something unique to my system or I was doing something wrong.  I tried it on a different system, but got similar results.

 

I realise Vince's post on page one, about artifacts in normal maps made by NormalMapPlus, is old but I thought it worth commenting on in case it helps others.

 

I've seen the odd line like artifacts described when using NormalMapPlus sometimes & found that it was due to being zoomed in to a part of the image that the effect is being applied to. If you zoom into a corner for example & run NormalMapPlus you can end up with a normal height line were the edge of the zoomed in portion meets the edge of Paint.net's display.

 

Simple solution is to ensure the image is set to "Zoom to Window", either in the View pull-down or the little screen icon at the bottom right of Paint.net.

That avoids those artifacts all together.

 

Hope this helps!

Edited by Prensa

Share this post


Link to post
Share on other sites
4 hours ago, Prensa said:

 

I realise Vince's post on page one, about artifacts in normal maps made by NormalMapPlus, is old but I thought it worth commenting on in case it helps others.

 

Unfortunately, thanks to Photobucket, I can't see the artifacts being discussed. If you could describe them, or better yet post a picture, it would helpful. In theory, the zoom level should have no effect. The plugin has no idea what the zoom level is, and always gets and sends the same data no matter what it is.

Share this post


Link to post
Share on other sites
5 hours ago, MJW said:

 

Unfortunately, thanks to Photobucket, I can't see the artifacts being discussed. If you could describe them, or better yet post a picture, it would helpful. In theory, the zoom level should have no effect. The plugin has no idea what the zoom level is, and always gets and sends the same data no matter what it is.

 

Hello MJW!

 

Sure, no problem!

 

As I say in the earlier post, you can avoid these artifacts simply by not being zoomed in when applying the NormalMapPlus effect.

You just need to be aware of the issue first. :)

 

Here's a visual demonstration of the described artifacts & how to trigger them:

 

46551336_artifact-example-series-1-copy.

 

Image, not zoomed into fitting the Paint.net window.

 

46551365_artifact-example-series-2-copy.

 

Apply NormalMapPlus & result is how it should be.

 

46551403_artifact-example-series-3-copy.

 

Start again, this time zoomed into a portion of the image

 

46551434_artifact-example-series-4-copy.

 

Apply NormalMapPlus while zoomed in.

 

46551466_artifact-example-series-5-copy.

 

Zoom out & you will see the artifacts (circled in red by me), two lines that match up with where the edges of the Paint.net window was when NormalMapPlus was applied.

 

You can also get artifacts on top & bottom of the zoomed in frame.

 

It's odd because, as you say, it is not normal behaviour for other tools that I've noticed.

 

Still it's easily avoided once you're aware of it, don't be zoomed in when you apply NormalMapPlus. :)

 

Hope this helps!

Edited by Prensa
  • Upvote 1

Share this post


Link to post
Share on other sites
10 hours ago, Prensa said:

Hope this helps!

 

It definitely helps, though I can't off hand explain how that could happen. I wonder if it's a less extreme manifestation of the problem that causes the crash. Maybe I'll be able to reproduce it on my system so I can investigate its cause more easily.

 

EDIT: I had a thought (and this will only be meaningful to plugin writers). Perhaps for some reason PDN passes different ROIs depending on the zoom level. I don't know why it would, but it would explain this problem. I don't think it would expalin the crash, but I'll have to give that more thought.

 

EDT 2: My assumption is now that PDN passes the ROIs for the visible region separately, perhaps either before or after the ROIs for the invisible region. That would explain the line-artifact problem. I don't have time to fix it now, but I'll try to make a new CodeLab version tonight. The code could then be easily converted to the non-CodeLab version.

  • Upvote 1

Share this post


Link to post
Share on other sites

Hello MJW!

 

"I wonder if it's a less extreme manifestation of the problem that causes the crash."

 

I would guess not as it's a glitch that's been in NormalMapPlus ever since I first started using it, long before the recent clash with Net Framework 4.7.

But I'm no coder. :)

 

I've now used the plugin many times since null54's fix & the crash is totally gone.

 

An interesting thing I did notice when recreating the artifact bug for those demo images, although I can reproduce it on any picture zoomed in on when activating NormalMapPlus, the artifacts won't form on a blank image.

 

Run Paint.net with no loaded image, zoom in to the empty canvas & run NormalMapPlus & no artifacts form.

 

Only forms when it's applied to an actual image.

Most peculiar.

 

Thought I'd mention it, in case it helps though it probably deepens the mystery. :)

 

"I don't have time to fix it now, but I'll try to make a new CodeLab version tonight. The code could then be easily converted to the non-CodeLab version."

 

No worries, there's an effective workaround that's painless, go out of zoom, so it's not an issue to those aware of it.

 

I do appreciate skilled coders taking interest in this plugin though, it's an essential tool to many of us who make textures for game mods like Fallout 3.

 

Prensa

 

Share this post


Link to post
Share on other sites

I know exactly why the artifacts occur, and why they don't occur with a blank image. It's just a matter of fixing the problem. I knew the problem existed in the code, and added a comment about it:

UL = (Float4)src[0, y]; // (This, and similar statements are wrong. Should be something like src[left, y].)

Because I didn't know the visible ROIs are passed separately, I thought it would not matter as long as there wasn't a selection, which is almost always the case for this type of plugin. In line with my "direct port" approach, I didn't fix it.

 

  • Upvote 1

Share this post


Link to post
Share on other sites

I finally found time to complete my changes to the CodeLab implementation of Normal Map Plus. The goal was to fix the edge-line artifacts when the zoom level doesn't show the entire image, but I'm also interested in whether by some chance it fixes the crash. My previous version now crashes on my system, though I made no changes to it since when it worked. Presumably that's due to a Windows update on my computer. So far, my new version doesn't crash for me. Perhaps some change I made happens to avoid the problem (or perhaps not).

 

I made quite a few changes, and I haven't had time to do a thorough test, so nothing is guaranteed.

 

For the DLL and the source code, please see the thread :  NormalMapPlus (CodeLab Implementation)

Edited by MJW
Removed DLL and source code, which have been moved to a separate thread.
  • Like 1
  • Upvote 3

Share this post


Link to post
Share on other sites
Guest
This topic is now closed to further replies.
Sign in to follow this