A couple GPU image effect questions

[frio]

I decided to attempt to roll my own height map->normal map plugin, and since at its heart it just involves some convolutions combined in a specific manner I can get it done as a GPU image effect filter chain. Coming along pretty well, just need to get a good set of kernels to make it a decent replacement of the non-native plugins from NVIDIA and xNormal.

 

Mostly out of curiosity, it's not necessary for performance nor exactly matching the results of existing plugins: the plugin works in 32-bit float precision since it's dealing with vectors. Is there a way to *round* the final output from float 0-1 to uint8 0-255  instead of truncating? With pixel effects I learned to use ColorBbgra32.Round(ColorRgba128Float, MidPointRounding.AwayFromZero) to get the everyday rounding behavior. I'm not privy to the internals of the normal map generators I've used before but they seem to do rounding instead of truncation, my pixel values are often like 1 unit off even if I got the kernels and scaling exact. At least my normal map normalizer leaves the results untouched, i.e. they are properly computed and normalized as is.

 

Secondly, is there a more efficient way that lets me get the dot product of the pixels in the image? Currently I use a GammaTransferEffect on an unclamped input (let's call that A = r, g, b) and raise it to the second power (giving B = r^2, g^2, b^2), then a ColorMatrixEffect to sum everything with the matrix {1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1} (giving S = r^2+g^2+b^2 in all channels), doing a HLSL unary op to take a square root (giving M = sqrt(r^2+g^2+b^2) in all channels) and finally doing a HLSL division op A/M, (giving N = r/M, g/M, b/M, a normalized vector) - can't do a division blend since that clamps the result and it needs further processing to become a proper pixel. Not that I'm seeing any performance issue there or any real complications, but the gamma transfer and color matrix feel a little clunky compared to the HLSL ops allowing me to do the rest.

Edited November 16 by frio

[_koh_]

1 hour ago, frio said:

Is there a way to *round* the final output from float 0-1 to uint8 0-255  instead of truncating?

 

While they are not exactly truncated, D2D denormalization allows relatively large rounding error which is +-.6 values could be rounded to the final value. So .4 to .6 range can go ether way, and I found good amount of .5 to .55 values actually rounded down. So when I need exact values, I put this kind of shader at the end of the chain.

 

float4 Execute() => Hlsl.Floor(D2D.GetInput(0) * 255 + 0.5f) / 255;
 

1 hour ago, frio said:

but the gamma transfer and color matrix feel a little clunky compared to the HLSL ops allowing me to do the rest.

 

HlslBinaryFunctionEffect has Pow and Dot so you can replace them with those I guess. But if you are familiar with HLSL, just move to SDK and writing your own shader might be more convenient.

[frio]

1 hour ago, _koh_ said:

While they are not exactly truncated, D2D denormalization allows relatively large rounding error which is +-.6 values could be rounded to the final value. So .4 to .6 range can go ether way, and I found good amount of .5 to .55 values actually rounded down.

Ah, gotcha. Fortunately normal maps from height maps are an approximation to begin with, so matching the results from other apps is not a must and it doesn't really matter if the RGB numbers are one off from different rounding.

 

1 hour ago, _koh_ said:

HlslBinaryFunctionEffect has Pow and Dot so you can replace them with those I guess. But if you are familiar with HLSL, just move to SDK and writing your own shader might be more convenient.

I'll take a further look, the API documentation looked confusing at a glance (it didn't specify where those vectors are exactly coming from since it has a Vector4Float input, suggesting it's not from the pixel data). I'm not familiar with HLSL or making plugins outside Codelab, so I'll stick with what's available in there for now.

 

I might switch to ArithmeticCompositionEffect instead of Gamma since it has pixel1 * pixel2 as one of its values, assuming both the inputs can be same that'll give me a square in a way that I imagine to be a little more efficient than a power function. Edit: yup, the default coeffecients are the pixel1 * pixel2 mode even, works fine.

Edited November 16 by frio

[_koh_]

Just read the math part, and sounds like this will just do that.

 

edit:

Ah, need to set alpha = 0 beforehand.

var output = new HlslUnaryFunctionEffect(DC, HlslUnaryFunction.Normalize, input);

 

And if you are processing data, you should put this method in your code which will bypass color space conversion and such, but maybe you already know.

protected override void OnInitializeRenderInfo(IGpuImageEffectRenderInfo renderInfo) {
    renderInfo.ColorContext = GpuEffectColorContext.WorkingSpace;
    renderInfo.InputAlphaMode = renderInfo.OutputAlphaMode = GpuEffectAlphaMode.Straight;
    base.OnInitializeRenderInfo(renderInfo);
}

[frio]

59 minutes ago, _koh_ said:

Just read the math part, and sounds like this will just do that.

Thanks, that actually does the job with least effort! Somehow I glazed over that one existing, I was just looking at the binary effect version. Just gotta remember to zero out the alpha channel first or else it'll contribute to the vector.

 

1 hour ago, _koh_ said:

if you are processing data, you should put this method in your code which will bypass color space conversion and such, but maybe you already know

Yup, I looked that up on this forum earlier when I wanted to make sure there's no colorspace stuff happening.

[Rick Brewster]

2 hours ago, frio said:

I'll take a further look, the API documentation looked confusing at a glance (it didn't specify where those vectors are exactly coming from since it has a Vector4Float input, suggesting it's not from the pixel data)

 

The Hlsl____Effects let you specify the parameter type as Input or Value, and then you put what you want into the corresponding Properties.InputN or Properties.ValueN.

 

https://paintdotnet.github.io/apidocs/api/PaintDotNet.Direct2D1.Effects.HlslBinaryFunctionEffect.Props.html