Imperfect Edges (+Video)

[SERAPHIM]

For the place of a smooth gradient, Paint.NET generates ladders ↴

 

Spoiler

Original language ↴

____________________

Вместо плавного градиента, Paint.NET генерирует лестницы.

 

Edited December 12, 2022 by SERAPHIM

[BoltBait]

No one knows what you're trying to demonstrate.

[MJW]

Though I hesitate to provide an explanation for a rather rudely expressed comment, I think the point is that the gradient along edge that results from antialiasing the tilted rectangle isn't continuous. It stays the same value for a span of several pixels, then jumps, instead of changing continuously by a smaller amount. I'd guess that may be due to the number of subsamples used for antialiasing (assuming that's how the antialiasing is achieved). If there are, say, 6 samples per side, there will be only 36 possible levels of gray, which would result in a non-continuous gradient. On the other hand, 16 samples per side would require 256 subsamples, which would probably lead to poor performance.

[BoltBait]

OK, I tinkered with this and I think the OP is saying...

 

When Paint.NET draws a single pixel width line that is one pixel off from vertical, Paint.NET doesn't use very many shades of gray:

 

The Paint.NET drawn line (zoomed in) is shown above (8-9 steps) along with a line drawn with a gradient tool (256 steps).

 

One could argue that when zoomed out, the Paint.NET line is "good enough"™.

 

[Rick Brewster]

The rendering for a shape doesn't use sub-/super-sampling, it's regular Direct2D per-primitive antialiasing. So either it's doing that at a coarser granularity than you'd expect, or some part of Paint.NET's composition engine is losing detail along the way. Probably not the latter, since you'd see weird things happening elsewhere too if that were the case.

 

Direct2D's per-primitive antialiasing is based on some kind of coverage calculation, it's not based on samples, but it's hard to find any real information about it. The best quality solution would be, as @MJW suggests, to use super sampling. Basically you render at 2x 4x 16x (or whatever) size without antialiasing, and then downsample. But you'd need to render at a very large size to get that to look perfect (16x, which is 256 samples per output pixel), and that's not really feasible. Intermediate sizes, say 4x (for 16 samples) would still look very good but the performance cost compared to the quality improvement likely wouldn't be worth it.

 

It may even be that Direct2D is just doing a poor job of rendering geometry to Alpha8 render targets, which is what's used in this case. It wouldn't surprise me. If that's the case, quality could be improved by rendering to a regular PBGRA32 target and then extracting the alpha channel, which would not be very expensive.

 

Edit: Just tried rendering to PBGRA32 and extracting alpha. No quality improvement.

 

Edit: Just tried both 2x and 4x super sampling. Quality improved, but not to the level as expected by OP. Performance also tanked.

[Rick Brewster]

Actually, I just tried supersampling again ... and it does seem to improve the antialiasing quality. The trick was to make sure D2D is rendering with antialiasing, not aliased as I said before.

 

On the left is a line at a -87 degree angle, drawn with 4.3.11. On the right is a prototyped version that renders at 2x the resolution and then downscales using Linear interpolation.

 

I'll look into this for the 4.4 release. Hopefully performance doesn't tank too bad, and quality can be maintained (sometimes things get offset or blurred when you do this sort of thing).

 

 

 

 

 

[Rick Brewster]

Okay this is in for the 4.4 release.

 

I'm using 2x super sampling -- that is, render at 2x size and then downsample using Linear interpolation. In my testing, quality was improved even further with higher sampling factors, e.g. 4x 8x 16x, but performance was also astonishingly degraded at that point. 2x seems to be the sweet spot.

[Rick Brewster]

I may be able to get 4x super sampling by first upgrading the Line/Curve and Shapes tools to use hardware rendering. That might even result in higher performance overall. I've already done this for the Move Selected Pixels tool (for v4.4), so there's precedent.

[Rick Brewster]

I was able to get 4x super sampling with no real reduction in performance, and it even improved temporary memory usage because it's not going through Direct2D's effects system, and because it's rendering directly to Alpha8 instead of going through a PBGRA32 intermediary.

 

I already had a very high-performance 4x Alpha8 downsampler which is used for the selection mask, so I just reused that.

[Red ochre]

@Rick Brewster

Will the super sampling improve the AA on lines drawn by an effect that uses G.D.I.+?

 

[Rick Brewster]

This does not affect effects, only the Shapes and Line/Curve tools. If you want to perform super sampling in your own effect then you have to do it yourself.

 

I recommend waiting for PDN 4.4 because then you can use Direct2D and everything will be much better.

[Red ochre]

Thanks for the reply.

[Rick Brewster]

Super sampling will be pretty easy to do with Direct2D. You'll use a command list to store the drawing commands, which you'll need to draw with a 4x scaling transform (easy), and then you will plug that command list into a ScaleEffect with 1/4 scaling. Replace "4" with your favorite scaling factor, although I recommend not going too crazy with it so as to not adversely destroy performance and memory usage.