'Callout Selection' plugin development

[midora]

Some may remember there was a plugin feature request to create a callout of the current selection.

Because plugins can only write inside of the selection this is not possible.

My basic idea was to ask the user to invert the selection first and use the hole content for the callout because read from the hole is allowed.

 

But I.m fighting with the basics and can not just fine tune and finish it.

 

<<<Callout1.png>>>

 

The image shows the inverted selection and some coloring inside of the hole which should not happen.

The green rectangle shows the the bounds of the inverted PdnRegion. So the calculation of the inverted selection seems to be correct.

The red ellipse is the scaled inverted region (via matrix transformation. The I call Clear on the inverted PdnRegion and there are stripes in the hole (different pattern depending on the scale factor.

 

var invertedSelectionBounds = optInvertedSelection.GetBounds();
var invertedSelectionCenterX = invertedSelectionBounds.X + invertedSelectionBounds.Width / 2;
var invertedSelectionCenterY = invertedSelectionBounds.Y + invertedSelectionBounds.Height / 2;
var dstRegion = optInvertedSelection.Clone();
var matrix = new Matrix();
matrix.Translate(+invertedSelectionCenterX, +invertedSelectionCenterY);
matrix.Scale(optZoomFactor, optZoomFactor);
matrix.Translate(-invertedSelectionCenterX, -invertedSelectionCenterY);
dstRegion.Transform(matrix);
dst.Clear(dstRegion, optBorderColor);

 

There is a second issue if the scale forces the inverted selection to intersect with the surface border then the plugin crashes.

 

 

 

 

 

 

 

 

 

 

[BoltBait]

5 hours ago, midora said:

ask the user to invert the selection first and use the hole content for the callout

Aren’t you describing my Effects > Selection > Outline selection plugin?

[MJW]

10 minutes ago, BoltBait said:

Aren’t you describing my Effects > Selection > Outline selection plugin?

 

My impression is that this plugin makes an offset, magnified version of the (un)selected region, with  a border around it.

[midora]

1 hour ago, BoltBait said:

Aren’t you describing my Effects > Selection > Outline selection plugin?

I don't know because I never installed but from the name I would say there is a difference between an outline and a callout. I guess MJW got the idea. $the picture just tries to show the basic issue that the plugin writes outside of the selection and I'm asking me if there is a bug in my code or in paint.net.

[BoltBait]

In your code, I don't see that you are taking the intersection of the recalculated selected region rectangle and the canvas... to be sure that your rectangle doesn't go outside of the canvas.

 

[midora]

@BoltBaitThis may explain issue 2: the crash. But if I'm could do the same with a standard region and then it doesn't crash. No problem with scaling.

 

var invertedSelectionCenterX = invertedSelectionBounds.X + invertedSelectionBounds.Width / 2;
var invertedSelectionCenterY = invertedSelectionBounds.Y + invertedSelectionBounds.Height / 2;
Region dstRegion = new Region(optInvertedSelection.GetRegionData());
var matrix = new Matrix();
matrix.Translate(+invertedSelectionCenterX, +invertedSelectionCenterY);
matrix.Scale(optZoomFactor, optZoomFactor);
matrix.Translate(-invertedSelectionCenterX, -invertedSelectionCenterY);
dstRegion.Transform(matrix);
g.FillRegion(Brushes.Red, dstRegion);

 

[midora]

OK, I can avoid both issues by cutting out the problematic areas. But for me this looks like that there are problems in dst.Clear.

 

var matrix = new Matrix();
matrix.Translate(-optInvertedSelectionCenter.X, -optInvertedSelectionCenter.Y, MatrixOrder.Append);
matrix.Scale(optZoomFactor, optZoomFactor, MatrixOrder.Append);
matrix.Translate(optCalloutCenter.X, optCalloutCenter.Y, MatrixOrder.Append);
var dstRegion = optInvertedSelection.Clone();
dstRegion.Transform(matrix);
dstRegion.Exclude(optInvertedSelection); // This avoids writing into the not selected part of the image
dstRegion.Intersect(dst.Bounds); // This avoids the PdnRegion crash if the transformed region is outside of dst.Bounds
dst.Clear(dstRegion, optBorderColor);

 

[midora]

There is something I missed totally.

The main issue now with the plugin request in the topic ZoomIn is the quality of the callout in the example image . This just not possible if the resolution of the original image is too small.  Sometimes you totally miss one requirement 😉. I'm no longer sure that it's worth to spent too much time with this. For high quality images it would work somehow. What do you think?

 

<<<Callout2.png>>>

 

<<<Callout3.png>>>

 

 

 

[BoltBait]

Looking good!

[Reptillian]

Selection is essentially set of pixel information. If you're referring to the shape of the callout image, that may be why. Maybe some pixel-art upscale algorithm, in addition to anti-aliasing algorithm could be your solution.

 

Another method would be using predefined shape, but that's gonna take a while. I should let you know in case that you might think of using predefined shape that I already did that method, and I could help you there.

Edited March 22, 2021 by Reptillian

[midora]

I'm using Bicubic upscaling, a little bit sharpen may help. But we all know, there is no magic to create a better quality if there are not enough pixels.

 

It would be really god if the selection could be provided also as an GraphicsPath and not just as a region. Would make some things much easier (like insetting the selection and keeping beziers and sharp corners).

 

[BoltBait]

24 minutes ago, midora said:

It would be really god if the selection could be provided also as an GraphicsPath and not just as a region.

 

There could be a way...

 

1) Make a selection

2) Edit > Copy Selection

 

3) Edit > Select All (or Deselect)

4) Run your effect which reads the selection from the clipboard

 

The above selection on the clipboard looks something like this:

 

{
  "polygonList": [
    "171,164,171.02302551269531,162.89015197753906,171.09173583984375,
  161.7872314453125,171.20556640625,160.69157409667969,171.36398315429688,
  159.603515625,171.56642150878906,158.52339172363281,171.81231689453125,
  157.4515380859375,172.10113525390625,156.3883056640625,172.43231201171875,
  155.33401489257813,172.8052978515625,154.28901672363281,173.21952819824219,
  153.25364685058594,173.67446899414063,152.22824096679688,174.1695556640625,
  151.213134765625,174.7042236328125,150.20866394042969,175.27793884277344,
  149.21517944335938,175.89013671875,148.23300170898438,176.54026794433594,
  147.26248168945313,177.22776794433594,146.303955078125,177.95208740234375,
  145.35775756835938,178.71267700195313,144.42422485351563,179.50898742675781,
  143.50369262695313,180.3404541015625,142.59649658203125,181.20652770996094,
  141.70298767089844,182.10665893554688,140.823486328125,183.04029846191406,
  139.95834350585938,184.00688171386719,139.10789489746094,185.005859375,
  138.27247619628906,186.03668212890625,137.45242309570313,187.09878540039063,
  136.6480712890625,188.19161987304688,135.85975646972656,189.31463623046875,
  135.08782958984375,190.46726989746094,134.33262634277344,191.64898681640625,
  133.59446716308594,192.85922241210938,132.87370300292969,194.097412109375,
  132.17066955566406,195.36302185058594,131.4857177734375,196.65548706054688,
  130.81916809082031,197.9742431640625,130.17135620117188,199.31875610351563,
  129.54263305664063,200.68846130371094,128.93333435058594,202.08280944824219,
  128.34378051757813,203.50125122070313,127.77433013916016,204.94322204589844,
  127.22531127929688,206.40817260742188,126.69706726074219,207.89555358886719,
  126.18992614746094,209.40480041503906,125.70423126220703,210.93536376953125,
  125.24032592773438,212.4866943359375,124.79853820800781,214.05824279785156,
  124.37921142578125,215.64944458007813,123.98268127441406,217.25975036621094,
  123.60928344726563,218.88861083984375,123.25936126708984,220.53547668457031,
  122.93325042724609,222.19978332519531,122.63128662109375,223.8809814453125,
  122.35381317138672,225.57850646972656,122.10115814208984,227.29182434082031,
  121.87366485595703,229.02037048339844,121.67166900634766,230.76358032226563,
  121.49551391601563,232.52091979980469,121.34553527832031,234.29182434082031,
  121.22206878662109,236.07574462890625,121.12545013427734,237.87211608886719,
  121.05601501464844,239.68040466308594,121.01410675048828,241.5,121,241.5,121,
  243.31964111328125,121.01404571533203,245.1279296875,121.05595397949219,
  246.92430114746094,121.12538909912109,248.70822143554688,121.22200775146484,
  250.4791259765625,121.34547424316406,252.23646545410156,121.49545288085938,
  253.97967529296875,121.67160797119141,255.70822143554688,121.87360382080078,
  257.42153930664063,122.10109710693359,259.11907958984375,122.35375213623047,
  260.80026245117188,122.63123321533203,262.46456909179688,122.93319702148438,
  264.11141967773438,123.25930786132813,265.74029541015625,123.60923004150391,
  267.35061645507813,123.98262786865234,268.94183349609375,124.37915802001953,
  270.51336669921875,124.79848480224609,272.064697265625,125.24027252197266,
  273.59527587890625,125.70417785644531,275.10452270507813,126.18987274169922,
  276.59188842773438,126.69701385498047,278.05682373046875,127.22525787353516,
  279.498779296875,127.77427673339844,280.91720581054688,128.34373474121094,
  282.31155395507813,128.93328857421875,283.6812744140625,129.54258728027344,
  285.02578735351563,130.17131042480469,286.34454345703125,130.81912231445313,
  287.63699340820313,131.48567199707031,288.902587890625,132.17062377929688,
  290.14077758789063,132.8736572265625,291.35101318359375,133.59442138671875,
  292.53271484375,134.33258056640625,293.68536376953125,135.08778381347656,
  294.80838012695313,135.85971069335938,295.90121459960938,136.64802551269531,
  296.96331787109375,137.45237731933594,297.994140625,138.27243041992188,
  298.99310302734375,139.10784912109375,299.95968627929688,139.95829772949219,
  300.893310546875,140.82344055175781,301.79345703125,141.70294189453125,
  302.6595458984375,142.59645080566406,303.49102783203125,143.50364685058594,
  304.28732299804688,144.42417907714844,305.04791259765625,145.35771179199219,
  305.77224731445313,146.30390930175781,306.45974731445313,147.26243591308594,
  307.10986328125,148.23295593261719,307.7220458984375,149.21513366699219,
  308.29574584960938,150.2086181640625,308.83041381835938,151.21308898925781,
  309.32550048828125,152.22819519042969,309.78042602539063,153.25360107421875,
  310.19467163085938,154.28897094726563,310.56765747070313,155.33396911621094,
  310.89883422851563,156.38825988769531,311.18765258789063,157.45149230957031,
  311.43356323242188,158.52334594726563,311.635986328125,159.60346984863281,
  311.79440307617188,160.6915283203125,311.90823364257813,161.78718566894531,
  311.9769287109375,162.89010620117188,312,164,312,164,311.97695922851563,
  165.10984802246094,311.90826416015625,166.2127685546875,311.79443359375,
  167.30842590332031,311.63601684570313,168.396484375,311.43359375,
  169.47660827636719,311.18768310546875,170.5484619140625,310.89886474609375,
  171.6116943359375,310.56768798828125,172.66598510742188,310.1947021484375,
  173.71098327636719,309.78045654296875,174.74635314941406,309.32553100585938,
  175.77175903320313,308.8304443359375,176.786865234375,308.2957763671875,
  177.79133605957031,307.72207641601563,178.78482055664063,307.10989379882813,
  179.76699829101563,306.45977783203125,180.73751831054688,305.77227783203125,
  181.696044921875,305.04794311523438,182.64224243164063,304.287353515625,
  183.57577514648438,303.49105834960938,184.49630737304688,302.65957641601563,
  185.40350341796875,301.79348754882813,186.29701232910156,300.89334106445313,
  187.176513671875,299.959716796875,188.04165649414063,298.99313354492188,
  188.89210510253906,297.99417114257813,189.72752380371094,296.96334838867188,
  190.54757690429688,295.9012451171875,191.3519287109375,294.80841064453125,
  192.14024353027344,293.68539428710938,192.91217041015625,292.53274536132813,
  193.66737365722656,291.35104370117188,194.40553283691406,290.14080810546875,
  195.12629699707031,288.90261840820313,195.82933044433594,287.63702392578125,
  196.5142822265625,286.34457397460938,197.18083190917969,285.02581787109375,
  197.82864379882813,283.68130493164063,198.45736694335938,282.31158447265625,
  199.06666564941406,280.917236328125,199.65621948242188,279.49880981445313,
  200.22567749023438,278.05682373046875,200.77468872070313,276.59185791015625,
  201.30293273925781,275.1044921875,201.81007385253906,273.59524536132813,
  202.29576110839844,272.06466674804688,202.75967407226563,270.51333618164063,
  203.20146179199219,268.9417724609375,203.62078857421875,267.35055541992188,
  204.01731872558594,265.740234375,204.39071655273438,264.11135864257813,
  204.74063110351563,262.46450805664063,205.06674194335938,260.80020141601563,
  205.36869812011719,259.1190185546875,205.64617919921875,257.42147827148438,
  205.89883422851563,255.70816040039063,206.12632751464844,253.9796142578125,
  206.32832336425781,252.23640441894531,206.50448608398438,250.47906494140625,
  206.65446472167969,248.70816040039063,206.77793884277344,246.92424011230469,
  206.87455749511719,245.12785339355469,206.94398498535156,243.31956481933594,
  206.98588562011719,241.5,207,241.5,207,239.68035888671875,206.9859619140625,
  237.8720703125,206.94406127929688,236.07569885253906,206.8746337890625,
  234.29177856445313,206.77801513671875,232.5208740234375,206.654541015625,
  230.76353454589844,206.50456237792969,229.02032470703125,206.32839965820313,
  227.29177856445313,206.12640380859375,225.57846069335938,205.89891052246094,
  223.88093566894531,205.64625549316406,222.19973754882813,205.3687744140625,
  220.53543090820313,205.06681823730469,218.88856506347656,204.74070739746094,
  217.25970458984375,204.39079284667969,215.64939880371094,204.01739501953125,
  214.05819702148438,203.62086486816406,212.48664855957031,203.2015380859375,
  210.93531799316406,202.75975036621094,209.40475463867188,202.29583740234375,
  207.8955078125,201.81015014648438,206.40812683105469,201.30300903320313,
  204.94317626953125,200.77476501464844,203.50120544433594,200.22575378417969,
  202.082763671875,199.65629577636719,200.68841552734375,199.06674194335938,
  199.31871032714844,198.45744323730469,197.97419738769531,197.82872009277344,
  196.65544128417969,197.180908203125,195.36297607421875,196.51435852050781,
  194.09736633300781,195.82940673828125,192.85917663574219,195.12637329101563,
  191.64894104003906,194.40560913085938,190.46722412109375,193.66744995117188,
  189.31459045410156,192.91224670410156,188.19157409667969,192.14031982421875,
  187.09873962402344,191.35200500488281,186.03663635253906,190.54765319824219,
  185.00581359863281,189.72760009765625,184.0068359375,188.89218139648438,
  183.04025268554688,188.04173278808594,182.10661315917969,187.17658996582031,
  181.20648193359375,186.29708862304688,180.34040832519531,185.40357971191406,
  179.50894165039063,184.49638366699219,178.71263122558594,183.57585144042969,
  177.95204162597656,182.64231872558594,177.22772216796875,181.69612121582031,
  176.54022216796875,180.73759460449219,175.89009094238281,179.76707458496094,
  175.27789306640625,178.78489685058594,174.70417785644531,177.79141235351563,
  174.16950988769531,176.78694152832031,173.67442321777344,175.77183532714844,
  173.219482421875,174.74642944335938,172.80525207519531,173.7110595703125,
  172.43226623535156,172.66606140136719,172.10108947753906,171.61177062988281,
  171.81227111816406,170.54853820800781,171.56637573242188,169.4766845703125,
  171.36393737792969,168.39656066894531,171.20552062988281,167.30850219726563,
  171.0916748046875,166.21284484863281,171.02296447753906,165.10992431640625,
  171,164"
  ]
}

 

 

[midora]

@BoltBaitThanks for the hint. Yeah I checked this some days ago also with holes, but it looks like that it is just a list of RegionScans and not the Graphics path. There is more on the clipboard. I remember that 5 years ago I was able to get the path in a dirty way. But this code is blocked in the moment.

[MJW]

1 hour ago, midora said:

But we all know, there is no magic to create a better quality if there are not enough pixels.

 

 Obviously you never watched CSI. Four pixels were sufficient to recreate the entire image in high-def.

[MJW]

1 hour ago, midora said:

Thanks for the hint. Yeah I checked this some days ago also with holes, but it looks like that it is just a list of RegionScans and not the Graphics path.

 I'm almost certain the data on the clipboard is the Graphics path, from which the exact selection can be recreated. (Actually, I can say I'm certain; I just want to avoid looking too foolish if somehow I'm wrong.)

[midora]

Then we have to wait for the documentation. I could see only rectangles in the text output. no other markers or indication. But as I said there are other formats. As I remember there was some serialized data. When I did pdn2XXX for Irfanview I wrote my own dezerializer.

[MJW]

1 hour ago, midora said:

Then we have to wait for the documentation. I could see only rectangles in the text output.

 

Were you perhaps looking at the clipboard data from a selection created with the Magic Wand? That would explain why you would only see rectangles along pixel boundaries.  Create a rectangular selection, tilt it, and copy the selection to the clipboard. I believe you will see a four-sided path defining the tilted rectangle.

 

EDIT: -- Sorry I linked to something that's not accessible. ---

 

 

 

[Reptillian]

@MJW: On your documentation, "Sorry, we can't show this content because you do not have permission to see it. "

[midora]

I can not access this item.

 

I started with a rectangle in the top left border and get

 

{
  "polygonList": [
    "0,0,  200,0,  200,150,  0,150,  0,0"
  ]
}

 

Obviously this is a closed list of corner points.

 

Then I created a small rectangular hole in it.

 

  "polygonList": [
    "75,59,  19,59,  19,28,  75,28",
    "200,0,  0,0,  0,150,  200,150"
  ]

 

Also a list of point. The inner rectangle first and then starting rectangle. No closing point. Maybe CR has a special meaning in separating the rectangles.

 

But ellipses create tons of data...

 

And there is no selection tool in paint.net where part of the selection is a bezier curve.

 

 

 

 

[Reptillian]

On ellipse, you could create your own surface of ellipse or circle.

[MJW]

48 minutes ago, Reptillian said:

@MJW: On your documentation, "Sorry, we can't show this content because you do not have permission to see it. "

 

Sorry, I linked to something that's not accessible.

[null54]

28 minutes ago, MJW said:

I updated the link.

 

You are still linking to a forum thread that not everyone will be able to access.

Quoting the relevant portion of Rick's post:

 

On 8/17/2018 at 7:42 PM, Rick Brewster said:

When a selection is active, you can use Edit -> Copy Selection to copy just the selection to the clipboard. In a mechanical sense, this is copying the geometry of the selection to the clipboard. And you can see this in what actually ends up in the clipboard, as it's just a wad of JSON text that describes the polygon(s) of the selection.

 

For example, this selection was drawn with two strokes of the Rectangle Selection tool. The 2nd rectangle was drawn using "Invert ("xor")" mode (Ctrl + Right Mouse Button, btw).

 

 

It results in this JSON being put on the clipboard after clicking on Edit -> Copy Selection:

 

{
  "polygonList": [
    "152,63,112,63,112,106,67,106,67,143,152,143",
    "67,63,112,63,112,26,32,26,32,106,67,106"
  ]
}

 

Why JSON? Because it's really simple, and there's already a lot of tooling out there for processing it. With regard to simplicity, if you want to (for example) save 5 selections that you're needing to reuse, then you can just open up 5 copies of Notepad and use them as scratchpads. You could also write a script (python? PowerShell? whatever :)) to do processing on the selection data. Maybe implement a script to do erode/dilate, shrink/expand, combine/extract, etc. The point is that you're not limited to the functionality that Paint.NET has baked into it. Some of this stuff will possibly get baked into Paint.NET directly so that everyone can benefit from it -- for now it'll be the realm of enthusiasts and hackers

 

The format of the JSON should be relatively self-evident, at least it is to me  but I wrote it. The format is: a JSON object containing 1 property called polygonList whose value is an array. Each element in the array is a string. The format of those strings is an interleaved list of X and Y values that make up the points in the polygon. Each polygon is conceptually XOR'd with the previous one in the array, which follows the standard rules that fall out of polygon rasterization using the even-odd rule (which is oddly described best at the Wikipedia entry for its close cousin, the nonzero-rule).

 

You can use Ctrl+Alt+Shift+V to paste the selection directly in replacement mode. The other selection combine modes are also available from within the Edit -> Paste Selection sub-menu. If you want to access them via the keyboard, you can use the standard keyboard accessibility chords based on the underlined letters at each step of the way, e.g. Alt+E (open Edit menu), then N (Paste Selection), then E (for Exclude).

 

Also: You can use Paste Selection when a regular image is on the clipboard. That is, if you've used Edit -> Copy (Ctrl+C), then both Edit -> Paste and Edit -> Paste Selection will work. The latter will extract only the mask geometry and will use it as the selection geometry; if it's just a regular image, like something copied from another app, then the image bounds will be used. That is, a rectangle starting at (0,0) with the same width and height of the image.

 

[MJW]

1 hour ago, midora said:

Also a list of point. The inner rectangle first and then starting rectangle. No closing point. Maybe CR has a special meaning in separating the rectangles.

 

That is strange that the single rectangle is explicitly closed while the two rectangles aren't. However, I believe the the explicit closure is -- if not a minor bug -- at least redundant. Polygons are implicitly closed; and if I edit out the final 0,0 in the first example and copy it to the clipboard, Paste Selection still works.

 

EDIT: Closing polygons in GraphicPaths is optional, says the documentation: "The points in the points array specify the vertices of a polygon. If the first point in the array is not the same as the last point, those two points are connected to close the polygon."

[midora]

Still getting strange results while the dialog is open.

 

 

But after pressing OK the result is ?always? as expected (no red artifacts, no writing to the hole as visible in the screenshot.

Typically I always cancelled before...

Edited November 14, 2021 by midora

[midora]

The XOR even-odd rules make sense. This is a kind of standard. So for rectangles this will be fine. But it seems difficult to recreate a GraphicsPath from curved or rotated stuff.