Jump to content

Perspective Transformation


MJW

Recommended Posts

This is a beta version of a perspective transformation plugin. Unfortunately, I can't seem to post images using my flickr account (I get the message, "You are not allowed to use that image extension on this community.")  The plugin uses four double-vector controls to place the transformation control points. Due to CodeLab restrictions, the double vectors are all initially at the center of the window, and have to be moved to reasonable positions to produce a picture. It's not too hard to do, but it's very inelegant. Eventually, I'll convert it a Visual Studio project and fix that problem. There are also scale and offset controls to adjust the image size and position.

 

If the scales and offsets are in their default positions, the image is perspectively mapped into the quadrilateral formed by the control points, surrounded by transparency. The image isn't clipped to the quadrilateral, it's just that there's nothing to display. If the scaling factor is greater than 1, the image will extend beyond the control-point quadrilateral. If the control points are moved to invalid positions, such as a non-convex quadrilateral, the entire image will be transparent.

 

The effect is in the Effects>Distort menu.

 

EDIT: New version uploaded 2/8/2015. There are three changes. First, I replaced the X and Y Scale controls with Scale and XY Proportion controls. Second, I added optional anti-aliasing using supersampling. Third, I used floats instead of doubles for almost all the calculations. The source code is also updated.

 

One thing that might be noticed is that if the checkerboard or grid is displayed with the control points in the corners, so the image is undistorted, then the image will appear to be a bit darker when anti-aliasing is enabled. This is an (as far as I know) unavoidable consequence of using supersampling and bilinear interpolation. If you zoom in on the image, you'll see what amounts to a one-pixel blur, so the black-white edge is now dark gray where it was black, and light gray where it was white. If the image is moved or rescaled a bit, all the edges will be shades of gray, and the darkening effect will more or less disappear, as it will for other images that don't have a bunch of black-to-white transitions. So while I wouldn't call it a feature, it's not much of a bug, either.

 

EDIT: Fixed spelling error in plugin name.

 

A checkerboard pattern and a grid pattern to demonstrate anti-aliasing is here:

Checkerboard.png

Grid.png

 

The plugin is here (beta 1.1): PerspectiveTransformation1_1.zip (the old version)

 

This is version Beta 1.2: PerspectiveTransformation1_2.zip (the current version)


Here is the source code for the older version:

Spoiler

 


// Author: MJW
// Name: Perspective Transformation
// Title: Perspective Transformation
// Submenu: Distort
// Desc: Perspective transform of selected pixels
// Keywords: perspective transformation
#region UICode
Pair<double, double> Amount1 = Pair.Create(-1.0 , -1.0); // Upper Left
Pair<double, double> Amount2 = Pair.Create(1.0 , -1.0 ); // Upper Right
Pair<double, double> Amount3 = Pair.Create(1.0 , 1.0); // Lower Right
Pair<double, double> Amount4 = Pair.Create(-1.0 , 1.0); // Lower Left
double Amount5 = 1; // [0,10] Scale
double Amount6 = 0; // [-1,1] XY Proportion
double Amount7 = 0; // [-2,2] X Offset
double Amount8 = 0; // [-2,2] Y Offset
bool Amount9 = false; //[0,1] Anti-Alias
int Amount10 = 4; //[2, 6] Subsamples Per Direction
#endregion

ColorBgra transparent = ColorBgra.FromBgra(0, 0, 0, 0);

float Mxx, Mxy, Mxw, Myx, Myy, Myw, Mwx, Mwy, Mww;
bool antialias;
private Surface Src = null;
const double maxPropScale = 8.0;

void Render(Surface dst, Surface src, Rectangle rect)
{  
#if false
    // Area preserving version.
    double angle = 0.25 * Math.PI * (1.0 + 0.95 * Amount6);
    double sqrtHalf = Math.Sqrt(0.5);
    float uiXScale = (float)(sqrtHalf * Amount5 / Math.Cos(angle));
    float uiYScale = (float)(sqrtHalf * Amount5 / Math.Sin(angle));
#else
    // Simple version.
    float uiXScale = (float)Amount5;
    float uiYScale = uiXScale;
    if (Amount6 < 0.0)
        uiYScale *= (float)(1.0 - (maxPropScale - 1.0) * Amount6);
    else
        uiXScale *= (float)(1.0 + (maxPropScale - 1.0) * Amount6);   
#endif
    float uiXOffset = (float)Amount7, uiYOffset = (float)Amount8;
        
    float xScale = 0.5f * src.Width, yScale = 0.5f * src.Height;
    float x0 = xScale * ((float)Amount1.First + 1.0f);
    float y0 = yScale * ((float)Amount1.Second + 1.0f);
    float x1 = xScale * ((float)Amount2.First + 1.0f);
    float y1 = yScale * ((float)Amount2.Second + 1.0f);
    float x2 = xScale * ((float)Amount3.First + 1.0f);
    float y2 = yScale * ((float)Amount3.Second + 1.0f);
    float x3 = xScale * ((float)Amount4.First + 1.0f);
    float y3 = yScale * ((float)Amount4.Second + 1.0f);

    float T012 = TripleProduct(x0, y0, x1, y1, x2, y2);
    float T123 = TripleProduct(x1, y1, x2, y2, x3, y3);
    float T230 = TripleProduct(x2, y2, x3, y3, x0, y0);
    
    float Sx = T012 * T123;
    float Sy = T123 * T230;
    float Sw = T230 * T012;
    
    // The initial calculations must be done with local variables
    // to avoid modifying the matrix while in use by other threads.
    float Txx, Txy, Txw, Tyx, Tyy, Tyw;
    Txx = Sx * (y3 - y0);
    Txy = Sx * (x0 - x3);
    Txw = Sx * (x3 * y0 - x0 * y3);
    Tyx = Sy * (y0 - y1);    
    Tyy = Sy * (x1 - x0);        
    Tyw = Sy * (x0 * y1 - x1 * y0);
    Mwx = Txx + Tyx + Sw * (y1 - y3);
    Mwy = Txy + Tyy + Sw * (x3 - x1);
    Mww = Txw + Tyw + Sw * (x1 * y3 - x3 * y1);
   
    float xRescale, yRescale, xShift, yShift;
    xRescale = (uiXScale < 0.01f) ? 100.0f : 1.0f / uiXScale;
    yRescale = (uiYScale < 0.01f) ? 100.0f : 1.0f / (float)uiYScale;
    // The shifts are adjusted so scaling is from the center.
    xShift = 0.5f * ((1.0f - xRescale) - uiXOffset * (1.0f + xRescale)) * src.Width;  
    yShift = 0.5f * ((1.0f - yRescale) - uiYOffset * (1.0f + yRescale)) * src.Height;
    xRescale *= src.Width;
    yRescale *= src.Height;
    Mxx = xRescale * Txx + xShift * Mwx;
    Mxy = xRescale * Txy + xShift * Mwy;
    Mxw = xRescale * Txw + xShift * Mww;
    Myx = yRescale * Tyx + yShift * Mwx;
    Myy = yRescale * Tyy + yShift * Mwy;
    Myw = yRescale * Tyw + yShift * Mww;
    
    antialias = Amount9;
    SetupForSubPixels(Amount10, Amount10);
    Src = src;

    for (int y = rect.Top; y < rect.Bottom; y++)
    {
        for (int x = rect.Left; x < rect.Right; x++)
        {
            ColorBgra CurrentPixel;
            if (!antialias)
            {
                float fx = (float)x, fy = (float)y;
                float tx = Mxx * fx + Mxy * fy + Mxw;
                float ty = Myx * fx + Myy * fy + Myw;
                float tw = Mwx * fx + Mwy * fy + Mww;
                if (tw > 0.0f)
                {
                    tx /= tw;
                    ty /= tw;
                    CurrentPixel = src.GetBilinearSample(tx, ty);
                }
                else
                {
                    CurrentPixel = transparent;
                }
            }
            else
            {
                CurrentPixel = GetAntialiasedPerspectivePixel(x, y);
            }            
            dst[x, y] = CurrentPixel;
        }
    }
}


int ssXSamples, ssYSamples, ssSamples, ssTwiceSamples;
float ssXStart, ssYStart, ssXStep, ssYStep;

void SetupForSubPixels(int xSamples, int ySamples)
{
    ssXSamples = xSamples;
    ssYSamples = ySamples;
    ssSamples = xSamples * ySamples;
    ssTwiceSamples = ssSamples << 1;
    ssXStep = 1.0f / (float)xSamples;
    ssYStep = 1.0f / (float)ySamples;
    ssXStart = 0.5f * (1.0f - ssXStep);
    ssYStart = 0.5f * (1.0f - ssYStep);  
}

ColorBgra GetAntialiasedPerspectivePixel(int x, int y)
{
    float bx = (float)x - ssXStart, fy = (float)y - ssYStart;
    
    int b = 0, g = 0, r = 0, a = 0;
    for (int i = 0; i < ssYSamples; i++)
    {
        float fx = bx;
        for (int j = 0; j < ssXSamples; j++)
        {
            float tx = Mxx * fx + Mxy * fy + Mxw;
            float ty = Myx * fx + Myy * fy + Myw;
            float tw = Mwx * fx + Mwy * fy + Mww;
            if (tw > 0.0f)
            {
                tx /= tw;
                ty /= tw;
                ColorBgra CurrentPixel = Src.GetBilinearSample(tx, ty);
                int alpha;
                if ((alpha = CurrentPixel.A) != 0)
                {
                    b += alpha * CurrentPixel.B;
                    g += alpha * CurrentPixel.G;
                    r += alpha * CurrentPixel.R;
                    a += alpha;
                }
            }               
            fx += ssXStep;    
        }
        fy += ssYStep;
    }
    if (a == 0)
    {
        return transparent;
    }
    else
    {
        // Compute the (rounded) averages.
        int twiceA = a << 1;
        b = ((b << 1) + a) / twiceA;
        g = ((g << 1) + a) / twiceA;
        r = ((r << 1) + a) / twiceA;
        a = (twiceA + ssSamples) / ssTwiceSamples;
        return ColorBgra.FromBgra((byte)b, (byte)g, (byte)r, (byte)a);
    }    
}

// The triple product of three vectors, considered as 3D homogeneous vectors with 1 as
// the homogeneous coordinate.
float TripleProduct(float x1, float y1, float x2, float y2, float x3, float y3)
{
    return (x1 - x3) * (y2 - y3) - (x2 - x3) * (y1 - y3);
}

 

 

 

EDIT: Beta version 1.2.

Changes include:

Control points start in the corners instead of the center of the screen.

The corner points can be moved outside the window boundary (the current limit is -1.5 to 1.5).

The corner controls, and the other double controls, are 3 decimal places instead of two, for more precise control.

The XY Proportion control works differently, increasing one dimension while simultaneous decreasing the other. I thought it would be simpler to use if it only changed one dimension at a time, but found it to be difficult and confusing..

A Help menu is added.

Some minor cosmetic changes, such as renaming the "Anti-Alias" control "Antialias."

 

This is a VS project instead of CodeLab. I didn't include the source code, but if anyone wants it I'll post it. I will post the code when I release the non-beta version. I didn't remove either the CodeLab source or the old CodeLab DLL.

 

There are a few enhancements I hope to add to the eventual non-beta plugin, but this version is, I think, a substantial improvement from the previous version.

Please let me know what features you'd like to see (though you may not see them). Also, let me know if there are any spelling errors, grammar errors, or improvements I could make to the Help menu.

I could gray-out the Antialias Quality when Antialiasing is disabled, but I thought it might be useful to be able to adjust it before enabling it. Opinions about this are welcome.

  • Upvote 1
Link to comment
Share on other sites

Very interesting! I'll d/l this and have a play. Thanks.

Link to comment
Share on other sites

I hope those who try this effect will offer any suggestions for improvements. One thing I need to change is the resolution of the double-vector controls. The control points can't be positioned as precisely as they should be. I believe I can change this once I convert it to a VS project. I want to get the user interface nailed down before converting.

 

One idea I had is to change the X and Y scale controls to an overall scale, and a separate control to adjust the aspect ratio. The control would probably have a range of -1 to 1, with a default of 0. Moving to the right would increase the X size; moving to the left would increase the Y size. The disadvantage is that there's no obvious physical meaning to the numeric value. On the other hand, I'm not sure the scale values mean anything, and I think the scale and aspect ratio controls are quite intuitive and would be easier to use.

Edited by MJW
Link to comment
Share on other sites

Works just as you described. Initial thoughts were that the X and Y scale was too severe, but I quickly found it quite usable.

If you want to change the vector control, why not roll your own all-in-one control? Something like this would give you a great deal more accuracy AND be easier to visualize the transformation (see UI for Quadrilateral Reshape - which your effect is similar to).

yhsjjie-1412.png

Quad Reshape doesn't use the numeric coordinates - but I think they're useful to have.

Link to comment
Share on other sites

I haven't downloaded this yet but it looks interesting.  Kind of an upgrade to the Quad reshape plugin and could come in handy in some of my work. Will it work on the older version of pdn or just the new one?

 

                                                              http://forums.getpaint.net/index.php?/topic/21233-skullbonz-art-gallery

Link to comment
Share on other sites

Ergo Eram Reputo: If Quadrilateral Reshape does a perspective transformation, there's probably not much need for my version. If it doesn't, evanolds is certainly welcome to adapt my code to add the feature. I like the look of your interface, but I doubt I'd get around to writing the custom controls anytime soon, so unless they already exist, I'll probably stick with the Indirect IU controls.

 

skullbonz: I assume the posted DLL will only work on the new pdn version, but the source code will work in an older version, assuming it has an old version of CodeLab to build a DLL.

Link to comment
Share on other sites

Eli, what, in particular, makes it difficult to use? I'm not sure that I've got any ideas to make it much easier to use, besides putting the control points in more reasonable starting positions. I'll say (perhaps defensively) that I don't find it difficult to use. The post-beta ideas I'd planned are mostly some additional features, not a greatly improved user interface.

Link to comment
Share on other sites

First, I was surprised when I saw nothing but a transparent layer. Then I could not figure out which slider move and in what direction. I am a senior with very little logic :D  so I think that the graphical /visual interface prosed by EER would make it user friendly. It would be nice if I could move the nodes (A, B, C and D) directly on the visual interface and then fine tune their position with the numeric coordinates.

Link to comment
Share on other sites

First, I was surprised when I saw nothing but a transparent layer. Then I could not figure out which slider move and in what direction. I am a senior with very little logic  :D  so I think that the graphical /visual interface proposed by EER would make it user friendly. It would be nice if I could move the nodes (A, B, C and D) directly on the visual interface and then fine tune their position with the numeric coordinates.

 

Post edit: I was getting all kinds of shapes that did not make sense. I was going the wrong way and could not get what I wanted. It is agreat plugin once you know how it works :)  . I found out that it is possible to flip the object and that is what was throwing me off. But as I said before, I have very little logic and so far I do not know how flipping occurs.  :lol:

Link to comment
Share on other sites

Goonfella, it works ! you just need to drag the black crosses to the proper corner and you will see your entire canvas. Then you can start to move the little black crosses until you find the desired transformation.   :)

Be careful! At some point your image will flip but that is another story that I do not understand yet. 

perspective-tarnsformation-49d1a1a.png

Edited by Eli
Link to comment
Share on other sites

The lack of an initial picture is, indeed, very ugly. It's due to the fact that Codelab doesn't allow initialization values for double vectors - they're always at the center. Once I convert the plugin to a VS project, that will be fixed. I haven't done that because it's easier to change or add controls while it's still in CodeLab.

 

I would certainty like to use an interface like Ergo Eram Reputo suggested (perhaps with the position controls arranged, instead, in a rectangle to better correspond to the points). I'm not sure I want to take the time to write all the controls to do it. This originated as a little personal project to check the transformation math. Someone asked for a plugin to transform some text into a quadrilateral shape, so I decided to add scale and offset controls and post it. I want it to be useful, but writing the new controls, with all the expected PDN behavior, seems rather daunting.

 

Perhaps someone can answer a related question: Is preserving the control positions from one invocation in a PDN session to the next somehow automatic, or does it have to be incorporated into the control design? I really don't know how that works.

Link to comment
Share on other sites

Great example Eli!

@MJW: The problem is not the effect. Its that the default settings (vector locations) produce a blank canvas. For new users this is intimidating.

While its fairly easy to work out the UI, the goal should be for it to be user friendly (intuitive) as possible. You already know this because you appreciate how the default control locations need to be improved.

Porting the code to VS and resetting the locations will help a great deal. A custom UI and quad control would make it awesome. See how intuitive Quad Reshape is to use?

Please don't take these suggestions as criticisms. Your effect is a really nice one and worthy of some effort. A fine addition to my paint.net toolbox.

Please give all the above suggestions some thought. If you wish to develop a custom UI I'm more than willing to assist.

Link to comment
Share on other sites

Thanks Eli. At least now I know what I was doing wrong. :P

 

MJW - sounds awfully complicated to make any plugin - especially to someone like me who knows zilch about coding.

 

( tried it again and it works great. )

Edited by Goonfella

 

 

Please feel free to visit my Gallery on PDNFans

And my Alternatives to PDN

Link to comment
Share on other sites

The values of a control persist because they are tokenized and stored by the plugin. This happens behind the scenes when using CodeLab. In VS you can take control of this process yourself. Indeed when using a Winform UI you have to code the variables yourself in order to persist the plugin settings. This is not difficult ;)

Link to comment
Share on other sites

I've seen many examples of plugins with fancy interfaces like Ergo Eram Reputo proposes. I wish someone would put a library of PDN controls in a DLL so other people could use them. I promise to do that if I write the controls for this plugin.

Edited by MJW
Link to comment
Share on other sites

Does anyone know offhand if the Quadrilateral Reshape plugin does a perspective transformation? If it does, I'm not sure there's much point in developing a plugin that does about the same thing. I read through the Evan's Effects thread, and didn't see any information one way or the other on what transform(s) it uses to map the image into the quadrilateral.

Link to comment
Share on other sites

Some controls have already been published:

 

http://forums.getpaint.net/index.php?/topic/31390-hex-color-wheel

 

 

Here's some other interesting links for you...

 

 

BTW  You know Evan open-sourced his plugins didn't you?  The link is broken - and I can't find a copy.  Maybe someone else will be nice and post it.

Link to comment
Share on other sites

BTW  You know Evan open-sourced his plugins didn't you?  The link is broken - and I can't find a copy.  Maybe someone else will be nice and post it.

 

The source code is available from the Internet Archive at http://web.archive.org/web/20130830101215/http://evanolds.com/pdn_effects.html.

The license from that page and a direct link are included below:

The license for the source code is public domain. This means that you can use it pretty much however you want and do not have to give me credit (although if you want to, feel free). The only thing you cannot do is claim legal ownership and prevent other people from using it.

Think of it like algebra. You can always use algebra however you want to solve problems, but you can't say that you own a patent on algebra and charge other people for using it.

 

http://web.archive.org/web/20130830101215/http://evanolds.com/dl/EOEffectsSrc_Oct16_2012.zip

  • Upvote 1

PdnSig.png

Plugin Pack | PSFilterPdn | Content Aware Fill | G'MICPaint Shop Pro Filetype | RAW Filetype | WebP Filetype

The small increase in performance you get coding in C++ over C# is hardly enough to offset the headache of coding in the C++ language. ~BoltBait

 

Link to comment
Share on other sites

Thank you for the link, null54. I looked at the quadrilateral code. It's quite confusing, but as far as I can tell, doesn't seem to be doing a perspective transformation. For some reason, I can't locate the Quadrilateral Reshape UI. All I get is a mostly empty form with a couple of buttons and a "Feather with radius" control. Too bad; I really hoped to swipe some of the code.

Link to comment
Share on other sites

I've seen many examples of plugins with fancy interfaces like Ergo Eram Reputo proposes. I wish someone would put a library of PDN controls in a DLL so other people could use them. I promise to do that if I write the controls for this plugin.

 

OptionBasedLibrary contains all the IndirectUI controls and more (like tabs and panels).

A lot of option controls provide extended features. I.e.NumericUpDown which supports multiple units in one control (i.e px, cm, and in).

OptionBasedLibrary is HighDPI aware.

OptionBasedLibrary supports as many languages as you like.

Your are creating OptionBased UIs simular to PropertyBased ones. I.e.

   new OptionDoubleSlider(OptionNames.MySlider, optContext, 0, -100, 100)

creates the typical slider object with a NumericUpDown and the reset button (range [-100 100], default value 0)

You may add custom option controls local to your project.

OptionBasedDialogs may remember position and size. The current token (set of visible options) can be saved and loaded by the user. The user may even define the default option settings for a paint.net session.

 

Examples? See 'Parallel Lines and Patterns' or 'PrintIt'. Or

 

RichTextDialog.jpg

 

Still you should have an idea about PropertyBased effects first and switch to OptionBased later if you need more flexibiltity.

Edited by midora

midoras signature.gif

Link to comment
Share on other sites

Thank you, midora. I must say, I find this subject pretty confusing. Perhaps it's just what I'm used to, but what I understand are VS-type controls with properties and event handlers. I know VS allows custom controls, which can be either combinations of standard controls or new controls using GDI+. I have a relatively simple VS project from somewhere for an angle-selector control. What I'd like to do someday (assuming it hasn't been done, and I just don't realize it) is make a standard VS library of PDN-type controls that can be used from the control toolbox in the form designer just like standard VS controls. Most of them seem like they'd be pretty straight forward. The reset arrow is either a quite simple custom control, or perhaps could be done as a custom button. Most of the other controls, such as the double slider, would just seem to be combos, such as a slider and a reset button, along with whatever code is necessary to preserve the position during the PDN session.

 

The problems I have with the IndirectUI controls are, first, that their placement is so inflexible, making for rather space-inefficient and somewhat inelegant UIs, and second, that they're very easy to use as long as I have CodeLab writing most of the code for me, but everything's so spread out and indirectly done, that they're totally confusing (at least to me) to add and modify in VS. (Of course the same is true for VS controls: it'd be a mess to have to write all the underlying support code; fortunately, the form designer does the work for us.)

 

(When I say IndirectUI user interfaces are inelegant, I mean for plugins with a large number of controls. For plugins with, say, six or fewer controls, it's fine, and provides a standard look to the UI. With a bunch of controls, the inability to have them side-by-side, or to grouped (in, for instance, a group box) according to function, makes the interface less than ideal.)

Edited by MJW
Link to comment
Share on other sites

The other thing that would need to be dealt with is to cause a re-rendering when a control changes. That magically happens with IndirectUI controls. I assume it's reasonably simple to incorporate into VS controls. I just don't know much about how that part works in PDN. There might have to be some sort of time delay so that every little change in a control being adjusted doesn't trigger a re-render.

 

(Actually, now that I think about it, I believe I knew at one time how that works. In fact, I think I discovered it by looking at some of your (midora's) code. [it may have been Red ochre's code])

Edited by MJW
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...