Tiling Effect Perspective woes

[Red ochre]

I'm trying to keep things in perspective but they keep being blown out of all proportion!



I've written tiling effect that I'm pleased with. It uses MJW's super-sampling ideas and some ideas I had for brick like and reflective tiling and yields good results apart from the tilt. It really needs perspective tilting and I cannot solve the problem without creating unwanted curved perspective lines?

If the perspective problem is solvable, I would then (hopefully) do the calculations in Render so that it could benefit from super-sampling at the 'furthest' distance (Capped to a maximum of say 100 samples). 'Simple' backwards tilting would be ideal (not around the cross hairs, as enlarging below the axis never looks good IMO).


The dropbox link below contains the codelab code, the .dll and an image I've been using for testing.

I've been driving myself nuts trying to get the tilting to work correctly so any help will be greatly appreciated.

Btw I chose the name Wat Tyler as a play on words and as a hero to revolting peasants like myself   


https://www.dropbox.com/sh/grkg1wvp4ioyve8/AABAf4OAJ1D9RGLnTM4hq0F3a?dl=0

[MJW]

I believe the only way to get the perspective correct is to use homogeneous coordinates. That may sound complicated, but it just involves addition, multiplication, and division.
 
This explanation probably won't be enough to write code from, but it may make it clearer how it works.

You start of with the image coordinates, (x, y). There's an assumed "w" coordinate of 1, so it can be considered to be (x, y, 1). (I call it "w", but others might call it "z". It represents the distance from the viewer.)

The coordinates are transformed to other coordinates using a transformation:

x' = Mxx * x + Mxy * y + Mxw
y' = Myx * x + Myy * y + Myw
w' = Mwx * x + Mwy * y + Mww

The nine values, Mxx, Mxy, Mxw, etc., depend on how the image is transformed; that is, how it's rotated, translated, and scaled.

This is usually thought of as a matrix multiplication, written:

 / x' \     / Mxx Mxy Mxw \   / x \
|  y'  | = |  Myx Myy Myw  | |  y  |
 \ w' /     \ Mwx Mwy Mww /   \ 1 /

Now this has three components, x', y', w', yet you can only display two. The two coordinates are produced by dividing x' and y' by w':

x" = x'/w'
y" = y'/w'

The division is called the "perspective division," and it's what causes the size of objects to decrease as they get farther away.

Here's the tricky part. In PDN plugins, you need to solve the opposite problem. You start with (x", y"), which are the coordinates of the destination pixel, and you need to find the (x, y) coordinates in the source image that will transform to them. Fortunately, it's not too difficult, because the whole process is reversible. The matrix can be "inverted" to produce a new matrix such that:
 
 / x* \     / Mxx' Mxy' Mxw' \   / x" \
|  y*  | = |  Myx' Myy' Myw'  | |  y"  |
 \ w* /     \ Mwx' Mwy' Mww' /   \ 1  /

or,

x* = Mxx' * x" + Mxy' * y" + Mxw"
y* = Myx' * x" + Myy' * y" + Myw"
w* = Mwx' * x" + Mwy' * y" + Mww"

Then the points that are used in src.GetBilinearSampleClamped(x, y) are:

x = x*/w*
y = y*/w*

(if w* is less than or equal to 0, the point is behind the viewer, so it's invisible.)

So, once the values for Mxx', Mxy', Mxw', Myx', Myy', Myw', Mwx', Mwy', Mww' are known, it's just a matter of some multiplications, additions, and divisions.

I don't understand what you're doing in the plugin well enough yet to tell you how to find the values. I downloaded the code, so I can probably figure it out, but it would help if you'd explain it a little.

 

(In case it isn't clear, the primes (' and ") and star (*) are just used to distinguish between different versions of the coordinates.)

 

(Also, one small technical point: because the x and y values are divided by w, all the matrix entries can be scaled by the same amount without affecting the result. Therefore, the transposed co-factor matrix can be used instead of the inverted matrix, which simplifies the computation a bit. Also, the total transformation is usually a combination of easily invertible operations, such as "rotate about the x axis," followed by  "translate in the x and y direction," etc.  The total inverse is just the individual inverses in reverse order.  (Don't worry if that's not clear. I'm just trying to show that the computations involved are usually not particularly difficult.))

[Red ochre]

Thank you MJW for the explanation and the P.M. Unfortunately my knowledge of matrices is terrible!

 

I downloaded the code, so I can probably figure it out, but it would help if you'd explain it a little.

Sorry, there are explanations in the codelab code.
Basically, it calculates the X & Y distances from each src pixel to a moveable centre point. These values are then multiplied for zooming and/or rotation and fed through a method which does the tiling.
Tiling is achieved basically by using the remainder when divided by width or height to access the src image. This part of the effect works perfectly.

The problem is the tilting back into the Z direction.
I was hoping for a simpler 'trigonometric' algorithm for tilt based on the src Y coordinate but fear this is not possible.
Well, it should be possible but I'm clearly going wrong somewhere.'Close but no cigar'.

I will have a think about your notes above and P.M.
If the two systems are compatible it would be useful to have a choice of tiling pattern combined with high quality (super-sampled) perspective tilting.

Thank you again - I will be in touch via P.M. when I've studied your post further (I better read those text books from school in the 1970's !)

[MJW]

I've had a chance to look at the code, and I'll try to fix the perspective transformation soon (sometime today, I hope), then post the modified version or put it somewhere you can retrieve it.

[Red ochre]

Many thanks in advance!
If you can get a result without the perspective lines curving at higher tilt angles that would be brilliant.
Hopefully I could then adapt the existing sub-pixel sampling method to include it.
Then add 'prop rules', icon, sample image etc.

[MJW]

The sub-pixel sampling should be no problem. Just as it currently does, it will take a destination pixel location and transform it into a source pixel location, which can then accessed with the bilinear sample routines. The destination locations can be for sub-pixels as well as pixels.

[Ego Eram Reputo]

I used these transformation routines in my (unfinished) faux landscape generator.

 

               
//    Rotational transformation.
                // Rotates the layer rectangle into a diamond, with top-left corner rotated to the top-middle (i.e. by 45 degrees clockwise).
                // float x1 = (0.5f * (Width + x - y));
                // float y1 = (0.5f * (x + y));
                // However, I'm not rotating...
                float x1 = (float)x;
                float y1 = (float)y;

                //    Project flat 2d point into 3d.
                // Simple 3d projection to convert x, y & height into perspective on a 2d screen.
                // The basic idea behind any perspective projection is to divide horizonal and vertical position by depth so
                // that objects further away appear smaller and vice-versa.

                float x2 = HalfWidth;
                float y2 = HeightMod;
                float z2 = HalfWidth - cVN + (y1 * (float)Amount10); // last float is the apparent view height.  Larger = higher view, smaller = sea level view.
                float x3 = (x1 - HalfWidth) * (float)Amount11; // last is the view width.  Larger = wider field of view.
                float y3 = (Width - y1) * (float)(Amount12/10) + 1.0f; // small float is the distance.  Larger = further away, smaller = closer

                x1 = x2 + x3 / y3;
                x1 = (x1<0) ? 0 : (x1>=Width) ? Width-1 : x1;
                y1 = y2 + z2 / y3;
                y1 = (y1<0) ? 0 : (y1>=Height) ? Height-1 : y1;

                dst[(int)x1,(int)y1] = srcBgra;

cVN is a Value Noise integer. Basically a heightmap value.

[MJW]

So I don't keep you waiting, here is a version that seems to work. I'm still trying to reconcile some differences in this code and Rotate/Zoom in how tilt angles are treated. The Rotate/Zoom angles have to be a little larger to produce the same image. I added an extra control, purely for experimental purposes. It's called Perspective Scale, and amounts to how far away the viewer is from the plane. The larger the value, the less rapidly the lines will converge. I'm not sure the way I handle this is the most sensible, and I don't currency understand how the equivalent feature is handled by Rotate/Zoom. (It's constant in Rotate/Zoom, but I don't know how that constant is determined.)
 

To produce the same distortion as Rotate/Zoom with a 75 degree tilt, I used a 72.3 degree tilt, and a Perspective Scale of 3.43. I don't yet fully understand the computations in Rotate/Zoom. They're kind of confusing, and of course, the disassembly has no comments.

 

As you can see, the changes are pretty minor. Just computing two values in Render, and replacing the perspective code in the transformation. Note I compute W in the perspective code, but don't divide by it till the rotate and zoom are complete. W is unaffected by those transformations, and doesn't affect them.
 

Hidden Content:

// Name: Wat Tyler
// Author: Red ochre (John Robbins)
// Submenu: Test
// URL: http://www.getpaint.net/redirect/plugins.html
// Title: Wat Tyler Bad Tilt Beta    9/5/16 Red Ochre

#region UICode
Pair<double, double> Amount1 = Pair.Create( 0.0 , 0.0 ); // Centre
byte Amount2 = 1; // Tiling options|reflect|repeat|reflect brick|repeat brick|clamp|none
bool Amount3 = true; // [0,1] Limit to integers
double Amount4 = 1; // [1,20] Horizontal(X) zoom out
double Amount5 = 1; // [1,20] Vertical  (Y) zoom out
bool Amount6 = true; // [0,1] Link X & Y zoom
double Amount7 = 0; // [0,90] Tilt back angle BAD!
double Amount8 = 0; // [-180,180] Rotation
bool Amount9 = true; // [0,1] Limit rotation (15 degree steps)
bool Amount10 = false; // [0,1] Faster (lower quality)
double Amount11 = 3.0; //[1,20] Perspective Scale (for demonstation purposes)
#endregion

double PI = Math.PI;
double PI2 = Math.PI/2;
// Methods here

private ColorBgra move (Surface src,float X,float Y,float xoffset,float yoffset,float Xzoom,float Yzoom,float cosZ,float sinZ, double Tangle)

{   
    Rectangle sel = this.EnvironmentParameters.GetSelection(src.Bounds).GetBoundsInt();
    int sL = sel.Left;
    int sR = sel.Right;
    int sT = sel.Top;
    int sB = sel.Bottom;
    int H = sel.Height;
    int W = sel.Width;
    float hW = (float)(W)/2;
    float hH = (float)(H)/2;
 
    ColorBgra Np;
    //let C be the centre of zoom and rotation at a moveable but static position.
    //1. subtract selection top or left
    //2. work out distance to (moveable but static) crosshairs
    //3. distort new xdist & ydist for rotation.
    //4. distort for zoom out
    //5. currently distort for tilt back VERY WRONG!!!
    //   Ideally this would be calculated in Render to change the number of samples made.
    //6. feed new x & y through tesselation methods.
    //7. add back selection top and left and access pixel values to feed back to SSpix method,
    //   where average taken and fed back to Render
    
    
    float Cx =  xoffset + hW;
    float Cy =  yoffset + hH;

    float xdist = X - Cx;//selection.Left not in X (only in x)
    float ydist = Y - Cy;
   //try tilt here to get maths correct--------------------
   //DOES NOT WORK
   // I have tried every possible calculation I can think of here:
   // Tanh, powers etc etc etc. 
   // I always end up with curved perspective lines?
   // It really hurts me to give up but I just don't have the understanding of mathematics to do this correctly
   // 
    //float xwarpT = xdist;
    //float ywarpT = ydist;
    //float Hrat = (H -Y)/H;
    //float iHrat = 1 - Hrat;
  
    //xwarpT = (float)((xdist + iHrat) + (Hrat * Math.Tan(Tangle) * xdist));//curved
    //ywarpT = (float)((ydist + iHrat) + (Hrat * Math.Tan(Tangle) * ydist * 8));//all rubbish
  

    float ywarpT = yTiltScale * ydist;
    float w = wTiltScale * ydist + 1.0f;

    //----------------------------Ends Tilt -------------------
    float xwarpR = (cosZ * xdist) - (sinZ * ywarpT);//Rotation
    float ywarpR = (cosZ * ywarpT) + (sinZ * xdist);

    float xwarpZ = xwarpR * Xzoom;//Zoom out
    float ywarpZ = ywarpR * Yzoom;

    if (w <= 0.0f)
    {
        Np = ColorBgra.Transparent;
    }
    else
    {
        float recipW = 1.0f / w;
        xwarpZ *= recipW;
        ywarpZ *= recipW;
        
        float nxP = xwarpZ + hW;
        float nyP = ywarpZ + hH;
     
        bool Oob = false;//out of bounds for case 5 only
      
        
        float repx = (int)Math.Abs(nxP/W);// NOW FLOAT! (really int but saves loads of boxing/casting)
        float repy = (int)Math.Abs(nyP/H);
       
        //Syntax,value of this = condition ? value if true:value if false
        //REMEMBER selection irrelevant here!!! use W, H & zero only.
   
        switch(Amount2)
        { 
           case 0://reflect - default
            if(nxP < 0 ){nxP =  - nxP;}
            if(nxP >= W ){nxP = repx%2 >= 1 ? W - nxP%W: nxP%W;}
            if(nyP < 0 ){nyP =  - nyP;}
            if(nyP >= H ){nyP = repy%2 >= 1 ? H - nyP%H: nyP%H;}
           break;
    
         
           case 1://repeat
            if(nxP < 0){nxP = W - Math.Abs(nxP%W);}
            if(nxP >= W){nxP = nxP%W;}
            if(nyP < 0){nyP = H - Math.Abs(nyP%H);}
            if(nyP >= H){nyP = nyP%H;}
           break;
    
           case 2://brick reflect
            if(nyP >= 0){repy += 1;}//must be first
            if(repy%2 == 0 ){nxP = hW + nxP;}
            
            repy = (int)Math.Abs(nyP/H);//must be re-calculated as changed values
            repx = (int)Math.Abs(nxP/W);
            if(nxP < 0){nxP = - nxP;}
            if(nxP >= W){nxP = repx%2 >= 1 ? W - nxP%W: nxP%W;}
            if(nyP < 0){nyP =  - nyP;}
            if(nyP >= H){nyP = repy%2 >= 1 ? H - nyP%H: nyP%H ;}
           break;
    
           case 3://brick repeat
            nyP = nyP + 0.0000001f;//bodge!!!! - no idea where the real bug is?
            if(nyP > 0){repy += 1;}
            if(repy%2 == 0){nxP = hW + nxP;}
            if(nxP < 0){nxP = W - Math.Abs(nxP%W);}
            nxP = nxP%W;
            if(nyP < 0){nyP = H - Math.Abs(nyP%H);}
            nyP = nyP%H;
           break;
    
           case 4://clamp
            if(nxP < sL){nxP = sL;}
            if(nxP > sR){nxP = sR;}
            if(nyP < sT){nyP = sT;}
            if(nyP > sB){nyP = sB;}
           break;
           case 5://none
            if(nxP < sL || nxP > sR || nyP < sT || nyP > sB){Oob = true;}
           break;
           } 
        
        nxP = nxP + sL;//add selection top and left back in
        nyP = nyP + sT;
      
        ColorBgra klear = ColorBgra.Transparent;
        Np = src.GetBilinearSampleClamped(nxP,nyP);
        if(Oob){Np = klear;}
    }
    return Np;
}

 private ColorBgra SSpix(Surface src,int x, int y,float Xstep,float Ystep, int xsamples, int ysamples,float xoffset,float yoffset,float Xzoom,float Yzoom,float cosZ,float sinZ,double Tangle)
{    //Sub-pixel Sampling method
     ColorBgra SStemp = ColorBgra.Aquamarine;
     int SSno = xsamples * ysamples;
     int B = 0;int G = 0;int R = 0;int A = 0;
     float tempB = 0;float tempG = 0;float tempR = 0;float tempA = 0;
     float X = 0;float Y = 0;
     double Trat = 0;
     Rectangle sel = EnvironmentParameters.GetSelection(src.Bounds).GetBoundsInt();
     double H = sel.Height;
      for (int Ys = 0; Ys < ysamples;Ys++)
            {
               Y = (y - sel.Top) + (Ys * Ystep);
             
        
              for (int Xs = 0; Xs < xsamples;Xs++)
                {
                    X = (x - sel.Left) + (Xs * Xstep);//note: now float

                    SStemp = move(src, X, Y, xoffset, yoffset, Xzoom, Yzoom, cosZ, sinZ, Tangle);

                    tempB += SStemp.B;
                    tempG += SStemp.G;
                    tempR += SStemp.R;
                    tempA += SStemp.A;// sum values 
                }
  
            }
             
            B = (int)(tempB/SSno);//end of SPS loop... divide by number of pixels sampled to find average
            G = (int)(tempG/SSno);
            R = (int)(tempR/SSno);
            A = (int)(tempA/SSno);

     return ColorBgra.FromBgra(Int32Util.ClampToByte(, Int32Util.ClampToByte(G), Int32Util.ClampToByte(R), Int32Util.ClampToByte(A));

}

// There's nothing wrong with using "global" variables (which are actually class variables)
// to communicate with subroutines, provided that the values never change during the
// rendering process. The values are state, not arguments. It seems to me to be so much
// neater than passing a huge number of unchanging arguments.
float yTiltScale;
float wTiltScale;

void Render(Surface dst, Surface src, Rectangle rect)
{
    
    Rectangle sel = EnvironmentParameters.GetSelection(src.Bounds).GetBoundsInt();
    float W = (float) sel.Width;
    float hW = (float)(sel.Width/2);
    float hH = (float)(sel.Height/2);
    float H = (float)sel.Height;
   
    int B = 0;
    int G = 0;
    int R = 0;
    int A = 0;
    int tempB = 0;
    int tempG = 0;
    int tempR = 0;
    int tempA = 0;

    ColorBgra cp = ColorBgra.Aquamarine; //just to declare a value
  
    float xoffset = (float)(Amount1.First * hW);
    float yoffset = (float)(Amount1.Second * hH);
    float Xzoom = (float)Amount4;
    float Yzoom = (float)Amount5;if(Amount6){Yzoom = (float)Amount4;}
    if(Amount3){Xzoom = (int)(Xzoom);Yzoom = (int)(Yzoom);}
    int xsamples = (int)Xzoom;
    int ysamples = (int)Yzoom;
    float Xstep = (float)(1.0/xsamples);
    float Ystep = (float)(1.0/ysamples);

    double Zrot = PI * Amount8/180;//all in radians now
    if(Amount9){int uaf = (int)(Amount8/15);Zrot = PI * (uaf * 15)/180;}//confine to multiples of 7.5 degrees
    float cosZ = (float)Math.Cos(-Zrot);
    float sinZ = (float)Math.Sin(-Zrot);//just prefer things rotating as the slider does

    double Tangle = PI * Amount7/180;//-ve halfPI to +ve PI 
    double tanT = Math.Tan(Tangle), cosT = Math.Cos(Tangle);
    yTiltScale = (float)(1.0 / cosT);
    wTiltScale = (float)(tanT) / ((float)Amount11 * hH);
    
    for (int y = rect.Top; y < rect.Bottom; y++)
    {
      
        for (int x = rect.Left; x < rect.Right; x++)
        {
          double Trat = (H - (y - sel.Top))/H;
             
            if(Amount10){xsamples = ysamples = 1;}
            cp = SSpix(src,x,y,Xstep,Ystep,xsamples,ysamples,xoffset,yoffset,Xzoom,Yzoom,cosZ,sinZ,Tangle);//call Sub-pixel sampling method here
          

            dst[x,y] = cp;
        }
    }
}

 

EDIT: I'm convinced I'm not doing the perspective transformation correctly. The foreshortening is wrong, so the image stretches in Y as the tilt increases. I have an idea why, but I'll need to do a little math. I tried to derive it using 3x3 matrices instead of the usual 4x4, and I think it led me astray.

 

EDIT 2: What a silly mistake! The posted code seems just fine. I was experimenting with something in my version, and managed to use the wrong version of the Y coordinate at one step in the process. Arghh!

Edited May 13, 2016 by MJW

[Red ochre]

I'm a little pushed for time at the moment but couldn't resist downloading and compiling.
Excellent initial results. Straight perspective lines! Hurray!
I will get back to you when I've studied it a bit more but in the meantime many, many thanks! :star: :star: