Image Warping(Affine transformation)

What is Image Warping?
It is a technique to manipulate the digital images by changing its shape ,size and dimensions.It can be used to distort or to correct distortness depending upon the need of the task.
So we need a transformation matrix (often called geometric transform) which maps the input image(source image) to a transformed output one(target image). There are various transformation functions present but we will discuss about Affine transformation here.

Affine Transformation:

It is a Linear transformation function which performs Scaling , Rotation , Shear , Translation or combination of all the above on the source image by preserving its structure and distance ratio.It uses a 2*3 matrix for transformation.

Before applying Affine Transformation pixel coordinates is converted into Homogenous Coordinates for the ease of Transforamtion. By saying Homogenous coordinate it simply means adding a dimension to the raw coordinate of the image. Image works on a 2-D workspace so its homogenous coordinate will be extended to 3 dimensions.

                         X = (x,y)       ⇒     X’ = (x,y,1)

Once coordinates has been converted into Homogenous coordinates now Affine transformation can be applied to it for different types of manipulation on the source image

If you see deeply in these transformation you will observe that it only maps the coordinate of the source image into a new coordinate based on a particular transformation and the pixel intensity for each previous coordinate will now belong to newer one after the transformation in the target image.

Let’s take an example of scaling an image uniformly with a factor of 3 so an image of (3,3) will now become (9,9)

As you can see that most of the coordinates doesn’t get any value and hence information is lost due to the digitalisation and sampling of the image.
For calculating the values of missing coordinate different interpolation is used like Nearest neighbour, Bilinear, Bicubic e.t.c.

In Nearest neighbour the missing coordinate gets the value of its nearest coordinate while in Bilinear interpolation the value is obtained by the weighted sum of its nearest coordinate depending upon its distance from it.

Calculating the transformation matrix:

Till now we transform the image using the transformation matrix which is already known to us.But here we will try to find out the coefficient of the transformation matrix by observing the target image and the source image.
Let’s suppose that we have a source image and its transformed image and we want to obtain the transformation matrix regarding that. Since for 2-D image transformation matrix has 6 degree of freedom i.e 6 unknown values so to determine it we need 6 equations.
Each point can give us 2 equation (one for x and one for y) so we need 3 points in the source image and same 3 points after transformation in the target image. So by comparing these 3 points in both image we can get all 6 values of the matrix and hence transformation matrix can be calculated.

Where does it used in Computer vision problem…….?

In Object Detection algorithm R-CNN , We use a deep ConvNet as a feature extractor for the image.The extracted feature vector for each image get fed to the SVM classifier for the recognition purpose.
However the ConvNet which we are using as a feature extractor is trained on a large databases of images of fixed dimension. For extracting features of the test images(or the proposed region in case of R-CNN which can be of varying dimension) we need to first map it into the size in which that Network has been trained on , so here image warping is used specially affine transformation.

References:

Learning OpenCV — Gary Bradski and Adrian Kaehler
CS6640: Image Processing(Affine Transformation, Landmarks registration, Non linear Warping)
— Arthur Coste: coste.arthur@gmail.com