Week 4 of The Build: Part 1 - Paper Tracking & Projection

This week, we made big steps in tracking and projecting onto paper. We also started developing the use-cases that we will demonstrate. I'll start with paper tracking.

Paper tracking & projection:

After calibration, we use the Kinect and OpenCV to detect the paper. The contour functions of OpenCV are good for detecting the paper. To find contours, OpenCV defines a curve that joins all the continuous points along the boundary of an object that has the same colour or intensity [1]. To work best, there should be a big contrast between the object you want to detect and the background. Therefore, we bought a black cloth to put behind the white paper. When a threshold [2] is applied, there is a clear distinction between the paper and the background.

The contours found by OpenCV can be many. Therefore, we applied a contour approximation method to remove the redundant points on the contour. With the points of the contour and the Kinect depth information, we get coordinates for the paper in world space. With a set of transformations using the view matrix and projection matrix, the projector is then able to project onto the paper.

Below is a video demonstrating that we are able to track the paper and project onto it. To further develop this, we need to handle rotations as currently, the system doesn't handle the paper being rotated well.

[1] "Contours : Getting Started", OpenCV 3.0.0 Documentation, http://docs.opencv.org/trunk/doc/py_tutorials/py_imgproc/py_contours/py_contours_begin/py_contours_begin.html
[2] "Basic Thresholding Operations", OpenCV 2.4.9.0 Documentation, http://docs.opencv.org/doc/tutorials/imgproc/threshold/threshold.html