AR vs VR

Augmented reality (AR) is a live direct or indirect view of a physical, real-world environment whose elements are augmented (or supplemented) by computer-generated sensory input such as sound, video, graphics or GPS data. By contrast, virtual reality replaces the real world with a simulated one.

Stages

• Registration
• Recognition
• Tracking

Sensor-based AR 

• GPS
• RFID
• Bluetooth/Beacons
• e.t.c

Sensor-based AR 

Coordinate system

• Latitude
• Longitude
• Altitude

Sensors (1)

• TYPE_ACCELEROMETER
• TYPE_GRAVITY
• TYPE_GYROSCOPE
• TYPE_LINEAR_ACCELERATION
• TYPE_ROTATION_VECTOR
• TYPE_GEOMAGNETIC_ROTATION_VECTOR 

Sensors (2)

Sensors (3)

Camera

https://www.built.io/blog/2013/05/applying-low-pass-filter-to-android-sensors-readings/

https://www.netguru.co/blog/augmented-reality-mobile-android

http://www.wikitude.com/?gclid=CP6AhZbGk84CFUQo0wodec8IWA

https://www.layar.com/

http://www.pokemon.com/us/pokemon-video-games/pokemon-go/

https://play.google.com/store/apps/details?id=com.nianticproject.ingress

• Calibration
• Pre-processing
• Recognition
• Tracking

Computer vision-based AR

Theory

        mFOVY = cameraParameters.getVerticalViewAngle();
        mFOVX = cameraParameters.getHorizontalViewAngle();
        
        mHeightPx = imageSize.height;
        mWidthPx = imageSize.width;
        
        mProjectionDirtyGL = true;
        mProjectionDirtyCV = true;


 final float fovAspectRatio = mFOVX / mFOVY;
 final double diagonalPx = Math.sqrt(
              (Math.pow(mWidthPx, 2.0) +
               Math.pow(mWidthPx / fovAspectRatio, 2.0)));
 final double focalLengthPx = 0.5 * diagonalPx / Math.sqrt(
              Math.pow(Math.tan(0.5 * mFOVX * Math.PI / 180f), 2.0) +
               Math.pow(Math.tan(0.5 * mFOVY * Math.PI / 180f), 2.0));
            

Calibration

http://docs.opencv.org/2.4/doc/tutorials/calib3d/camera_calibration/camera_calibration.html#cameracalibrationopencv

Pre-processing(1)

http://docs.opencv.org/2.4/doc/tutorials/introduction/load_save_image/load_save_image.html​

Pre-processing(2)

Canny edge detector

Blob detector

http://docs.opencv.org/trunk/da/d22/tutorial_py_canny.html#gsc.tab=0

http://www.shervinemami.info/blobs.html

Pre-processing(3)

Feature Detection

• Harris Corner Detection
• Shi-Tomasi Corner Detector
• SIFT (Scale-Invariant Feature Transform)
• SURF (Speeded-Up Robust Features)
• FAST Algorithm for Corner Detection
• BRIEF (Binary Robust Independent Elementary Features)
• ORB (Oriented FAST and Rotated BRIEF)

http://docs.opencv.org/3.0-beta/doc/py_tutorials/py_feature2d/py_table_of_contents_feature2d/py_table_of_contents_feature2d.html

Recognition(1)

Common case

https://www.ece.cmu.edu/~ee551/projects/S12/Final_Report_Group7.pdf

Recognition(2)

Face detection

Haar Vs LBP

http://docs.opencv.org/2.4/doc/user_guide/ug_traincascade.html

Tracking

RANSAC Vs Least Median

http://www.ee.ucl.ac.uk/lcs/previous/LCS2011/LCS1115.pdf

http://opencv.org/

http://www.vuforia.com/

https://www.thingworx.com/

Virtual Reality

Cardboard

@Override
public void onDrawEye(Eye eye) {
    GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT);
    ...
    // Apply the eye transformation to the camera.
    Matrix.multiplyMM(mView, 0, eye.getEyeView(), 0, mCamera, 0);

    // Set the position of the light
    Matrix.multiplyMV(mLightPosInEyeSpace, 0, mView, 0, LIGHT_POS_IN_WORLD_SPACE, 0);

    // Build the ModelView and ModelViewProjection matrices
    // for calculating cube position and light.
    float[] perspective = eye.getPerspective(Z_NEAR, Z_FAR);
    Matrix.multiplyMM(mModelView, 0, mView, 0, mModelCube, 0);
    Matrix.multiplyMM(mModelViewProjection, 0, perspective, 0, mModelView, 0);
    drawCube();

    // Draw the rest of the scene.
    ...
}

https://developers.google.com/vr/