Introduction into Augmented Reality
Illya Rodin
AR vs VR
Augmented reality (AR) is a live direct or indirect view of a physical, real-world environment whose elements are augmented (or supplemented) by computer-generated sensory input such as sound, video, graphics or GPS data. By contrast, virtual reality replaces the real world with a simulated one.
Stages
- Registration
- Recognition
- Tracking
Sensor-based AR
- GPS
- RFID
- Bluetooth/Beacons
- e.t.c
Sensor-based AR
Coordinate system
- Latitude
- Longitude
- Altitude
Sensors (1)
- TYPE_ACCELEROMETER
- TYPE_GRAVITY
- TYPE_GYROSCOPE
- TYPE_LINEAR_ACCELERATION
- TYPE_ROTATION_VECTOR
- TYPE_GEOMAGNETIC_ROTATION_VECTOR
Sensors (2)
Sensors (3)
Camera
- Calibration
- Pre-processing
- Recognition
- Tracking
Computer vision-based AR
Theory
mFOVY = cameraParameters.getVerticalViewAngle();
mFOVX = cameraParameters.getHorizontalViewAngle();
mHeightPx = imageSize.height;
mWidthPx = imageSize.width;
mProjectionDirtyGL = true;
mProjectionDirtyCV = true;
final float fovAspectRatio = mFOVX / mFOVY;
final double diagonalPx = Math.sqrt(
(Math.pow(mWidthPx, 2.0) +
Math.pow(mWidthPx / fovAspectRatio, 2.0)));
final double focalLengthPx = 0.5 * diagonalPx / Math.sqrt(
Math.pow(Math.tan(0.5 * mFOVX * Math.PI / 180f), 2.0) +
Math.pow(Math.tan(0.5 * mFOVY * Math.PI / 180f), 2.0));
Calibration
Pre-processing(1)
Pre-processing(2)
Canny edge detector
Blob detector
Pre-processing(3)
Feature Detection
- Harris Corner Detection
- Shi-Tomasi Corner Detector
- SIFT (Scale-Invariant Feature Transform)
- SURF (Speeded-Up Robust Features)
- FAST Algorithm for Corner Detection
- BRIEF (Binary Robust Independent Elementary Features)
- ORB (Oriented FAST and Rotated BRIEF)
Recognition(1)
Common case
Recognition(2)
Face detection
Haar Vs LBP
Tracking
RANSAC Vs Least Median
Virtual Reality
Cardboard
@Override
public void onDrawEye(Eye eye) {
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT);
...
// Apply the eye transformation to the camera.
Matrix.multiplyMM(mView, 0, eye.getEyeView(), 0, mCamera, 0);
// Set the position of the light
Matrix.multiplyMV(mLightPosInEyeSpace, 0, mView, 0, LIGHT_POS_IN_WORLD_SPACE, 0);
// Build the ModelView and ModelViewProjection matrices
// for calculating cube position and light.
float[] perspective = eye.getPerspective(Z_NEAR, Z_FAR);
Matrix.multiplyMM(mModelView, 0, mView, 0, mModelCube, 0);
Matrix.multiplyMM(mModelViewProjection, 0, perspective, 0, mModelView, 0);
drawCube();
// Draw the rest of the scene.
...
}
Q&A
Introduction into Augmented Reality
By ris
Introduction into Augmented Reality
- 3,017