Dmytro Danylyk

Android Platform Software Developer

at  Lemberg Solutions Limited


In love with Git and IntelliJ IDEA.
Big fun of stackoverflow with more than 4000 reputation.
Author of technical articles for androidweekly.net, mobile.tutsplus.com, blog.lemberg.co.uk. 


@dmytrodanylyk
@lemberg_co_uk


To Intent, or not to Intent, that is the question?


William Shakespeare, Hamlet 
(re-factored)


For never was a story of more woe than this of SurfaceView and Me.


William Shakespeare, Romeo and Juliet 
(re-factored)


Act 1, Scene: 1
First meet


Act 1, Scene: 2

More than friends?


"Surface view creates a new window, placed behind your application’s window, to manage content."

Act 1, Scene: 3

Oh God, Why?


  • unexpectable behavior inside scrollable container
  • unexpectable behavior with animations
  • no way to crop content

Act 1, Scene: 4 
Serious conversation


Act 2, Scene: 1

Texture View


  • API level 14
  • Hardware Acceleration

Act 2, Scene: 2

Benefits


"Texture View - does not create a separate window but behaves as a regular view. This key difference allows a texture view to be moved, transformed, animated, etc."

"Because it uses hardware accelerated 2D rendering - it is so fast and efficient."

Act 3: Playing video from assets


AndroidManifest.xml
 <uses-sdk android:minSdkVersion="14"
           android:targetSdkVersion="14"/>

texture_video_simple.xml
<TextureView
	android:id="@+id/textureView"
	android:layout_width="wrap_content"
	android:layout_height="wrap_content">

Activity Structure

public class CameraSimpleActivity extends Activity
        implements TextureView.SurfaceTextureListener {
// Log tag
private static final String TAG = VideoAssetActivity.class.getName();

// Asset video file name.
private static final String FILE_NAME = "big_buck_bunny.mp4";

// MediaPlayer instance to control playback of video file.
private MediaPlayer mMediaPlayer;

@Override
public void onCreate(Bundle savedInstanceState) {
	super.onCreate(savedInstanceState);
	setContentView(R.layout.texture_video_simple);

	initView();
}
private void initView() {
	TextureView textureView = findViewById(R.id.textureView);
	textureView.setSurfaceTextureListener(this);
}

@Override
public void onSurfaceTextureSizeChanged(SurfaceTexture surfaceTexture, int i, int i2) {
    // ignore
}
@Override
public void onSurfaceTextureUpdated(SurfaceTexture surfaceTexture) {
    // ignore
}
@Override
public boolean onSurfaceTextureDestroyed(SurfaceTexture surfaceTexture) {
    return true; 
}

@Override
public void onSurfaceTextureAvailable(SurfaceTexture surfaceTexture, int i, int i2) {     

Surface surface = new Surface(surfaceTexture);
AssetFileDescriptor afd = getAssets().openFd(FILE_NAME);
mMediaPlayer = new MediaPlayer();
mMediaPlayer.setDataSource(
    afd.getFileDescriptor(), afd.getStartOffset(), afd.getLength());
mMediaPlayer.setSurface(surface);
mMediaPlayer.setLooping(true);
mMediaPlayer.prepareAsync();
// Play video when the media source is ready for playback.
mMediaPlayer.setOnPreparedListener(new MediaPlayer.OnPreparedListener() {
    @Override
    public void onPrepared(MediaPlayer mediaPlayer) {
    	mediaPlayer.start();
	}
});
}  

Memory cleanup


@Override
protected void onDestroy() {
    super.onDestroy();
    if (mMediaPlayer != null) {
        mMediaPlayer.stop();
        mMediaPlayer.release();
        mMediaPlayer = null;
    }
}

Act 4: Video Cropping


When user click on screen:
  • resize view
  • crop video

Video Resize

Video Crop

XML

 <FrameLayout 
		xmlns:android="http://schemas.android.com/apk/res/android"
		android:layout_width="match_parent"
		android:layout_height="match_parent"
		android:id="@+id/rootView">

    <TextureView
            android:id="@+id/textureView"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"/>
</FrameLayout>

View resizing

initView();
mTextureView = (TextureView)findViewById(R.id.textureView); mTextureView.setSurfaceTextureListener(this);
FrameLayout rootView = (FrameLayout) findViewById(R.id.rootView);
rootView.setOnTouchListener(new View.OnTouchListener() {
    @Override
    public boolean onTouch(View view, MotionEvent motionEvent) {
		switch (motionEvent.getAction()) {
			case MotionEvent.ACTION_UP:
	           updateTextureViewSize(                       motionEvent.getX(),                        motionEvent.getY());
	        break;
	    }
	return true;
    }
});
private void updateTextureViewSize(float viewWidth, float viewHeight) {
    mTextureView.setLayoutParams(new FrameLayout.LayoutParams(           (int)viewWidth, (int)viewHeight));
}

 

Preview

To crop video we need to know original video size
// Original video size
private float mVideoWidth;
private float mVideoHeight;
Call this method to initialize mVideoWidth and mVideoHeight variables
private void calculateVideoSize() {
	AssetFileDescriptor afd = getAssets().openFd(FILE_NAME);
	
	MediaMetadataRetriever metaRetriever 
	 	= new MediaMetadataRetriever();
	metaRetriever.setDataSource(afd.getFileDescriptor(), 
		afd.getStartOffset(), afd.getLength());
	
	String height = metaRetriever.extractMetadata(
	 	MediaMetadataRetriever.METADATA_KEY_VIDEO_HEIGHT);
	
	String width = metaRetriever.extractMetadata(
	 	MediaMetadataRetriever.METADATA_KEY_VIDEO_WIDTH);
	
	mVideoHeight = Float.parseFloat(height);
	mVideoWidth = Float.parseFloat(width);
}
Change updateTextureViewSize method
 private void updateTextureViewSize(float viewWidth, float viewHeight) {
float scaleX = 1.0f;
float scaleY = 1.0f;
if (mVideoWidth > viewWidth && mVideoHeight > viewHeight) {
	scaleX = mVideoWidth / viewWidth;
	scaleY = mVideoHeight / viewHeight;
} else if (mVideoWidth < viewWidth && mVideoHeight < viewHeight) {
	scaleY = viewWidth / mVideoWidth;
	scaleX = viewHeight / mVideoHeight;
} else if (viewWidth > mVideoWidth) {
	scaleY = (viewWidth / mVideoWidth) / (viewHeight / mVideoHeight);
} else if (viewHeight > mVideoHeight) {
	scaleX = (viewHeight / mVideoHeight) / (viewWidth / mVideoWidth);
}
int pivotPointX = (int) (viewWidth / 2); // Calculate pivot points, 
int pivotPointY = (int) (viewHeight / 2;) // in our case crop from center
Matrix matrix = new Matrix();
matrix.setScale(scaleX, scaleY, pivotPointX, pivotPointY);

mTextureView.setTransform(matrix);
mTextureView.setLayoutParams(new FrameLayout.LayoutParams((int) viewWidth, (int) viewHeight));

Preview

Act 5: Camera

Example from: developer.android.com

AndroidManifest.xml
<uses-sdk
            android:minSdkVersion="14"
            android:targetSdkVersion="14"/>

<uses-feature android:name="android.hardware.camera"/><uses-permission android:name="android.permission.CAMERA"/>

Activity

public class CameraSimpleActivity extends Activity
        implements TextureView.SurfaceTextureListener {
// Log tag
private static final String TAG = CameraSimpleActivity.class.getName();

private Camera mCamera;

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    TextureView textureView = new TextureView(this);
    textureView.setSurfaceTextureListener(this);

    setContentView(textureView);
}
@Override
public void onSurfaceTextureAvailable(SurfaceTexture surface, int width, int height) {
    Log.d(TAG, "SurfaceTexture width: " + width + " height: " + height);

    mCamera = Camera.open();
    try {
        mCamera.setPreviewTexture(surface);
        mCamera.startPreview();
    } catch (IOException e) {
        Log.d(TAG, e.getMessage());
    }
}
@Override
public void onSurfaceTextureSizeChanged(SurfaceTexture surface, int width, int height) {
    // Ignored, Camera does all the work for us
}
 @Override
public void onSurfaceTextureUpdated(SurfaceTexture surface) {
    // Invoked every time there's a new Camera preview frame
}

Memory cleanup


@Override
public boolean onSurfaceTextureDestroyed(SurfaceTexture surface) {
    if (mCamera != null) {
        mCamera.stopPreview();
        mCamera.release();
    }
    return true;
}

How to fight 

camera issues?

Prevent from appearing in 

Google Play


Adding camera features to your manifest causes Google Play to prevent your application from being installed to devices that do not include a camera.


<uses-feature android:name="android.hardware.camera" android:required="true" />

Check if device has camera


User may install and launch application on the device which doesn't have camera, to prevent your app from crash you should check if device has camer.  


if (getPackageManager().hasSystemFeature(PackageManager.FEATURE_CAMERA)) {
    // This device has a Camera
} else {
    // This device doesn't have a Camera
}

Check if camera is available


Camera could be locked by another application, you should wait until it becomes available. Consider initializing camera in Activity onResume() method.


Camera c = null;
try {
    c = Camera.open(); // attempt to get a Camera instance
}catch (Exception e){
    // Camera is not available
}

Lock activity screen orientation


While camera is launched user may change screen orientation which lead to activity re-creation. To prevent this, lock activity in landscape or portrait mode.


<activity
        android:name=".CameraActivity"
        android:screenOrientation="landscape"/>

Calculate camera display orientation

Calculate preview size

private Camera.Size getBestPreviewSize( int requiredWidth, 
        int requiredHeight, Camera.Parameters parameters) {
Camera.Size result = null;
for (Camera.Size currentSize : parameters.getSupportedPreviewSizes()) {
    if (currentSize.width <= requiredWidth
            && currentSize.height <= requiredHeight) {

        if (result == null) {
            result = currentSize;
        } else {
            int resultArea = result.width * result.height;
            int newArea = currentSize.width * currentSize.height;

            if (newArea > resultArea) {
                result = currentSize;
            }
        }
}
   return result;}

Check if feature is available before using it


 // get Camera parameters
Camera.Parameters params = mCamera.getParameters();

List<String> focusModes = params.getSupportedFocusModes();
if (focusModes.contains(Camera.Parameters.FOCUS_MODE_AUTO)) {
    // Auto-focus mode is supported
    params.setFocusMode(Camera.Parameters.FOCUS_MODE_AUTO);
    // set Camera parameters
    mCamera.setParameters(params);
}

Don't lock camera forever, you are not alone

User may not exit your app, but just switch to another application. If you didn't release camera in Activity onPause() method you will prevent camera use for any other application.


@Override
protected void onPause() {
    // release the camera for other applications   
    if (mCamera != null) {
        mCamera.release();        
        mCamera = null;
    }    
}

Real-life project: AR. Drone

by Lemberg Solutions


AR. Drone: Piloting


AR. Drone: Media Gallery


Questions?


Detailed article about how to use Texture View will be published on

&

GDG Dev Fest

By Dmytro Danylyk

Loading comments...

More from Dmytro Danylyk