16714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis/* 26714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * Copyright (C) 2010 The Android Open Source Project 36714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * 46714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * Licensed under the Apache License, Version 2.0 (the "License"); 56714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * you may not use this file except in compliance with the License. 66714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * You may obtain a copy of the License at 76714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * 86714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * http://www.apache.org/licenses/LICENSE-2.0 96714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * 106714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * Unless required by applicable law or agreed to in writing, software 116714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * distributed under the License is distributed on an "AS IS" BASIS, 126714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 136714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * See the License for the specific language governing permissions and 146714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * limitations under the License. 156714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis */ 166714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis 176714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennispackage android.graphics; 186714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis 19ba873d21718b0c0bbeabbb3b4796e54073739700Jeff Brownimport android.annotation.Nullable; 20376590d668e22a918439877b55faf075427b13f3Jamie Gennisimport android.os.Handler; 21376590d668e22a918439877b55faf075427b13f3Jamie Gennisimport android.os.Looper; 22376590d668e22a918439877b55faf075427b13f3Jamie Gennisimport android.os.Message; 23a86ab640f7bb0bf3cb4eaed80473ca8c5d131903Igor Murashkinimport android.view.Surface; 24376590d668e22a918439877b55faf075427b13f3Jamie Gennis 25f4612191855b999be7b8b2402fd7b45f07aa95f4Chris Craikimport java.lang.ref.WeakReference; 26f4612191855b999be7b8b2402fd7b45f07aa95f4Chris Craik 276714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis/** 286714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * Captures frames from an image stream as an OpenGL ES texture. 296714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * 30b942b05093d2b1cee59ac73196a4b99962f10addEino-Ville Talvala * <p>The image stream may come from either camera preview or video decode. A 31b942b05093d2b1cee59ac73196a4b99962f10addEino-Ville Talvala * {@link android.view.Surface} created from a SurfaceTexture can be used as an output 32b942b05093d2b1cee59ac73196a4b99962f10addEino-Ville Talvala * destination for the {@link android.hardware.camera2}, {@link android.media.MediaCodec}, 33b942b05093d2b1cee59ac73196a4b99962f10addEino-Ville Talvala * {@link android.media.MediaPlayer}, and {@link android.renderscript.Allocation} APIs. 34b942b05093d2b1cee59ac73196a4b99962f10addEino-Ville Talvala * When {@link #updateTexImage} is called, the contents of the texture object specified 35b942b05093d2b1cee59ac73196a4b99962f10addEino-Ville Talvala * when the SurfaceTexture was created are updated to contain the most recent image from the image 36b942b05093d2b1cee59ac73196a4b99962f10addEino-Ville Talvala * stream. This may cause some frames of the stream to be skipped. 37b942b05093d2b1cee59ac73196a4b99962f10addEino-Ville Talvala * 38b942b05093d2b1cee59ac73196a4b99962f10addEino-Ville Talvala * <p>A SurfaceTexture may also be used in place of a SurfaceHolder when specifying the output 39b942b05093d2b1cee59ac73196a4b99962f10addEino-Ville Talvala * destination of the older {@link android.hardware.Camera} API. Doing so will cause all the 40b942b05093d2b1cee59ac73196a4b99962f10addEino-Ville Talvala * frames from the image stream to be sent to the SurfaceTexture object rather than to the device's 41b942b05093d2b1cee59ac73196a4b99962f10addEino-Ville Talvala * display. 426714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * 435f8b6653e0b5b99097db7d8f41d5251f7b398704Jamie Gennis * <p>When sampling from the texture one should first transform the texture coordinates using the 448f0095cd33558e9cc8a440047908e53b68906f5fRomain Guy * matrix queried via {@link #getTransformMatrix(float[])}. The transform matrix may change each 458f0095cd33558e9cc8a440047908e53b68906f5fRomain Guy * time {@link #updateTexImage} is called, so it should be re-queried each time the texture image 468f0095cd33558e9cc8a440047908e53b68906f5fRomain Guy * is updated. 475f8b6653e0b5b99097db7d8f41d5251f7b398704Jamie Gennis * This matrix transforms traditional 2D OpenGL ES texture coordinate column vectors of the form (s, 485f8b6653e0b5b99097db7d8f41d5251f7b398704Jamie Gennis * t, 0, 1) where s and t are on the inclusive interval [0, 1] to the proper sampling location in 495f8b6653e0b5b99097db7d8f41d5251f7b398704Jamie Gennis * the streamed texture. This transform compensates for any properties of the image stream source 505f8b6653e0b5b99097db7d8f41d5251f7b398704Jamie Gennis * that cause it to appear different from a traditional OpenGL ES texture. For example, sampling 515f8b6653e0b5b99097db7d8f41d5251f7b398704Jamie Gennis * from the bottom left corner of the image can be accomplished by transforming the column vector 525f8b6653e0b5b99097db7d8f41d5251f7b398704Jamie Gennis * (0, 0, 0, 1) using the queried matrix, while sampling from the top right corner of the image can 535f8b6653e0b5b99097db7d8f41d5251f7b398704Jamie Gennis * be done by transforming (1, 1, 0, 1). 545f8b6653e0b5b99097db7d8f41d5251f7b398704Jamie Gennis * 556714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * <p>The texture object uses the GL_TEXTURE_EXTERNAL_OES texture target, which is defined by the 56858873271250e80a704c29c71ff0578a87bd9d31Jamie Gennis * <a href="http://www.khronos.org/registry/gles/extensions/OES/OES_EGL_image_external.txt"> 57858873271250e80a704c29c71ff0578a87bd9d31Jamie Gennis * GL_OES_EGL_image_external</a> OpenGL ES extension. This limits how the texture may be used. 58858873271250e80a704c29c71ff0578a87bd9d31Jamie Gennis * Each time the texture is bound it must be bound to the GL_TEXTURE_EXTERNAL_OES target rather than 59858873271250e80a704c29c71ff0578a87bd9d31Jamie Gennis * the GL_TEXTURE_2D target. Additionally, any OpenGL ES 2.0 shader that samples from the texture 60858873271250e80a704c29c71ff0578a87bd9d31Jamie Gennis * must declare its use of this extension using, for example, an "#extension 61858873271250e80a704c29c71ff0578a87bd9d31Jamie Gennis * GL_OES_EGL_image_external : require" directive. Such shaders must also access the texture using 62858873271250e80a704c29c71ff0578a87bd9d31Jamie Gennis * the samplerExternalOES GLSL sampler type. 6337cec0fc50760fe89614863eded14011f9412534Jamie Gennis * 6437cec0fc50760fe89614863eded14011f9412534Jamie Gennis * <p>SurfaceTexture objects may be created on any thread. {@link #updateTexImage} may only be 6537cec0fc50760fe89614863eded14011f9412534Jamie Gennis * called on the thread with the OpenGL ES context that contains the texture object. The 6637cec0fc50760fe89614863eded14011f9412534Jamie Gennis * frame-available callback is called on an arbitrary thread, so unless special care is taken {@link 6737cec0fc50760fe89614863eded14011f9412534Jamie Gennis * #updateTexImage} should not be called directly from the callback. 686714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis */ 696714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennispublic class SurfaceTexture { 70c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown private final Looper mCreatorLooper; 71c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown private Handler mOnFrameAvailableHandler; 72376590d668e22a918439877b55faf075427b13f3Jamie Gennis 738f0095cd33558e9cc8a440047908e53b68906f5fRomain Guy /** 74c99db2bc460cc795947d99076da380e22a21e493Igor Murashkin * These fields are used by native code, do not access or modify. 758f0095cd33558e9cc8a440047908e53b68906f5fRomain Guy */ 7672aa313ff4c91e7b2aae3d37067f9201b2b0fdbeAshok Bhat private long mSurfaceTexture; 770dc146be5a6cd0c33910d5b18885df46873a93cbDan Stoza private long mProducer; 7872aa313ff4c91e7b2aae3d37067f9201b2b0fdbeAshok Bhat private long mFrameAvailableListener; 796714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis 80aff2f949395725f4bb82802bd12201b65d514e68Pablo Ceballos private boolean mIsSingleBuffered; 81aff2f949395725f4bb82802bd12201b65d514e68Pablo Ceballos 826714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis /** 836714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * Callback interface for being notified that a new stream frame is available. 846714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis */ 856714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis public interface OnFrameAvailableListener { 866714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis void onFrameAvailable(SurfaceTexture surfaceTexture); 876714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis } 886714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis 896714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis /** 90a86ab640f7bb0bf3cb4eaed80473ca8c5d131903Igor Murashkin * Exception thrown when a SurfaceTexture couldn't be created or resized. 91a86ab640f7bb0bf3cb4eaed80473ca8c5d131903Igor Murashkin * 92c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown * @deprecated No longer thrown. {@link android.view.Surface.OutOfResourcesException} 93c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown * is used instead. 946714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis */ 95a86ab640f7bb0bf3cb4eaed80473ca8c5d131903Igor Murashkin @SuppressWarnings("serial") 96a86ab640f7bb0bf3cb4eaed80473ca8c5d131903Igor Murashkin @Deprecated 976714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis public static class OutOfResourcesException extends Exception { 986714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis public OutOfResourcesException() { 996714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis } 1006714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis public OutOfResourcesException(String name) { 1016714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis super(name); 1026714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis } 1036714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis } 1046714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis 1056714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis /** 1066714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * Construct a new SurfaceTexture to stream images to a given OpenGL texture. 1076714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * 1086714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * @param texName the OpenGL texture object name (e.g. generated via glGenTextures) 109a86ab640f7bb0bf3cb4eaed80473ca8c5d131903Igor Murashkin * 110b8d152f5cd4628685ec5acfd321b43a422b0cce7John Reck * @throws android.view.Surface.OutOfResourcesException If the SurfaceTexture cannot be created. 1116714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis */ 1126714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis public SurfaceTexture(int texName) { 113c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown this(texName, false); 114e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian } 115e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian 116e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian /** 117e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian * Construct a new SurfaceTexture to stream images to a given OpenGL texture. 118e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian * 119e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian * In single buffered mode the application is responsible for serializing access to the image 120e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian * content buffer. Each time the image content is to be updated, the 121e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian * {@link #releaseTexImage()} method must be called before the image content producer takes 122e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian * ownership of the buffer. For example, when producing image content with the NDK 123e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian * ANativeWindow_lock and ANativeWindow_unlockAndPost functions, {@link #releaseTexImage()} 124e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian * must be called before each ANativeWindow_lock, or that call will fail. When producing 125e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian * image content with OpenGL ES, {@link #releaseTexImage()} must be called before the first 126e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian * OpenGL ES function call each frame. 127e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian * 128e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian * @param texName the OpenGL texture object name (e.g. generated via glGenTextures) 129e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian * @param singleBufferMode whether the SurfaceTexture will be in single buffered mode. 130a86ab640f7bb0bf3cb4eaed80473ca8c5d131903Igor Murashkin * 131b8d152f5cd4628685ec5acfd321b43a422b0cce7John Reck * @throws android.view.Surface.OutOfResourcesException If the SurfaceTexture cannot be created. 132e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian */ 133e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian public SurfaceTexture(int texName, boolean singleBufferMode) { 134c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown mCreatorLooper = Looper.myLooper(); 135aff2f949395725f4bb82802bd12201b65d514e68Pablo Ceballos mIsSingleBuffered = singleBufferMode; 136493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza nativeInit(false, texName, singleBufferMode, new WeakReference<SurfaceTexture>(this)); 137493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza } 138493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza 139493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza /** 140493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza * Construct a new SurfaceTexture to stream images to a given OpenGL texture. 141493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza * 142493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza * In single buffered mode the application is responsible for serializing access to the image 143493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza * content buffer. Each time the image content is to be updated, the 144493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza * {@link #releaseTexImage()} method must be called before the image content producer takes 145493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza * ownership of the buffer. For example, when producing image content with the NDK 146493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza * ANativeWindow_lock and ANativeWindow_unlockAndPost functions, {@link #releaseTexImage()} 147493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza * must be called before each ANativeWindow_lock, or that call will fail. When producing 148493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza * image content with OpenGL ES, {@link #releaseTexImage()} must be called before the first 149493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza * OpenGL ES function call each frame. 150493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza * 151493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza * Unlike {@link #SurfaceTexture(int, boolean)}, which takes an OpenGL texture object name, 152493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza * this constructor creates the SurfaceTexture in detached mode. A texture name must be passed 153493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza * in using {@link #attachToGLContext} before calling {@link #releaseTexImage()} and producing 154493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza * image content using OpenGL ES. 155493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza * 156493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza * @param singleBufferMode whether the SurfaceTexture will be in single buffered mode. 157493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza * 158b8d152f5cd4628685ec5acfd321b43a422b0cce7John Reck * @throws android.view.Surface.OutOfResourcesException If the SurfaceTexture cannot be created. 159493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza */ 160493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza public SurfaceTexture(boolean singleBufferMode) { 161493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza mCreatorLooper = Looper.myLooper(); 162aff2f949395725f4bb82802bd12201b65d514e68Pablo Ceballos mIsSingleBuffered = singleBufferMode; 163493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza nativeInit(true, 0, singleBufferMode, new WeakReference<SurfaceTexture>(this)); 1646714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis } 1656714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis 1666714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis /** 1676714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * Register a callback to be invoked when a new image frame becomes available to the 168c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown * SurfaceTexture. 169c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown * <p> 170ba873d21718b0c0bbeabbb3b4796e54073739700Jeff Brown * The callback may be called on an arbitrary thread, so it is not 1716714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * safe to call {@link #updateTexImage} without first binding the OpenGL ES context to the 1726714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * thread invoking the callback. 173c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown * </p> 174c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown * 175ba873d21718b0c0bbeabbb3b4796e54073739700Jeff Brown * @param listener The listener to use, or null to remove the listener. 1766714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis */ 177ba873d21718b0c0bbeabbb3b4796e54073739700Jeff Brown public void setOnFrameAvailableListener(@Nullable OnFrameAvailableListener listener) { 178c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown setOnFrameAvailableListener(listener, null); 179c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown } 180c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown 181c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown /** 182c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown * Register a callback to be invoked when a new image frame becomes available to the 183c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown * SurfaceTexture. 184c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown * <p> 185ba873d21718b0c0bbeabbb3b4796e54073739700Jeff Brown * If a handler is specified, the callback will be invoked on that handler's thread. 186ba873d21718b0c0bbeabbb3b4796e54073739700Jeff Brown * If no handler is specified, then the callback may be called on an arbitrary thread, 187c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown * so it is not safe to call {@link #updateTexImage} without first binding the OpenGL ES 188c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown * context to the thread invoking the callback. 189c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown * </p> 190c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown * 191ba873d21718b0c0bbeabbb3b4796e54073739700Jeff Brown * @param listener The listener to use, or null to remove the listener. 192c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown * @param handler The handler on which the listener should be invoked, or null 193c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown * to use an arbitrary thread. 194c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown */ 195ba873d21718b0c0bbeabbb3b4796e54073739700Jeff Brown public void setOnFrameAvailableListener(@Nullable final OnFrameAvailableListener listener, 196ba873d21718b0c0bbeabbb3b4796e54073739700Jeff Brown @Nullable Handler handler) { 197c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown if (listener != null) { 198c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown // Although we claim the thread is arbitrary, earlier implementation would 199c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown // prefer to send the callback on the creating looper or the main looper 200c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown // so we preserve this behavior here. 201c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown Looper looper = handler != null ? handler.getLooper() : 202c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown mCreatorLooper != null ? mCreatorLooper : Looper.getMainLooper(); 203c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown mOnFrameAvailableHandler = new Handler(looper, null, true /*async*/) { 204c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown @Override 205c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown public void handleMessage(Message msg) { 206c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown listener.onFrameAvailable(SurfaceTexture.this); 207c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown } 208c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown }; 209c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown } else { 210c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown mOnFrameAvailableHandler = null; 211c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown } 2126714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis } 2136714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis 2146714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis /** 2152aafe742e5d2d63d77c49df032ec580966661597Jamie Gennis * Set the default size of the image buffers. The image producer may override the buffer size, 2162aafe742e5d2d63d77c49df032ec580966661597Jamie Gennis * in which case the producer-set buffer size will be used, not the default size set by this 2172aafe742e5d2d63d77c49df032ec580966661597Jamie Gennis * method. Both video and camera based image producers do override the size. This method may 2182aafe742e5d2d63d77c49df032ec580966661597Jamie Gennis * be used to set the image size when producing images with {@link android.graphics.Canvas} (via 2192aafe742e5d2d63d77c49df032ec580966661597Jamie Gennis * {@link android.view.Surface#lockCanvas}), or OpenGL ES (via an EGLSurface). 220050316184b01c0d1a01c46afae7429b89a27c31btedbo * 2212aafe742e5d2d63d77c49df032ec580966661597Jamie Gennis * The new default buffer size will take effect the next time the image producer requests a 2222aafe742e5d2d63d77c49df032ec580966661597Jamie Gennis * buffer to fill. For {@link android.graphics.Canvas} this will be the next time {@link 2232aafe742e5d2d63d77c49df032ec580966661597Jamie Gennis * android.view.Surface#lockCanvas} is called. For OpenGL ES, the EGLSurface should be 2242aafe742e5d2d63d77c49df032ec580966661597Jamie Gennis * destroyed (via eglDestroySurface), made not-current (via eglMakeCurrent), and then recreated 2252aafe742e5d2d63d77c49df032ec580966661597Jamie Gennis * (via eglCreateWindowSurface) to ensure that the new default size has taken effect. 226a86ab640f7bb0bf3cb4eaed80473ca8c5d131903Igor Murashkin * 227b89d88f531ee39927f8f554baaae5ecc9101ba9dMathias Agopian * The width and height parameters must be no greater than the minimum of 228a86ab640f7bb0bf3cb4eaed80473ca8c5d131903Igor Murashkin * GL_MAX_VIEWPORT_DIMS and GL_MAX_TEXTURE_SIZE (see 229b89d88f531ee39927f8f554baaae5ecc9101ba9dMathias Agopian * {@link javax.microedition.khronos.opengles.GL10#glGetIntegerv glGetIntegerv}). 230b89d88f531ee39927f8f554baaae5ecc9101ba9dMathias Agopian * An error due to invalid dimensions might not be reported until 231b89d88f531ee39927f8f554baaae5ecc9101ba9dMathias Agopian * updateTexImage() is called. 232050316184b01c0d1a01c46afae7429b89a27c31btedbo */ 233050316184b01c0d1a01c46afae7429b89a27c31btedbo public void setDefaultBufferSize(int width, int height) { 234050316184b01c0d1a01c46afae7429b89a27c31btedbo nativeSetDefaultBufferSize(width, height); 235050316184b01c0d1a01c46afae7429b89a27c31btedbo } 236050316184b01c0d1a01c46afae7429b89a27c31btedbo 237050316184b01c0d1a01c46afae7429b89a27c31btedbo /** 2386714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis * Update the texture image to the most recent frame from the image stream. This may only be 2392b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * called while the OpenGL ES context that owns the texture is current on the calling thread. 2402b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * It will implicitly bind its texture to the GL_TEXTURE_EXTERNAL_OES texture target. 2416714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis */ 242376590d668e22a918439877b55faf075427b13f3Jamie Gennis public void updateTexImage() { 24333efb231cb92065c40c019319adae36abc413863Jamie Gennis nativeUpdateTexImage(); 244c6d993077761fc737bbb0f4db44b961a4e7b6bbbJamie Gennis } 245c6d993077761fc737bbb0f4db44b961a4e7b6bbbJamie Gennis 246c6d993077761fc737bbb0f4db44b961a4e7b6bbbJamie Gennis /** 247e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian * Releases the the texture content. This is needed in single buffered mode to allow the image 248e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian * content producer to take ownership of the image buffer. 249cb92a8d9428c2e4a9f038cd5f10e9d61bf265cd4Mathias Agopian * For more information see {@link #SurfaceTexture(int, boolean)}. 250e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian */ 251e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian public void releaseTexImage() { 252e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian nativeReleaseTexImage(); 253e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian } 254e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian 255e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian /** 2562b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * Detach the SurfaceTexture from the OpenGL ES context that owns the OpenGL ES texture object. 2572b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * This call must be made with the OpenGL ES context current on the calling thread. The OpenGL 2582b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * ES texture object will be deleted as a result of this call. After calling this method all 2592b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * calls to {@link #updateTexImage} will throw an {@link java.lang.IllegalStateException} until 2602b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * a successful call to {@link #attachToGLContext} is made. 2612b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * 2622b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * This can be used to access the SurfaceTexture image contents from multiple OpenGL ES 2632b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * contexts. Note, however, that the image contents are only accessible from one OpenGL ES 2642b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * context at a time. 265c6d993077761fc737bbb0f4db44b961a4e7b6bbbJamie Gennis */ 266c6d993077761fc737bbb0f4db44b961a4e7b6bbbJamie Gennis public void detachFromGLContext() { 267c6d993077761fc737bbb0f4db44b961a4e7b6bbbJamie Gennis int err = nativeDetachFromGLContext(); 268c6d993077761fc737bbb0f4db44b961a4e7b6bbbJamie Gennis if (err != 0) { 269c6d993077761fc737bbb0f4db44b961a4e7b6bbbJamie Gennis throw new RuntimeException("Error during detachFromGLContext (see logcat for details)"); 270c6d993077761fc737bbb0f4db44b961a4e7b6bbbJamie Gennis } 271c6d993077761fc737bbb0f4db44b961a4e7b6bbbJamie Gennis } 272c6d993077761fc737bbb0f4db44b961a4e7b6bbbJamie Gennis 273c6d993077761fc737bbb0f4db44b961a4e7b6bbbJamie Gennis /** 2742b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * Attach the SurfaceTexture to the OpenGL ES context that is current on the calling thread. A 2752b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * new OpenGL ES texture object is created and populated with the SurfaceTexture image frame 2762b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * that was current at the time of the last call to {@link #detachFromGLContext}. This new 2772b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * texture is bound to the GL_TEXTURE_EXTERNAL_OES texture target. 2782b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * 2792b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * This can be used to access the SurfaceTexture image contents from multiple OpenGL ES 2802b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * contexts. Note, however, that the image contents are only accessible from one OpenGL ES 2812b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * context at a time. 2822b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * 2832b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * @param texName The name of the OpenGL ES texture that will be created. This texture name 2842b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis * must be unusued in the OpenGL ES context that is current on the calling thread. 285c6d993077761fc737bbb0f4db44b961a4e7b6bbbJamie Gennis */ 286c6d993077761fc737bbb0f4db44b961a4e7b6bbbJamie Gennis public void attachToGLContext(int texName) { 287c6d993077761fc737bbb0f4db44b961a4e7b6bbbJamie Gennis int err = nativeAttachToGLContext(texName); 288c6d993077761fc737bbb0f4db44b961a4e7b6bbbJamie Gennis if (err != 0) { 289425c3da61ad39f5dd4aaba9fd0235f46ff079245Jonathan Dixon throw new RuntimeException("Error during attachToGLContext (see logcat for details)"); 290b89d88f531ee39927f8f554baaae5ecc9101ba9dMathias Agopian } 291376590d668e22a918439877b55faf075427b13f3Jamie Gennis } 292b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis 293b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis /** 294b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis * Retrieve the 4x4 texture coordinate transform matrix associated with the texture image set by 295b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis * the most recent call to updateTexImage. 296b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis * 297b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis * This transform matrix maps 2D homogeneous texture coordinates of the form (s, t, 0, 1) with s 298b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis * and t in the inclusive range [0, 1] to the texture coordinate that should be used to sample 299b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis * that location from the texture. Sampling the texture outside of the range of this transform 300b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis * is undefined. 301b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis * 302b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis * The matrix is stored in column-major order so that it may be passed directly to OpenGL ES via 303b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis * the glLoadMatrixf or glUniformMatrix4fv functions. 304b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis * 305b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis * @param mtx the array into which the 4x4 matrix will be stored. The array must have exactly 306b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis * 16 elements. 307b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis */ 308b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis public void getTransformMatrix(float[] mtx) { 309cc562a3576a6a8096626387472e05e8bee03352aGlenn Kasten // Note we intentionally don't check mtx for null, so this will result in a 310cc562a3576a6a8096626387472e05e8bee03352aGlenn Kasten // NullPointerException. But it's safe because it happens before the call to native. 311b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis if (mtx.length != 16) { 312b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis throw new IllegalArgumentException(); 313b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis } 314376590d668e22a918439877b55faf075427b13f3Jamie Gennis nativeGetTransformMatrix(mtx); 315b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis } 316b0ba48c95ea8768a051100c5adb4c906caa1e080Jamie Gennis 317c5f94d8a4779050125145396ca83fbc862c7ed6bEino-Ville Talvala /** 318c5f94d8a4779050125145396ca83fbc862c7ed6bEino-Ville Talvala * Retrieve the timestamp associated with the texture image set by the most recent call to 319c5f94d8a4779050125145396ca83fbc862c7ed6bEino-Ville Talvala * updateTexImage. 320c5f94d8a4779050125145396ca83fbc862c7ed6bEino-Ville Talvala * 3215c2faf3dc310b100707eb9e32e1e5ae8ceffd0c6Glenn Kasten * This timestamp is in nanoseconds, and is normally monotonically increasing. The timestamp 3225c2faf3dc310b100707eb9e32e1e5ae8ceffd0c6Glenn Kasten * should be unaffected by time-of-day adjustments, and for a camera should be strictly 3235c2faf3dc310b100707eb9e32e1e5ae8ceffd0c6Glenn Kasten * monotonic but for a MediaPlayer may be reset when the position is set. The 324c5f94d8a4779050125145396ca83fbc862c7ed6bEino-Ville Talvala * specific meaning and zero point of the timestamp depends on the source providing images to 325c5f94d8a4779050125145396ca83fbc862c7ed6bEino-Ville Talvala * the SurfaceTexture. Unless otherwise specified by the image source, timestamps cannot 326c5f94d8a4779050125145396ca83fbc862c7ed6bEino-Ville Talvala * generally be compared across SurfaceTexture instances, or across multiple program 327c5f94d8a4779050125145396ca83fbc862c7ed6bEino-Ville Talvala * invocations. It is mostly useful for determining time offsets between subsequent frames. 328c5f94d8a4779050125145396ca83fbc862c7ed6bEino-Ville Talvala */ 3295c2faf3dc310b100707eb9e32e1e5ae8ceffd0c6Glenn Kasten 330c5f94d8a4779050125145396ca83fbc862c7ed6bEino-Ville Talvala public long getTimestamp() { 331c5f94d8a4779050125145396ca83fbc862c7ed6bEino-Ville Talvala return nativeGetTimestamp(); 332c5f94d8a4779050125145396ca83fbc862c7ed6bEino-Ville Talvala } 333c5f94d8a4779050125145396ca83fbc862c7ed6bEino-Ville Talvala 334ec46b4e1ca89d7c3a9ad70ded58da08b5e19f08fMathias Agopian /** 335ec46b4e1ca89d7c3a9ad70ded58da08b5e19f08fMathias Agopian * release() frees all the buffers and puts the SurfaceTexture into the 336ec46b4e1ca89d7c3a9ad70ded58da08b5e19f08fMathias Agopian * 'abandoned' state. Once put in this state the SurfaceTexture can never 337ec46b4e1ca89d7c3a9ad70ded58da08b5e19f08fMathias Agopian * leave it. When in the 'abandoned' state, all methods of the 338d47f7d8b5fe3a3861d7cbdc5f912235407823c8eAndy McFadden * IGraphicBufferProducer interface will fail with the NO_INIT error. 339ec46b4e1ca89d7c3a9ad70ded58da08b5e19f08fMathias Agopian * 340ec46b4e1ca89d7c3a9ad70ded58da08b5e19f08fMathias Agopian * Note that while calling this method causes all the buffers to be freed 341ec46b4e1ca89d7c3a9ad70ded58da08b5e19f08fMathias Agopian * from the perspective of the the SurfaceTexture, if there are additional 342ec46b4e1ca89d7c3a9ad70ded58da08b5e19f08fMathias Agopian * references on the buffers (e.g. if a buffer is referenced by a client or 343ec46b4e1ca89d7c3a9ad70ded58da08b5e19f08fMathias Agopian * by OpenGL ES as a texture) then those buffer will remain allocated. 344ec46b4e1ca89d7c3a9ad70ded58da08b5e19f08fMathias Agopian * 345ec46b4e1ca89d7c3a9ad70ded58da08b5e19f08fMathias Agopian * Always call this method when you are done with SurfaceTexture. Failing 346ec46b4e1ca89d7c3a9ad70ded58da08b5e19f08fMathias Agopian * to do so may delay resource deallocation for a significant amount of 347ec46b4e1ca89d7c3a9ad70ded58da08b5e19f08fMathias Agopian * time. 348f4612191855b999be7b8b2402fd7b45f07aa95f4Chris Craik * 349f4612191855b999be7b8b2402fd7b45f07aa95f4Chris Craik * @see #isReleased() 350ec46b4e1ca89d7c3a9ad70ded58da08b5e19f08fMathias Agopian */ 351ec46b4e1ca89d7c3a9ad70ded58da08b5e19f08fMathias Agopian public void release() { 352ec46b4e1ca89d7c3a9ad70ded58da08b5e19f08fMathias Agopian nativeRelease(); 353ec46b4e1ca89d7c3a9ad70ded58da08b5e19f08fMathias Agopian } 354ec46b4e1ca89d7c3a9ad70ded58da08b5e19f08fMathias Agopian 3556d8371e73ff6452be5a23089e7edeb8d6d96f065John Reck /** 356f4612191855b999be7b8b2402fd7b45f07aa95f4Chris Craik * Returns true if the SurfaceTexture was released. 357f4612191855b999be7b8b2402fd7b45f07aa95f4Chris Craik * 358f4612191855b999be7b8b2402fd7b45f07aa95f4Chris Craik * @see #release() 3596d8371e73ff6452be5a23089e7edeb8d6d96f065John Reck */ 3606d8371e73ff6452be5a23089e7edeb8d6d96f065John Reck public boolean isReleased() { 3616d8371e73ff6452be5a23089e7edeb8d6d96f065John Reck return nativeIsReleased(); 3626d8371e73ff6452be5a23089e7edeb8d6d96f065John Reck } 3636d8371e73ff6452be5a23089e7edeb8d6d96f065John Reck 364a86ab640f7bb0bf3cb4eaed80473ca8c5d131903Igor Murashkin @Override 365376590d668e22a918439877b55faf075427b13f3Jamie Gennis protected void finalize() throws Throwable { 366376590d668e22a918439877b55faf075427b13f3Jamie Gennis try { 367376590d668e22a918439877b55faf075427b13f3Jamie Gennis nativeFinalize(); 368376590d668e22a918439877b55faf075427b13f3Jamie Gennis } finally { 369376590d668e22a918439877b55faf075427b13f3Jamie Gennis super.finalize(); 370376590d668e22a918439877b55faf075427b13f3Jamie Gennis } 371376590d668e22a918439877b55faf075427b13f3Jamie Gennis } 372376590d668e22a918439877b55faf075427b13f3Jamie Gennis 3738f0095cd33558e9cc8a440047908e53b68906f5fRomain Guy /** 3748f0095cd33558e9cc8a440047908e53b68906f5fRomain Guy * This method is invoked from native code only. 3758f0095cd33558e9cc8a440047908e53b68906f5fRomain Guy */ 3768f0095cd33558e9cc8a440047908e53b68906f5fRomain Guy @SuppressWarnings({"UnusedDeclaration"}) 377c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown private static void postEventFromNative(WeakReference<SurfaceTexture> weakSelf) { 378c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown SurfaceTexture st = weakSelf.get(); 379c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown if (st != null) { 380c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown Handler handler = st.mOnFrameAvailableHandler; 381c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown if (handler != null) { 382c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown handler.sendEmptyMessage(0); 383c7282e57cd01f1576baac04356bf99bee34e4c18Jeff Brown } 384e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian } 385e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian } 386e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian 387aff2f949395725f4bb82802bd12201b65d514e68Pablo Ceballos /** 388aff2f949395725f4bb82802bd12201b65d514e68Pablo Ceballos * Returns true if the SurfaceTexture is single-buffered 389aff2f949395725f4bb82802bd12201b65d514e68Pablo Ceballos * @hide 390aff2f949395725f4bb82802bd12201b65d514e68Pablo Ceballos */ 391aff2f949395725f4bb82802bd12201b65d514e68Pablo Ceballos public boolean isSingleBuffered() { 392aff2f949395725f4bb82802bd12201b65d514e68Pablo Ceballos return mIsSingleBuffered; 393aff2f949395725f4bb82802bd12201b65d514e68Pablo Ceballos } 394aff2f949395725f4bb82802bd12201b65d514e68Pablo Ceballos 395493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza private native void nativeInit(boolean isDetached, int texName, 396493f2e11909e2d5839ca81ddc66d48d538192478Dan Stoza boolean singleBufferMode, WeakReference<SurfaceTexture> weakSelf) 397a86ab640f7bb0bf3cb4eaed80473ca8c5d131903Igor Murashkin throws Surface.OutOfResourcesException; 398376590d668e22a918439877b55faf075427b13f3Jamie Gennis private native void nativeFinalize(); 399376590d668e22a918439877b55faf075427b13f3Jamie Gennis private native void nativeGetTransformMatrix(float[] mtx); 400c5f94d8a4779050125145396ca83fbc862c7ed6bEino-Ville Talvala private native long nativeGetTimestamp(); 401050316184b01c0d1a01c46afae7429b89a27c31btedbo private native void nativeSetDefaultBufferSize(int width, int height); 4022b4bfa5efec7df408b4db127961cfc9aca9e57cfJamie Gennis private native void nativeUpdateTexImage(); 403e591b49de038a9942cbcc77540c03e85c96e3dcbMathias Agopian private native void nativeReleaseTexImage(); 404c6d993077761fc737bbb0f4db44b961a4e7b6bbbJamie Gennis private native int nativeDetachFromGLContext(); 405c6d993077761fc737bbb0f4db44b961a4e7b6bbbJamie Gennis private native int nativeAttachToGLContext(int texName); 406ec46b4e1ca89d7c3a9ad70ded58da08b5e19f08fMathias Agopian private native void nativeRelease(); 4076d8371e73ff6452be5a23089e7edeb8d6d96f065John Reck private native boolean nativeIsReleased(); 4086714efc5e0c52953b65e774de0003e22377e7d39Jamie Gennis} 409