R
R
Roman Bazhenov2020-05-01 13:00:58
Android
Roman Bazhenov, 2020-05-01 13:00:58

Android MediaPlayer - how to render video file to texture over background texture from camera?

Hello everyone, I'm making a simple application: an object (Vuforia User Defined Target) is detected in a running AR session, a surface is attached to it, into which an external video should be rendered. Everything is fine with detection, but a frame from the camera is rendered instead of a frame from the video file. In principle, Vuforia does not seem to depend much here, so the question can be abstracted from it.

Here is the media player initialization code:

videoPlayer = MediaPlayer()
        videoPlayer.setSurface(mRenderer?.planeSurface)
        videoPlayer.isLooping = true
        val afd: AssetFileDescriptor
        try {
            afd = assets.openFd("test.mp4")
            videoPlayer.setDataSource(afd.fileDescriptor, afd.startOffset, afd.length)
            videoPlayer.prepare()
            videoPlayer.start()
        } catch (e: IOException) {
            e.printStackTrace()
        }


Here is the texture initialization code:
GLES20.glGenTextures(1, planeSurfaceTextureId, 0)
        GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, planeSurfaceTextureId[0])
        GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR)
        GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR)
        GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE)
        GLES20.glTexParameteri(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE)
        planeSurfaceTexture = SurfaceTexture(planeSurfaceTextureId[0])
        planeSurfaceTexture.setOnFrameAvailableListener(mActivity)
        planeSurface = Surface(planeSurfaceTexture)


Here is the actual rendering of the surface with the frame:
planeSurfaceTexture.updateTexImage()
        GLES20.glActiveTexture(GLES20.GL_TEXTURE1)
        GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, planeSurfaceTextureId[0])
        GLES20.glUniformMatrix4fv(mvpMatrixHandle, 1, false, modelViewProjection, 0)
        GLES20.glUniform1i(texSampler2DHandle, 0)
        GLES20.glDrawElements(GLES20.GL_TRIANGLES, contentPlane.numObjectIndex, GLES20.GL_UNSIGNED_SHORT, contentPlane.meshIndices)
        GLES20.glDisableVertexAttribArray(vertexHandle)
        GLES20.glDisableVertexAttribArray(textureCoordHandle)


I can show you the whole code if needed.
I'm new to android, maybe some kind of stupid mistake, please help me figure it out :) Maybe here you can do without GLES at all.

Thanks in advance!

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question