com.jogamp.opengl.util.av.GLMediaPlayer Maven / Gradle / Ivy
/**
* Copyright 2012 JogAmp Community. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without modification, are
* permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice, this list of
* conditions and the following disclaimer.
*
* 2. Redistributions in binary form must reproduce the above copyright notice, this list
* of conditions and the following disclaimer in the documentation and/or other materials
* provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY JogAmp Community ``AS IS'' AND ANY EXPRESS OR IMPLIED
* WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
* FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL JogAmp Community OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
* SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
* ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
* ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*
* The views and conclusions contained in the software and documentation are those of the
* authors and should not be interpreted as representing official policies, either expressed
* or implied, of JogAmp Community.
*/
package com.jogamp.opengl.util.av;
import java.net.URI;
import javax.media.opengl.GL;
import javax.media.opengl.GLException;
import jogamp.opengl.Debug;
import com.jogamp.opengl.util.texture.Texture;
import com.jogamp.opengl.util.texture.TextureSequence;
import com.jogamp.opengl.util.TimeFrameI;
/**
* GLMediaPlayer interface specifies a {@link TextureSequence} state machine
* using a multiplexed audio/video stream as it's source.
*
* Audio maybe supported and played back internally or via an {@link AudioSink} implementation.
*
*
* Audio and video streams can be selected or muted via {@link #initStream(URI, int, int, int)}
* using the appropriate stream id's.
*
*
* Camera input can be selected using the {@link #CameraInputScheme} URI.
*
*
* StreamWorker Decoding Thread
*
* Most of the stream processing is performed on the decoding thread, a.k.a. StreamWorker:
*
* - Stream initialization triggered by {@link #initStream(URI, int, int, int) initStream(..)} - User gets notified whether the stream has been initialized or not via {@link GLMediaEventListener#attributesChanged(GLMediaPlayer, int, long) attributesChanges(..)}.
* - Stream decoding - User gets notified of a new frame via {@link GLMediaEventListener#newFrameAvailable(GLMediaPlayer, com.jogamp.opengl.util.texture.TextureSequence.TextureFrame, long) newFrameAvailable(...)}.
* - Caught exceptions on the decoding thread are delivered as {@link StreamException}s.
*
* StreamWorker generates it's own {@link GLContext}, shared with the one passed to {@link #initGL(GL)}.
* The shared {@link GLContext} allows the decoding thread to push the video frame data directly into
* the designated {@link TextureFrame}, later returned via {@link #getNextTexture(GL)} and used by the user.
*
* StreamWorker Error Handling
*
* Caught exceptions on StreamWorker are delivered as {@link StreamException}s,
* which either degrades the {@link State} to {@link State#Uninitialized} or {@link State#Paused}.
*
*
* An occurring {@link StreamException} triggers a {@link GLMediaEventListener#EVENT_CHANGE_ERR EVENT_CHANGE_ERR} event,
* which can be listened to via {@link GLMediaEventListener#attributesChanged(GLMediaPlayer, int, long)}.
*
*
* An occurred {@link StreamException} can be read via {@link #getStreamException()}.
*
*
*
* GLMediaPlayer Lifecycle
*
*
* Action {@link State} Before {@link State} After {@link GLMediaEventListener Event} {@link #initStream(URI, int, int, int)} {@link State#Uninitialized Uninitialized} {@link State#Initialized Initialized}1, {@link State#Uninitialized Uninitialized} {@link GLMediaEventListener#EVENT_CHANGE_INIT EVENT_CHANGE_INIT} or ( {@link GLMediaEventListener#EVENT_CHANGE_ERR EVENT_CHANGE_ERR} + {@link GLMediaEventListener#EVENT_CHANGE_UNINIT EVENT_CHANGE_UNINIT} ) {@link #initGL(GL)} {@link State#Initialized Initialized} {@link State#Paused Paused}, , {@link State#Uninitialized Uninitialized} {@link GLMediaEventListener#EVENT_CHANGE_PAUSE EVENT_CHANGE_PAUSE} or ( {@link GLMediaEventListener#EVENT_CHANGE_ERR EVENT_CHANGE_ERR} + {@link GLMediaEventListener#EVENT_CHANGE_UNINIT EVENT_CHANGE_UNINIT} ) {@link #play()} {@link State#Paused Paused} {@link State#Playing Playing} {@link GLMediaEventListener#EVENT_CHANGE_PLAY EVENT_CHANGE_PLAY} {@link #pause(boolean)} {@link State#Playing Playing} {@link State#Paused Paused} {@link GLMediaEventListener#EVENT_CHANGE_PAUSE EVENT_CHANGE_PAUSE} {@link #seek(int)} {@link State#Paused Paused}, {@link State#Playing Playing} {@link State#Paused Paused}, {@link State#Playing Playing} none {@link #getNextTexture(GL)} {@link State#Paused Paused}, {@link State#Playing Playing} {@link State#Paused Paused}, {@link State#Playing Playing} none {@link #getLastTexture()} {@link State#Paused Paused}, {@link State#Playing Playing} {@link State#Paused Paused}, {@link State#Playing Playing} none {@link TextureFrame#END_OF_STREAM_PTS END_OF_STREAM} {@link State#Playing Playing} {@link State#Paused Paused} {@link GLMediaEventListener#EVENT_CHANGE_EOS EVENT_CHANGE_EOS} + {@link GLMediaEventListener#EVENT_CHANGE_PAUSE EVENT_CHANGE_PAUSE} {@link StreamException} ANY {@link State#Paused Paused}, {@link State#Uninitialized Uninitialized} {@link GLMediaEventListener#EVENT_CHANGE_ERR EVENT_CHANGE_ERR} + ( {@link GLMediaEventListener#EVENT_CHANGE_PAUSE EVENT_CHANGE_PAUSE} or {@link GLMediaEventListener#EVENT_CHANGE_UNINIT EVENT_CHANGE_UNINIT} ) {@link #destroy(GL)} ANY {@link State#Uninitialized Uninitialized} {@link GLMediaEventListener#EVENT_CHANGE_UNINIT EVENT_CHANGE_UNINIT}
*
*
* Audio and video Stream IDs
*
*
* value request get {@link #STREAM_ID_NONE} mute not available {@link #STREAM_ID_AUTO} auto unspecified ≥0 specific stream specific stream
*
*
* Current implementations (check each API doc link for details):
*
* - {@link jogamp.opengl.util.av.NullGLMediaPlayer}
* - {@link jogamp.opengl.util.av.impl.OMXGLMediaPlayer}
* - {@link jogamp.opengl.util.av.impl.FFMPEGMediaPlayer}
* - {@link jogamp.opengl.android.av.AndroidGLMediaPlayerAPI14}
*
*
*
* Implementations of this interface must implement:
*
* public static final boolean isAvailable();
*
* to be properly considered by {@link GLMediaPlayerFactory#create(ClassLoader, String)}
* and {@link GLMediaPlayerFactory#createDefault()}.
*
* Timestamp Accuracy
*
*
* Timestamp type and value range has been chosen to suit embedded CPUs
* and characteristics of audio and video streaming. See {@link TimeFrameI}.
*
*
* Audio and video synchronization
*
* The class follows a passive A/V synchronization pattern.
* Audio is being untouched, while {@link #getNextTexture(GL)} delivers a new video frame
* only, if its timestamp is less than {@link #MAXIMUM_VIDEO_ASYNC} ahead of time.
* If its timestamp is more than {@link #MAXIMUM_VIDEO_ASYNC} ahead of time,
* the previous frame is returned.
* If its timestamp is more than {@link #MAXIMUM_VIDEO_ASYNC} after time,
* the frame is dropped and the next frame is being fetched.
*
*
* https://en.wikipedia.org/wiki/Audio_to_video_synchronization
*
* d_av = v_pts - a_pts;
*
*
*
* Recommendation of audio/video pts time lead/lag at production:
*
* - Overall: +40ms and -60ms audio ahead video / audio after video
* - Each stage: +5ms and -15ms. audio ahead video / audio after video
*
*
*
* Recommendation of av pts time lead/lag at presentation:
*
* - TV: +15ms and -45ms. audio ahead video / audio after video.
* - Film: +22ms and -22ms. audio ahead video / audio after video.
*
*
*
* Test Streams
*
*
* Big Buck Bunny 24f 16:9 Big Buck Bunny 320p h264 aac 48000Hz 2 chan http://download.blender.org/peach/bigbuckbunny_movies/BigBuckBunny_320x180.mp4 Big Buck Bunny 720p mpeg4 ac3 48000Hz 5.1 chan http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_surround.avi Big Buck Bunny 720p msmpeg4v2 mp3 48000Hz 2 chan http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_stereo.avi Big Buck Bunny 720p theora vorbis 48000Hz 2 chan http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_stereo.ogg Big Buck Bunny 1080p mpeg4 ac3 48000Hz 5.1 chan http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_1080p_surround.avi WebM/Matroska (vp8/vorbis) Big Buck Bunny Trailer 640p vp8 vorbis 44100Hz 1 chan http://video.webmfiles.org/big-buck-bunny_trailer.webm Elephants Dream 540p vp8 vorbis 44100Hz 1 chan http://video.webmfiles.org/elephants-dream.webm You Tube http/rtsp Sintel http://www.youtube.com/watch?v=eRsGyueVLvQ rtsp://v3.cache1.c.youtube.com/CiILENy73wIaGQn0LpXnygYbeRMYDSANFEgGUgZ2aWRlb3MM/0/0/0/video.3gp Audio/Video Sync Five-minute-sync-test1080p https://www.youtube.com/watch?v=szoOsG9137U rtsp://v7.cache8.c.youtube.com/CiILENy73wIaGQm133VvsA46sxMYDSANFEgGUgZ2aWRlb3MM/0/0/0/video.3gp Audio-Video-Sync-Test-Calibration-23.98fps-24fps https://www.youtube.com/watch?v=cGgf_dbDMsw sound_in_sync_test https://www.youtube.com/watch?v=O-zIZkhXNLE