A few years ago, I've added a code into upstream GStreamer's androidmedia to utilize NDK APIs for MediaCodec, in hope that one day we'll be able to utilize that in Ubuntu Touch.
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4115
This holiday, due to a brief exchange with @thekit during the last devsync, my mind wandered again and I've decided to implement additional codepath which utilize NDK APIs to render decoded buffers with OpenGL instead of Java-based SurfaceTexture.
https://gitlab.freedesktop.org/peat-psuwit/gstreamer/-/tree/aimagereader?ref_type=heads
The code "works" but is still littered with TODOs in error handlings and handling of pause state (also see TODOs in the commit message itself). Essentially this is a demo-level code.
With that branch, the only thing that still require Java/JNI is the codec listing. While NDK recently added the APIs for that, it lacks some function which GStreamer expects (e.g. the ability to query supported color format), so it'll stay as Java/JNI for now (or use libhybris-specific APIs if we bring that to Ubuntu Touch).
Also this won't quite work with current cross-process media decoding pipeline in Ubuntu Touch's media-hub. Some additional work is needed to transport buffers cross-process (and add necessary synchronization). But that will have to wait lots, lots longer...
But if you happen to have an Android phone and want to toy with this code, the steps are:
Clone Cerbero, GStreamer's cross-compilation environment, and point it to the mentioned Git branch.
https://gitlab.freedesktop.org/gstreamer/cerbero#how-to-build-a-custom-gstreamer-repository-or-branch
Build GStreamer package using:
./cerbero-uninstalled -c config/cross-android-arm64.cbc -c ../localconfig.cbc -v norust package gstreamer-1.0
Make a new directory, then unpack the resulting .zip file inside another folder. Rename the resulting directories so that they look like this:
gst-android (or any folder)
- arm64
- lib
- share
- [...]
Clone the GStreamer locally again, and apply following patch:
diff --git a/subprojects/gst-examples/playback/player/android/app/build.gradle b/subprojects/gst-examples/playback/player/android/app/build.gradle
index d8e000598e7..a66163a9a87 100644
--- a/subprojects/gst-examples/playback/player/android/app/build.gradle
+++ b/subprojects/gst-examples/playback/player/android/app/build.gradle
@@ -27,7 +27,7 @@ android {
targets "gstplayer"
// All archs except MIPS and MIPS64 are supported
- abiFilters 'armeabi-v7a', 'arm64-v8a', 'x86', 'x86_64'
+ abiFilters 'arm64-v8a'
}
}
}
@@ -47,6 +47,10 @@ android {
ndkVersion '25.2.9519653'
namespace 'org.freedesktop.gstreamer.play'
+
+ packagingOptions{
+ doNotStrip "**/libgstreamer_android.so"
+ }
}
afterEvaluate {
diff --git a/subprojects/gst-examples/playback/player/android/app/src/main/jni/CMakeLists.txt b/subprojects/gst-examples/playback/player/android/app/src/main/jni/CMakeLists.txt
index 79bdf8b80fd..fa027e35c58 100644
--- a/subprojects/gst-examples/playback/player/android/app/src/main/jni/CMakeLists.txt
+++ b/subprojects/gst-examples/playback/player/android/app/src/main/jni/CMakeLists.txt
@@ -26,10 +26,12 @@ set(GSTREAMER_NDK_BUILD_PATH "${GSTREAMER_ROOT}/share/gst-android/ndk-build/")
include("${GSTREAMER_NDK_BUILD_PATH}/plugins.cmake")
set(GSTREAMER_PLUGINS ${GSTREAMER_PLUGINS_CORE} ${GSTREAMER_PLUGINS_PLAYBACK} ${GSTREAMER_PLUGINS_CODECS} ${GSTREAMER_PLUGINS_NET} ${GSTREAMER_PLUGINS_SYS} ${GSTREAMER_PLUGINS_CODECS_RESTRICTED} ${GSTREAMER_CODECS_GPL} ${GSTREAMER_PLUGINS_ENCODING} ${GSTREAMER_PLUGINS_VIS} ${GSTREAMER_PLUGINS_EFFECTS} ${GSTREAMER_PLUGINS_NET_RESTRICTED})
set(GStreamer_EXTRA_DEPS gstreamer-play-1.0 gstreamer-player-1.0 gstreamer-video-1.0 glib-2.0)
+set(G_IO_MODULES openssl)
+
find_library(ANDROID_LIB android REQUIRED)
find_library(LOG_LIB log REQUIRED)
-find_package(GStreamerMobile COMPONENTS ${GSTREAMER_PLUGINS} fonts REQUIRED)
+find_package(GStreamerMobile COMPONENTS ${GSTREAMER_PLUGINS} fonts ca_certificates REQUIRED)
add_library(gstplayer SHARED player.c dummy.cpp)
target_link_libraries(gstplayer
Edit subprojects/gst-examples/playback/player/android/gradle.properties to point gstAndroidRoot to the directory you make in step 3.).
Open subprojects/gst-examples/playback/player/android in Android Studio and build the project.
Launch the example with:
adb shell am start -a android.intent.action.VIEW -d "https://test-videos.co.uk/vids/bigbuckbunny/webm/vp8/1080/Big_Buck_Bunny_1080_10s_5MB.webm" org.freedesktop.gstreamer.play
Feel free to replace the URL with another video, depending on what video codec is supported by your device. Note that currently H.26[45] doesn't work because of another GStreamer bug, so you'll likely want a VP[89] or AV1 videos.
Enjoy, I guess?
