What's Inside the Black Box of Ubuntu Touch?

rob · 29 Jan 2020, 22:06

So is the first executable on Ubuntu /sbin/init? Or does something else happen before that?

Also, I noticed on the Development release, that Libertine applications are now mixed in with regular applications. Is it safe to manually create *.desktop files in /usr/share/applications on the Libertine container?

UniSuperBox · 30 Jan 2020, 00:58

If you'd like to read exactly how Ubuntu Touch is booted, this is the script: https://github.com/Halium/initramfs-tools-halium/blob/halium/scripts/halium

The steps are:

Android bootloader(s) load Linux kernel, dts, and initramfs into memory
Android bootloader executes Linux
Linux finds initramfs (ours is basically a Debian initramfs with the script I linked above added)
Linux runs /sbin/init in the initramfs

The init in an Debian initramfs is Busybox configured to run a few scripts. The one that handles actually booting Ubuntu Touch is, again, linked above. It mounts the system image (/data/ubuntu.img) at /root, does a few more mounts for Android filesystems (for example, your /persist partition on Android goes to /android/persist/), mounts the read-write filesystem parts (like your home folder), then chroots into /root, running /root/sbin/init.

PID 1 on an Ubuntu Touch system is Upstart provided by the 16.04 rootfs. Android runs in a lxc container.

None of these are required on the PinePhone, that basically boots like any embedded Linux system, with Pine64's u-boot and ARM Trusted Firmware build taking the system up to a point where it can load and execute Linux (steps 1 and 2 above). Then, we currently don't have a custom rootfs compared to Ubuntu, basically executing /sbin/init and going with it.

You can create your own desktop files in your Libertine containers, I guess, but I don't expect they'll do what you want them to do.

rob · 30 Jan 2020, 01:41

@UniSuperBox This is great! Exactly the detail I'm looking for. Thanks!

UniSuperBox · 30 Jan 2020, 01:53

By "Pine64's u-boot", I meant the one that all of us in the Pine64 community maintain together. All of the projects that are shared among almost all of the OSs are here: https://gitlab.com/pine64-org

rob · 30 Jan 2020, 02:25

@UniSuperBox didn't realize you were Dalton. Good to talk to you!

"PID 1 on an Ubuntu Touch system is Upstart provided by the 16.04 rootfs. Android runs in a lxc container."

You say Android runs in an lxc container. Is this what is referred to as the Halium container?

UniSuperBox · 30 Jan 2020, 02:50

@rob, the stealth is not entirely accidental! I can turn on special badges, but prefer not to. Seems like a weird status symbol. Not having a profile picture is sloth, though.

Yes, that could be referred to as the Halium container I suppose. It's really just a stripped-down Android build with no Java stack, though. It exists to allow our software to communicate with Android Services (capital-S Services) through Unix domain sockets. The camera Service and rild (Radio Interface Layer Daemon) are two examples that we communicate to. Using these services directly allows us to reuse closed-source drivers from Android and bring up Ubuntu Touch on a variety of Android phones faster than if we had to reverse-engineer drivers.

rob · 30 Jan 2020, 03:35

@UniSuperBox LOL. I didn't identify myself here before either but someone outed me. Haha..,

The specific details of what you have to access with halium is definitely very interesting and I'll get to that! LOL. My focus right now is understanding the big picture (by understanding the detailed fundamentals). I'm not originally a Linux developer so some of the internal processes for each project are definitely illuminating.

So is the Pinephone Kernel an actual mainline image or is it custom-compiled by someone? And if so where did ubports acquire it?

UniSuperBox · 30 Jan 2020, 03:45

@rob, the PinePhone's Linux kernel is tracked at https://gitlab.com/pine64-org/linux and compiled by GitLab CI for consumption. More information on how our images are built can be found in the README at https://gitlab.com/ubports/community-ports/pinephone

rob · 30 Jan 2020, 04:02

I see the familiar names now in that Kernel project! Thank you for passing that!

Now in an earlier question on the Libertine Container, you implied that there's more to running an application to just creating it in the Libertine-Container. And I've tried non-graphical python applications and they appear to work fine.

So back-pedaling here, I presume that the issue is the display server mapping from the container to Mir? This is clearly very vague to me.

Is there code that I can look at that shows what needs to be done, first in a manual way? 90% of the time adding a package from the Libertine UI doesn't tell me what's happening. It just stops without an error message so I prefer to install things manually so I understand what the limitations are.

For example, I've had issues running Python3 apps due to Gtk dependency issues. My guess is that Xenial doesn't support them. Although Xenial supposedly supports Gtk3, Python in Xenial does not.

If I can unlock the complexities of Libertine-Containers, a lot of people will start to see Ubuntu touch to be more open that it is perceived to be.

Right now, without full understanding, the difficulty in installing Desktop apps will push people to PostmarketOS since it won't have this lxc/chroot jail built in.

dobey · 30 Jan 2020, 16:21

@rob said in What's Inside the Black Box of Ubuntu Touch?:

Although Xenial supposedly supports Gtk3, Python in Xenial does not.

It certainly does via https://packages.ubuntu.com/xenial/python3-gi though the necessary GObject Introspection libraries are probably not installed by default in the container, and if you haven't installed any apps via packages which depend on them in the container, you would need to install them manually to run your own python3 apps that use GTK+.

rob · 30 Jan 2020, 18:42

@dobey I converted my app to Python 2.7 for other compatibility reasons (with Xenial) but even after successfully installing gi and gobject, I get this error on my python program:

gi.require_version('Gtk',3.0)
raise ValueError('Namespace %s not available' % namespace)
ValueError: Namespace Gtk not available

I realize that it is recommended that I learn Qt/QML and rewrite the app but I was just testing what's possible on Libertine-Containers.

And so far I haven't had success using Libertine-Launch. I'm only starting things from bash within the container itself,

rob · 30 Jan 2020, 19:57

If running a GUI in a Libertine-Container, does it use Xmir or Xwayland or is there something else that has to be enabled on the host, after setting the DISPLAY environment variable? This is one of those Ubuntu specific things that just make it different and a little bit frustrating for those who don't know.

I've tried searching all over to understand this but there is so little documentation on this. And all I'm trying to do is demonstrate that I can make my own app only load it on my own phone.

dobey · 30 Jan 2020, 20:29

@rob said in What's Inside the Black Box of Ubuntu Touch?:

ValueError: Namespace Gtk not available

Yes, this means that the package which provides the GObject Introspection code for GTK+ (gir1.2-gtk-3.0) is not installed. Each gir1.2-foo package must be installed for the libraries you want to use in your app, if using python and gtk libraries.

dobey · 30 Jan 2020, 20:33

@rob said in What's Inside the Black Box of Ubuntu Touch?:

If running a GUI in a Libertine-Container, does it use Xmir or Xwayland or is there something else that has to be enabled on the host, after setting the DISPLAY environment variable? This is one of those Ubuntu specific things that just make it different and a little bit frustrating for those who don't know.

Currently, Xmir is used, however, you cannot simply run GUI apps from the command line, such as just typing firefox as Unity 8 authenticates apps in a certain way. Xwayland will eventually replace Xmir, when Wayland replaces MirClient as the protocol used, and the work to make Xwayland work properly in Unity 8 is done. You will need a .desktop file and icon, for any GUI app, and it will need to be started from the Unity 8 app drawer (or the legacy apps scope if using earlier versions), or with ubuntu-app-launch on the command line.

rob · 30 Jan 2020, 20:45

Thank you @dobey. Everything I learn will be shared with others in a simplified way so please consider me a time investment for UT.

Is there further documentation on ubuntu-app-launch? I've been searching for this since early today and haven't found much.

Yes, I understand about handling .desktop which I can see is stored in the container in the regular /usr/share/applications.

Is this launch command handling the display server? Is the handling of Xmir/Xwayland (future) done automatically? Or do I have to do something else either on the host or in the container?

I'm seeing prompts for setting the DISPLAY env variable and I see a -E parameter. But the sparse documentation doesn't make it clear what I have to do.

rob · 30 Jan 2020, 20:48

On the Gobject installation, why did the apt install not automatically handle those dependencies?

dobey · 30 Jan 2020, 20:53

@rob said in What's Inside the Black Box of Ubuntu Touch?:

Is there further documentation on ubuntu-app-launch? I've been searching for this since early today and haven't found much.

There is some API documentation for developers working on the system, but the ubuntu-app-launch tool itself is quite trivial. It only takes one argument, the appid that is being launched. ubuntu-app-launch-appids will list the installed appids available to launch.

@rob said in What's Inside the Black Box of Ubuntu Touch?:

Is this launch command handling the display server? Is the handling of Xmir/Xwayland (future) done automatically? Or do I have to do something else either on the host or in the container?

Xmir does not work in the same way that Xwayland does. A server is started for each X11 app that is being run (so that X11 apps cannot pry into the $DISPLAY and steal data from other X11 apps) with Xmir, while Xwayland has a single display server. Therefore, starting of Xmir for each app is handled by the ubuntu-app-launch code, hence why you need to launch GUI apps with it.

@rob said in What's Inside the Black Box of Ubuntu Touch?:

On the Gobject installation, why did the apt install not automatically handle those dependencies?

Why should it, if you did not install anything which doesn't need them? As you are the developer in this case, you need to install the dependencies you need when developing an application. The libertine container doesn't know what you need in it, for your own code.

rob · 30 Jan 2020, 20:57

@dobey said in What's Inside the Black Box of Ubuntu Touch?:

Why should it, if you did not install anything which doesn't need them? As you are the developer in this case, you need to install the dependencies you need when developing an application. The libertine container doesn't know what you need in it, for your own code.

It's only a comment related to consistency between UT/Libertine-Container vs let's say an Ubuntu 18.04 desktop which is what I'm using. I documented the dependencies needed to install it on Ubuntu 18.04 and am discovering that it is differently packaged for UT. Maybe this is a 16.04 thing..

rob · 30 Jan 2020, 21:04

And the ubuntu-app-launch worked on something I installed from apt! Thanks! Slowly making sense.

My python app is still failing on some other aspect of gtk so I'll research that some more. But glad to have made some progress.

rob · 30 Jan 2020, 22:53

I moved now to a different approach which is to try to make a Python QT app. So I get the error:
This application failed to start because it could not find or load the Qt platform plugin "ubuntumirclient".

Then I see that this is now deprecated and that this will replaced by Xwayland. But this is just a basic Qt app.

I can't even get a simple python Hello World in QT to work in a Libertine Container. That''s all I'm doing.