What's Inside the Black Box of Ubuntu Touch?

rob · 28 Jan 2020, 21:28

Thanks Florian!

That explains this uname (Nexus 5):
Linux ubuntu-phablet 3.4.0-cyanogenmod-gc95a726 #1 SMP PREEMPT Wed Oct 16 04:30:42 CEST 2019 armv7l armv7l armv7l GNU/Linux

But where is the actual Kernel located, on let's say, a Nexus 5.

BTW - I'm only looking for a boot sequence that's pre-Init. Once Ubuntu Touch actually gets control, it looks pretty straightforward.

flohack · 28 Jan 2020, 21:35

@rob said in What's Inside the Black Box of Ubuntu Touch?:

ly looking for a boot sequence that's pre-Init. Once Ubuntu Touch actually gets control, it looks pretty straightforward.

The kernel is packed together with the small initramfs and flashed into the boot partition of each device with the usual fastboot flash command thats also used for Android kernels, and replaces that part entirely.

rob · 29 Jan 2020, 16:35

Here's part 1 of the video series, Exploring Ubuntu Touch - Dissecting the Black Box
https://youtu.be/rMmWmNyDKG8

There's a lot more to explore but this is a beginning.

rob · 29 Jan 2020, 17:07

So exploring this further, Florian, it sounds to me like a Nexus 5 (for example) is really following an Android boot sequence like this

https://elinux.org/Android_Booting

and then Ubuntu 16.04 takes over somewhere wherever its Init is...which then initializes Ubuntu Touch. Right?

(Just for my own edification, it would be great to know what the first code is for Ubuntu in this kind of environment)

I would guess then that a Pinephone would have a more traditional bootloader and starts the kernel from /boot?

flohack · 29 Jan 2020, 19:24

@rob Yes you are absolutely right in your understanding. On Android devices, fastboot takes care of loading the kernel and the initrd, while on a Pinephone this task is done by uBoot. Thats kinda standard bootloader now for arm based open hardware.

rob · 29 Jan 2020, 22:06

So is the first executable on Ubuntu /sbin/init? Or does something else happen before that?

Also, I noticed on the Development release, that Libertine applications are now mixed in with regular applications. Is it safe to manually create *.desktop files in /usr/share/applications on the Libertine container?

UniSuperBox · 30 Jan 2020, 00:58

If you'd like to read exactly how Ubuntu Touch is booted, this is the script: https://github.com/Halium/initramfs-tools-halium/blob/halium/scripts/halium

The steps are:

Android bootloader(s) load Linux kernel, dts, and initramfs into memory
Android bootloader executes Linux
Linux finds initramfs (ours is basically a Debian initramfs with the script I linked above added)
Linux runs /sbin/init in the initramfs

The init in an Debian initramfs is Busybox configured to run a few scripts. The one that handles actually booting Ubuntu Touch is, again, linked above. It mounts the system image (/data/ubuntu.img) at /root, does a few more mounts for Android filesystems (for example, your /persist partition on Android goes to /android/persist/), mounts the read-write filesystem parts (like your home folder), then chroots into /root, running /root/sbin/init.

PID 1 on an Ubuntu Touch system is Upstart provided by the 16.04 rootfs. Android runs in a lxc container.

None of these are required on the PinePhone, that basically boots like any embedded Linux system, with Pine64's u-boot and ARM Trusted Firmware build taking the system up to a point where it can load and execute Linux (steps 1 and 2 above). Then, we currently don't have a custom rootfs compared to Ubuntu, basically executing /sbin/init and going with it.

You can create your own desktop files in your Libertine containers, I guess, but I don't expect they'll do what you want them to do.

rob · 30 Jan 2020, 01:41

@UniSuperBox This is great! Exactly the detail I'm looking for. Thanks!

UniSuperBox · 30 Jan 2020, 01:53

By "Pine64's u-boot", I meant the one that all of us in the Pine64 community maintain together. All of the projects that are shared among almost all of the OSs are here: https://gitlab.com/pine64-org

rob · 30 Jan 2020, 02:25

@UniSuperBox didn't realize you were Dalton. Good to talk to you!

"PID 1 on an Ubuntu Touch system is Upstart provided by the 16.04 rootfs. Android runs in a lxc container."

You say Android runs in an lxc container. Is this what is referred to as the Halium container?

UniSuperBox · 30 Jan 2020, 02:50

@rob, the stealth is not entirely accidental! I can turn on special badges, but prefer not to. Seems like a weird status symbol. Not having a profile picture is sloth, though.

Yes, that could be referred to as the Halium container I suppose. It's really just a stripped-down Android build with no Java stack, though. It exists to allow our software to communicate with Android Services (capital-S Services) through Unix domain sockets. The camera Service and rild (Radio Interface Layer Daemon) are two examples that we communicate to. Using these services directly allows us to reuse closed-source drivers from Android and bring up Ubuntu Touch on a variety of Android phones faster than if we had to reverse-engineer drivers.

rob · 30 Jan 2020, 03:35

@UniSuperBox LOL. I didn't identify myself here before either but someone outed me. Haha..,

The specific details of what you have to access with halium is definitely very interesting and I'll get to that! LOL. My focus right now is understanding the big picture (by understanding the detailed fundamentals). I'm not originally a Linux developer so some of the internal processes for each project are definitely illuminating.

So is the Pinephone Kernel an actual mainline image or is it custom-compiled by someone? And if so where did ubports acquire it?

UniSuperBox · 30 Jan 2020, 03:45

@rob, the PinePhone's Linux kernel is tracked at https://gitlab.com/pine64-org/linux and compiled by GitLab CI for consumption. More information on how our images are built can be found in the README at https://gitlab.com/ubports/community-ports/pinephone

rob · 30 Jan 2020, 04:02

I see the familiar names now in that Kernel project! Thank you for passing that!

Now in an earlier question on the Libertine Container, you implied that there's more to running an application to just creating it in the Libertine-Container. And I've tried non-graphical python applications and they appear to work fine.

So back-pedaling here, I presume that the issue is the display server mapping from the container to Mir? This is clearly very vague to me.

Is there code that I can look at that shows what needs to be done, first in a manual way? 90% of the time adding a package from the Libertine UI doesn't tell me what's happening. It just stops without an error message so I prefer to install things manually so I understand what the limitations are.

For example, I've had issues running Python3 apps due to Gtk dependency issues. My guess is that Xenial doesn't support them. Although Xenial supposedly supports Gtk3, Python in Xenial does not.

If I can unlock the complexities of Libertine-Containers, a lot of people will start to see Ubuntu touch to be more open that it is perceived to be.

Right now, without full understanding, the difficulty in installing Desktop apps will push people to PostmarketOS since it won't have this lxc/chroot jail built in.

dobey · 30 Jan 2020, 16:21

@rob said in What's Inside the Black Box of Ubuntu Touch?:

Although Xenial supposedly supports Gtk3, Python in Xenial does not.

It certainly does via https://packages.ubuntu.com/xenial/python3-gi though the necessary GObject Introspection libraries are probably not installed by default in the container, and if you haven't installed any apps via packages which depend on them in the container, you would need to install them manually to run your own python3 apps that use GTK+.

rob · 30 Jan 2020, 18:42

@dobey I converted my app to Python 2.7 for other compatibility reasons (with Xenial) but even after successfully installing gi and gobject, I get this error on my python program:

gi.require_version('Gtk',3.0)
raise ValueError('Namespace %s not available' % namespace)
ValueError: Namespace Gtk not available

I realize that it is recommended that I learn Qt/QML and rewrite the app but I was just testing what's possible on Libertine-Containers.

And so far I haven't had success using Libertine-Launch. I'm only starting things from bash within the container itself,

rob · 30 Jan 2020, 19:57

If running a GUI in a Libertine-Container, does it use Xmir or Xwayland or is there something else that has to be enabled on the host, after setting the DISPLAY environment variable? This is one of those Ubuntu specific things that just make it different and a little bit frustrating for those who don't know.

I've tried searching all over to understand this but there is so little documentation on this. And all I'm trying to do is demonstrate that I can make my own app only load it on my own phone.

dobey · 30 Jan 2020, 20:29

@rob said in What's Inside the Black Box of Ubuntu Touch?:

ValueError: Namespace Gtk not available

Yes, this means that the package which provides the GObject Introspection code for GTK+ (gir1.2-gtk-3.0) is not installed. Each gir1.2-foo package must be installed for the libraries you want to use in your app, if using python and gtk libraries.

dobey · 30 Jan 2020, 20:33

@rob said in What's Inside the Black Box of Ubuntu Touch?:

If running a GUI in a Libertine-Container, does it use Xmir or Xwayland or is there something else that has to be enabled on the host, after setting the DISPLAY environment variable? This is one of those Ubuntu specific things that just make it different and a little bit frustrating for those who don't know.

Currently, Xmir is used, however, you cannot simply run GUI apps from the command line, such as just typing firefox as Unity 8 authenticates apps in a certain way. Xwayland will eventually replace Xmir, when Wayland replaces MirClient as the protocol used, and the work to make Xwayland work properly in Unity 8 is done. You will need a .desktop file and icon, for any GUI app, and it will need to be started from the Unity 8 app drawer (or the legacy apps scope if using earlier versions), or with ubuntu-app-launch on the command line.

rob · 30 Jan 2020, 20:45

Thank you @dobey. Everything I learn will be shared with others in a simplified way so please consider me a time investment for UT.

Is there further documentation on ubuntu-app-launch? I've been searching for this since early today and haven't found much.

Yes, I understand about handling .desktop which I can see is stored in the container in the regular /usr/share/applications.

Is this launch command handling the display server? Is the handling of Xmir/Xwayland (future) done automatically? Or do I have to do something else either on the host or in the container?

I'm seeing prompts for setting the DISPLAY env variable and I see a -E parameter. But the sparse documentation doesn't make it clear what I have to do.