Why Miroil

Ubports relies on Mir for a large part of the graphics stack and Mir has been changing in ways that make that problematic. In particular:

1. Mir 2.x has made some server APIs private and these are used by the UBPorts QtMir and unity-system-compositor (unless that's been renamed to lomiri-system-compositor); and,
2. The other problem is that some features exposed by the mirclient API are not supported by Mir's Wayland support.

Andrew Koenig's "Fundamental Theorem of Software Engineering" says that "we can solve any problem by introducing an extra level of indirection." Miroil is an extra level of indirection that solves the problem of Lomiri depending on features that Mir no longer offers.

A bit of history

While Canonical was working on Ubuntu Touch there was a plan to update QtMir to remove the direct dependencies on mirserver and migrate some code into the (then separate) MirAL project that would provide the functionality QtMir needed from mirserver. As a first step much of the code that depended on mirserver was moved to a src/platforms/mirserver/miral directory in QtMir. This has been updated and extended over the years, but the intended move of the code out of QtMir into MirAL never happened.

A similar exercise for code in unity-system-compositor was started by @WebDrake and that too stalled before code was moved.

Introducing Miroil

Because of events this year I have a few more vacation days to use by the end of the year than I normally have. So I've decided to start a project to complete the work described above. (Although Miroil lives in the Mir repository it isn't going to be a significant part of my "day job" as Mir development lead.)

The point of Miroil is to provide the support from Mir needed for Ubuntu Touch to move to Mir 2.x and Wayland. This will require further input from downstream before it is fully functional and I'm appealing for any interested developers to help.

Current status of Miroil

I've created a feature branch from Mir "master" that included this new library and APIs and added the work already completed in QtMir.

I've taken a look at the unity-system-compositor work, but that is still incomplete and I don't currently have the time to bring that to a state ready to move into Miroil.

The future of Miroil

For any interested developers, it is possible to work on Miroil on "desktop" and test with QtMir, Lomiri and unity-system-compositor.

While the current work in progress is based on Mir 2.x it is sufficiently isolated from the rest of Mir to make a backport to Mir 1.x a simple "cherry-pick" once it reaches a sufficient level of maturity to be of interest to Lomiri development. (And doing that would simplify Lomiri migration to Mir 2.x.)

My plans

With my "hobby project" hat on, I've still got some (but not much) vacation time left to spend on this, and plan to look into providing a Wayland extension that maps easily to the "trusted prompt session" APIs from mirclient.

With my "Mir team lead" hat on, I'm happy to accept something like Miroil into the Mir source tree to support Lomiri development. And ensuring that it continues to build, pass tests and deploy. But further development will be a very low priority for a small team with other responsibilities.

Miroil needs you!

I won't have time to do everything that needs doing. But I will try to help anyone interested in joining this effort and am trying to provide a template to follow for the different aspects of work needed, in particular, server APIs and Wayland extensions.

[edit]
If anyone has the time to document how to build QtMir on the desktop and how to run the test program the project contains that would reduce the barrier for others interested in helping. (Also the same for unity-system-compositor and Lomiri.)
[end-edit]

---

References

General advice on developing Mir	https://mir-server.io/doc/getting_involved_in_mir.html
The Miroil PR	https://github.com/MirServer/mir/pull/1820
QtMir	https://github.com/ubports/qtmir
The unity-system-compositor PR	https://github.com/ubports/unity-system-compositor/pull/4

Those that have been following along will know that MirOil is a library intended to be added to Mir that will enable Lomiri to migrate to Mir 2.x.

The development of MirOil stalled last year as the people involved found the time available for it in short supply. Over the holidays I found some time to revisit this and made some progress.

I've been experimenting on building Mir+MirOil and QtMir on 20.04 with the UBports repository (http://repo2.ubports.com/) added. (Note this is not recommended for your main machine.)

The current situation is that there are now two branches in the Mir repository: "miroil" and "1.x/miroil" which add miroil to the main and release/1.8 branches of Mir respectively.

There's also a corresponding fork of UBport's QtMir branch "focal_-miroil-2.0" (from https://gitlab.com/ubports/core/qtmir). I've put this on https://github.com/MirOil-for-Lomiri/qtmir.git and called it "ubports/focal-_miroil-2.0".

The current status is that (with the right dependencies installed) that this version of QtMir builds against both the Mir branches mentioned above and runs against the 1.x branch. It fails to run against the 2.x branch because there are dependencies like lomiri-ui-toolkit that don't currently build against 2.x.

If you set up this environment, build and install Mir from "1.x/miroil" and then build and install QtMir from "ubports/focal_-_miroil-2.0" you can run a the demo program as follows:

export QML2_IMPORT_PATH=/usr/local/lib/qt5/qml
export QT_QPA_PLATFORM_PLUGIN_PATH=/usr/local/lib/qt5/plugins/platforms
QT_PLUGIN_PATH=/usr/local/lib/qt5/plugins
WAYLAND_DISPLAY=wayland-99 MIR_SERVER_CURSOR=null QT_QPA_PLATFORM=mirserver cmake-build-debug/demos/qtmir-demo-shell/qtmir-demo-shell&

And run, e.g. gedit with:

WAYLAND_DISPLAY=wayland-99  gedit

It will likely be some time before I do much more on this, so if anyone else wants to help push things forward: Please don't wait for me!

I've noticed a few discussions recently regarding developers getting involved in developing Unity8. I don't know all the answers myself, and don't have the free time to find them, but I think I can start a constructive discussion.

A bit of history

The Unity8 shell comprises a lot of inter-related projects and dependencies. These have to be coordinated and installed together, this worked best in Ubuntu 17.04 at which point it was possible to install a usable Unity8 desktop from the archive with a single command.

When Canonical withdrew from the project they understandably stopped maintaining these projects and dropped them and a number of the dependencies from the archives. There have also been some incompatible changes to some of the dependencies.

One of the big changes has been to the Mesa graphics drivers: Ubuntu used to carry a "Mir EGL" distro patch that enabled EGL clients to use the "mirclient API". Because Mesa changed in ways that broke that patch badly it was dropped in Ubuntu 18.04. I'll discuss this further below under "Going Wayland".

The UBports team has done its best to maintain the projects and dependencies in the UBports repo, but this hasn't been the most urgent thing to work on and they have not dealt with everything that has changed.

Getting started

I've never built the majority of Unity8 components from source, not am I sure which branches are canon in the UBports source repositories. I'm sure developers could figure this out, but it would save everyone time if there were a blogpost.

The bits I do know:

1. The first, best hope is 18.04LTS.
   Other distros and more recent series are possible, but let's make it work in one place before fixing everywhere.
2. The UBports repo at http://repo.ubports.com/ contains the dependencies, and needs to be added to apt sources.
   echo "deb http://repo.ubports.com/ $UBUNTU_CODENAME main" | sudo tee /etc/apt/sources.list.d/ubports.list
wget http://repo.ubports.com/keyring.gpg -O - | sudo apt-key add -
3. This is messy enough accidentally break your system: use test partition, VM, or secondary laptop as appropriate.
4. There are more detailed instructions in the README: https://github.com/ubports/unity8

Going Wayland

This is the bit I understand best (because I work on Mir). It is also, I think the biggest cause of deterioration since 17.04.

Unity8 was originally built using the "mirclient API" which was designed for convergence. But for "mirclient API" to be useful this needs support in client toolkits and even with Canonical pushing it this was never great. Since then, it has been dropped from GTK+ and SDL2.

Unity8 is based on Mir and @mariogrip had done the work needed for Unity8 to run on Ubuntu 18.04 without "Mir EGL" by using Wayland instead of the "mirclient API" (which is supported by recent versions of Mir).

However, Wayland is not a "drop-in" replacement for the "mirclient API" and there are some limitations apparent on Unity8 desktop:

1. GTK+ applications assume CSD and add shadows which Unity8 is "not expecting". That produces weird effects, like a disconnected, second titlebar.
   There is, theoretically, a solution to this upstream: https://github.com/MirServer/mir/issues/664
2. For X11applications the integration provided by Xwayland is even worse than that previously provided by Xmir
   Part of the solution to this is upstream: https://github.com/MirServer/mir/labels/Experimental
   2.1 But this isn't all: https://github.com/ubports/unity8/issues/56

I am biassed by my involvement in Mir, but these are issues that an interested developer could tackle to make a significant improvement to the Unity8 desktop. In addition, they are easier to work on than Unity8 as the Mir project has significantly fewer dependencies (all of which are in the Ubuntu archives).

Another effect of the loss of "Mir EGL" in Mesa and the consequent switch to Wayland ls that the "Mir-on-Mir" support needed to use the Unity greeter is no longer available. Some work has been done on providing a "Mir-on-Wayland" graphics platform in Mir, but that needs significant rework (it is on my list of things I want to get to, but if you fancy taking it on get in touch).

Note: There are additional limitations to using Wayland on the phone that need to be addressed, but I'm keeping this discussion to Unity8 desktop.

Other stuff

Unity8 was not finished in 17.04, and quite a few of the things that were "done" need to be updated. I'm not an expert on this so I'll leave detailing a list to others.

What I do know:

1. The issues list: https://github.com/ubports/unity8/issues

2019: Time to make Unity8 great again?

I hope this provides a bit of impetus to developers interested in contributing to Unity8 to progress further. Even if this post is simply a place for potential developers to declare their interest and meet each other it will have served a purpose.

Some context

A year ago I set out The Way to Wayland (for Unity8) which consisted of three pieces of work:

1. Updating to Mir 1.x
2. Updating Mir-on-Mir
3. Updating to Wayland

There's been progress on all fronts and, although none of these is entirely finished, it is time to look ahead again.

1. On the UBports side, The "devel" channel is working on moving Ubuntu Touch to a Mir 1.x base (Mir 1.2 AIUI)
2. On the Mir side Mir 1.6 introduced a Mir-on-Wayland platform that can, in theory, be used instead of Mir-on-Mir
3. On the UBports side there's been some initial work to use Wayland in Unity8 for the 18.04 desktop

The big piece of work outstanding is "Updating to Wayland":

Updating to Wayland

 
Using the mirclient API is also largely restricted to the phone and Mesa on 16.04LTS because GL rendering, like “Mir-on-MIr” depends upon Mir-EGL. That is a limitation on other distros, and other drivers.
 
The natural solution is to switch to Wayland and, with a recent Mir version, that can work for a lot of applications. However, Unity8 and the associated applications used by Ubuntu Touch also make use of Mir features designed to support “Convergence” and the security model Canonical designed for the phone.
 
This means that it isn’t possible to switch to everything Wayland without additional work. On the current phones that doesn’t matter for moving to Mir 1.x, as the mirclient APIs are still available and supported, but in the longer term there needs to be a way to support any features that prove necessary through Wayland. Fortunately, Wayland is designed to support protocol extensions and I would recommend this as the way forward.
 
Mir doesn’t (yet) have a clean API for adding Wayland protocol extensions in a server and, in any case, those that are needed by Ubuntu Touch are simply exposing existing Mir functionality. So, once the necessary protocol extensions are identified, the Mir code is the place to implement them.
 
A prerequisite for adding Wayland extensions is Wayland support in Mir, and that requires a recent version of Mir.

The point of the Wayland transition is to move away from the mirclient based APIs that Mir will be dropping in further development.

Mir 1.x is new and shiny, what is this "and beyond"?

For the immediate future, UBports will continue using Mir 1.x and the Mir team will try to provide backports of useful features to the 1.x series. But that will become increasingly difficult as time goes by.

The Mir team is in the process of removing mirclient specific stuff and that means there are already changes in Mir "master" that break all graphics using the mirclient API (if new features depend on those changes they won't appear in a Mir 1.x release).

Because of the history of Mir, the mirclient API actually affects other Mir APIs, for example Taking the mirclient out of MirAL. That means the impact of dropping mirclient on UBports goes further than simply switching qtubuntu-[desktop|android] to qtwayland for all applications.

QtMir

The principle component of the Unity8 stack that is affected by all this is QtMir. This the library responsible for joining Qt to the Mir server to customise the scenegraph and rendering. The way it does this is inelegant for historical reasons and everyone that has ever touched it has felt the urge to "clean it up". (Indeed, @greyback and I had started on this before Canonical withdrew from the project.)

Having these customisation points (animations, compositing and rendering) was one of the goals of Mir, and something the Mir team aims to provide in a more elegant way in the future. Ideally, one that isn't tied to Qt or GTK but can support alternatives. (Indeed @mariogrip has independently suggested the same idea.)

This seems like an area that would, once other priorities are cleared by both teams, would benefit from collaboration.

After another discussion around this topic I felt it time to write up something for the forum. However, I have to declare an interest: my "day job" is developing Mir. You have to decide if that means I know what I'm talking about, or if I'm promoting my own goals.

The Past

When the Unity8 project started Canonical needed a rich enough "language" for applications and server to "talk" to each other to support convergence. They chose to develop this in the form of an API: Mir. The idea was that Mir would provide a way for various shells (Unity8 and other shells) and applications to co-operate.

It may sound ambitions to expect that every existing application should stop talking X11 and start "talking Mir", but really there were reasons to move away from X11 and applications largely use one of a few "toolkits" instead of using X11 directly. Get support into those toolkits and the job is mostly done. For other applications they built Xmir - which is an X11 server that translates for applications that only talk X11. Xmir could never be perfect for the converged scenario, but applications would be able to run.

At least that was the theory. In practice things didn't work out that way: toolkit support was variable and, in general, was never good enough for convergence. While Qt got pretty close and SDL2 is usable, there were still serious problems with gtk3 and Xmir.

The Present

UBports has not given up on the idea of convergence and of running desktop applications on Unity8. But if those applications do not support "talking Mir" then there are currently no good options. The least bad option is to use Xmir, but the result isn't great.

Canonical is no longer developing Unity8, but it has continued with Mir because of the "other shells" mentioned above. That is good for UBPorts, as Mir is a substantial part of the functionality Unity8 needs. However, there is a problem both for Canonical and for UBPorts: applications are not going to "talk Mir".

The approach Canonical is taking is to support applications that talk Wayland in Mir. Wayland is another X11 replacement that is being taken up by toolkits. While the approach taken is different (there is a limited "core" protocol that all Wayland servers must support and an extension mechanism for servers that need something richer) there are enough parallels for this to work. There's even an Xmir replacement: Xwayland, plays the same role of interpreter.

The Future

UBports cannot rely on seeing improvements to the Mir support in toolkits nor improvements to Xmir. The way forward for UBports is to leverage the continuing work Canonical is doing on Mir to support Wayland clients.

That requires one, non-trivial, thing: Moving to the current, supported, version of Mir.

There are several reasons why that is non-trivial:

1. Mir is not supported on Vivid
2. Mir has dropped support for "android" graphics drivers
3. Code changes are needed to reflect changes to the Mir API
4. Some further changes are needed to support Wayland applications

Now for the good news:

1. Mir is supported on Xenial, and that is the next target for UBports
2. The code for using "android graphics" drivers is not lost, it is alive and well thanks to UBports developers
3. The code changes have been made before (Canonical made these changes for unity8-desktop-session on Zesty)
4. The bad news: while unity8-desktop-session on Zesty works with the latest Mir, Wayland applications are not rendered on the display.

Updating UBports to the latest version of Mir will take time and effort, but it is the most feasible way of getting desktop applications working.

Where we are now

While Unity8 has been most complete and successful on phone and tablet it was always intended that it would also encompass desktop operation.

After Canonical stopped work on Unity8 UBports has focused on supporting and developing phones and a separate project Yunit was working on the desktop case. However, nobody has been working on Yunit for a while now and people are asking on the UBports telegram group about Unity8 on desktop. This is an attempt to give a high level view of the implications.

There are two sets of differences between phone and desktop: the graphics "stack" and the input and display capabilities. Attaching external displays, keyboards and mice to phones allows a switch to "desktop mode" but uses the same libhybris/android graphics stack as for normal phone operation. On "desktop" architectures a different mesa-kms graphics stack is used.

For Unity8 or application developers the differences between these, or other possible, graphics stacks are hidden beneath the Mir libraries, but the distinction is important when installing onto a computer or device.

While all the software needed for the "desktop" installation exists in some form it hasn't been maintained since the Ubuntu 17.04 release. Indeed Canonical has dropped many of the packages from the upcoming 18.04LTS.

The way forward for "desktop"

The UBports developers are demonstrably sympathetic to a desktop variant, but their priority should be progressing the supported deployment onto the phone "stack". If people are interested in desktop they should try to progress it in a way that doesn't distract from these existing efforts.

Late last year I outlined a plan on the Yunit forum. It was probably too late for Yunit but I think much of that discussion is still valid.

There are differences between the UBports and Yunit codebases (and there's likely useful code on Canonical branches) this needs to be resolved.

In short, if there is the will, there is work that could be done.

tl;dr: There's nothing to worry about, this was expected and planned for.

The mirclient APIs used in Unity8 and 16.04 based applications is not dropped, and does not require a build-time option to enable it.

It simply needs switching on with a configuration option at runtime. (There are many ways that can be supplied, for example, adding code to QtMir and unity-system-compositor that fakes passing this as a "command-line option" to Mir.)

For a bit of perspective, these APIs have been deprecated for two years, and the UBports developers have been aware of that. Similarly, the Mir developers have been aware that they are used by UBports and have continued to build and test them.

Everyone working on this software stack has always known that UBports needs to move off these APIs at some point, but that cannot happen until there is a viable alternative. And that requires moving to a more recent version of Mir (at least Mir 1.3 for this Mir support for Wayland).

The change to "new Mir" is progressing, and is available on the "edge" channel. Delivering that will allow the migration to Wayland to start.

There have been discussions (here and here) about this migration and, when the time comes, the Mir team has the expertise, willingness and availability to help UBport move from mirclient APIs to Wayland protocols.

There may be a time during this transition where UBports cannot ship with the latest release of Mir, but the way forward is clear and the problems to be addressed are far smaller than ones that have already been overcome.

@advocatux it isn't just "now".

Even after 16.04 ships do we really want to spend a month moving everything instead of...

• Porting to new phone(s)?
• Getting anbox support released?
• Getting Unity8 desktop working again?
• Delivering other cool stuff?
• Fixing bugs?
• Promoting the project?

Really, unless we realistically expect things to be better after the move this has to be a bad idea. (Getting 50% more work done afterwards would be a good argument - that's break-even in 3 months.)

You did great for UBports, now do great for yourself!

It isn't frozen in time. When the project was forked the base was the already EOL 15.04 and a big task faced by UBports was to change that to 16.04. That was done a couple of years back.

The next big task was updating other dependencies in the stack to supported upstream versions. This may never be totally complete, but is largely done in OTA-12. This was more urgent than migrating to 18.04 as 16.04 was, and is still supported.

A stated goal is to update to 20.04LTS base before 16.04 goes EOL next year.

@psingh said in Ubuntu Touch Q&A 120 This Saturday 30th July At 19:00 UTC:

I recall though how when Canonical maintained the OS they had a "pages" feature

I think this was called "scopes".

@duncancragg said in Vulkan and USC 0.9.0 (Mir):

do you mean there was work done inside /Mesa/ to support mirclient's Vulkan buffers

Vulkan, like GL and GLES, is an API.

Mesa is an implementation of graphics API specifications like GL, GLES and Vulkan.

For each API work is needed to support mirclient.

Sorry, your phone has a niche graphics stack based on libhybris and android hardware drivers. The only folks that understand that are working on Ubuntu Touch (or other Halium based phone OSs).

I have no direct knowledge of the current status of Wayland (or Vulkan) support over this graphics stack. Better to ask someone that works on it.

@duncancragg firstly, the mirclient API has been deprecated for five years and I've done nothing with it in that time. So you will be very much on your own in attempting this.

Next, by way of context, Vulkan, GL, GLE, GLES, etc are ways for a client to form graphical content ("buffers") to pass to the server. (Yes, "the server" would be Lomiri, etc.)

For GL* there were changes to Mesa to submit the buffers using mirclient.

When Canonical dropped Unity8 and related projects there had been some initial work done for using mirclient via Vulkan, but it was never merged.

At a minimum, you'd need to update and complete the mirclient support in the Vulkan driver. But I don't know if that would be enough.

What you may have seen elsewhere, is that Vulkan applications using Wayland work with Mir. That doesn't help you with getting Vulkan working on mirclient. (And I suspect that is a lot more work than you hoped for.)

To some extent it depends on the actual goal. Is this intended as a test harness for an app in a controlled environment? Or as a means of automation on a desktop?

Faking udev ought to work across different compositors. The virtual keyboard extensions might work for some compositors but different compositors support different protocol extensions. (E.g. Mir supports virtual_keyboard_unstable_v1 but not org_kde_kwin_fake_input.)

If this is just for an application test harness, then it should be possible to test against a specific, even a bespoke, compositor.

This could be complicated for most people I know as they have employers, and work (especially paid) outside that raises contract concerns.

Maybe ask on the wayland-devel mailing list?

@ahmed_samy a phone is not a PC and Ubuntu Touch won't make it one. (Nor would postmarketOS, but that would be closer.)

Ubuntu Touch will make a phone a phone. Phone calls, messaging, web access, email, etc all working well enough for daily use. But it doesn't have the finish of iOS or Android (both of which have had enormously more investment.)

Generally, it doesn't run applications designed for Windows, iOS, Android or traditional Linux desktop distros. (Typically the developers of these apps have no interest in getting them to work on UT.)

It sounds as though what you want is a new Android phone. Ubuntu Touch isn't that.

@lakotaubp as I understand it, it is just the core team doesn't have the capacity to work on this for the foreseeable future. Other people interested in progressing the pinephone can and should work on it.

You did great for UBports, now do great for yourself!

There are a lot of related issues here.

Firstly, Mir is not "similar to Wayland", that confuses a protocol (Wayland) with an implementation of that protocol (Mir). Mir started out by providing an API (mirclient) to applications that API is similar to Wayland.

Mir supported both mirclient and Wayland for a while, but dropped mirclient as it is not widely used.

Ubuntu Touch uses an old version of Mir (1.8) that supports both Wayland and mirclient. But Ubuntu Touch only uses mirclient (except on PinePhone where Wayland is used, but the work to use Wayland is incomplete and the "gaps" cause problems).

Electron (and probably Chromium as it is based on the same engine) has some bugs in the way it implements Wayland and some work needed to get Electron/Wayland working with Mir 2.5.

One day Ubuntu Touch will update to a more recent version of Mir and applications based on Wayland will work. But this is not trivial (not all of the features of mirclient are available with Wayland and replacements are needed), and not as urgent as the move to Ubuntu 20.04.

UBPorts: phone technology supporting humanity