Why Miroil

Ubports relies on Mir for a large part of the graphics stack and Mir has been changing in ways that make that problematic. In particular:

1. Mir 2.x has made some server APIs private and these are used by the UBPorts QtMir and unity-system-compositor (unless that's been renamed to lomiri-system-compositor); and,
2. The other problem is that some features exposed by the mirclient API are not supported by Mir's Wayland support.

Andrew Koenig's "Fundamental Theorem of Software Engineering" says that "we can solve any problem by introducing an extra level of indirection." Miroil is an extra level of indirection that solves the problem of Lomiri depending on features that Mir no longer offers.

A bit of history

While Canonical was working on Ubuntu Touch there was a plan to update QtMir to remove the direct dependencies on mirserver and migrate some code into the (then separate) MirAL project that would provide the functionality QtMir needed from mirserver. As a first step much of the code that depended on mirserver was moved to a src/platforms/mirserver/miral directory in QtMir. This has been updated and extended over the years, but the intended move of the code out of QtMir into MirAL never happened.

A similar exercise for code in unity-system-compositor was started by @WebDrake and that too stalled before code was moved.

Introducing Miroil

Because of events this year I have a few more vacation days to use by the end of the year than I normally have. So I've decided to start a project to complete the work described above. (Although Miroil lives in the Mir repository it isn't going to be a significant part of my "day job" as Mir development lead.)

The point of Miroil is to provide the support from Mir needed for Ubuntu Touch to move to Mir 2.x and Wayland. This will require further input from downstream before it is fully functional and I'm appealing for any interested developers to help.

Current status of Miroil

I've created a feature branch from Mir "master" that included this new library and APIs and added the work already completed in QtMir.

I've taken a look at the unity-system-compositor work, but that is still incomplete and I don't currently have the time to bring that to a state ready to move into Miroil.

The future of Miroil

For any interested developers, it is possible to work on Miroil on "desktop" and test with QtMir, Lomiri and unity-system-compositor.

While the current work in progress is based on Mir 2.x it is sufficiently isolated from the rest of Mir to make a backport to Mir 1.x a simple "cherry-pick" once it reaches a sufficient level of maturity to be of interest to Lomiri development. (And doing that would simplify Lomiri migration to Mir 2.x.)

My plans

With my "hobby project" hat on, I've still got some (but not much) vacation time left to spend on this, and plan to look into providing a Wayland extension that maps easily to the "trusted prompt session" APIs from mirclient.

With my "Mir team lead" hat on, I'm happy to accept something like Miroil into the Mir source tree to support Lomiri development. And ensuring that it continues to build, pass tests and deploy. But further development will be a very low priority for a small team with other responsibilities.

Miroil needs you!

I won't have time to do everything that needs doing. But I will try to help anyone interested in joining this effort and am trying to provide a template to follow for the different aspects of work needed, in particular, server APIs and Wayland extensions.

[edit]
If anyone has the time to document how to build QtMir on the desktop and how to run the test program the project contains that would reduce the barrier for others interested in helping. (Also the same for unity-system-compositor and Lomiri.)
[end-edit]

---

References

General advice on developing Mir	https://mir-server.io/doc/getting_involved_in_mir.html
The Miroil PR	https://github.com/MirServer/mir/pull/1820
QtMir	https://github.com/ubports/qtmir
The unity-system-compositor PR	https://github.com/ubports/unity-system-compositor/pull/4

I've noticed a few discussions recently regarding developers getting involved in developing Unity8. I don't know all the answers myself, and don't have the free time to find them, but I think I can start a constructive discussion.

A bit of history

The Unity8 shell comprises a lot of inter-related projects and dependencies. These have to be coordinated and installed together, this worked best in Ubuntu 17.04 at which point it was possible to install a usable Unity8 desktop from the archive with a single command.

When Canonical withdrew from the project they understandably stopped maintaining these projects and dropped them and a number of the dependencies from the archives. There have also been some incompatible changes to some of the dependencies.

One of the big changes has been to the Mesa graphics drivers: Ubuntu used to carry a "Mir EGL" distro patch that enabled EGL clients to use the "mirclient API". Because Mesa changed in ways that broke that patch badly it was dropped in Ubuntu 18.04. I'll discuss this further below under "Going Wayland".

The UBports team has done its best to maintain the projects and dependencies in the UBports repo, but this hasn't been the most urgent thing to work on and they have not dealt with everything that has changed.

Getting started

I've never built the majority of Unity8 components from source, not am I sure which branches are canon in the UBports source repositories. I'm sure developers could figure this out, but it would save everyone time if there were a blogpost.

The bits I do know:

1. The first, best hope is 18.04LTS.
   Other distros and more recent series are possible, but let's make it work in one place before fixing everywhere.
2. The UBports repo at http://repo.ubports.com/ contains the dependencies, and needs to be added to apt sources.
   echo "deb http://repo.ubports.com/ $UBUNTU_CODENAME main" | sudo tee /etc/apt/sources.list.d/ubports.list
wget http://repo.ubports.com/keyring.gpg -O - | sudo apt-key add -
3. This is messy enough accidentally break your system: use test partition, VM, or secondary laptop as appropriate.
4. There are more detailed instructions in the README: https://github.com/ubports/unity8

Going Wayland

This is the bit I understand best (because I work on Mir). It is also, I think the biggest cause of deterioration since 17.04.

Unity8 was originally built using the "mirclient API" which was designed for convergence. But for "mirclient API" to be useful this needs support in client toolkits and even with Canonical pushing it this was never great. Since then, it has been dropped from GTK+ and SDL2.

Unity8 is based on Mir and @mariogrip had done the work needed for Unity8 to run on Ubuntu 18.04 without "Mir EGL" by using Wayland instead of the "mirclient API" (which is supported by recent versions of Mir).

However, Wayland is not a "drop-in" replacement for the "mirclient API" and there are some limitations apparent on Unity8 desktop:

1. GTK+ applications assume CSD and add shadows which Unity8 is "not expecting". That produces weird effects, like a disconnected, second titlebar.
   There is, theoretically, a solution to this upstream: https://github.com/MirServer/mir/issues/664
2. For X11applications the integration provided by Xwayland is even worse than that previously provided by Xmir
   Part of the solution to this is upstream: https://github.com/MirServer/mir/labels/Experimental
   2.1 But this isn't all: https://github.com/ubports/unity8/issues/56

I am biassed by my involvement in Mir, but these are issues that an interested developer could tackle to make a significant improvement to the Unity8 desktop. In addition, they are easier to work on than Unity8 as the Mir project has significantly fewer dependencies (all of which are in the Ubuntu archives).

Another effect of the loss of "Mir EGL" in Mesa and the consequent switch to Wayland ls that the "Mir-on-Mir" support needed to use the Unity greeter is no longer available. Some work has been done on providing a "Mir-on-Wayland" graphics platform in Mir, but that needs significant rework (it is on my list of things I want to get to, but if you fancy taking it on get in touch).

Note: There are additional limitations to using Wayland on the phone that need to be addressed, but I'm keeping this discussion to Unity8 desktop.

Other stuff

Unity8 was not finished in 17.04, and quite a few of the things that were "done" need to be updated. I'm not an expert on this so I'll leave detailing a list to others.

What I do know:

1. The issues list: https://github.com/ubports/unity8/issues

2019: Time to make Unity8 great again?

I hope this provides a bit of impetus to developers interested in contributing to Unity8 to progress further. Even if this post is simply a place for potential developers to declare their interest and meet each other it will have served a purpose.

Some context

A year ago I set out The Way to Wayland (for Unity8) which consisted of three pieces of work:

1. Updating to Mir 1.x
2. Updating Mir-on-Mir
3. Updating to Wayland

There's been progress on all fronts and, although none of these is entirely finished, it is time to look ahead again.

1. On the UBports side, The "devel" channel is working on moving Ubuntu Touch to a Mir 1.x base (Mir 1.2 AIUI)
2. On the Mir side Mir 1.6 introduced a Mir-on-Wayland platform that can, in theory, be used instead of Mir-on-Mir
3. On the UBports side there's been some initial work to use Wayland in Unity8 for the 18.04 desktop

The big piece of work outstanding is "Updating to Wayland":

Updating to Wayland

 
Using the mirclient API is also largely restricted to the phone and Mesa on 16.04LTS because GL rendering, like “Mir-on-MIr” depends upon Mir-EGL. That is a limitation on other distros, and other drivers.
 
The natural solution is to switch to Wayland and, with a recent Mir version, that can work for a lot of applications. However, Unity8 and the associated applications used by Ubuntu Touch also make use of Mir features designed to support “Convergence” and the security model Canonical designed for the phone.
 
This means that it isn’t possible to switch to everything Wayland without additional work. On the current phones that doesn’t matter for moving to Mir 1.x, as the mirclient APIs are still available and supported, but in the longer term there needs to be a way to support any features that prove necessary through Wayland. Fortunately, Wayland is designed to support protocol extensions and I would recommend this as the way forward.
 
Mir doesn’t (yet) have a clean API for adding Wayland protocol extensions in a server and, in any case, those that are needed by Ubuntu Touch are simply exposing existing Mir functionality. So, once the necessary protocol extensions are identified, the Mir code is the place to implement them.
 
A prerequisite for adding Wayland extensions is Wayland support in Mir, and that requires a recent version of Mir.

The point of the Wayland transition is to move away from the mirclient based APIs that Mir will be dropping in further development.

Mir 1.x is new and shiny, what is this "and beyond"?

For the immediate future, UBports will continue using Mir 1.x and the Mir team will try to provide backports of useful features to the 1.x series. But that will become increasingly difficult as time goes by.

The Mir team is in the process of removing mirclient specific stuff and that means there are already changes in Mir "master" that break all graphics using the mirclient API (if new features depend on those changes they won't appear in a Mir 1.x release).

Because of the history of Mir, the mirclient API actually affects other Mir APIs, for example Taking the mirclient out of MirAL. That means the impact of dropping mirclient on UBports goes further than simply switching qtubuntu-[desktop|android] to qtwayland for all applications.

QtMir

The principle component of the Unity8 stack that is affected by all this is QtMir. This the library responsible for joining Qt to the Mir server to customise the scenegraph and rendering. The way it does this is inelegant for historical reasons and everyone that has ever touched it has felt the urge to "clean it up". (Indeed, @greyback and I had started on this before Canonical withdrew from the project.)

Having these customisation points (animations, compositing and rendering) was one of the goals of Mir, and something the Mir team aims to provide in a more elegant way in the future. Ideally, one that isn't tied to Qt or GTK but can support alternatives. (Indeed @mariogrip has independently suggested the same idea.)

This seems like an area that would, once other priorities are cleared by both teams, would benefit from collaboration.

After another discussion around this topic I felt it time to write up something for the forum. However, I have to declare an interest: my "day job" is developing Mir. You have to decide if that means I know what I'm talking about, or if I'm promoting my own goals.

The Past

When the Unity8 project started Canonical needed a rich enough "language" for applications and server to "talk" to each other to support convergence. They chose to develop this in the form of an API: Mir. The idea was that Mir would provide a way for various shells (Unity8 and other shells) and applications to co-operate.

It may sound ambitions to expect that every existing application should stop talking X11 and start "talking Mir", but really there were reasons to move away from X11 and applications largely use one of a few "toolkits" instead of using X11 directly. Get support into those toolkits and the job is mostly done. For other applications they built Xmir - which is an X11 server that translates for applications that only talk X11. Xmir could never be perfect for the converged scenario, but applications would be able to run.

At least that was the theory. In practice things didn't work out that way: toolkit support was variable and, in general, was never good enough for convergence. While Qt got pretty close and SDL2 is usable, there were still serious problems with gtk3 and Xmir.

The Present

UBports has not given up on the idea of convergence and of running desktop applications on Unity8. But if those applications do not support "talking Mir" then there are currently no good options. The least bad option is to use Xmir, but the result isn't great.

Canonical is no longer developing Unity8, but it has continued with Mir because of the "other shells" mentioned above. That is good for UBPorts, as Mir is a substantial part of the functionality Unity8 needs. However, there is a problem both for Canonical and for UBPorts: applications are not going to "talk Mir".

The approach Canonical is taking is to support applications that talk Wayland in Mir. Wayland is another X11 replacement that is being taken up by toolkits. While the approach taken is different (there is a limited "core" protocol that all Wayland servers must support and an extension mechanism for servers that need something richer) there are enough parallels for this to work. There's even an Xmir replacement: Xwayland, plays the same role of interpreter.

The Future

UBports cannot rely on seeing improvements to the Mir support in toolkits nor improvements to Xmir. The way forward for UBports is to leverage the continuing work Canonical is doing on Mir to support Wayland clients.

That requires one, non-trivial, thing: Moving to the current, supported, version of Mir.

There are several reasons why that is non-trivial:

1. Mir is not supported on Vivid
2. Mir has dropped support for "android" graphics drivers
3. Code changes are needed to reflect changes to the Mir API
4. Some further changes are needed to support Wayland applications

Now for the good news:

1. Mir is supported on Xenial, and that is the next target for UBports
2. The code for using "android graphics" drivers is not lost, it is alive and well thanks to UBports developers
3. The code changes have been made before (Canonical made these changes for unity8-desktop-session on Zesty)
4. The bad news: while unity8-desktop-session on Zesty works with the latest Mir, Wayland applications are not rendered on the display.

Updating UBports to the latest version of Mir will take time and effort, but it is the most feasible way of getting desktop applications working.

tl;dr: There's nothing to worry about, this was expected and planned for.

The mirclient APIs used in Unity8 and 16.04 based applications is not dropped, and does not require a build-time option to enable it.

It simply needs switching on with a configuration option at runtime. (There are many ways that can be supplied, for example, adding code to QtMir and unity-system-compositor that fakes passing this as a "command-line option" to Mir.)

For a bit of perspective, these APIs have been deprecated for two years, and the UBports developers have been aware of that. Similarly, the Mir developers have been aware that they are used by UBports and have continued to build and test them.

Everyone working on this software stack has always known that UBports needs to move off these APIs at some point, but that cannot happen until there is a viable alternative. And that requires moving to a more recent version of Mir (at least Mir 1.3 for this Mir support for Wayland).

The change to "new Mir" is progressing, and is available on the "edge" channel. Delivering that will allow the migration to Wayland to start.

There have been discussions (here and here) about this migration and, when the time comes, the Mir team has the expertise, willingness and availability to help UBport move from mirclient APIs to Wayland protocols.

There may be a time during this transition where UBports cannot ship with the latest release of Mir, but the way forward is clear and the problems to be addressed are far smaller than ones that have already been overcome.

Where we are now

While Unity8 has been most complete and successful on phone and tablet it was always intended that it would also encompass desktop operation.

After Canonical stopped work on Unity8 UBports has focused on supporting and developing phones and a separate project Yunit was working on the desktop case. However, nobody has been working on Yunit for a while now and people are asking on the UBports telegram group about Unity8 on desktop. This is an attempt to give a high level view of the implications.

There are two sets of differences between phone and desktop: the graphics "stack" and the input and display capabilities. Attaching external displays, keyboards and mice to phones allows a switch to "desktop mode" but uses the same libhybris/android graphics stack as for normal phone operation. On "desktop" architectures a different mesa-kms graphics stack is used.

For Unity8 or application developers the differences between these, or other possible, graphics stacks are hidden beneath the Mir libraries, but the distinction is important when installing onto a computer or device.

While all the software needed for the "desktop" installation exists in some form it hasn't been maintained since the Ubuntu 17.04 release. Indeed Canonical has dropped many of the packages from the upcoming 18.04LTS.

The way forward for "desktop"

The UBports developers are demonstrably sympathetic to a desktop variant, but their priority should be progressing the supported deployment onto the phone "stack". If people are interested in desktop they should try to progress it in a way that doesn't distract from these existing efforts.

Late last year I outlined a plan on the Yunit forum. It was probably too late for Yunit but I think much of that discussion is still valid.

There are differences between the UBports and Yunit codebases (and there's likely useful code on Canonical branches) this needs to be resolved.

In short, if there is the will, there is work that could be done.

@advocatux it isn't just "now".

Even after 16.04 ships do we really want to spend a month moving everything instead of...

• Porting to new phone(s)?
• Getting anbox support released?
• Getting Unity8 desktop working again?
• Delivering other cool stuff?
• Fixing bugs?
• Promoting the project?

Really, unless we realistically expect things to be better after the move this has to be a bad idea. (Getting 50% more work done afterwards would be a good argument - that's break-even in 3 months.)

It isn't frozen in time. When the project was forked the base was the already EOL 15.04 and a big task faced by UBports was to change that to 16.04. That was done a couple of years back.

The next big task was updating other dependencies in the stack to supported upstream versions. This may never be totally complete, but is largely done in OTA-12. This was more urgent than migrating to 18.04 as 16.04 was, and is still supported.

A stated goal is to update to 20.04LTS base before 16.04 goes EOL next year.

Is there any plan to replace Mir?

An oft asked question has arisen on the Telegram channel once again and I thought I'd post an answer here for posterity:

Is there any plan to replace Mir? Since it's completely abandoned, it may take too much effort to maintain it, and it may better to switch to Wayland, if it's possible, since it has way more support. - so manpower could be used somewhere else (PWA maybe?)

Greatly exaggerated

"I can understand perfectly how the report of my illness got about, I have even heard on good authority that I was dead. [A cousin] was ill in London two or three weeks ago, but is well now. The report of my illness great out of his illness,. The report of my death was an exaggeration." - Mark Twain

Like the reports of Mark Twain's death, Mir's abandonment is based on a misconception.

There's a lot of confusion about this because "the internet" has confused toolkits supporting the mirclient API with servers using the Mir libraries. These are two different stories.

The mirclient API

The mirclient API is a way for application toolkits (and other client-side libraries) to use Mir. This API is abandoned: it is no longer under development and the Mir developers have deprecated it and will remove it "someday". I think UBports is the only remaining project to use it (the Mir support in Kodi, SDL2, gtk3, etc has been dropped).

UBports do need to switch from using the mirclient API to using Wayland before "someday" becomes "today". That is a multi-step process described here: The Way to Wayland (for Unity8)

The Mir server libraries

Mir was the core of Canonical's effort to enable "Convergence". [See: https://forums.ubports.com/post/18392 for my take on "Convergence"]. These libraries are still supported and developed by Canonical and there are a number of projects using them or planning to use them. Canonical has IoT projects, some experimental "desktop" work (including with MATE).

There is no reason for UBports to switch from using the Mir server libraries, they are the foundation on which Unity8 (and some other, less visible, components) are built. Even with the issues presented by the switch to using Wayland for applications the support for "Convergence" is unique to Mir and core to UBports.

Status 2020-12-02

Firstly, thanks for all the interest. I hope to make it easier for others to get involved but haven't achieved that yet. It won't get done in just the bits of time I am able to spare.

Today's the last opportunity I'll have to spend time on miroil until "the holidays" (and I don't yet know how much I'll able to do then), so I'll spend it writing a status report.

Wayland extensions

There hasn't been much change in the miroil code since last week's post, the only thing is that I started drafting a Wayland extension protocol to support the "trusted prompt sessions" used by Lomiri. I've not yet resolved all the issues with the protocol definition and not started the code to implement or use it. However, what I have done might provide a skeleton for anyone wanting to implement additional protocols for Lomiri.

QtMir

QtMir is the key thing to get going with miroil as that will "magically" make Lomiri work with Mir 2.x.

However, it is not ready to use with miroil at present, especially the Mir-2.x based branch I've been working on. There are some dependencies on both mirclient and mirserver that need the "extra level of indirection" to be written before it can be reimplemented in miroil.

I've been thinking of rebasing miroil on Mir-1.8 to make this work a bit easier, but not found the time to do this. (Rebasing should be trivial though.)

AFAIK nobody has yet stepped up and documented a development process for using QtMir and Mir/miroil on the desktop. Although I did get one volunteer. (You know who you are!)

unity-system-compositor

No change here.

@jbriones the code currently used on devel is being worked on for OTA16, but it isn't finished or final. Only once it has been tested on RC and released to stable will it be OTA16.

@jonius said in Introducing Miroil:

@jonius next issue is a failing test (ApplicationManager), which fails here.

I had a look over the weekend. The test is failing in the branch I based this off, and I didn't see an obvious cause for it.

I probably need to find a better base branch for this work. I really find all these QtMir branches a mystery. Maybe the best one is one of those in the gitlab repo?

@jonius said in Introducing Miroil:

@alan_g does that mean no need for a PR from my side?

Well, I've fixed the compile error, but you modernized the code as well. I think that's still worth a PR.

@aury88 said in Introducing Miroil:

@jonius i tried again after the git pull but obtain the same error

Sorry, you'll need a git fetch first.

@aury88 I'm not sure when I'll be at a computer where I can fix this. In the meantime you could google enough git magic to cherry-pick @jonius's fix.

UPDATE: I've pushed a fix for the build error. There's still the test failure noted above.

@Aury88 You can get the fix with git pull

@jonius can you PR your fix? It looks to tidy up the cmake code too.

@gizmochicken said in Ubuntu Touch Q&A 92 Saturday 16th January at 19:00 UTC:

QUESTION: I realize you have many things on your plate now, and so I'm not asking about your immediate plans, but would you consider bringing Lomiri to Ubuntu Core sometime in the future?

Ubuntu Core only supports confined snaps. (Not debs, "classic" snaps, or other packaging formats.)

Running any shell under snap confinement is an unsolved problem that I and others are working on. That needs to be addressed before it would be feasible to deploy Lomiri this way.

@kevin before you can install on a device you need a "port" for that device.

There are some "unsupported" ports (typically ones that haven't been completed).

The entry point to the porting process is: https://docs.halium.org/en/latest/porting/first-steps.html

@phoenixlandpirat said in What about Matrix?:

Wayland had much if anything to do with flutter

Flutter is similar to toolkits like GTK, Qt, SDL and so on. The application code talks to Flutter and Flutter talks to the graphical shell. Flutter has "platforms" for various stacks e.g. Android and iOS. The Flutter "Linux" platform will work on shells that support Wayland or shells that support X11.

It is no secret that the Mir team is helping with the Wayland support for Flutter and the Wayland support for Lomiri. So there's every reason to expect Flutter apps to work once the Wayland support has been released by both projects.