Why Miroil

Ubports relies on Mir for a large part of the graphics stack and Mir has been changing in ways that make that problematic. In particular:

1. Mir 2.x has made some server APIs private and these are used by the UBPorts QtMir and unity-system-compositor (unless that's been renamed to lomiri-system-compositor); and,
2. The other problem is that some features exposed by the mirclient API are not supported by Mir's Wayland support.

Andrew Koenig's "Fundamental Theorem of Software Engineering" says that "we can solve any problem by introducing an extra level of indirection." Miroil is an extra level of indirection that solves the problem of Lomiri depending on features that Mir no longer offers.

A bit of history

While Canonical was working on Ubuntu Touch there was a plan to update QtMir to remove the direct dependencies on mirserver and migrate some code into the (then separate) MirAL project that would provide the functionality QtMir needed from mirserver. As a first step much of the code that depended on mirserver was moved to a src/platforms/mirserver/miral directory in QtMir. This has been updated and extended over the years, but the intended move of the code out of QtMir into MirAL never happened.

A similar exercise for code in unity-system-compositor was started by @WebDrake and that too stalled before code was moved.

Introducing Miroil

Because of events this year I have a few more vacation days to use by the end of the year than I normally have. So I've decided to start a project to complete the work described above. (Although Miroil lives in the Mir repository it isn't going to be a significant part of my "day job" as Mir development lead.)

The point of Miroil is to provide the support from Mir needed for Ubuntu Touch to move to Mir 2.x and Wayland. This will require further input from downstream before it is fully functional and I'm appealing for any interested developers to help.

Current status of Miroil

I've created a feature branch from Mir "master" that included this new library and APIs and added the work already completed in QtMir.

I've taken a look at the unity-system-compositor work, but that is still incomplete and I don't currently have the time to bring that to a state ready to move into Miroil.

The future of Miroil

For any interested developers, it is possible to work on Miroil on "desktop" and test with QtMir, Lomiri and unity-system-compositor.

While the current work in progress is based on Mir 2.x it is sufficiently isolated from the rest of Mir to make a backport to Mir 1.x a simple "cherry-pick" once it reaches a sufficient level of maturity to be of interest to Lomiri development. (And doing that would simplify Lomiri migration to Mir 2.x.)

My plans

With my "hobby project" hat on, I've still got some (but not much) vacation time left to spend on this, and plan to look into providing a Wayland extension that maps easily to the "trusted prompt session" APIs from mirclient.

With my "Mir team lead" hat on, I'm happy to accept something like Miroil into the Mir source tree to support Lomiri development. And ensuring that it continues to build, pass tests and deploy. But further development will be a very low priority for a small team with other responsibilities.

Miroil needs you!

I won't have time to do everything that needs doing. But I will try to help anyone interested in joining this effort and am trying to provide a template to follow for the different aspects of work needed, in particular, server APIs and Wayland extensions.

[edit]
If anyone has the time to document how to build QtMir on the desktop and how to run the test program the project contains that would reduce the barrier for others interested in helping. (Also the same for unity-system-compositor and Lomiri.)
[end-edit]

References

Those that have been following along will know that MirOil is a library intended to be added to Mir that will enable Lomiri to migrate to Mir 2.x.

The development of MirOil stalled last year as the people involved found the time available for it in short supply. Over the holidays I found some time to revisit this and made some progress.

I've been experimenting on building Mir+MirOil and QtMir on 20.04 with the UBports repository (http://repo2.ubports.com/) added. (Note this is not recommended for your main machine.)

The current situation is that there are now two branches in the Mir repository: "miroil" and "1.x/miroil" which add miroil to the main and release/1.8 branches of Mir respectively.

There's also a corresponding fork of UBport's QtMir branch "focal_-miroil-2.0" (from https://gitlab.com/ubports/core/qtmir). I've put this on https://github.com/MirOil-for-Lomiri/qtmir.git and called it "ubports/focal-_miroil-2.0".

The current status is that (with the right dependencies installed) that this version of QtMir builds against both the Mir branches mentioned above and runs against the 1.x branch. It fails to run against the 2.x branch because there are dependencies like lomiri-ui-toolkit that don't currently build against 2.x.

If you set up this environment, build and install Mir from "1.x/miroil" and then build and install QtMir from "ubports/focal_-_miroil-2.0" you can run a the demo program as follows:

export QML2_IMPORT_PATH=/usr/local/lib/qt5/qml
export QT_QPA_PLATFORM_PLUGIN_PATH=/usr/local/lib/qt5/plugins/platforms
QT_PLUGIN_PATH=/usr/local/lib/qt5/plugins
WAYLAND_DISPLAY=wayland-99 MIR_SERVER_CURSOR=null QT_QPA_PLATFORM=mirserver cmake-build-debug/demos/qtmir-demo-shell/qtmir-demo-shell&

And run, e.g. gedit with:

WAYLAND_DISPLAY=wayland-99  gedit

It will likely be some time before I do much more on this, so if anyone else wants to help push things forward: Please don't wait for me!

I've noticed a few discussions recently regarding developers getting involved in developing Unity8. I don't know all the answers myself, and don't have the free time to find them, but I think I can start a constructive discussion.

A bit of history

The Unity8 shell comprises a lot of inter-related projects and dependencies. These have to be coordinated and installed together, this worked best in Ubuntu 17.04 at which point it was possible to install a usable Unity8 desktop from the archive with a single command.

When Canonical withdrew from the project they understandably stopped maintaining these projects and dropped them and a number of the dependencies from the archives. There have also been some incompatible changes to some of the dependencies.

One of the big changes has been to the Mesa graphics drivers: Ubuntu used to carry a "Mir EGL" distro patch that enabled EGL clients to use the "mirclient API". Because Mesa changed in ways that broke that patch badly it was dropped in Ubuntu 18.04. I'll discuss this further below under "Going Wayland".

The UBports team has done its best to maintain the projects and dependencies in the UBports repo, but this hasn't been the most urgent thing to work on and they have not dealt with everything that has changed.

Getting started

I've never built the majority of Unity8 components from source, not am I sure which branches are canon in the UBports source repositories. I'm sure developers could figure this out, but it would save everyone time if there were a blogpost.

The bits I do know:

1. The first, best hope is 18.04LTS.
   Other distros and more recent series are possible, but let's make it work in one place before fixing everywhere.
2. The UBports repo at http://repo.ubports.com/ contains the dependencies, and needs to be added to apt sources.
   echo "deb http://repo.ubports.com/ $UBUNTU_CODENAME main" | sudo tee /etc/apt/sources.list.d/ubports.list
wget http://repo.ubports.com/keyring.gpg -O - | sudo apt-key add -
3. This is messy enough accidentally break your system: use test partition, VM, or secondary laptop as appropriate.
4. There are more detailed instructions in the README: https://github.com/ubports/unity8

Going Wayland

This is the bit I understand best (because I work on Mir). It is also, I think the biggest cause of deterioration since 17.04.

Unity8 was originally built using the "mirclient API" which was designed for convergence. But for "mirclient API" to be useful this needs support in client toolkits and even with Canonical pushing it this was never great. Since then, it has been dropped from GTK+ and SDL2.

Unity8 is based on Mir and @mariogrip had done the work needed for Unity8 to run on Ubuntu 18.04 without "Mir EGL" by using Wayland instead of the "mirclient API" (which is supported by recent versions of Mir).

However, Wayland is not a "drop-in" replacement for the "mirclient API" and there are some limitations apparent on Unity8 desktop:

1. GTK+ applications assume CSD and add shadows which Unity8 is "not expecting". That produces weird effects, like a disconnected, second titlebar.
   There is, theoretically, a solution to this upstream: https://github.com/MirServer/mir/issues/664
2. For X11applications the integration provided by Xwayland is even worse than that previously provided by Xmir
   Part of the solution to this is upstream: https://github.com/MirServer/mir/labels/Experimental
   2.1 But this isn't all: https://github.com/ubports/unity8/issues/56

I am biassed by my involvement in Mir, but these are issues that an interested developer could tackle to make a significant improvement to the Unity8 desktop. In addition, they are easier to work on than Unity8 as the Mir project has significantly fewer dependencies (all of which are in the Ubuntu archives).

Another effect of the loss of "Mir EGL" in Mesa and the consequent switch to Wayland ls that the "Mir-on-Mir" support needed to use the Unity greeter is no longer available. Some work has been done on providing a "Mir-on-Wayland" graphics platform in Mir, but that needs significant rework (it is on my list of things I want to get to, but if you fancy taking it on get in touch).

Note: There are additional limitations to using Wayland on the phone that need to be addressed, but I'm keeping this discussion to Unity8 desktop.

Other stuff

Unity8 was not finished in 17.04, and quite a few of the things that were "done" need to be updated. I'm not an expert on this so I'll leave detailing a list to others.

What I do know:

1. The issues list: https://github.com/ubports/unity8/issues

2019: Time to make Unity8 great again?

I hope this provides a bit of impetus to developers interested in contributing to Unity8 to progress further. Even if this post is simply a place for potential developers to declare their interest and meet each other it will have served a purpose.

Some context

A year ago I set out The Way to Wayland (for Unity8) which consisted of three pieces of work:

1. Updating to Mir 1.x
2. Updating Mir-on-Mir
3. Updating to Wayland

There's been progress on all fronts and, although none of these is entirely finished, it is time to look ahead again.

1. On the UBports side, The "devel" channel is working on moving Ubuntu Touch to a Mir 1.x base (Mir 1.2 AIUI)
2. On the Mir side Mir 1.6 introduced a Mir-on-Wayland platform that can, in theory, be used instead of Mir-on-Mir
3. On the UBports side there's been some initial work to use Wayland in Unity8 for the 18.04 desktop

The big piece of work outstanding is "Updating to Wayland":

Updating to Wayland

 
Using the mirclient API is also largely restricted to the phone and Mesa on 16.04LTS because GL rendering, like “Mir-on-MIr” depends upon Mir-EGL. That is a limitation on other distros, and other drivers.
 
The natural solution is to switch to Wayland and, with a recent Mir version, that can work for a lot of applications. However, Unity8 and the associated applications used by Ubuntu Touch also make use of Mir features designed to support “Convergence” and the security model Canonical designed for the phone.
 
This means that it isn’t possible to switch to everything Wayland without additional work. On the current phones that doesn’t matter for moving to Mir 1.x, as the mirclient APIs are still available and supported, but in the longer term there needs to be a way to support any features that prove necessary through Wayland. Fortunately, Wayland is designed to support protocol extensions and I would recommend this as the way forward.
 
Mir doesn’t (yet) have a clean API for adding Wayland protocol extensions in a server and, in any case, those that are needed by Ubuntu Touch are simply exposing existing Mir functionality. So, once the necessary protocol extensions are identified, the Mir code is the place to implement them.
 
A prerequisite for adding Wayland extensions is Wayland support in Mir, and that requires a recent version of Mir.

The point of the Wayland transition is to move away from the mirclient based APIs that Mir will be dropping in further development.

Mir 1.x is new and shiny, what is this "and beyond"?

For the immediate future, UBports will continue using Mir 1.x and the Mir team will try to provide backports of useful features to the 1.x series. But that will become increasingly difficult as time goes by.

The Mir team is in the process of removing mirclient specific stuff and that means there are already changes in Mir "master" that break all graphics using the mirclient API (if new features depend on those changes they won't appear in a Mir 1.x release).

Because of the history of Mir, the mirclient API actually affects other Mir APIs, for example Taking the mirclient out of MirAL. That means the impact of dropping mirclient on UBports goes further than simply switching qtubuntu-[desktop|android] to qtwayland for all applications.

QtMir

The principle component of the Unity8 stack that is affected by all this is QtMir. This the library responsible for joining Qt to the Mir server to customise the scenegraph and rendering. The way it does this is inelegant for historical reasons and everyone that has ever touched it has felt the urge to "clean it up". (Indeed, @greyback and I had started on this before Canonical withdrew from the project.)

Having these customisation points (animations, compositing and rendering) was one of the goals of Mir, and something the Mir team aims to provide in a more elegant way in the future. Ideally, one that isn't tied to Qt or GTK but can support alternatives. (Indeed @mariogrip has independently suggested the same idea.)

This seems like an area that would, once other priorities are cleared by both teams, would benefit from collaboration.

After another discussion around this topic I felt it time to write up something for the forum. However, I have to declare an interest: my "day job" is developing Mir. You have to decide if that means I know what I'm talking about, or if I'm promoting my own goals.

The Past

When the Unity8 project started Canonical needed a rich enough "language" for applications and server to "talk" to each other to support convergence. They chose to develop this in the form of an API: Mir. The idea was that Mir would provide a way for various shells (Unity8 and other shells) and applications to co-operate.

It may sound ambitions to expect that every existing application should stop talking X11 and start "talking Mir", but really there were reasons to move away from X11 and applications largely use one of a few "toolkits" instead of using X11 directly. Get support into those toolkits and the job is mostly done. For other applications they built Xmir - which is an X11 server that translates for applications that only talk X11. Xmir could never be perfect for the converged scenario, but applications would be able to run.

At least that was the theory. In practice things didn't work out that way: toolkit support was variable and, in general, was never good enough for convergence. While Qt got pretty close and SDL2 is usable, there were still serious problems with gtk3 and Xmir.

The Present

UBports has not given up on the idea of convergence and of running desktop applications on Unity8. But if those applications do not support "talking Mir" then there are currently no good options. The least bad option is to use Xmir, but the result isn't great.

Canonical is no longer developing Unity8, but it has continued with Mir because of the "other shells" mentioned above. That is good for UBPorts, as Mir is a substantial part of the functionality Unity8 needs. However, there is a problem both for Canonical and for UBPorts: applications are not going to "talk Mir".

The approach Canonical is taking is to support applications that talk Wayland in Mir. Wayland is another X11 replacement that is being taken up by toolkits. While the approach taken is different (there is a limited "core" protocol that all Wayland servers must support and an extension mechanism for servers that need something richer) there are enough parallels for this to work. There's even an Xmir replacement: Xwayland, plays the same role of interpreter.

The Future

UBports cannot rely on seeing improvements to the Mir support in toolkits nor improvements to Xmir. The way forward for UBports is to leverage the continuing work Canonical is doing on Mir to support Wayland clients.

That requires one, non-trivial, thing: Moving to the current, supported, version of Mir.

There are several reasons why that is non-trivial:

1. Mir is not supported on Vivid
2. Mir has dropped support for "android" graphics drivers
3. Code changes are needed to reflect changes to the Mir API
4. Some further changes are needed to support Wayland applications

Now for the good news:

1. Mir is supported on Xenial, and that is the next target for UBports
2. The code for using "android graphics" drivers is not lost, it is alive and well thanks to UBports developers
3. The code changes have been made before (Canonical made these changes for unity8-desktop-session on Zesty)
4. The bad news: while unity8-desktop-session on Zesty works with the latest Mir, Wayland applications are not rendered on the display.

Updating UBports to the latest version of Mir will take time and effort, but it is the most feasible way of getting desktop applications working.

tl;dr: There's nothing to worry about, this was expected and planned for.

The mirclient APIs used in Unity8 and 16.04 based applications is not dropped, and does not require a build-time option to enable it.

It simply needs switching on with a configuration option at runtime. (There are many ways that can be supplied, for example, adding code to QtMir and unity-system-compositor that fakes passing this as a "command-line option" to Mir.)

For a bit of perspective, these APIs have been deprecated for two years, and the UBports developers have been aware of that. Similarly, the Mir developers have been aware that they are used by UBports and have continued to build and test them.

Everyone working on this software stack has always known that UBports needs to move off these APIs at some point, but that cannot happen until there is a viable alternative. And that requires moving to a more recent version of Mir (at least Mir 1.3 for this Mir support for Wayland).

The change to "new Mir" is progressing, and is available on the "edge" channel. Delivering that will allow the migration to Wayland to start.

There have been discussions (here and here) about this migration and, when the time comes, the Mir team has the expertise, willingness and availability to help UBport move from mirclient APIs to Wayland protocols.

There may be a time during this transition where UBports cannot ship with the latest release of Mir, but the way forward is clear and the problems to be addressed are far smaller than ones that have already been overcome.

Where we are now

While Unity8 has been most complete and successful on phone and tablet it was always intended that it would also encompass desktop operation.

After Canonical stopped work on Unity8 UBports has focused on supporting and developing phones and a separate project Yunit was working on the desktop case. However, nobody has been working on Yunit for a while now and people are asking on the UBports telegram group about Unity8 on desktop. This is an attempt to give a high level view of the implications.

There are two sets of differences between phone and desktop: the graphics "stack" and the input and display capabilities. Attaching external displays, keyboards and mice to phones allows a switch to "desktop mode" but uses the same libhybris/android graphics stack as for normal phone operation. On "desktop" architectures a different mesa-kms graphics stack is used.

For Unity8 or application developers the differences between these, or other possible, graphics stacks are hidden beneath the Mir libraries, but the distinction is important when installing onto a computer or device.

While all the software needed for the "desktop" installation exists in some form it hasn't been maintained since the Ubuntu 17.04 release. Indeed Canonical has dropped many of the packages from the upcoming 18.04LTS.

The way forward for "desktop"

The UBports developers are demonstrably sympathetic to a desktop variant, but their priority should be progressing the supported deployment onto the phone "stack". If people are interested in desktop they should try to progress it in a way that doesn't distract from these existing efforts.

Late last year I outlined a plan on the Yunit forum. It was probably too late for Yunit but I think much of that discussion is still valid.

There are differences between the UBports and Yunit codebases (and there's likely useful code on Canonical branches) this needs to be resolved.

In short, if there is the will, there is work that could be done.

@advocatux it isn't just "now".

Even after 16.04 ships do we really want to spend a month moving everything instead of...

• Porting to new phone(s)?
• Getting anbox support released?
• Getting Unity8 desktop working again?
• Delivering other cool stuff?
• Fixing bugs?
• Promoting the project?

Really, unless we realistically expect things to be better after the move this has to be a bad idea. (Getting 50% more work done afterwards would be a good argument - that's break-even in 3 months.)

Is there any plan to replace Mir?

An oft asked question has arisen on the Telegram channel once again and I thought I'd post an answer here for posterity:

Is there any plan to replace Mir? Since it's completely abandoned, it may take too much effort to maintain it, and it may better to switch to Wayland, if it's possible, since it has way more support. - so manpower could be used somewhere else (PWA maybe?)

Greatly exaggerated

"I can understand perfectly how the report of my illness got about, I have even heard on good authority that I was dead. [A cousin] was ill in London two or three weeks ago, but is well now. The report of my illness great out of his illness,. The report of my death was an exaggeration." - Mark Twain

Like the reports of Mark Twain's death, Mir's abandonment is based on a misconception.

There's a lot of confusion about this because "the internet" has confused toolkits supporting the mirclient API with servers using the Mir libraries. These are two different stories.

The mirclient API

The mirclient API is a way for application toolkits (and other client-side libraries) to use Mir. This API is abandoned: it is no longer under development and the Mir developers have deprecated it and will remove it "someday". I think UBports is the only remaining project to use it (the Mir support in Kodi, SDL2, gtk3, etc has been dropped).

UBports do need to switch from using the mirclient API to using Wayland before "someday" becomes "today". That is a multi-step process described here: The Way to Wayland (for Unity8)

The Mir server libraries

Mir was the core of Canonical's effort to enable "Convergence". [See: https://forums.ubports.com/post/18392 for my take on "Convergence"]. These libraries are still supported and developed by Canonical and there are a number of projects using them or planning to use them. Canonical has IoT projects, some experimental "desktop" work (including with MATE).

There is no reason for UBports to switch from using the Mir server libraries, they are the foundation on which Unity8 (and some other, less visible, components) are built. Even with the issues presented by the switch to using Wayland for applications the support for "Convergence" is unique to Mir and core to UBports.

It isn't frozen in time. When the project was forked the base was the already EOL 15.04 and a big task faced by UBports was to change that to 16.04. That was done a couple of years back.

The next big task was updating other dependencies in the stack to supported upstream versions. This may never be totally complete, but is largely done in OTA-12. This was more urgent than migrating to 18.04 as 16.04 was, and is still supported.

A stated goal is to update to 20.04LTS base before 16.04 goes EOL next year.

What is the state of the Lomiri desktop environment for the upcoming release of Ubuntu 23.10?

Support for the Wayland extensions (zwp_input_method_v1 and zwp_input_panel_v1) needed by maliit-keyboard (as used by Lomiri) have landed upstream in Mir. They are not in the Mir 2.15 release currently being tested, but will be available in subsequent releases.

As this was requested for Lomiri (by @dobey) I hope this of interest here. There will be some Lomiri work needed to use this in Lomiri so I hope the following will include enough information to guilde this.

Using Mir's support for maliit-keyboard

The easiest way to test this support is to install Miriway from edge and configure it to support maliit-keyboard:

sudo snap install miriway --classic --edge
sudo apt install maliit-keyboard
cat /etc/xdg/xdg-miriway/miriway-shell.config/ > ~/.config/miriway-shell.config
cat <<EOT >> ~/.config/miriway-shell.config
shell-add-wayland-extension=zwp_input_method_v1
shell-add-wayland-extension=zwp_input_panel_v1
shell-add-wayland-extension=zwp_primary_selection_device_manager_v1 # it isn't obvious why maliit-keyboard needs this
shell-component=miriway-unsnap maliit-keyboard
EOT

You can then log out and log back in into Miriway from the greeter. maliit-keyboard will show when input is on a text field*.

[*] The client needs to support one of the text-input-v{1,2,3} Wayland extensions.

Please report any issues on Mir's github: https://github.com/MirServer/mir/issues

@extraymond said in Possibility of basing on ubuntu core?:

run lomiri as it is on mir which I assume is possible with ubuntu core

If by "on mir" you mean using the mirclient IPC, then no. That won't work on Ubuntu Core.

Helpfully , that leaves you with just one option:

learn about confined-shell and mirway and see if I can get lomiri running as a standalone snap on a ubuntu core device.

Here's the code for these:

https://github.com/MirServer/confined-shell-wip
https://github.com/Miriway/Miriway

@extraymond I'd love to see this happen, but are you volunteering to work on it? Or hoping someone else does?

I've some knowledge of both Ubuntu Touch and Snaps and am willing to advise.

For anyone interested, the POC described would be a huge amount of work. Probably less than the 20.04 transition but certainly more than a few months:

• Ubuntu Touch uses a custom kernel for each phone so a kernel snap would need to be maintained for each phone. I also suspect that work would be needed to have hallium "play nice" with snapd.
  Anyone tackling this has a huge learning curve to surmount: You likely have as much knowledge as anyone else here about getting hallium working in a kernel snap

• The base doesn't really matter for this, but I'd suggest core20 to match the 20.04 base of other UBports projects

• Lomiri (the shell used by the project) is not designed to work with snap confinement, so this would involve more than "just packaging".
  And "just packaging" as a snap isn't trivial for shells - I speak as one that has done it! (See below)

• The only snap interface between Lomiri and "an snap" would have to be wayland (There is no Snap interface for the legacy mirclient-based IPC).
  The client bit is simple, there are loads of snaps available that consume the wayland slot. However, work on transitioning Lomiri to Wayland is incomplete (it has been in progress for several years)

If you're on Ubuntu desktop, you can experiment with a couple of relevant snaps of another Mir based shell (Miriway) that I've packaged: you can install both of these on your system:

• confined-shell is a strictly confined snap which is what is needed to install on Ubuntu Core; and,
• miriway an unconfined snap that doesn't have to deal with the limitations imposed by strict confinement

Probably the most tractable part of this POC would be to package Lomiri as a snap. This could also be useful for Lomiri's desktop ambitions. You could get an idea of what's involved by starting with a "classic" snap modelled on miriway.

Update: the weird colours are gone now

I too can't find any settings to change the horrible colour scheme. But I found the colours are OK (unchanged?) on an iPhone. Time to switch?

A better discussion of implementing this feature is:

https://forums.ubports.com/topic/2848/what-s-the-situation-with-blue-light-filtering-on-ubuntu-touch/

The tl;dr: it needs to be implemented (in Mir) and someone who thinks it important needs to their spend time doing that.

@arubislander said in Ubuntu Pro:

Only ESM support does not include ARM packages, so it turns out.

That's true for 14.04 and 16.04 but has changed since:

https://ubuntu.com/security/esm

Can you tell us how the integration of extended support for Ubuntu 16.04 went and what it means for both developers and users of Ubuntu Touch?

@rugby the developers (I think @Flohack is driving it) have been liaising with Canonical to get security updates from them. And, once this is in place, these will be incorporated in the OTA updates.