Introducing Miroil

alan_g · 2 Dec 2020, 15:26

Status 2020-12-02

Firstly, thanks for all the interest. I hope to make it easier for others to get involved but haven't achieved that yet. It won't get done in just the bits of time I am able to spare.

Today's the last opportunity I'll have to spend time on miroil until "the holidays" (and I don't yet know how much I'll able to do then), so I'll spend it writing a status report.

Wayland extensions

There hasn't been much change in the miroil code since last week's post, the only thing is that I started drafting a Wayland extension protocol to support the "trusted prompt sessions" used by Lomiri. I've not yet resolved all the issues with the protocol definition and not started the code to implement or use it. However, what I have done might provide a skeleton for anyone wanting to implement additional protocols for Lomiri.

QtMir

QtMir is the key thing to get going with miroil as that will "magically" make Lomiri work with Mir 2.x.

However, it is not ready to use with miroil at present, especially the Mir-2.x based branch I've been working on. There are some dependencies on both mirclient and mirserver that need the "extra level of indirection" to be written before it can be reimplemented in miroil.

I've been thinking of rebasing miroil on Mir-1.8 to make this work a bit easier, but not found the time to do this. (Rebasing should be trivial though.)

AFAIK nobody has yet stepped up and documented a development process for using QtMir and Mir/miroil on the desktop. Although I did get one volunteer. (You know who you are!)

unity-system-compositor

No change here.

alan_g · 10 Dec 2020, 09:17

In the latest Q&A @mariogrip said that he's been working on documenting and improving the process of building the Lomiri stack. Once that is published (following editorial checks by @UniSuperBox) I expect it to include details for building QtMir and, hopefully, also running the test example in that project.

alan_g · 23 Dec 2020, 17:22

Some work-in-progress notes

I've had a little time to document some things about building Mir and QtMir to work on this. I've not yet got all the way to where it is simple, but I'll document where I've got to in the hope that it is useful.

I've set up a new MirOil-for-Lomiri github project for this work. This contains forks of (so far) Mir and QtMir. The ultimate intention is, of course, to merge the changes upstream but this keeps the ongoing work in one place.

The Mir project has the default branch set to a version of Miroil based on Mir 1.8 and that will be the initial focus of work.

The QtMir project has the default branch set to a copy of the "xenial" branch and that will be the initial focus of work.

Where to start from?

I decided that an Ubuntu 16.04 base was the best place to start as everything worked there at one time. But I soon found that there were a lot of QtMir dependencies that are more recent that the 16.04 archive. So I then decided to add the UBports repo.

So, steps to follow along:

Install 16.04 either on a VM or (as I did) as a partition on a development laptop. (What follows is probably not a good idea on your main computer.)
Add the UBports archive

sudo apt-add-repository http://repo.ubports.com/
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 4BD4B4D6DBB583F1
sudo apt update
sudo apt dist-upgrade

This allows you to use apt to install any dependencies.

Building Mir

You’ll need a few development tools installed:

sudo apt install devscripts equivs git

With these installed you can checkout Mir, get the remaining dependencies and build:

git clone --recursive https://github.com/MirOil-for-Lomiri/mir.git
cd mir
sed -i /.*wlcs.*/d debian/control
mk-build-deps -i -s sudo
mkdir build
cd  build
cmake -DMIR_ENABLE_WLCS_TESTS=off ..
make

Then install in /usr/local:

sudo make install
sudo ldconfig

Building QtMir

You’ll need a few development tools installed:

sudo apt install devscripts equivs git

With these installed you can checkout QtMir, get the remaining dependencies and build:

git clone https://github.com/MirOil-for-Lomiri/qtmir.git
cd qtmir
mk-build-deps -i -s sudo
mkdir -p cmake-build-debug
cd cmake-build-debug
cmake -DCMAKE_INSTALL_PREFIX=/usr/local ..
make -j 6 all test

Then install in /usr/local:

sudo make install
sudo ldconfig

Now it should be possible to run a demo program using something like the following:

cd ../demos
QT_QPA_PLATFORM=mirserver qmlscene qml-demo-shell/qml-demo-shell.qml

But this didn't work for me (it failed to find the mirserver plugin) and I've not figured out the next step. Yet...

alan_g · 24 Dec 2020, 12:48

Finally, the QtMir demo runs!

What follows may not be the best way to hack Qt, so suggestions are welcome. Starting from the previous post:

sudo ln -s /usr/local/lib/qt5/plugins/platforms/libqpa-mirserver.so /usr/lib/x86_64-linux-gnu/qt5/plugins/platforms/

Allows Qt to find the platform.

Then, Mir fails to start because "zero sized software cursor image is invalid", so remove the cursor and...

$ MIR_SERVER_CURSOR=null QT_QPA_PLATFORM=mirserver qmlscene qml-demo-shell/qml-demo-shell.qml
...
qtmir.screens: Added Screen with id 1 and geometry QRect(0,0 1280x930)
qtmir.screens: Screen::setMirDisplayBuffer Screen(0x7f48c4541260) 0x7f48c4532118 0x7f48c4532108
qtmir.screens: =======================================
qtmir.screens: Screen(0x7f48c4541260) - id: 1 geometry: QRect(0,0 1280x930) window: 0x0 type: "Unknown" scale: 1
qtmir.screens: =======================================
file:///home/alan/CLionProjects/qtmir/demos/qml-demo-shell/qml-demo-shell.qml:2 module "Unity.Screens" is not installed

[2020-12-24 12:12:40.033503] <information> mirserver: Stopping
qtmir.screens: QtCompositor::stop

Now unityscreensplugin is something QtMir provides and it is built...

find /usr/local/ -name libunityscreensplugin.so
/usr/local/lib/qt5/qml/Unity/Screens/libunityscreensplugin.so

So Let's make that easier to find too:

sudo ln -s /usr/local/lib/qt5/qml/Unity/ /usr/lib/x86_64-linux-gnu/qt5/qml/

And that gets another error:

(process:5405): GLib-GIO-ERROR **: Settings schema 'com.canonical.qtmir' is not installed

Which can be solved:

sudo ln -s /usr/local/share/glib-2.0/schemas/com.canonical.qtmir.gschema.xml /usr/share/glib-2.0/schemas/
sudo glib-compile-schemas /usr/share/glib-2.0/schemas/
MIR_SERVER_CURSOR=null QT_QPA_PLATFORM=mirserver qmlscene qml-demo-shell/qml-demo-shell.qml

mardy · 25 Dec 2020, 21:04

@alan_g said in Introducing Miroil:

What follows may not be the best way to hack Qt, so suggestions are welcome. Starting from the previous post:
sudo ln -s /usr/local/lib/qt5/plugins/platforms/libqpa-mirserver.so /usr/lib/x86_64-linux-gnu/qt5/plugins/platforms/
Allows Qt to find the platform.

You can also try:

export QT_PLUGIN_PATH=/usr/local/lib/qt5/plugins

instead of symlinking the Qt plugins. I believe there's no need appending the other system plugins directory as it should be somehow already embedded in the Qt libs.

Though, if you are using a (virtual) marchine specifically setup for this goal, it might be better to just set the install prefix to the system Qt:

cmake \
    -DCMAKE_INSTALL_PREFIX=/usr/ \
    -DCMAKE_INSTALL_LIBDIR=lib/x86_64-linux-gnu \
    ..

alan_g · 30 Dec 2020, 15:50

@mardy thanks, the environment variable looks like a good option. But even though this is a setup specifically for this development I like to comfort of being able to nuke /usr/local, so I'll continue to install there.

alan_g · 30 Dec 2020, 16:15

Running some clients

I've been doing some experimentation with the setup above.

Firstly, I had to revise the QtMir commandline to make it possible to connect mirclient based clients:

MIR_SERVER_ENABLE_MIRCLIENT= MIR_SERVER_CURSOR=null QT_QPA_PLATFORM=mirserver qmlscene qml-demo-shell/qml-demo-shell.qml

After this I could connect, for example the "egltriangle" example:

~$ mir_demo_client_egltriangle --desktop_file_hint=/usr/share/applications/thunderbird.desktop
Window is on output 1: 96 DPI, scale 1.0x, monitor form factor, 60.00Hz
Window exposed

I also found that I could (somewhat surprisingly) connect s/w rendered Wayland clients:

$ mir_demo_client_wayland --desktop_file_hint=/usr/share/applications/thunderbird.desktop 
Got geometry: (338mm × 245mm)@(0, 0)
Got mode: 1280×930@60000 (flags: 3)
Output scale: 1
Output events done
^CSignal 2 received. Good night.

(The --desktop_file_hint... argument is something that QtMir consumes - it just has to point at a .desktop file that exists on the system.)

Sadly, EGL based Wayland clients crash the server:

$ mir_demo_client_wayland_egl_spinner --desktop_file_hint=/usr/share/applications/thunderbird.desktop
~~~
qtmir.surfaces: MirSurface[0x214cc80,"thunderbird"]::registerView(35079680) after=1
qtmir.surfaces: MirSurface[0x214cc80,"thunderbird"]::setReady()
qtmir.applications: Application["thunderbird"]::setInternalState(state=Running)
qtmir.surfaces: MirSurface[0x214cc80,"thunderbird"]::updateExposure(true)
terminate called after throwing an instance of 'std::logic_error'
  what():  Buffer does not support GL rendering
Aborted (core dumped)

I'm going to try a couple of other QtMir branches to see if I can get a more stable starting point.

NB: So far, I've not made any changes to QtMir, I've just shown how to build Mir and QtMir locally and run one against the other.

branch	what works	what doesn't
`xenial`	mirclient and s/w wayland	egl wayland
`xenial_-_edge_-_wayland`	mirclient and egl wayland	s/w wayland
`xenial_-_edge_-_wayland_-_mir18`	mirclient and s/w wayland and egl wayland	(none)

That's reason enough to base this work on xenial_-_edge_-_wayland_-_mir18, I'll update the miroil branch accordingly.

alan_g · 30 Dec 2020, 18:32

@alan_g said in Introducing Miroil:

NB: So far, I've not made any changes to QtMir, I've just shown how to build Mir and QtMir locally and run one against the other.

I've now made one change. Using miroil::GLBuffer in QtMir in place of the original version from QtMir. But that's all I'll achieve today. (PRs welcome!)

jonius · 6 Jan 2021, 17:25

At building QtMir I run into

/home/miroil/qtmir/src/modules/Unity/Application/mirbuffersgtexture.h:20:30: fatal error: miroil/mirbuffer.h: No such file or directory

the file is installed at /usr/local/include/miroil/miroil/mirbuffer.h. It looks like there is one miroil too much in there. Or is GCC supposed to find it in a sub folder?

alan_g · 6 Jan 2021, 17:34

@jonius I don't have access to my test system just now, but:

pkg-config --cflags miroil

Should return -I/usr/include/miroil (amongst others), and that should be picked up by cmake when run in the QtMir directory.

jonius · 6 Jan 2021, 18:55

@alan_g pkg-config lists the include dir correctly (/usr/local/include/miroil). I also printed the include dirs in CMake and the directory was there. Running make with VERBOSE=1 shows that it does not get passed to the compiler for some reason:

cd /home/miroil/qtmir/cmake-build-debug/tests/modules/SurfaceManager && /usr/bin/c++   -DGTEST_VERSION_MAJOR=1 -DGTEST_VERSION_MINOR=7 -DGTEST_VERSION_PATCH=0 -DQT_CORE_LIB -DQT_DBUS_LIB -DQT_DISABLE_DEPRECATED_BEFORE=0x050900 -DQT_GUI_LIB -DQT_NETWORK_LIB -DQT_NO_DEBUG -DQT_NO_KEYWORDS -DQT_QML_LIB -DQT_QUICK_LIB -DQT_SENSORS_LIB -DQT_TESTCASE_BUILDDIR=\"/home/miroil/qtmir/cmake-build-debug\" -DQT_TESTLIB_LIB -DQT_USE_QSTRINGBUILDER -DQT_USING_OPENGL -I/home/miroil/qtmir/cmake-build-debug/tests/modules/SurfaceManager -I/home/miroil/qtmir/tests/modules/SurfaceManager -isystem /usr/include/x86_64-linux-gnu/qt5/QtCore -isystem /usr/include/x86_64-linux-gnu/qt5 -isystem /usr/include/libdrm -I/home/miroil/qtmir/tests/include -I/home/miroil/qtmir/src/common -I/home/miroil/qtmir/src/platforms/mirserver -I/home/miroil/qtmir/src/modules -I/home/miroil/qtmir/tests/framework -isystem /usr/local/include/miral -isystem /usr/local/include/mirclient -isystem /usr/local/include/mircookie -isystem /usr/local/include/mircore -isystem /usr/local/include/mirtest -isystem /usr/local/include/mirserver -isystem /usr/local/include/mirplatform -isystem /usr/local/include/mircommon -isystem /usr/local/include/mirrenderer -isystem /usr/include/uuid -isystem /usr/lib/x86_64-linux-gnu/qt5/mkspecs/linux-g++ -isystem /usr/include/x86_64-linux-gnu/qt5/QtDBus -isystem /usr/include/x86_64-linux-gnu/qt5/QtQml -isystem /usr/include/x86_64-linux-gnu/qt5/QtNetwork -isystem /usr/include/x86_64-linux-gnu/qt5/QtQuick -isystem /usr/include/x86_64-linux-gnu/qt5/QtGui -isystem /usr/include/x86_64-linux-gnu/qt5/QtFontDatabaseSupport/5.12.9 -isystem /usr/include/x86_64-linux-gnu/qt5/QtThemeSupport/5.12.9 -isystem /usr/include/x86_64-linux-gnu/qt5/QtEventDispatcherSupport/5.12.9 -isystem /usr/include/x86_64-linux-gnu/qt5/QtEglSupport/5.12.9 -isystem /usr/include/x86_64-linux-gnu/qt5/QtServiceSupport/5.12.9 -isystem /usr/include/x86_64-linux-gnu/qt5/QtSensors -isystem /usr/include/x86_64-linux-gnu/qt5/QtTest  -fPIC -Wall -fno-strict-aliasing -Werror -Wextra -O2 -g -DNDEBUG   -fPIC -DQT_GUI_LIB -DQT_CORE_LIB -std=gnu++14 -o CMakeFiles/surfacemanager_test.dir/surface_manager_test.cpp.o -c /home/miroil/qtmir/tests/modules/SurfaceManager/surface_manager_test.cpp

jonius · 1 Jun 2021, 19:34

I can workaround the issue by adding the include directories to the target directly (see Commit). Weird issue...

jonius · 1 Jun 2021, 20:36

@jonius next issue is a failing test (ApplicationManager), which fails here.

The demo runs into a segfault as well. (MIR_SERVER_CURSOR=null QT_QPA_PLATFORM=mirserver qmlscene qml-demo-shell/qml-demo-shell.qml).

btw, I am on Ubuntu 16.04 in a VM.

alan_g · 7 Jan 2021, 12:22

@jonius Not sure why you're seeing different things. Maybe something with the graphics setup in your VM?

Can you run the mir examples? For example:

$ mir-smoke-test-runner
...
I: Smoke testing complete with returncode 0

If that also shows problems, this solution might help (depending on your choice of VM):

https://discourse.ubuntu.com/t/problems-running-electron-app-in-mir-kiosk-on-kvm/16196/6

jonius · 7 Jan 2021, 19:17

@alan_g said in Introducing Miroil:

mir-smoke-test-runner

I: The following clients failed to execute successfully:
I:     mir_demo_client_prerendered_frames
I:     mir_demo_client_animated_cursor
I:     mir_demo_client_render_surface
I:     mir_demo_client_basic
I:     mir_demo_client_flicker
I:     mir_demo_client_fingerpaint
I:     mir_demo_client_release_at_exit
I:     mir_demo_client_eglstateswitcher
I:     mir_demo_client_wayland
I:     mir_demo_client_target
I:     mir_demo_client_tooltip
I:     mir_demo_client_eglsquare
I:     mir_demo_client_cursors
I:     mir_demo_client_egldiamond_render_surface
I:     mir_demo_client_egltriangle
I:     mir_demo_client_chain_jumping_buffers
I:     mir_demo_client_touch_validator
I:     mir_demo_client_wayland_egl_spinner
I:     mir_demo_client_progressbar
I:     mir_demo_client_camera
I:     mir_demo_client_prompt_session
I:     mir_demo_client_multiwin
I:     mir_demo_client_scroll
I:     mir_demo_client_display_config
I:     mir_demo_client_screencast
I:     mir_demo_client_eglcounter
I:     mir_demo_client_multistream
I:     mir_demo_client_input_shapes
I:     mir_demo_client_eglplasma
I:     mir_demo_client_eglflash
I:     mir_demo_client_pointer_confinement
I: Smoke testing complete with returncode -1

I am using VirtualBox. I am not sure how to apply the suggestions in the link. Which VM solution would you suggest? Otherwise I'll install the system again on a separate partition without VM.

alan_g · 7 Jan 2021, 20:26

@jonius I don't have VirtualBox to hand to give detailed instructions. But it should work if you enable "the right" graphics options. Not sure what it's called. Something like 'passthrough' or '3D' maybe?

jonius · 1 Aug 2021, 19:00

@alan_g
I was hit by this bug. Now with 3D acceleration enabled most of the smoke tests succeed. Only mir_demo_client_eglplasma still failes. And the unit test I mentioned above continues to fail as well. Will test on separate installation without VM this weekend.

MIR_SERVER_CURSOR=null QT_QPA_PLATFORM=mirserver qmlscene qml-demo-shell/qml-demo-shell.qml works partially. The circle rotates, but the mouse cursor is not visible and clicking the scale buttons results in file:///home/miroil/qtmir/demos/qml-demo-shell/qml-demo-shell.qml:28: Error: Unknown method parameter type: FormFactor.

alan_g · 8 Jan 2021, 23:53

@jonius said in Introducing Miroil:

the mouse cursor is not visible

That's sort of expected. I disabled the cursor in Mir because QtMir tries to set a zero sized image instead of hiding it. And Mir treats that as an error.

I've not looked at how it works for Lomiri. I suspect it renders its own cursor.

jonius · 9 Jan 2021, 23:48

@alan_g installed Ubuntu 16.04 on my notebook directly via Ubuntu Touch (ISODrive is a great tool). Unfortunately the situation here is worse than in my VM on an Ubuntu 20.04 host. I guess the outdated amdgpu for my Radeon Vega 8 Graphics in Ubuntu 16.04 is to blame. The demo runs like in the VM, the same unit test fails, but many smoke tests fail:

I:     mir_demo_client_camera
I:     mir_demo_client_tooltip
I:     mir_demo_client_eglsquare
I:     mir_demo_client_eglplasma
I:     mir_demo_client_eglstateswitcher
I:     mir_demo_client_target
I:     mir_demo_client_scroll
I:     mir_demo_client_cursors
I:     mir_demo_client_egltriangle
I:     mir_demo_client_egldiamond_render_surface
I:     mir_demo_client_display_config
I:     mir_demo_client_eglcounter
I:     mir_demo_client_touch_validator
I:     mir_demo_client_animated_cursor
I:     mir_demo_client_pointer_confinement
I:     mir_demo_client_input_shapes
I:     mir_demo_client_eglflash

So I guess best is to switch back to my VM? Is the failing tests something I could ignore for now?

alan_g · 10 Jan 2021, 00:12

@jonius said in Introducing Miroil:

Is the failing tests something I could ignore for now?

Of course. It will get annoying, but if you know you didn't break the code...

You've clearly got enough working to try stuff out.