Building on device with small root partition

anhilde · 28 Jan 2021, 17:21

Hi,

I have been trying to follow the steps outlined here: https://docs.ubports.com/en/latest/systemdev/testing-locally.html#building-on-the-device-itself

Unfortunately this approach does not work for me since the root partition is to small and resizing it according to the information in this thread https://forums.ubports.com/topic/1729/set-partition-sizes-when-flashing/8?_=1611848970365 failed.

Is this approach deprecated or is there another trick one should know about to make it work?

Best Regards,

anhilde

dobey · 28 Jan 2021, 20:08

@anhilde You should be able to create a libertine container, install all the necessary build dependencies in it using libertine-container-manager, and then building inside the container to generate a deb build.

anhilde · 28 Jan 2021, 22:27

@dobey Thanks, I already started going down that route after some initial head scratching.

anhilde · 29 Jan 2021, 17:47

Hi,

after some fiddling I managed to mostly build the address-book-app in the libertine container. Unfortunately I know next to nothing about debian package build at this moment in time. The build fails with:

-- Installing: /home/phablet/address-book-app/debian/tmp/usr/share/content-hub/peers/address-book-app
make[2]: Leaving directory '/home/phablet/address-book-app/obj-arm-linux-gnueabihf'
dh_apparmor -paddress-book-app --profile-name=address-book-app
make[1]: Leaving directory '/home/phablet/address-book-app'
dh_install -O--parallel -O--fail-missing
dh_install: address-book-app-autopilot missing files: usr/lib/python*/dist-packages/address_book_app/*
dh_install: missing files, aborting
debian/rules:9: recipe for target 'binary' failed
make: *** [binary] Error 2
dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2

Perhaps some of the knowledgeable people here can help me out where to look for the cause of this error.

Best Regards,

anhilde

dobey · 29 Jan 2021, 23:10

@anhilde How exactly was it built? Do you have a complete build log you can post on a pastebin or such? It looks like somehow you managed to try to build a deb but also build as a click for some reason.

anhilde · 30 Jan 2021, 17:04

@dobey I must have messed up something in the repo clone by earlier attempts to get something to build. I reset it then build again inside the container using:

DEB_BUILD_OPTIONS="parallel=2 debug" dpkg-buildpackage -rfakeroot -b

The build went fine, and a set of *.deb files is created:

address-book-app_0.4~0ubports1_armhf.deb
qtdeclarative5-ubuntu-addressbook0.1_0.4~0ubports1_armhf.deb
address-book-app-autopilot_0.4~0ubports1_all.deb
qtdeclarative5-ubuntu-contacts0.1_0.4~0ubports1_armhf.deb
address-book-app-dbg_0.4~0ubports1_armhf.deb

From the documentation (https://docs.ubports.com/en/latest/systemdev/testing-locally.html) I gather I need to install the *.deb files using dpkg using:

sudo dpkg -i ../<package>.deb [../<package2>.deb ...]

I is not clear though in which context, in the conainer or outside of it? If I pull the files from the container and install outside of it I get dependecy problems such as:

phablet@ubuntu-phablet:~$ sudo dpkg -i address-book-app_0.4~0ubports1_armhf.deb
(Reading database ... 51249 files and directories currently installed.)
Preparing to unpack address-book-app_0.4~0ubports1_armhf.deb ...
Unpacking address-book-app (0.4~0ubports1) over (0.4~0ubports1) ...
dpkg: dependency problems prevent configuration of address-book-app:
 address-book-app depends on libqt5core5a (>= 5.10.0); however:
  Version of libqt5core5a:armhf on system is 5.9.5+dfsg-0ubports3.

dpkg: error processing package address-book-app (--install):
 dependency problems - leaving unconfigured
Processing triggers for mime-support (3.59ubuntu1) ...
Errors were encountered while processing:
 address-book-app

This is not surprising really I guess. Setting up the container was manual work, as I could not get the command

sudo apt build-dep address-book-app

to work inside it and could not find an alternative (I wanted to retest that once I managed to actually get the process going from end to end)
However inside the container installation is not possible since there is no sudo and even if I install it, it does not work:

sudo: /usr/bin/sudo must be owned by uid 0 and have the setuid bit set

Any further help would be greatly appreciated. Once I understand the container based app development process (which I think is a great idea) I would like to offer to try and document it better.

dobey · 30 Jan 2021, 19:08

@anhilde You will need to switch to devel channel to be able to test anything you build now, as we have merged Qt 5.12, and so you've built against that when compiling.

As for sudo inside the container, you should never use it and never install it inside libertine. It will not work and can break other things if installed.

doniks · 1 Feb 2021, 16:29

@anhilde maybe you want make a PR to update the "on the device" section in the docs to use libertine. It probably makes more sense to suggest libertine first and then there can be some footnote, that depending on resources one can also attempt on the root partition ...

anhilde · 1 Feb 2021, 16:29

@doniks Yes that makes sense, however I have not yet succeeded building within the container.

To be specific, the build went through, but at runtime there is a library missing (libGLES), which is also the case when running the tests (I commented them out). Note, I did switch to the Development channel. So while the build goes through and I can theoretically start the build result from outside the container, the app fails to run due to the dependecy problem.

My assumption was, since I tried all sorts of approaches in my explorative stage of app development with the container, that I messed something up. Therefore I gave it another go with a fresh container. However I get new troubles. My main problem, I think, that it is not clear to me how to get the container configured efficiently for a build of a specific component. In the documentation on how to build on the device on the root partition, there are these steps:

sudo apt update
sudo apt build-dep address-book-app

However if I do this, the packet installation always fails with the message, that the package installation requires root access:

phablet@ubuntu-phablet:~/address-book-app$ apt build-dep address-book-app
Reading package lists... Done
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  libaudio2 libmng2 libmysqlclient20 libpcre16-3 mysql-common qtcore4-l10n
Use 'apt autoremove' to remove them.
The following NEW packages will be installed:
  libegl1-mesa-dev libflac8 libfontenc1 libgl1-mesa-dev libgles2-mesa-dev libgsettings-qt1 libgstreamer-plugins-base1.0-0
  libgstreamer1.0-0 libhistoryservice0 libleveldb1v5 libmirclient-dev libmircommon-dev libmircookie-dev libmircookie2 libmircore-dev
  libogg0 liborc-0.4-0 libphonenumber7 libprotobuf-dev libprotobuf9v5 libpthread-stubs0-dev libpulse0 libqofono-dev libqofono-qt5-0
  libqt5concurrent5 libqt5contacts5 libqt5feedback5 libqt5multimedia5 libqt5organizer5 libqt5printsupport5 libqt5quickparticles5
  libqt5quickshapes5 libqt5quicktest5 libqt5quickwidgets5 libqt5sql5 libqt5systeminfo5 libqt5test5 libqt5versit5 libqt5versitorganizer5
  libqt5xml5 libqt5xmlpatterns5 libsnappy1v5 libsndfile1 libtag1v5 libtag1v5-vanilla libtelepathy-qt5-0 libubuntugestures5
  libubuntumetrics5 libubuntutoolkit5 libunity-api1 libvorbis0a libvorbisenc2 libvulkan-dev libvulkan1 libwayland-bin libwayland-dev
  libwrap0 libx11-dev libx11-xcb-dev libxau-dev libxaw7 libxcb-dri2-0-dev libxcb-dri3-dev libxcb-glx0-dev libxcb-present-dev
  libxcb-randr0-dev libxcb-render0-dev libxcb-shape0-dev libxcb-sync-dev libxcb-xfixes0-dev libxcb1-dev libxdamage-dev libxdmcp-dev
  libxext-dev libxfixes-dev libxfont1 libxkbcommon-dev libxkbfile1 libxmu6 libxshmfence-dev libxxf86vm-dev mesa-common-dev pkg-config
  qml-module-qtcontacts qml-module-qtfeedback qml-module-qtgraphicaleffects qml-module-qtquick-layouts qml-module-qtquick-window2
  qml-module-qtquick2 qml-module-qttest qml-module-ubuntu-components qml-module-ubuntu-components-labs qml-module-ubuntu-layouts
  qml-module-ubuntu-performancemetrics qml-module-ubuntu-test qt5-default qt5-qmake qt5-qmake-bin qt5-qmltooling-plugins qtbase5-dev
  qtbase5-dev-tools qtdeclarative5-buteo-syncfw0.1 qtdeclarative5-dev qtdeclarative5-dev-tools qtdeclarative5-gsettings1.0
  qtdeclarative5-ofono0.2 qtdeclarative5-qtcontacts-plugin qtdeclarative5-ubuntu-content1 qtdeclarative5-ubuntu-history0.1
  qtdeclarative5-ubuntu-keyboard-extensions0.1 qtdeclarative5-ubuntu-telephony-phonenumber0.1 qtdeclarative5-ubuntu-ui-toolkit-plugin
  qtpim5-dev suru-icon-theme thumbnailer-service ttf-ubuntu-font-family ubuntu-mobile-icons ubuntu-ui-toolkit-theme x11-xkb-utils
  x11proto-core-dev x11proto-damage-dev x11proto-dri2-dev x11proto-fixes-dev x11proto-gl-dev x11proto-input-dev x11proto-kb-dev
  x11proto-xext-dev x11proto-xf86vidmode-dev xorg-sgml-doctools xserver-common xtrans-dev xvfb zlib1g-dev
0 upgraded, 133 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/32,3 MB of archives.
After this operation, 122 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
debconf: delaying package configuration, since apt-utils is not installed
dpkg: error: requested operation requires superuser privilege
E: Sub-process /usr/bin/dpkg returned an error code (2)
E: Failed to process build dependencies

dobey · 2 Jan 2021, 16:35

@anhilde said in Building on device with small root partition:

However if I do this, the packet installation always fails with the message, that the package installation requires root access:

You need to enter the libertine container differently so that you are using it as "root" like so:

libertine-container-mainager exec -i <containerid> -c bash

This will give you a "root" prompt inside the container, and you should be able to use apt build-dep then (assuming you did not install sudo in the container).

After that, you can exit the "root" prompt, and run the standard container shell normally, to be able to build the packages.

anhilde · 1 Feb 2021, 18:34

@dobey Thanks, that was the missing link to getting the container set up. However the tests still fail with the libGLESv2.so.2 issue, same as I had before when trying with the "manually" configured container:

[100%] Linking CXX shared module libubuntu-contacts-qml.so
cd /home/phablet/address-book-app/obj-arm-linux-gnueabihf/src/imports/Ubuntu/Contacts && /usr/bin/cmake -E cmake_link_script CMakeFiles/ubuntu-contacts-qml.dir/link.txt --verbose=1
/usr/bin/arm-linux-gnueabihf-g++  -fPIC -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2  -std=c++11 -Wall  -Wl,-Bsymbolic-functions -Wl,-z,relro -shared  -o libubuntu-contacts-qml.so CMakeFiles/ubuntu-contacts-qml.dir/contacts.cpp.o CMakeFiles/ubuntu-contacts-qml.dir/imagescalethread.cpp.o CMakeFiles/ubuntu-contacts-qml.dir/plugin.cpp.o CMakeFiles/ubuntu-contacts-qml.dir/simcardcontacts.cpp.o CMakeFiles/ubuntu-contacts-qml.dir/ubuntu-contacts-qml_automoc.cpp.o -lqofono-qt5 /usr/lib/arm-linux-gnueabihf/libQt5Contacts.so.5.0.0 /usr/lib/arm-linux-gnueabihf/libQt5Quick.so.5.12.9 /usr/lib/arm-linux-gnueabihf/libQt5DBus.so.5.12.9 /usr/lib/arm-linux-gnueabihf/libQt5Qml.so.5.12.9 /usr/lib/arm-linux-gnueabihf/libQt5Network.so.5.12.9 /usr/lib/arm-linux-gnueabihf/libQt5Gui.so.5.12.9 /usr/lib/arm-linux-gnueabihf/libQt5Core.so.5.12.9 
make[3]: Leaving directory '/home/phablet/address-book-app/obj-arm-linux-gnueabihf'
[100%] Built target ubuntu-contacts-qml
make[2]: Leaving directory '/home/phablet/address-book-app/obj-arm-linux-gnueabihf'
/usr/bin/cmake -E cmake_progress_start /home/phablet/address-book-app/obj-arm-linux-gnueabihf/CMakeFiles 0
make[1]: Leaving directory '/home/phablet/address-book-app/obj-arm-linux-gnueabihf'
   dh_auto_test -O--parallel -O--fail-missing
	make -j2 test ARGS\+=-j2
make[1]: Entering directory '/home/phablet/address-book-app/obj-arm-linux-gnueabihf'
Running tests...
/usr/bin/ctest --force-new-ctest-process -j2
Test project /home/phablet/address-book-app/obj-arm-linux-gnueabihf
    Start 1: contact_list
    Start 2: Contact_list_model
1/9 Test #2: Contact_list_model ...............***Failed    2.20 sec
/usr/lib/arm-linux-gnueabihf/qt5/bin/qmltestrunner: error while loading shared libraries: libGLESv2.so.2: cannot open shared object file: No such file or directory

doniks · 2 Jan 2021, 19:11

@anhilde maybe you need to install more stuff

maybe libgles2-mesa-dev https://packages.ubuntu.com/search?suite=groovy&arch=any&mode=filename&searchon=contents&keywords=libGLESv2

anhilde · 1 Feb 2021, 19:30

@doniks I would have though, that that should be taken care off by:

apt build-dep

Is that assumption wrong? Nore, I'm not trying to get this working somehow, but rather want to try and understand how the application development process is intended.

anhilde · 2 Feb 2021, 07:57

Hi, I have a problem understanding, how the (if?) the build application should be runnable from within the container. When the build runs the tests I assume it uses the executable address-book-app found in the project tree.

phablet@ubuntu-phablet:~/address-book-app/obj-arm-linux-gnueabihf$ ./src/app/address-book-app 
./src/app/address-book-app: error while loading shared libraries: libGLESv2.so.2: cannot open shared object file: No such file or directory
phablet@ubuntu-phablet:~/address-book-app/obj-arm-linux-gnueabihf$ ldd ./src/app/address-book-app 
	libQt5Gui.so.5 => not found
	libQt5Qml.so.5 => not found
	libgcc_s.so.1 => not found
	libstdc++.so.6 => not found
	libQt5Core.so.5 => not found
	libQt5Network.so.5 => not found
	/lib/ld-linux-armhf.so.3 => /lib/ld-linux-armhf.so.3 (0x00000000)
	libQt5DBus.so.5 => not found
	libQt5Quick.so.5 => not found
	libc.so.6 => not found

Since that is not directly linked to libGLESv2.so.2 my assumption was that this dependency comes in via libQTGui.so.5. Why is this not found?

If I search for it logged into the container as root I find libQt5Gui.so.5:

root@ubuntu-phablet:/# ldd /usr/lib/arm-linux-gnueabihf/libQt5Gui.so.5
	linux-gate.so.1 =>  (0x00000000)
	libfakeroot-sysv.so => /usr/lib/arm-linux-gnueabihf/libfakeroot/libfakeroot-sysv.so (0x00000000)
	libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0x00000000)
	libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0x00000000)
	/lib/ld-linux-armhf.so.3 => /lib/ld-linux-armhf.so.3 (0x00000000)
	libfakechroot.so => /usr/lib/arm-linux-gnueabihf/fakechroot/libfakechroot.so (0x00000000)
	libQt5Core.so.5 => /usr/lib/arm-linux-gnueabihf/libQt5Core.so.5 (0x00000000)
	libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0x00000000)
	libz.so.1 => /lib/arm-linux-gnueabihf/libz.so.1 (0x00000000)
	libicui18n.so.55 => /usr/lib/arm-linux-gnueabihf/libicui18n.so.55 (0x00000000)
	libicuuc.so.55 => /usr/lib/arm-linux-gnueabihf/libicuuc.so.55 (0x00000000)
	libicudata.so.55 => /usr/lib/arm-linux-gnueabihf/libicudata.so.55 (0x00000000)
	libstdc++.so.6 => /usr/lib/arm-linux-gnueabihf/libstdc++.so.6 (0x00000000)
	libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0x00000000)
	libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0x00000000)
	libpcre2-16.so.0 => /usr/lib/arm-linux-gnueabihf/libpcre2-16.so.0 (0x00000000)
	libdouble-conversion.so.1 => /usr/lib/arm-linux-gnueabihf/libdouble-conversion.so.1 (0x00000000)
	libglib-2.0.so.0 => /lib/arm-linux-gnueabihf/libglib-2.0.so.0 (0x00000000)
	libpcre.so.3 => /lib/arm-linux-gnueabihf/libpcre.so.3 (0x00000000)
	libGLESv2.so.2 => /usr/lib/arm-linux-gnueabihf/mesa-egl/libGLESv2.so.2 (0x00000000)
	libglapi.so.0 => /usr/lib/arm-linux-gnueabihf/libglapi.so.0 (0x00000000)
	libpng12.so.0 => /lib/arm-linux-gnueabihf/libpng12.so.0 (0x00000000)
	libharfbuzz.so.0 => /usr/lib/arm-linux-gnueabihf/libharfbuzz.so.0 (0x00000000)
	libfreetype.so.6 => /usr/lib/arm-linux-gnueabihf/libfreetype.so.6 (0x00000000)
	libgraphite2.so.3 => /usr/lib/arm-linux-gnueabihf/libgraphite2.so.3 (0x00000000)

And libGLESv2.so.2 is available under the path reported by ldd and it reports:

root@ubuntu-phablet:/usr/lib/arm-linux-gnueabihf/mesa-egl# ldd libGLESv2.so.2
	linux-gate.so.1 =>  (0x00000000)
	libfakeroot-sysv.so => /usr/lib/arm-linux-gnueabihf/libfakeroot/libfakeroot-sysv.so (0x00000000)
	libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0x00000000)
	libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0x00000000)
	/lib/ld-linux-armhf.so.3 => /lib/ld-linux-armhf.so.3 (0x00000000)
	libfakechroot.so => /usr/lib/arm-linux-gnueabihf/fakechroot/libfakechroot.so (0x00000000)
	libglapi.so.0 => /usr/lib/arm-linux-gnueabihf/libglapi.so.0 (0x00000000)
	libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0x00000000)

Does anybody have a clue why then the tests fail with the message that libGLESv2.so.2 is missing? Note I do have dyslexia, so if there is an obvious name mix up here, please let me know.

dobey · 2 Feb 2021, 15:49

@anhilde Well GLES2 is there. There does seem to be a weird issue though, since it's not finding the libs when using the container normally. Especially odd given that it compiled and only failed when trying to run the tests.

As for what is run for the tests, no, it is not running the address book app itself, but other tests binaries, or qmltestrunner for QML tests.

doniks · 2 Feb 2021, 18:24

@anhilde said in Building on device with small root partition:

@doniks I would have though, that that should be taken care off by:
apt build-dep
Is that assumption wrong?

I would have assumed the same yes ...

Nore, I'm not trying to get this working somehow, but rather want to try and understand how the application development process is intended.

Understood. And very much appreciated! I was just thinking maybe there is a genuine bug in the dependencies.

Full disclosure, I've never done this on-device build myself. I'm just thinking out loud here.

Also, I just notice that there are two packages that have this lib: mesa and hybris. Maybe you actually need the other one. Or a related thought: Maybe you can run the tests from OUTSIDE the container, after the building INSIDE is complete.

dobey · 2 Feb 2021, 19:28

@doniks said in Building on device with small root partition:

, after the building INSIDE is complete.

The build is incomplete, because the tests failed to run.

anhilde · 2 Feb 2021, 19:59

@dobey I cannot get the build to run. I always end up with the libGLESv2.so.2 problem. I actually tried installing all packages via:

libertine-container-manager install-package -p $ALLPACKS_FROM_APT_BUILD_DEP_COMMAND

The very first build fails with:

phablet@ubuntu-phablet:~/address-book-app$ DEB_BUILD_OPTIONS="parallel=2 debug" dpkg-buildpackage -rfakeroot -b
dpkg-buildpackage: source package address-book-app
dpkg-buildpackage: source version 0.4~0ubports1
dpkg-buildpackage: source distribution xenial
dpkg-buildpackage: source changed by Alberto Mardegan <mardy@users.sourceforge.net>
dpkg-buildpackage: host architecture armhf
 dpkg-source --before-build address-book-app
dpkg-checkbuilddeps: error: Unmet build dependencies: dh-apparmor qml-module-ofono qml-module-qt-labs-platform qml-module-qtsysteminfo (>= 5.0~)
dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting
dpkg-buildpackage: warning: (Use -d flag to override.)

I installed these packages as well with the libertine-container-manager. Following that the build works, until the test execution fails.

I fiddled with the LD_LIBRARY_PATH, just to see if I can get anything going and ran the qmltestrunner manually, leading to this output:

phablet@ubuntu-phablet:~/address-book-app$ export LD_LIBRARY_PATH="/usr/lib/arm-linux-gnueabihf:/usr/lib/arm-linux-gnueabihf/mesa-egl:$LD_LIBRARY_PATH"
phablet@ubuntu-phablet:~/address-book-app$ /usr/lib/arm-linux-gnueabihf/qt5/bin/qmltestrunner
qt.qpa.xcb: could not connect to display 
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb.

Aborted

At that point I am wondering, with entering the container with DISPLAY= , there won't be a display available ever in there. So can these tests really run in the container? Is there a nightly build that I could try to investigate, to see how this should be done?

doniks · 3 Feb 2021, 07:11

@dobey said in Building on device with small root partition:

@doniks said in Building on device with small root partition:

, after the building INSIDE is complete.

The build is incomplete, because the tests failed to run.

well ok, but I would assume that the build process first creates executables and then tests them. so, if the executables exist, one could still try to use/test them outside the container.

and even if the failing tests are before some final step that puts the executables together maybe anhilde can relatively easily disable them for the moment, to get one step further

@anhilde

root@ubuntu-phablet:/# ldd /usr/lib/arm-linux-gnueabihf/libQt5Gui.so.5
	libGLESv2.so.2 => /usr/lib/arm-linux-gnueabihf/mesa-egl/libGLESv2.so.2 (0x00000000)

so this is the mesa version, maybe you need the hybris one https://packages.ubuntu.com/search?suite=xenial&arch=armhf&mode=filename&searchon=contents&keywords=libGLESv2 so I guess you could try to libertine-container-manager remove-package -p libgles2-mesa-dev and instead libertine-container-manager install-package -p libhybris

dobey · 3 Feb 2021, 19:48

@doniks said in Building on device with small root partition:

so, if the executables exist, one could still try to use/test them outside the container.

I mean, the packages aren't built yet, because the tests failed, so it's not so easy to do things with them outside the container.