Building on device with small root partition
I have been trying to follow the steps outlined here: https://docs.ubports.com/en/latest/systemdev/testing-locally.html#building-on-the-device-itself
Unfortunately this approach does not work for me since the root partition is to small and resizing it according to the information in this thread https://forums.ubports.com/topic/1729/set-partition-sizes-when-flashing/8?_=1611848970365 failed.
Is this approach deprecated or is there another trick one should know about to make it work?
@anhilde You should be able to create a libertine container, install all the necessary build dependencies in it using
libertine-container-manager, and then building inside the container to generate a deb build.
@dobey Thanks, I already started going down that route after some initial head scratching.
after some fiddling I managed to mostly build the address-book-app in the libertine container. Unfortunately I know next to nothing about debian package build at this moment in time. The build fails with:
-- Installing: /home/phablet/address-book-app/debian/tmp/usr/share/content-hub/peers/address-book-app
make: Leaving directory '/home/phablet/address-book-app/obj-arm-linux-gnueabihf'
dh_apparmor -paddress-book-app --profile-name=address-book-app
make: Leaving directory '/home/phablet/address-book-app'
dh_install -O--parallel -O--fail-missing
dh_install: address-book-app-autopilot missing files: usr/lib/python*/dist-packages/address_book_app/*
dh_install: missing files, aborting
debian/rules:9: recipe for target 'binary' failed
make: *** [binary] Error 2
dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2
Perhaps some of the knowledgeable people here can help me out where to look for the cause of this error.
@anhilde How exactly was it built? Do you have a complete build log you can post on a pastebin or such? It looks like somehow you managed to try to build a deb but also build as a click for some reason.
@dobey I must have messed up something in the repo clone by earlier attempts to get something to build. I reset it then build again inside the container using:
DEB_BUILD_OPTIONS="parallel=2 debug" dpkg-buildpackage -rfakeroot -b
The build went fine, and a set of *.deb files is created:
From the documentation (https://docs.ubports.com/en/latest/systemdev/testing-locally.html) I gather I need to install the *.deb files using dpkg using:
sudo dpkg -i ../<package>.deb [../<package2>.deb ...]
I is not clear though in which context, in the conainer or outside of it? If I pull the files from the container and install outside of it I get dependecy problems such as:
phablet@ubuntu-phablet:~$ sudo dpkg -i address-book-app_0.4~0ubports1_armhf.deb (Reading database ... 51249 files and directories currently installed.) Preparing to unpack address-book-app_0.4~0ubports1_armhf.deb ... Unpacking address-book-app (0.4~0ubports1) over (0.4~0ubports1) ... dpkg: dependency problems prevent configuration of address-book-app: address-book-app depends on libqt5core5a (>= 5.10.0); however: Version of libqt5core5a:armhf on system is 5.9.5+dfsg-0ubports3. dpkg: error processing package address-book-app (--install): dependency problems - leaving unconfigured Processing triggers for mime-support (3.59ubuntu1) ... Errors were encountered while processing: address-book-app
This is not surprising really I guess. Setting up the container was manual work, as I could not get the command
sudo apt build-dep address-book-app
to work inside it and could not find an alternative (I wanted to retest that once I managed to actually get the process going from end to end)
However inside the container installation is not possible since there is no sudo and even if I install it, it does not work:
sudo: /usr/bin/sudo must be owned by uid 0 and have the setuid bit set
Any further help would be greatly appreciated. Once I understand the container based app development process (which I think is a great idea) I would like to offer to try and document it better.
@anhilde You will need to switch to
develchannel to be able to test anything you build now, as we have merged Qt 5.12, and so you've built against that when compiling.
sudoinside the container, you should never use it and never install it inside libertine. It will not work and can break other things if installed.
@anhilde maybe you want make a PR to update the "on the device" section in the docs to use libertine. It probably makes more sense to suggest libertine first and then there can be some footnote, that depending on resources one can also attempt on the root partition ...
@doniks Yes that makes sense, however I have not yet succeeded building within the container.
To be specific, the build went through, but at runtime there is a library missing (libGLES), which is also the case when running the tests (I commented them out). Note, I did switch to the Development channel. So while the build goes through and I can theoretically start the build result from outside the container, the app fails to run due to the dependecy problem.
My assumption was, since I tried all sorts of approaches in my explorative stage of app development with the container, that I messed something up. Therefore I gave it another go with a fresh container. However I get new troubles. My main problem, I think, that it is not clear to me how to get the container configured efficiently for a build of a specific component. In the documentation on how to build on the device on the root partition, there are these steps:
sudo apt update sudo apt build-dep address-book-app
However if I do this, the packet installation always fails with the message, that the package installation requires root access:
phablet@ubuntu-phablet:~/address-book-app$ apt build-dep address-book-app Reading package lists... Done Reading package lists... Done Building dependency tree Reading state information... Done The following packages were automatically installed and are no longer required: libaudio2 libmng2 libmysqlclient20 libpcre16-3 mysql-common qtcore4-l10n Use 'apt autoremove' to remove them. The following NEW packages will be installed: libegl1-mesa-dev libflac8 libfontenc1 libgl1-mesa-dev libgles2-mesa-dev libgsettings-qt1 libgstreamer-plugins-base1.0-0 libgstreamer1.0-0 libhistoryservice0 libleveldb1v5 libmirclient-dev libmircommon-dev libmircookie-dev libmircookie2 libmircore-dev libogg0 liborc-0.4-0 libphonenumber7 libprotobuf-dev libprotobuf9v5 libpthread-stubs0-dev libpulse0 libqofono-dev libqofono-qt5-0 libqt5concurrent5 libqt5contacts5 libqt5feedback5 libqt5multimedia5 libqt5organizer5 libqt5printsupport5 libqt5quickparticles5 libqt5quickshapes5 libqt5quicktest5 libqt5quickwidgets5 libqt5sql5 libqt5systeminfo5 libqt5test5 libqt5versit5 libqt5versitorganizer5 libqt5xml5 libqt5xmlpatterns5 libsnappy1v5 libsndfile1 libtag1v5 libtag1v5-vanilla libtelepathy-qt5-0 libubuntugestures5 libubuntumetrics5 libubuntutoolkit5 libunity-api1 libvorbis0a libvorbisenc2 libvulkan-dev libvulkan1 libwayland-bin libwayland-dev libwrap0 libx11-dev libx11-xcb-dev libxau-dev libxaw7 libxcb-dri2-0-dev libxcb-dri3-dev libxcb-glx0-dev libxcb-present-dev libxcb-randr0-dev libxcb-render0-dev libxcb-shape0-dev libxcb-sync-dev libxcb-xfixes0-dev libxcb1-dev libxdamage-dev libxdmcp-dev libxext-dev libxfixes-dev libxfont1 libxkbcommon-dev libxkbfile1 libxmu6 libxshmfence-dev libxxf86vm-dev mesa-common-dev pkg-config qml-module-qtcontacts qml-module-qtfeedback qml-module-qtgraphicaleffects qml-module-qtquick-layouts qml-module-qtquick-window2 qml-module-qtquick2 qml-module-qttest qml-module-ubuntu-components qml-module-ubuntu-components-labs qml-module-ubuntu-layouts qml-module-ubuntu-performancemetrics qml-module-ubuntu-test qt5-default qt5-qmake qt5-qmake-bin qt5-qmltooling-plugins qtbase5-dev qtbase5-dev-tools qtdeclarative5-buteo-syncfw0.1 qtdeclarative5-dev qtdeclarative5-dev-tools qtdeclarative5-gsettings1.0 qtdeclarative5-ofono0.2 qtdeclarative5-qtcontacts-plugin qtdeclarative5-ubuntu-content1 qtdeclarative5-ubuntu-history0.1 qtdeclarative5-ubuntu-keyboard-extensions0.1 qtdeclarative5-ubuntu-telephony-phonenumber0.1 qtdeclarative5-ubuntu-ui-toolkit-plugin qtpim5-dev suru-icon-theme thumbnailer-service ttf-ubuntu-font-family ubuntu-mobile-icons ubuntu-ui-toolkit-theme x11-xkb-utils x11proto-core-dev x11proto-damage-dev x11proto-dri2-dev x11proto-fixes-dev x11proto-gl-dev x11proto-input-dev x11proto-kb-dev x11proto-xext-dev x11proto-xf86vidmode-dev xorg-sgml-doctools xserver-common xtrans-dev xvfb zlib1g-dev 0 upgraded, 133 newly installed, 0 to remove and 0 not upgraded. Need to get 0 B/32,3 MB of archives. After this operation, 122 MB of additional disk space will be used. Do you want to continue? [Y/n] y debconf: delaying package configuration, since apt-utils is not installed dpkg: error: requested operation requires superuser privilege E: Sub-process /usr/bin/dpkg returned an error code (2) E: Failed to process build dependencies
dobey last edited by dobey
However if I do this, the packet installation always fails with the message, that the package installation requires root access:
You need to enter the libertine container differently so that you are using it as "root" like so:
libertine-container-mainager exec -i <containerid> -c bash
This will give you a "root" prompt inside the container, and you should be able to use
apt build-depthen (assuming you did not install sudo in the container).
After that, you can exit the "root" prompt, and run the standard container shell normally, to be able to build the packages.
@dobey Thanks, that was the missing link to getting the container set up. However the tests still fail with the libGLESv2.so.2 issue, same as I had before when trying with the "manually" configured container:
[100%] Linking CXX shared module libubuntu-contacts-qml.so cd /home/phablet/address-book-app/obj-arm-linux-gnueabihf/src/imports/Ubuntu/Contacts && /usr/bin/cmake -E cmake_link_script CMakeFiles/ubuntu-contacts-qml.dir/link.txt --verbose=1 /usr/bin/arm-linux-gnueabihf-g++ -fPIC -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -std=c++11 -Wall -Wl,-Bsymbolic-functions -Wl,-z,relro -shared -o libubuntu-contacts-qml.so CMakeFiles/ubuntu-contacts-qml.dir/contacts.cpp.o CMakeFiles/ubuntu-contacts-qml.dir/imagescalethread.cpp.o CMakeFiles/ubuntu-contacts-qml.dir/plugin.cpp.o CMakeFiles/ubuntu-contacts-qml.dir/simcardcontacts.cpp.o CMakeFiles/ubuntu-contacts-qml.dir/ubuntu-contacts-qml_automoc.cpp.o -lqofono-qt5 /usr/lib/arm-linux-gnueabihf/libQt5Contacts.so.5.0.0 /usr/lib/arm-linux-gnueabihf/libQt5Quick.so.5.12.9 /usr/lib/arm-linux-gnueabihf/libQt5DBus.so.5.12.9 /usr/lib/arm-linux-gnueabihf/libQt5Qml.so.5.12.9 /usr/lib/arm-linux-gnueabihf/libQt5Network.so.5.12.9 /usr/lib/arm-linux-gnueabihf/libQt5Gui.so.5.12.9 /usr/lib/arm-linux-gnueabihf/libQt5Core.so.5.12.9 make: Leaving directory '/home/phablet/address-book-app/obj-arm-linux-gnueabihf' [100%] Built target ubuntu-contacts-qml make: Leaving directory '/home/phablet/address-book-app/obj-arm-linux-gnueabihf' /usr/bin/cmake -E cmake_progress_start /home/phablet/address-book-app/obj-arm-linux-gnueabihf/CMakeFiles 0 make: Leaving directory '/home/phablet/address-book-app/obj-arm-linux-gnueabihf' dh_auto_test -O--parallel -O--fail-missing make -j2 test ARGS\+=-j2 make: Entering directory '/home/phablet/address-book-app/obj-arm-linux-gnueabihf' Running tests... /usr/bin/ctest --force-new-ctest-process -j2 Test project /home/phablet/address-book-app/obj-arm-linux-gnueabihf Start 1: contact_list Start 2: Contact_list_model 1/9 Test #2: Contact_list_model ...............***Failed 2.20 sec /usr/lib/arm-linux-gnueabihf/qt5/bin/qmltestrunner: error while loading shared libraries: libGLESv2.so.2: cannot open shared object file: No such file or directory
@anhilde maybe you need to install more stuff
@doniks I would have though, that that should be taken care off by:
Is that assumption wrong? Nore, I'm not trying to get this working somehow, but rather want to try and understand how the application development process is intended.
Hi, I have a problem understanding, how the (if?) the build application should be runnable from within the container. When the build runs the tests I assume it uses the executable address-book-app found in the project tree.
phablet@ubuntu-phablet:~/address-book-app/obj-arm-linux-gnueabihf$ ./src/app/address-book-app ./src/app/address-book-app: error while loading shared libraries: libGLESv2.so.2: cannot open shared object file: No such file or directory phablet@ubuntu-phablet:~/address-book-app/obj-arm-linux-gnueabihf$ ldd ./src/app/address-book-app libQt5Gui.so.5 => not found libQt5Qml.so.5 => not found libgcc_s.so.1 => not found libstdc++.so.6 => not found libQt5Core.so.5 => not found libQt5Network.so.5 => not found /lib/ld-linux-armhf.so.3 => /lib/ld-linux-armhf.so.3 (0x00000000) libQt5DBus.so.5 => not found libQt5Quick.so.5 => not found libc.so.6 => not found
Since that is not directly linked to libGLESv2.so.2 my assumption was that this dependency comes in via libQTGui.so.5. Why is this not found?
If I search for it logged into the container as root I find libQt5Gui.so.5:
root@ubuntu-phablet:/# ldd /usr/lib/arm-linux-gnueabihf/libQt5Gui.so.5 linux-gate.so.1 => (0x00000000) libfakeroot-sysv.so => /usr/lib/arm-linux-gnueabihf/libfakeroot/libfakeroot-sysv.so (0x00000000) libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0x00000000) libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0x00000000) /lib/ld-linux-armhf.so.3 => /lib/ld-linux-armhf.so.3 (0x00000000) libfakechroot.so => /usr/lib/arm-linux-gnueabihf/fakechroot/libfakechroot.so (0x00000000) libQt5Core.so.5 => /usr/lib/arm-linux-gnueabihf/libQt5Core.so.5 (0x00000000) libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0x00000000) libz.so.1 => /lib/arm-linux-gnueabihf/libz.so.1 (0x00000000) libicui18n.so.55 => /usr/lib/arm-linux-gnueabihf/libicui18n.so.55 (0x00000000) libicuuc.so.55 => /usr/lib/arm-linux-gnueabihf/libicuuc.so.55 (0x00000000) libicudata.so.55 => /usr/lib/arm-linux-gnueabihf/libicudata.so.55 (0x00000000) libstdc++.so.6 => /usr/lib/arm-linux-gnueabihf/libstdc++.so.6 (0x00000000) libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0x00000000) libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0x00000000) libpcre2-16.so.0 => /usr/lib/arm-linux-gnueabihf/libpcre2-16.so.0 (0x00000000) libdouble-conversion.so.1 => /usr/lib/arm-linux-gnueabihf/libdouble-conversion.so.1 (0x00000000) libglib-2.0.so.0 => /lib/arm-linux-gnueabihf/libglib-2.0.so.0 (0x00000000) libpcre.so.3 => /lib/arm-linux-gnueabihf/libpcre.so.3 (0x00000000) libGLESv2.so.2 => /usr/lib/arm-linux-gnueabihf/mesa-egl/libGLESv2.so.2 (0x00000000) libglapi.so.0 => /usr/lib/arm-linux-gnueabihf/libglapi.so.0 (0x00000000) libpng12.so.0 => /lib/arm-linux-gnueabihf/libpng12.so.0 (0x00000000) libharfbuzz.so.0 => /usr/lib/arm-linux-gnueabihf/libharfbuzz.so.0 (0x00000000) libfreetype.so.6 => /usr/lib/arm-linux-gnueabihf/libfreetype.so.6 (0x00000000) libgraphite2.so.3 => /usr/lib/arm-linux-gnueabihf/libgraphite2.so.3 (0x00000000)
And libGLESv2.so.2 is available under the path reported by ldd and it reports:
root@ubuntu-phablet:/usr/lib/arm-linux-gnueabihf/mesa-egl# ldd libGLESv2.so.2 linux-gate.so.1 => (0x00000000) libfakeroot-sysv.so => /usr/lib/arm-linux-gnueabihf/libfakeroot/libfakeroot-sysv.so (0x00000000) libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0x00000000) libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0x00000000) /lib/ld-linux-armhf.so.3 => /lib/ld-linux-armhf.so.3 (0x00000000) libfakechroot.so => /usr/lib/arm-linux-gnueabihf/fakechroot/libfakechroot.so (0x00000000) libglapi.so.0 => /usr/lib/arm-linux-gnueabihf/libglapi.so.0 (0x00000000) libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0x00000000)
Does anybody have a clue why then the tests fail with the message that libGLESv2.so.2 is missing? Note I do have dyslexia, so if there is an obvious name mix up here, please let me know.
@anhilde Well GLES2 is there. There does seem to be a weird issue though, since it's not finding the libs when using the container normally. Especially odd given that it compiled and only failed when trying to run the tests.
As for what is run for the tests, no, it is not running the address book app itself, but other tests binaries, or qmltestrunner for QML tests.
@doniks I would have though, that that should be taken care off by:
Is that assumption wrong?
I would have assumed the same yes ...
Nore, I'm not trying to get this working somehow, but rather want to try and understand how the application development process is intended.
Understood. And very much appreciated! I was just thinking maybe there is a genuine bug in the dependencies.
Full disclosure, I've never done this on-device build myself. I'm just thinking out loud here.
Also, I just notice that there are two packages that have this lib: mesa and hybris. Maybe you actually need the other one. Or a related thought: Maybe you can run the tests from OUTSIDE the container, after the building INSIDE is complete.
@dobey I cannot get the build to run. I always end up with the libGLESv2.so.2 problem. I actually tried installing all packages via:
libertine-container-manager install-package -p $ALLPACKS_FROM_APT_BUILD_DEP_COMMAND
The very first build fails with:
phablet@ubuntu-phablet:~/address-book-app$ DEB_BUILD_OPTIONS="parallel=2 debug" dpkg-buildpackage -rfakeroot -b dpkg-buildpackage: source package address-book-app dpkg-buildpackage: source version 0.4~0ubports1 dpkg-buildpackage: source distribution xenial dpkg-buildpackage: source changed by Alberto Mardegan <email@example.com> dpkg-buildpackage: host architecture armhf dpkg-source --before-build address-book-app dpkg-checkbuilddeps: error: Unmet build dependencies: dh-apparmor qml-module-ofono qml-module-qt-labs-platform qml-module-qtsysteminfo (>= 5.0~) dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting dpkg-buildpackage: warning: (Use -d flag to override.)
I installed these packages as well with the libertine-container-manager. Following that the build works, until the test execution fails.
I fiddled with the LD_LIBRARY_PATH, just to see if I can get anything going and ran the qmltestrunner manually, leading to this output:
phablet@ubuntu-phablet:~/address-book-app$ export LD_LIBRARY_PATH="/usr/lib/arm-linux-gnueabihf:/usr/lib/arm-linux-gnueabihf/mesa-egl:$LD_LIBRARY_PATH" phablet@ubuntu-phablet:~/address-book-app$ /usr/lib/arm-linux-gnueabihf/qt5/bin/qmltestrunner qt.qpa.xcb: could not connect to display qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found. This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem. Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb. Aborted
At that point I am wondering, with entering the container with DISPLAY= , there won't be a display available ever in there. So can these tests really run in the container? Is there a nightly build that I could try to investigate, to see how this should be done?
, after the building INSIDE is complete.
The build is incomplete, because the tests failed to run.
well ok, but I would assume that the build process first creates executables and then tests them. so, if the executables exist, one could still try to use/test them outside the container.
and even if the failing tests are before some final step that puts the executables together maybe anhilde can relatively easily disable them for the moment, to get one step further
root@ubuntu-phablet:/# ldd /usr/lib/arm-linux-gnueabihf/libQt5Gui.so.5 libGLESv2.so.2 => /usr/lib/arm-linux-gnueabihf/mesa-egl/libGLESv2.so.2 (0x00000000)
so this is the mesa version, maybe you need the hybris one https://packages.ubuntu.com/search?suite=xenial&arch=armhf&mode=filename&searchon=contents&keywords=libGLESv2 so I guess you could try to
libertine-container-manager remove-package -p libgles2-mesa-devand instead
libertine-container-manager install-package -p libhybris