Qml tests using TestCase
-
I'm currently searching for a way to write automated tests for my Qml apps. I've heard about Autopilot but couldn't find a documentation how to use it. However there is the TestCase Component (https://doc.qt.io/qt-5/qml-qttest-testcase.html) for Qml and I've made it working by just switch the qmlscene to qmltestrunner. I've written a dirty script to run tests (run_tests.sh):
#!/bin/bash # Switch to test mode sed '3iExec=qmltestrunner' fluffychat.desktop.in -i # Test clickable clean build-libs desktop # Switch back to production mode sed '3d' fluffychat.desktop.in -i
So now I can write some tests like this one (tst_main.qml):
import QtQuick 2.0 import QtTest 1.2 TestCase { name: "MathTests" function test_math() { compare(2 + 2, 4, "2 + 2 = 4") } function test_fail() { compare(2 + 2, 5, "2 + 2 = 5") } }
which is working fine with this script. But I have two problems:
- The script needs to be executed in a graphical way. So I assume that will make problems for GitLab CI...
- The current solution is very dirty. It would be awesome to have something like a "clickable test" command. I have tried:
clickable run "qmltestrunner"
But this gives:
Using docker container "clickable/ubuntu-sdk:16.04-armhf-ec0904fa-133f-4754-bb80-2510473d4668" qmltestrunner: could not find a Qt installation of ''
Maybe we can fix this by installing some packages in the container?
-
@Krille said in Qml tests using TestCase:
The script needs to be executed in a graphical way. So I assume that will make problems for GitLab CI...
xvfb-run -a qmltestrunner
@Krille said in Qml tests using TestCase:
Maybe we can fix this by installing some packages in the container?
QT_SELECT=qt5 xvfb-run -a qmltestrunner
Or install the
qt5-default
package in the container. -
Thanks for your answer. So in summary I have tried the following command:
clickable --arch="amd64" run "xvfb-run -a qmltestrunner"
Now there is no more a graphical problem and no other error. But nothing happens after executing this command except:
Using docker container "clickable/ubuntu-sdk:16.04-amd64-5a95c706-276c-493d-9fff-1507a27ab289"
And then it hangs forever.
Brian said: "ok, running
xvfb-run -a qmltestrunner
directly in the docker container gives me: "QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root' Could not initialize GLX Aborted (core dumped)
-
Hmm... It's possible that you also need to set
QT_QPA_PLATFORM=xcb
either usingexport
or by putting it before your command. -
@UniSuperBox said in Qml tests using TestCase:
Hmm... It's possible that you also need to set
QT_QPA_PLATFORM=xcb
either usingexport
or by putting it before your command.No, that is not the problem. Qt will select that automatically.
-
@Krille Ah, sorry. I forgot the extra option one needs to ensure color depth of the virtual display.
Try
xvfb-run -a -s '-screen 0 540x960x24' qmltestrunner
instead.Not sure why clickable didn't show you the error and exit, though. Maybe a bug in clickable?
-
Okay, so now I've tested:
clickable --arch="amd64" run "xfvb-run -a -s '-screen 0 540x960x24' qmltestrunner"
Leads me to:
bash: xfvb-run: command not found
Maybe we need to install xfvb in the container? Is there a way how I can install this in the container by myself?
-
Sorry...
xvfb
, notxfvb
. It stands for "X Virtual FrameBuffer." -
@Krille Sorry, was a typo. I've edited the post now. It's
xvfb-run
-
Haha okay. So now:
clickable --arch="amd64" run "xvfb-run -a -s '-screen 0 540x960x24' qmltestrunner"
And again the previous result: Nothing happens anymore I will try to run the command in the container to get an error report...
-
@Krille Hi. I don't know anything about what you are doing but my eyes see again xfvb above whereas it is spoken about xvfb. Just to be sure it's normal, i prefered to prevent you.
-
@domubpkm Oh, sure sorry. But nothing changes...
-
Btw I have found that this is propably the same reason why the UI tests (at least) in the messages app are still disabled: https://github.com/ubports/messaging-app/blob/xenial/tests/qml/CMakeLists.txt#L18
-
@Krille said in Qml tests using TestCase:
Btw I have found that this is propably the same reason why the UI tests (at least) in the messages app are still disabled: https://github.com/ubports/messaging-app/blob/xenial/tests/qml/CMakeLists.txt#L18
The tests aren't disabled. If
xvfb
is installed and tests are run, then these tests which depend on xvfb should be run too. If however, you are usingcrossbuilder
then the tests will not be run, because tests do not get run during the build then, as you cannot runarm
binaries native onx86
.