Navigation

    UBports Robot Logo

    UBports Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search

    Qml tests using TestCase

    App Development
    4
    14
    764
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Krille
      Krille last edited by

      I'm currently searching for a way to write automated tests for my Qml apps. I've heard about Autopilot but couldn't find a documentation how to use it. However there is the TestCase Component (https://doc.qt.io/qt-5/qml-qttest-testcase.html) for Qml and I've made it working by just switch the qmlscene to qmltestrunner. I've written a dirty script to run tests (run_tests.sh):

      #!/bin/bash
      
      # Switch to test mode
      sed '3iExec=qmltestrunner' fluffychat.desktop.in -i
      
      # Test
      clickable clean build-libs desktop
      
      # Switch back to production mode
      sed '3d' fluffychat.desktop.in -i
      

      So now I can write some tests like this one (tst_main.qml):

      import QtQuick 2.0
      import QtTest 1.2
      
      TestCase {
          name: "MathTests"
      
          function test_math() {
              compare(2 + 2, 4, "2 + 2 = 4")
          }
      
          function test_fail() {
              compare(2 + 2, 5, "2 + 2 = 5")
          }
      }
      

      which is working fine with this script. But I have two problems:

      1. The script needs to be executed in a graphical way. So I assume that will make problems for GitLab CI...
      2. The current solution is very dirty. It would be awesome to have something like a "clickable test" command. I have tried:
      clickable run "qmltestrunner"
      

      But this gives:

      Using docker container "clickable/ubuntu-sdk:16.04-armhf-ec0904fa-133f-4754-bb80-2510473d4668"
      qmltestrunner: could not find a Qt installation of ''
      

      Maybe we can fix this by installing some packages in the container?

      dobey 1 Reply Last reply Reply Quote 1
      • dobey
        dobey @Krille last edited by

        @Krille said in Qml tests using TestCase:

        The script needs to be executed in a graphical way. So I assume that will make problems for GitLab CI...

        xvfb-run -a qmltestrunner

        @Krille said in Qml tests using TestCase:

        Maybe we can fix this by installing some packages in the container?

        QT_SELECT=qt5 xvfb-run -a qmltestrunner

        Or install the qt5-default package in the container.

        1 Reply Last reply Reply Quote 0
        • Krille
          Krille last edited by

          Thanks for your answer. So in summary I have tried the following command:

          clickable --arch="amd64" run "xvfb-run -a qmltestrunner"
          

          Now there is no more a graphical problem and no other error. But nothing happens after executing this command except:

          Using docker container "clickable/ubuntu-sdk:16.04-amd64-5a95c706-276c-493d-9fff-1507a27ab289"
          
          
          

          And then it hangs forever.

          Brian said: "ok, running xvfb-run -a qmltestrunner directly in the docker container gives me: "

          QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
          Could not initialize GLX
          Aborted (core dumped)
          
          dobey 1 Reply Last reply Reply Quote 0
          • U
            UniSuperBox last edited by

            Hmm... It's possible that you also need to set QT_QPA_PLATFORM=xcb either using export or by putting it before your command.

            dobey 1 Reply Last reply Reply Quote 0
            • dobey
              dobey @UniSuperBox last edited by

              @UniSuperBox said in Qml tests using TestCase:

              Hmm... It's possible that you also need to set QT_QPA_PLATFORM=xcb either using export or by putting it before your command.

              No, that is not the problem. Qt will select that automatically.

              1 Reply Last reply Reply Quote 0
              • dobey
                dobey @Krille last edited by dobey

                @Krille Ah, sorry. I forgot the extra option one needs to ensure color depth of the virtual display.

                Try xvfb-run -a -s '-screen 0 540x960x24' qmltestrunner instead.

                Not sure why clickable didn't show you the error and exit, though. Maybe a bug in clickable?

                1 Reply Last reply Reply Quote 0
                • Krille
                  Krille last edited by

                  Okay, so now I've tested:

                  clickable --arch="amd64" run "xfvb-run -a -s '-screen 0 540x960x24' qmltestrunner"
                  

                  Leads me to:

                  bash: xfvb-run: command not found
                  

                  Maybe we need to install xfvb in the container? Is there a way how I can install this in the container by myself?

                  dobey 1 Reply Last reply Reply Quote 0
                  • U
                    UniSuperBox last edited by UniSuperBox

                    Sorry... xvfb, not xfvb. It stands for "X Virtual FrameBuffer."

                    1 Reply Last reply Reply Quote 0
                    • dobey
                      dobey @Krille last edited by

                      @Krille Sorry, was a typo. I've edited the post now. It's xvfb-run

                      1 Reply Last reply Reply Quote 0
                      • Krille
                        Krille last edited by Krille

                        Haha okay. So now:

                        clickable --arch="amd64" run "xvfb-run -a -s '-screen 0 540x960x24' qmltestrunner"
                        

                        And again the previous result: Nothing happens anymore 😞 I will try to run the command in the container to get an error report...

                        D 1 Reply Last reply Reply Quote 0
                        • D
                          domubpkm @Krille last edited by

                          @Krille Hi. I don't know anything about what you are doing but my eyes see again xfvb above whereas it is spoken about xvfb. Just to be sure it's normal, i prefered to prevent you.

                          Krille 1 Reply Last reply Reply Quote 1
                          • Krille
                            Krille @domubpkm last edited by

                            @domubpkm Oh, sure sorry. But nothing changes...

                            1 Reply Last reply Reply Quote 0
                            • Krille
                              Krille last edited by

                              Btw I have found that this is propably the same reason why the UI tests (at least) in the messages app are still disabled: https://github.com/ubports/messaging-app/blob/xenial/tests/qml/CMakeLists.txt#L18

                              dobey 1 Reply Last reply Reply Quote 0
                              • dobey
                                dobey @Krille last edited by

                                @Krille said in Qml tests using TestCase:

                                Btw I have found that this is propably the same reason why the UI tests (at least) in the messages app are still disabled: https://github.com/ubports/messaging-app/blob/xenial/tests/qml/CMakeLists.txt#L18

                                The tests aren't disabled. If xvfb is installed and tests are run, then these tests which depend on xvfb should be run too. If however, you are using crossbuilder then the tests will not be run, because tests do not get run during the build then, as you cannot run arm binaries native on x86.

                                1 Reply Last reply Reply Quote 0
                                • First post
                                  Last post