ut-sysbench-qt-gui: Compare CPU performance of different devices

sven

@moncon said in ut-sysbench-qt-gui: Compare CPU performance of different devices:

Works on Cosmo Communicator, copes with landscape view as well.

Same device, OTA-19: the two lower buttons are not visible for me. How did you manage to see them?

kugiigi

@sven That seems to be a bug, probably because the app uses Widgets that aren't really suitable for touch and not responsive enough. You can rotate the screen and it'll be fixed after that.

johndoe

@domubpkm
Thank you really much.

The memory test shows the RAM speed and it is good to see that it works on some devices. So it is maybe not a problem with Ubuntu Touch and I maybe need to change the parameters, so that we get results on all devices.

The mutex test shows the speed of mutual exclusions (less time is better). Sometimes multiple processes want access to the same files at the same time and this access permissions need to be managed. This management is done by mutexes.

johndoe

This post is deleted!

johndoe

@sven said in ut-sysbench-qt-gui: Compare CPU performance of different devices:

@johndoe said in ut-sysbench-qt-gui: Compare CPU performance of different devices:

if some of you could share your device's name together with a screenshot of the result you get

A save (as plain text) button in the app would be great, so that people can post results for immediate processing via scripts or copy/paste.

I will work on it.

johndoe

@cliffcoggin
@kugiigi
I will try to use QtQuick components instead of QtWidgets in a later version to prevent bugs, but I am no expert when it comes to Qt.
Restarting the app or turning the screen will maybe fix graphical bugs.
I don't know why the PinePhone crashes, maybe this will also be fixed with QtQuick components instead of QtWidgets .

Thank you for testing the app.

johndoe

@sven Restarting the app or rotating the screen will maybe fix graphical bugs. I will work on a solution.
@MonCon

A Former User

@sven
@johndoe
The only graphical tweak I could think of that I've done to mine is to reduce the 'launcher width' in UT Tweak Tool from 8 to 6 to get a couple more pinned shortcuts on the left hand side of the screen. I reset the value to 8 and rebooted and the buttons disappeared, set the value back to 6 and the buttons reappeared, see attached screenshots.

johndoe

@kugiigi I made some suggested changes to the app (including 'QtQuick components instead of QtWidgets'). It would be nice if you could try to run it on the Pinephone again and try to performe "Multiple Benchmarks", but be aware, that it will maybe crash again.

kugiigi

@johndoe Yup it's working now on the Pinephone. Thanks

Here's the results. I'm surprised it has a lot better results than phones like Nexus 5

sysbench --test=cpu --num-threads=2 --cpu-max-prime=20000 run
    617 events per second

sysbench --test=cpu --num-threads=8 --cpu-max-prime=20000 run
    1190 events per second

sysbench --test=memory --num-threads=4 run
Operations performed: 104857600 (875326.88 ops/sec)
102400.00 MB transferred (854.81 MB/sec)

sysbench --test=mutex --num-threads=64 run
    total time:                          1.0368s

sven

The new version works perfectly on the Cosmo Communicator.

johndoe

@kugiigi Thank you. Yes, this is really interesting. I think the difference between older phones and newer phones is so big, because the older phones have 32bit ARM CPUs and maybe ARM made changes for their 64bit architecture, that allows the CPU to perform way better in the specific workload in the sysbench CPU test.
What is even more interesting to me, is that that some phones with Mediatek CPUs perform better than the Pixel 3a and that the Librem 5 has similar results to the PinePhone (and not better results with a CPU that costs way more than an Allwinner A64).
I don't think that represents the everyday performance and responsiveness of the phones and I will maybe add a disclaimer to the app, that the results only give a rough idea about the performance, that the sysbench results are not 100% perfect and also other tests should be looked at when comparing devices.

Here are the results of the Librem5 with PureOS and results coming from a newer version of sysbench (maybe this could also explain why it did not perform better than the PinePhone, but I do not think so) :

sysbench --test=cpu --num-threads=2 --cpu-max-prime=20000 run
events per second: 595.77

sysbench --test=cpu --num-threads=8 --cpu-max-prime=20000 run
events per second: 1190.77

sysbench --test=memory --num-threads=4 run
5643286.07 ops per second
5511.02 MiB/sec

sysbench --test=mutex --num-threads=64 run
total time: 16.1842s

Source: https://forums.puri.sm/t/performance-compare-librem-5-to-ubuntu-touch-devices/15425

johndoe

@sven Nice. I would appreciate it, if you could also test the app on your OnePlus 3T and share the results of the benchmarks, if you have time for that.

sven

@johndoe Unfortunately, I borrowed the OnePlus 3T to a colleague; so I cannot run any tests ... at the moment.

johndoe

@sven Ok, no problem.

Yoni

@johndoe
Xiaomi mi A2 results:

sysbench --test=cpu --num-threads=2 --cpu-max-prime=20000 run
1717 events per second

sysbench --test=cpu --num-threads=8 --cpu-max-prime=20000 run
4915 events per second

sysbench --test=memory --num-threads=4 run
Operations performed: 104857600 (6280574.81 ops/sec)
102400.00 MB transferred (6133.37 MB/sec)

sysbench --test=mutex --num-threads=64 run
total time: 1.1919s

sysbench --test=cpu --num-threads=2 --cpu-max-prime=20000 run
1706 events per second

sysbench --test=cpu --num-threads=8 --cpu-max-prime=20000 run
5307 events per second

sysbench --test=memory --num-threads=4 run
Operations performed: 104857600 (6196075.77 ops/sec)
102400.00 MB transferred (6050.86 MB/sec)

sysbench --test=mutex --num-threads=64 run
total time: 1.1151s

sysbench --test=cpu --num-threads=2 --cpu-max-prime=20000 run
1705 events per second

sysbench --test=cpu --num-threads=2 --cpu-max-prime=20000 run
1706 events per second

johndoe

@yoni thank you

A Former User

@johndoe
New version works perfectly on another Cosmo Communicator and another PinePhone.
New export and compare function works flawlessly on both.
Copy and paste results works on Cosmo, if only I could figure out how to Ctrl+V on the PinePhone touchscreen .
Really nice looking app. From my very basic understanding of the numbers I would say the app accurately reflects my experience on the two devices I have in their current state of play.

Cosmo Communicator
sysbench --test=cpu --num-threads=2 --cpu-max-prime=20000 run
1652 events per second

sysbench --test=cpu --num-threads=8 --cpu-max-prime=20000 run
5287 events per second

sysbench --test=memory --num-threads=4 run
Operations performed: 104857600 (2469862.22 ops/sec)
102400.00 MB transferred (2411.97 MB/sec)

sysbench --test=mutex --num-threads=64 run
total time: 1.2297s

PinePhone
sysbench --test=cpu --num-threads=2 --cpu-max-prime=20000 run
623 events per second

sysbench --test=cpu --num-threads=8 --cpu-max-prime=20000 run
1201 events per second

sysbench --test=memory --num-threads=4 run
Operations performed: 104857600 (932315.64 ops/sec)
102400.00 MB transferred (910.46 MB/sec)

sysbench --test=mutex --num-threads=64 run
total time: 1.1598s

johndoe

@moncon Thanks, I think you can touch and hold a text field and then tap "Paste" to paste the text, that you have in the clipboard (but I see that you somehow managed to get the data of the Pinephone into your comment). It is good to see that the results are overall consistent with other phones from the same model.

s710

@johndoe said in ut-sysbench-qt-gui: Compare CPU performance of different devices:

Hello,
I wrote an app to compare the performance of different devices and released it on the OpenStore: https://open-store.io/app/ut-sysbench-qt-gui
I hope the arm64 version works, because I can only test the armhf version.
It would be nice, if some of you could share your device's name together with the results you get, after clicking on "Multiple Benchmarks".
"Export and Compare" will copy the results to your clipboard and show you a bar chart with the performace of other devices.

Multiple results from different persons for the same model are also useful.

You are also free to share problems with my app. I know that the memory (RAM) test does not work on all devices.

----------------------------------------------------------------------------------------
Results of my LG Nexus 4 with Ubuntu 16.04 OTA-19:

sysbench --test=cpu --num-threads=2 --cpu-max-prime=20000 run
57 events per second

sysbench --test=cpu --num-threads=8 --cpu-max-prime=20000 run
113 events per second

sysbench --test=memory --num-threads=4 run
Operations performed: 0 ( 0.00 ops/sec)
0.00 MB transferred (0.00 MB/sec)

sysbench --test=mutex --num-threads=64 run
total time: 2.4057s

Nice!

Would it be possible to compile the user results into some kind of diagram/table which allows to easily compare device performance?