VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed
-
Here is a situation where I'm close to Out-of-Memory, (my phone freezed for 15s and then recovered). If I reboot the phone and open the same apps I will have a lot more swap availiable. Surely if I keep using it like that, it will crash at some point. I have not made a call today, but I've taken pictures. I don't understand why the swap use is so much higher than normal because top does not show any processes that use abnormal amount of Ram. Could the problem come from swap itself?
top - 13:31:14 up 1 day, 25 min, 2 users, load average: 21.65, 22.07, 20.99 Tasks: 640 total, 1 running, 587 sleeping, 2 stopped, 50 zombie %Cpu(s): 12.1 us, 4.9 sy, 0.1 ni, 82.1 id, 0.0 wa, 0.0 hi, 0.8 si, 0.0 st MiB Mem : 81.1/3498.4 [ ] MiB Swap: 82.5/1924.1 [ ] PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 3415 phablet 20 0 6229920 388144 303092 S 0.7 10.8 32:31.70 lomiri 22255 phablet 20 0 196.4g 162708 60184 S 0.3 4.5 0:26.95 signal-desktop 17763 phablet 20 0 3545708 152084 145080 S 0.3 4.2 1:46.33 qmlscene 17888 phablet 20 0 14.0g 130968 25928 S 0.7 3.7 4:06.44 QtWebEngineProc 22374 phablet 20 0 2414496 112092 61980 S 2.0 3.1 0:11.02 lomiri-camera-a 22131 phablet 20 0 204.4g 105492 23376 S 7.9 2.9 0:42.14 signal-desktop 4854 phablet 20 0 2283820 63164 59408 S 0.3 1.8 1:39.14 maliit-server 2551 1047 20 0 12.7g 57016 28064 S 99.3 1.6 3:21.46 camerahalserver 22343 phablet 20 0 1977936 42924 41036 S 2.3 1.2 0:05.77 qmlscene 22221 phablet 20 0 32.5g 40492 38812 S 0.0 1.1 0:05.71 signal-desktop 5103 phablet 20 0 2132348 40060 40060 T 0.0 1.1 1:53.34 lomiri-system-s 5222 phablet 20 0 1919912 39960 39960 T 0.0 1.1 0:34.41 lomiri-dialer-a 21580 phablet 20 0 293540 30952 30472 S 0.0 0.9 0:17.62 Xwayland 2520 1013 20 0 10.9g 13532 12296 S 8.9 0.4 0:16.99 minimediaservic 712 root 19 -1 83872 11136 10304 S 0.0 0.3 0:52.47 systemd-journal 2148 root 20 0 1214404 9508 7156 S 1.3 0.3 16:42.81 lomiri-system-c 1 root 20 0 24248 9496 5516 S 0.0 0.3 0:57.16 systemd 22745 phablet 20 0 178464 9164 7852 S 0.0 0.3 0:00.08 mtp-server 22738 phablet 20 0 326744 9056 8216 S 0.0 0.3 0:00.14 adbd 22755 root 20 0 1155836 8052 5828 S 0.0 0.2 0:00.05 adbd-pam-sessio 2465 phablet 20 0 22468 7728 5136 S 0.0 0.2 0:16.46 systemd 4858 phablet 20 0 2666196 7260 4108 S 0.0 0.2 0:31.06 lomiri-push-ser 1654 root 20 0 2286604 6788 1184 S 0.0 0.2 0:58.29 snapd 22773 phablet 20 0 57208 5608 3388 R 1.6 0.2 0:01.15 top 4855 phablet 20 0 294540 5496 4320 S 0.0 0.2 0:03.85 mtp-server-usb- 5261 phablet 20 0 350504 5320 4488 S 0.0 0.1 0:06.35 zeitgeist-fts 4857 phablet 20 0 405916 5188 4340 S 0.0 0.1 0:16.78 lomiri-indicato 22763 phablet 20 0 53404 5080 3584 S 0.0 0.1 0:00.08 bash -
Actually there is a very strange phenomenon as I navigate in Signal-Destkop (without doing anything else that navigating in the menu) then the Ram used by signal desktop increases rapidly:
%Cpu(s): 9.9 us, 2.1 sy, 0.0 ni, 87.5 id, 0.1 wa, 0.0 hi, 0.4 si, 0.0 st MiB Mem : 3498.4 total, 368.6 free, 2913.7 used, 793.1 buff/cache MiB Swap: 1924.1 total, 752.5 free, 1171.7 used. 584.7 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 22255 phablet 20 0 196.7g 649492 519052 S 46.4 18.1 1:19.36 signal-desktop 3415 phablet 20 0 6140780 418372 295348 S 10.5 11.7 32:59.29 lomiri 17763 phablet 20 0 3553392 198512 160808 S 0.0 5.5 1:50.02 qmlscene 17888 phablet 20 0 14.0g 159116 38372 S 0.0 4.4 4:15.90 QtWebEngineProc 22131 phablet 20 0 204.6g 134476 32876 S 1.6 3.8 1:13.70 signal-desktop 22974 phablet 20 0 2640604 108728 63600 T 0.0 3.0 0:05.26 lomiri-clock-ap 5222 phablet 20 0 1919932 91832 52740 T 0.0 2.6 0:35.71 lomiri-dialer-a 4854 phablet 20 0 2283820 71756 60872 S 0.0 2.0 1:40.69 maliit-server 22221 phablet 20 0 32.5g 60192 57196 S 21.7 1.7 0:23.63 signal-desktop 21580 phablet 20 0 277844 52512 50644 S 2.6 1.5 0:21.82 Xwayland 5103 phablet 20 0 2132348 40060 40060 T 0.0 1.1 1:53.34 lomiri-system-s 2551 1047 20 0 11.6g 14280 1848 S 0.0 0.4 7:02.53 camerahalserver 4858 phablet 20 0 2666196 8568 4996 S 0.3 0.2 0:31.43 lomiri-push-ser 2148 root 20 0 1214576 7852 5532 S 3.6 0.2 16:55.07 lomiri-system-c 1832 root 20 0 364824 6392 2300 S 0.0 0.2 1:43.98 upowerd 1654 root 20 0 2286604 6372 0 S 0.0 0.2 0:58.48 snapd 1 root 20 0 24248 6084 2104 S 0.0 0.2 0:57.18 systemd 3241 phablet 20 0 1175228 5704 3172 S 0.0 0.2 0:52.98 media-hub-serve 22745 phablet 20 0 178464 5524 4212 S 0.0 0.2 0:00.08 mtp-server 1777 polkitd 20 0 386452 5184 2404 S 0.0 0.1 0:54.95 polkitd 712 root 19 -1 83872 4988 4060 S 0.0 0.1 0:52.84 systemd-journal 2465 phablet 20 0 22468 4752 2156 S 0.0 0.1 0:16.53 systemd 1576 message+ 20 0 12588 4696 1608 S 0.3 0.1 5:39.31 dbus-daemon 22738 phablet 20 0 326744 4452 3840 S 0.0 0.1 0:00.27 adbd 22946 phablet 20 0 57096 4436 2280 R 1.6 0.1 0:02.28 top 4685 phablet 20 0 465292 4360 1556 S 0.0 0.1 0:03.44 libertined 22201 phablet 20 0 32.4g 4056 3468 S 0.0 0.1 0:00.34 signal-desktop 4805 phablet 20 0 688412 4040 3228 S 0.0 0.1 0:09.09 lomiri-content-But then if I go to Whatsweb, the Ram of Signal-Desktop decrease, and the ram of Whatsweb or rather QtWebEngineProc starts to increase as I navigate
%Cpu(s): 33.6 us, 8.9 sy, 0.1 ni, 56.2 id, 0.0 wa, 0.0 hi, 1.2 si, 0.0 st MiB Mem : 3498.4 total, 152.5 free, 2967.3 used, 608.5 buff/cache MiB Swap: 1924.1 total, 828.9 free, 1095.2 used. 531.1 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 17888 phablet 20 0 14.4g 480044 70424 R 194.7 13.4 6:20.35 QtWebEngineProc 3415 phablet 20 0 6140780 413956 296784 S 21.4 11.6 33:32.10 lomiri 17763 phablet 20 0 3765040 351616 273648 S 67.8 9.8 2:44.29 qmlscene 22255 phablet 20 0 196.3g 286284 143608 S 0.3 8.0 2:21.52 signal-desktop 22131 phablet 20 0 204.3g 111528 18760 S 0.0 3.1 1:15.51 signal-desktop 22974 phablet 20 0 2640604 94748 49620 T 0.0 2.6 0:05.26 lomiri-clock-ap 5222 phablet 20 0 1919932 86168 47080 T 0.0 2.4 0:35.71 lomiri-dialer-a 23243 phablet 20 0 9860.2m 70140 46040 S 0.0 2.0 0:00.82 QtWebEngineProc 4854 phablet 20 0 2283820 67364 56404 S 0.3 1.9 1:40.94 maliit-server 22221 phablet 20 0 32.5g 57860 55796 S 0.0 1.6 1:22.57 signal-desktop 21580 phablet 20 0 277844 53788 51232 S 0.0 1.5 0:27.78 Xwayland 23196 phablet 20 0 917052 48556 40872 S 0.0 1.4 0:00.22 QtWebEngineProc 5103 phablet 20 0 2132348 40060 40060 T 0.0 1.1 1:53.34 lomiri-system-s 1654 root 20 0 2286604 8428 1628 S 0.0 0.2 0:58.64 snapd 2551 1047 20 0 11.6g 8152 1168 S 0.0 0.2 7:02.53 camerahalserver 712 root 19 -1 83872 7700 6760 S 0.0 0.2 0:53.07 systemd-journal 1 root 20 0 24248 7500 3492 S 0.0 0.2 0:57.44 systemd 2148 root 20 0 1214576 6656 4568 S 16.8 0.2 17:11.24 lomiri-system-c 1832 root 20 0 364824 6372 2432 S 0.0 0.2 1:44.27 upowerd 2465 phablet 20 0 22468 5344 2740 S 0.0 0.1 0:16.70 systemd 22201 phablet 20 0 32.4g 5320 4700 S 0.0 0.1 0:00.40 signal-desktop 3241 phablet 20 0 1175228 4824 2144 S 0.0 0.1 0:52.98 media-hub-serve 23134 phablet 20 0 244000 4688 3368 S 0.0 0.1 0:00.06 mtp-server 1576 message+ 20 0 12588 4508 1348 S 0.0 0.1 5:39.96 dbus-daemon 23141 root 20 0 1155836 4452 2264 S 0.0 0.1 0:00.05 adbd-pam-sessio 1777 polkitd 20 0 386452 4116 1348 S 0.0 0.1 0:55.05 polkitd 4685 phablet 20 0 465292 4052 1248 S 0.0 0.1 0:03.44 libertined 23159 phablet 20 0 57096 4028 1876 R 2.0 0.1 0:02.68 topIs all this normal?
-
Ok I think I've found an action that make Ram/Swap disappear magically: import a picture from camera in contentHUB: take a picture and import it directly in contentHub. I've made it various times it a row, until the swap reached 0, and the phone crashed. Here is the last top I got (it was running in background, until it crashed at the same time as the phone interface).
Tasks: 633 total, 1 running, 580 sleeping, 2 stopped, 50 zombie %Cpu(s): 8.8 us, 22.7 sy, 0.0 ni, 32.2 id, 35.1 wa, 0.0 hi, 1.2 si, 0.0 st MiB Mem : 3498.4 total, 359.3 free, 2822.9 used, 250.4 buff/cache MiB Swap: 1924.1 total, 3.1 free, 1921.0 used. 675.5 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 3415 phablet 20 0 6163912 373664 279220 S 6.8 10.4 34:50.99 lomiri 17763 phablet 20 0 4100244 290944 173152 S 9.0 8.1 4:20.49 qmlscene 17888 phablet 20 0 14.4g 160896 9724 S 9.3 4.5 9:20.91 QtWebEngineProc 24419 phablet 20 0 1867740 83372 49932 S 12.2 2.3 0:07.12 lomiri-camera-a 22255 phablet 20 0 196.3g 68524 24476 S 2.9 1.9 4:10.62 signal-desktop 4854 phablet 20 0 2283820 50280 48904 S 0.0 1.4 1:42.24 maliit-server 5103 phablet 20 0 2132348 40060 40060 T 0.0 1.1 1:53.34 lomiri-system-s 22974 phablet 20 0 2640604 39232 39232 T 0.0 1.1 0:05.26 lomiri-clock-ap 24521 phablet 20 0 9734284 33260 0 S 5.5 0.9 0:00.94 QtWebEngineProc 22221 phablet 20 0 32.5g 24884 23172 S 0.0 0.7 2:05.98 signal-desktop 21580 phablet 20 0 278376 20920 20100 S 0.0 0.6 0:35.24 Xwayland 2551 1047 20 0 11.8g 17060 2724 S 21.5 0.5 8:33.31 camerahalserver 22131 phablet 20 0 204.3g 10172 0 S 0.6 0.3 1:36.64 signal-desktop 3241 phablet 20 0 1888500 3672 88 S 0.0 0.1 0:53.64 media-hub-serve 4857 phablet 20 0 405916 3272 2396 S 1.0 0.1 0:18.51 lomiri-indicato 4858 phablet 20 0 2666196 2948 0 S 0.3 0.1 0:34.58 lomiri-push-ser 2817 phablet 20 0 12140 2752 0 S 0.0 0.1 0:27.39 dbus-daemon 23320 phablet 20 0 57096 2584 1340 R 2.3 0.1 0:12.16 top 1576 message+ 20 0 12588 2040 0 S 0.3 0.1 5:42.26 dbus-daemon 2148 root 20 0 1214388 1948 88 S 3.2 0.1 17:47.24 lomiri-system-c 2807 phablet 9 -11 3893872 1660 84 S 0.0 0.0 3:10.13 pulseaudio 1832 root 20 0 364824 1616 0 S 0.3 0.0 1:45.96 upowerd 4805 phablet 20 0 688412 1528 0 S 0.0 0.0 0:10.97 lomiri-content- 2465 phablet 20 0 22468 1516 56 S 0.0 0.0 0:17.06 systemd 1 root 20 0 23992 1500 0 S 0.3 0.0 0:58.76 systemd 2368 system 20 0 10.5g 1496 96 S 0.3 0.0 0:09.51 mtkpower@1.0-se 1107 root 20 0 10.5g 1196 532 S 1.9 0.0 0:43.22 init 2520 1013 20 0 11.0g 952 132 S 4.5 0.0 0:44.70 minimediaservicAt reboot with the same apps opened I have a lot more "free" swap:
MiB Mem : 3498.4 total, 149.3 free, 2738.0 used, 889.3 buff/cache MiB Swap: 1924.1 total, 1833.6 free, 90.5 used. 760.4 avail Mem -
@pparent said in VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed:
196.4g
oh, the Ram eater !
Holmes, the game is afoot
-
@gpatel-fr said in VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed:
oh, the Ram eater !
Holmes, the game is afootHolmes would say that Signal-Desktop sure is an easy culprit but the easy culprit rarely is the real culprit!
In this case Signal has an alibi: I've just been able to reproduce the bug, without starting signal-desktop since boot.
It can be done more or less reproducibility by playing with the camera, and importing pictures to Whatsweb directly from the camera app in contentHub a bunch of times, and recording videos.
But also a new clue arrives in the investigation, I've realized that actually if I give the system a lot of time, like 5 - 15 minutes, without rebooting it, it seems to end up recovering (at least sometime, I will see if it is reproductible), and after the recovery free swap recovers to more or less normal levels in top ( 1268 ) .
-
@pparent said in VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed:
It can be done more or less reproducibility by playing with the camera, and importing pictures to Whatsweb directly from the camera app in contentHub
could you play a little more with 'top' using 'f' and 's' to sort by VIRT or RES to see what is growing exactly, the default sort (Cpu) not being of much use here. If signal-desktop is not the culprit in this case, it may be whatsweb for example.
Note that camerahalserver (that I don't seem to have on my FP5) is part of the Android container so it's not very instructive by itself: on my FP5 all the Android processes have also high level of VIRT (10-12Gb) without causing any swap use.
-
@gpatel-fr said in VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed:
could you play a little more with 'top' using 'f' and 's' to sort by VIRT or RES to see what is growing exactly, the default sort (Cpu) not being of much use here. If signal-desktop is not the culprit in this case, it may be whatsweb for example.
If you look at the last top I posted I sorted with %MEM which is equivalent than sorted by RES. And there is seemingly no process that eats too much RAM. I've never seen in a top any processes consuming outrageous amount of RAM.
I've also done a df -h , to see if a ramfs could use too much space but I've seen nothing.
As you tell me that "the swap handling is not the 'classical' sort, it's the more advanced type of 'compressed Ram'", I wonder if the swap itself could be buggy?
-
@pparent said in VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed:
I've never seen in a top any processes consuming outrageous amount of RAM.
That's very strange. I had kept a copy of top when the dialer had crashed my phone and it was clear that there was a culprit:
top - 21:30:13 up 12:42, 2 users, load average: 34,25, 47,73, 35,64 Tasks: 618 total, 16 running, 601 sleeping, 0 stopped, 1 zombie %Cpu(s): 0,5 us, 92,4 sy, 0,0 ni, 1,3 id, 1,7 wa, 2,3 hi, 1,8 si, 0,0 st MiB Mem : 7294,9 total, 62,8 free, 7216,6 used, 64,6 buff/cache MiB Swap: 4096,0 total, 0,0 free, 4096,0 used. 78,3 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4265 phablet 20 0 396220 1340 0 R 60,9 0,0 3:03.44 mission-control 130 root 20 0 0 0 0 R 44,0 0,0 6:10.27 kswapd-1:0 3334 root 20 0 0 0 0 R 38,5 0,0 6:10.26 kswapd0:1 4396 phablet 20 0 4640104 29844 17684 R 34,6 0,4 9:03.45 lomiri 2055 root 20 0 17868 616 0 R 34,3 0,0 1:35.34 wpa_supplicant 4212 phablet 20 0 2043428 1404 0 S 34,3 0,0 3:20.83 nuntium 4888 phablet 20 0 407480 1544 0 R 34,3 0,0 2:04.60 lomiri-indicato 1 root 20 0 25684 3884 0 R 33,0 0,1 2:36.44 systemd 1399 root 20 0 28280 804 0 R 32,4 0,0 1:09.57 systemd-udevd 4914 phablet 20 0 1869152 3080 0 S 31,9 0,0 2:53.33 ciborium 23599 phablet 20 0 9934948 5,0g 1936 R 30,5 70,4 3:16.95 lomiri-dialer-a 4429 phablet 20 0 957680 612 0 R 29,1 0,0 0:48.45 evolution-addre 24185 root 20 0 155816 29852 84 R 28,8 0,4 0:46.14 lomiri-system-cAlso do you see kswapd taking a lot of Cpu like in my top ?
@pparent said in VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed:
the swap itself could be buggy?
It's probably the same software and setup for all ports since it's not linked to Android.
-
@gpatel-fr said in VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed:
Also do you see kswapd taking a lot of Cpu like in my top ?
No but I did not pay attention either, I will next time.
It's probably the same software and setup for all ports since it's not linked to Android.
But I would be very curious to know if the action that I mentioned previously is reproductible for other VP22 users and non-VP22 users: importing a picture with contentHub directly from camera app to Whatsweb (probably works with another app than whatsweb), does reduce progressively the amount of available swap, when done several times, without any clear reason.
Because on my side it does it reproductibly, it's probably not the only action that triggers that, and it kind of explain why I end up without swap.
-
@pparent said in VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed:
importing a picture with contentHub directly from camera app to Whatsweb (probably works with another app than whatsweb), does reduce progressively the amount of available swap, when done several times,
I tried to repro on my FP5 without success, but of course there are many differences: other phone, more Ram (I have 8 Gb hence a swap of 4Gb) , I don't use Whatsapp or Signal so all the apps available to share in contentHub are files and messages. After a dozen of shared photos, zero swap, available memory in top goes down by one or two hundred of megabytes, then back up.
-
Have you tried to lower the swappiness of the system? Maybe this could help to slow down the use of the swap-partition. On my phones (and Laptops) i set the swappiness to 10:
sudo sysctl vm.swappiness=10If that works for you, you can make this permanent via a config-file
BQ Aquaris E4.5 (UT 16.04)
Vollaphone (UT 20.04)
Vollaphone 22 (UT 24.04) -
@gpatel-fr said in VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed:
I tried to repro on my FP5 without success, but of course there are many differences: other phone, more Ram (I have 8 Gb hence a swap of 4Gb) , I don't use Whatsapp or Signal so all the apps available to share in contentHub are files and messages. After a dozen of shared photos, zero swap, available memory in top goes down by one or two hundred of megabytes, then back up.
Thank's a lot but I not sure to see how you can do it with files apps (or maybe there are import features that I don't know about, but I really don't see where).
Just to make sure I was clear with messages it would be:
1-Open messages app
2-Go to a conversation and click the attachment icon, then image
3-In contentHUb select camera
4-Take your picture and validateI will to some more tests tonight including swapiness.
-
@kristatos said in VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed:
sudo sysctl vm.swappiness=10
It does not seem to have clear impact on swap loss when importing from camera. I will see if the crash still happens in normal use, with this option in the coming days.
-
thanks for the explanation, it's indeed faster than the more convoluted process I was following. So I did about 6 messages each having 5 photos, lomiri-messaging memory footprint increased steadily by about 100 K for each message (that I never actually sent, it was a dummy number). When I closed lomiri-messaging all this memory was reclaimed by the system, and if I open it again I don't get back the high memory use even if I open the same contact.
So there may be a leak but it hardly seems a blocker on my phone - that is, my current use of the phone is that as soon as I have finished using an app, I close it, and each evening I shutdown the phone for the night. You may use your device very differently of course.
