VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed

pparent

@gpatel-fr said in VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed:

oh, the Ram eater !
Holmes, the game is afoot

Holmes would say that Signal-Desktop sure is an easy culprit but the easy culprit rarely is the real culprit!

In this case Signal has an alibi: I've just been able to reproduce the bug, without starting signal-desktop since boot.

It can be done more or less reproducibility by playing with the camera, and importing pictures to Whatsweb directly from the camera app in contentHub a bunch of times, and recording videos.

But also a new clue arrives in the investigation, I've realized that actually if I give the system a lot of time, like 5 - 15 minutes, without rebooting it, it seems to end up recovering (at least sometime, I will see if it is reproductible), and after the recovery free swap recovers to more or less normal levels in top ( 1268 ) .

gpatel-fr

@pparent said in VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed:

It can be done more or less reproducibility by playing with the camera, and importing pictures to Whatsweb directly from the camera app in contentHub

could you play a little more with 'top' using 'f' and 's' to sort by VIRT or RES to see what is growing exactly, the default sort (Cpu) not being of much use here. If signal-desktop is not the culprit in this case, it may be whatsweb for example.

Note that camerahalserver (that I don't seem to have on my FP5) is part of the Android container so it's not very instructive by itself: on my FP5 all the Android processes have also high level of VIRT (10-12Gb) without causing any swap use.

pparent

@gpatel-fr said in VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed:

could you play a little more with 'top' using 'f' and 's' to sort by VIRT or RES to see what is growing exactly, the default sort (Cpu) not being of much use here. If signal-desktop is not the culprit in this case, it may be whatsweb for example.

If you look at the last top I posted I sorted with %MEM which is equivalent than sorted by RES. And there is seemingly no process that eats too much RAM. I've never seen in a top any processes consuming outrageous amount of RAM.

I've also done a df -h , to see if a ramfs could use too much space but I've seen nothing.

As you tell me that "the swap handling is not the 'classical' sort, it's the more advanced type of 'compressed Ram'", I wonder if the swap itself could be buggy?

gpatel-fr

@pparent said in VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed:

I've never seen in a top any processes consuming outrageous amount of RAM.

That's very strange. I had kept a copy of top when the dialer had crashed my phone and it was clear that there was a culprit:

top - 21:30:13 up 12:42,  2 users,  load average: 34,25, 47,73, 35,64
Tasks: 618 total,  16 running, 601 sleeping,   0 stopped,   1 zombie
%Cpu(s):  0,5 us, 92,4 sy,  0,0 ni,  1,3 id,  1,7 wa,  2,3 hi,  1,8 si,  0,0 st
MiB Mem :   7294,9 total,     62,8 free,   7216,6 used,     64,6 buff/cache
MiB Swap:   4096,0 total,      0,0 free,   4096,0 used.     78,3 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
   4265 phablet   20   0  396220   1340      0 R  60,9   0,0   3:03.44 mission-control
    130 root      20   0       0      0      0 R  44,0   0,0   6:10.27 kswapd-1:0
   3334 root      20   0       0      0      0 R  38,5   0,0   6:10.26 kswapd0:1
   4396 phablet   20   0 4640104  29844  17684 R  34,6   0,4   9:03.45 lomiri
   2055 root      20   0   17868    616      0 R  34,3   0,0   1:35.34 wpa_supplicant
   4212 phablet   20   0 2043428   1404      0 S  34,3   0,0   3:20.83 nuntium
   4888 phablet   20   0  407480   1544      0 R  34,3   0,0   2:04.60 lomiri-indicato
      1 root      20   0   25684   3884      0 R  33,0   0,1   2:36.44 systemd
   1399 root      20   0   28280    804      0 R  32,4   0,0   1:09.57 systemd-udevd
   4914 phablet   20   0 1869152   3080      0 S  31,9   0,0   2:53.33 ciborium
  23599 phablet   20   0 9934948   5,0g   1936 R  30,5  70,4   3:16.95 lomiri-dialer-a
   4429 phablet   20   0  957680    612      0 R  29,1   0,0   0:48.45 evolution-addre
  24185 root      20   0  155816  29852     84 R  28,8   0,4   0:46.14 lomiri-system-c

Also do you see kswapd taking a lot of Cpu like in my top ?

@pparent said in VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed:

the swap itself could be buggy?

It's probably the same software and setup for all ports since it's not linked to Android.

pparent

@gpatel-fr said in VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed:

Also do you see kswapd taking a lot of Cpu like in my top ?

No but I did not pay attention either, I will next time.

It's probably the same software and setup for all ports since it's not linked to Android.

But I would be very curious to know if the action that I mentioned previously is reproductible for other VP22 users and non-VP22 users: importing a picture with contentHub directly from camera app to Whatsweb (probably works with another app than whatsweb), does reduce progressively the amount of available swap, when done several times, without any clear reason.

Because on my side it does it reproductibly, it's probably not the only action that triggers that, and it kind of explain why I end up without swap.

gpatel-fr

@pparent said in VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed:

importing a picture with contentHub directly from camera app to Whatsweb (probably works with another app than whatsweb), does reduce progressively the amount of available swap, when done several times,

I tried to repro on my FP5 without success, but of course there are many differences: other phone, more Ram (I have 8 Gb hence a swap of 4Gb) , I don't use Whatsapp or Signal so all the apps available to share in contentHub are files and messages. After a dozen of shared photos, zero swap, available memory in top goes down by one or two hundred of megabytes, then back up.

kristatos

Have you tried to lower the swappiness of the system? Maybe this could help to slow down the use of the swap-partition. On my phones (and Laptops) i set the swappiness to 10:

sudo sysctl vm.swappiness=10

If that works for you, you can make this permanent via a config-file

pparent

@gpatel-fr said in VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed:

I tried to repro on my FP5 without success, but of course there are many differences: other phone, more Ram (I have 8 Gb hence a swap of 4Gb) , I don't use Whatsapp or Signal so all the apps available to share in contentHub are files and messages. After a dozen of shared photos, zero swap, available memory in top goes down by one or two hundred of megabytes, then back up.

Thank's a lot but I not sure to see how you can do it with files apps (or maybe there are import features that I don't know about, but I really don't see where).

Just to make sure I was clear with messages it would be:

1-Open messages app
2-Go to a conversation and click the attachment icon, then image
3-In contentHUb select camera
4-Take your picture and validate

I will to some more tests tonight including swapiness.

pparent

@kristatos said in VP22 Upgrade from 24.04-1.0 to 24.04-1.1 failed:

sudo sysctl vm.swappiness=10

It does not seem to have clear impact on swap loss when importing from camera. I will see if the crash still happens in normal use, with this option in the coming days.

gpatel-fr

@pparent

thanks for the explanation, it's indeed faster than the more convoluted process I was following. So I did about 6 messages each having 5 photos, lomiri-messaging memory footprint increased steadily by about 100 K for each message (that I never actually sent, it was a dummy number). When I closed lomiri-messaging all this memory was reclaimed by the system, and if I open it again I don't get back the high memory use even if I open the same contact.

So there may be a leak but it hardly seems a blocker on my phone - that is, my current use of the phone is that as soon as I have finished using an app, I close it, and each evening I shutdown the phone for the night. You may use your device very differently of course.