Daily Lockups - kswapd & Memory leaks
-
@standupmobile In general you dont want to use any swap at all on a devices backed by flash memory since it will wear out the flash card. So, swap should be as little as possible, and frankly, 4GB must be more than enough for a mobile device.
Memory leaks: I had such supicion but interestingly is this maybe device-dependent. It could be that our truncation of Android services lets a process continually eat through memory. I had same experience on the Nexus 6P, also a Google device. But so far on no other device.
Getting memleaks caught is one of the most challenging problems on a mobile device like a smartphone since every run of the OS will lead to different mem usage patterns.
-
But basically I have the same problem on N6P, resulting in loss of modem connectivity. Mostly happens when I open the camera or other intense operations like starting GPS. Its never a problem after iniital boot, only after a day or 2.
@fredldotme could it be that there is still a process (maybe visible in logcat) that is unhappy and maybe just filling internal memory with error messages. My Nexus 6P for example has one process thats a bit spammy in logcat. -
I've been using a Pixel 3a for the past days and so far I haven't encountered any slow downs. In fact, this js the best experience I have with UT. I can open a lot of apps simultaneously and I haven't seen app closed due to lack of memory. I also don't encounter my problem with crashing webviews. It's a fresh install and I didn't restore any data or app data from other UT device. I only use TELEports, Morph and Pesbuk most of the time. I've been trying out Waydroid in the past days too and even with that, I don't see slow downs. The only thing that made it really unresponsive was Genshing Impact
I'm on devel BTW. -
Thanks @flohack and @kugiigi ...
I use a lot of webapps created with Webber (News sites) ... Quick test shows they jump memory/swap up in increments by 4-8% (just opening them) ... From a fresh boot, opening 5 news webapps, Morph, Telegram and Axotl brought it up to ~ 85% Swap; ... then I switched to closing them (upswipe) to see if swap and memory got freed up, which it did - but not all the way back down. I suspect (and will test) that if I had left them open and come back to them later to keep reading, I would max out and lockup.
None of my apps get closed at any point due to the lack of memory ... The phone just becomes very slow (eg: swipe responds after 30 seconds) ... If I leave it for 5 minutes, It resolves itself (at least for a little while), but I typical reboot to get back to the snappy smooth 3a experience.
I would happily rid it of the swap partition if you felt it would be better ... I gather the installer just set it up that way as a standard.
Let me know if I can test out and report anything to help ID and pin down what might be the cause!
-
Same with N6P leading to hard lock up or it reboots. Have not noticed anything on Pixel2 XL.
-
I have had exactly the same issues for quite some time using OnePlus 3T, even before using Webber app. As it seems it's not limited to a specific device or brand. The 'lock-ups' occur like 3-4 times a day.
I can't figure out what causes it either, not with 100% certainty anyways.
If I would only know how to capture it I would be happy to contribute to get this matter solved.
-
@3t_ed If you enable USB debugging, next time it happens, you can plug into a computer (with adb tools) and run adb shell. This will bring up a terminal: run "Top" then hit "m" twice to see memory and swap % use.
-
One further update:
sudo cat /proc/sys/vm/swappiness returns 100 ...
Is this parameter used in UT? If so, that doesn't make sense.
The default value of vm.swappiness is 60 and represents the percentage of the free memory before activating swap. The lower the value, the less swapping is used and the more memory pages are kept in physical memory.
I set mine to 20 ... and my test of opening a ton of webapps starts using swap much later.... and I haven't hit lockup yet. Not sure how to persist it though...
sudo mount -o remount,rw /
sudo sysctl vm.swappiness=20 -
@standupmobile said in Daily Lockups - kswapd & Memory leaks:
sudo mount -o remount,rw /
sudo sysctl vm.swappiness=20On the Volla the value 20 does not survive a reboot. vm.swappiness returns to its default value of 60.
-
A bit too quick to declare success ... I locked up with 100% swap usage again today while my swappiness was set to 20. Still think it is a beneficial change for the life of the emmc ... however, issue persists.
@domubpkm ... Not sure yet how to persist this across reboots ... going to reach out to porters to see where this setting is established. -
So turns out the swap showing on our Pixel3's is ZRam (Fake swap on memory that is compressed). The 100 setting is appropriate for this setup. Will keep digging around for some way to prevent 100% use ... (which is the source of my lockups).
-
So I built a routine that allowed me to reproduce the lockups (where the swap use shown in top (zram) hits 100%, and memory is 95+% used) - it basically involves opening a bunch of webapps / apps that I use; but all at once in fast sequence. The camera is one app that needs a bunch of memory ... and can easily push the device over the edge).
I then tweaked memory control parameters as per various suggestions found online and repeated the test. I was able to get a lot further before lockup with the settings below. (The phone uses less zram as I progress through my test, I guess ultimately pushing back the time to lockup). I haven't noticed any performance hit (purely subjective), and I can run many days now without lockup. On the one occasion where I did lockup, recovery was faster (rather than 1-3 mins, it was ~10 seconds)l; however I did notice one of my open apps was killed (white page with green triangle saying refresh). Haven't figured out how to set this across boots - although my reboots are a lot less frequent!
Disclaimer: This is for the pixel3a - that has 4GB of RAM and ~ 1.6GB zram setup. Based on my reading, I think some of these settings are very specific towards hardware (memory, cpu, swap vs. zram) on the device... also, I am a hack wrt linux and have NO other experience with these settings:)
sudo sysctl vm.min_free_kbytes=29620 &&
sudo sysctl vm.vfs_cache_pressure=200 &&
sudo sysctl vm.dirty_background_ratio=3 &&
sudo sysctl vm.dirty_ratio=40 &&
sudo sysctl vm.admin_reserve_kbytes=16384 &&
sudo sysctl vm.swappiness=80Hope this helps ...
-
@standupmobile
Is this problem solved? No more freezes? -
@3t_ed yes, at least we found out what happened with apps locking up everything, while in reality there was no low-memory killer active. I enabled it and it's available in devel or even release candidate already.
-
@fredldotme
Great! Thank you for letting me know. That's good news so thank you for sorting out this issue. I'm still on stable channel but I assume we'll notice with the next OTA? -
Thank you @fredldotme !
What are the gitlab or github links that fix the problem(s). I don't seem to see this in OTA-23 project anyway. -
@3t_ed To elaborate a bit further: First, Alfred applied those changes to the port. One OR 2 did not stick properly and rather than chasing down the issue further, he remove zram completely. So the dev version of p3a is no longer using any swap. There are no more lockups, but (IMHO) there is a little less bandwidth for multitasking, especially after a few days of uptime (where presumably software memory leaks have chewed into the avail 4GB of ram. All in all a big improvement ... would be good if some other could test and chime in their thoughts.
-
@standupmobile
Thanks. I followed up on your advice and am now on RC 673 to check if daily lockups have gone and check for lagging in multitasking. Will report back after the weekend. -
The actual fix was this one: https://github.com/fredldotme/android_kernel_google_bonito/commit/944dd258c369ed224b37a9fa439a64c919ad4f27
-
@fredldotme On which phones does this fix apply? The Volla too ? If yes, on the current RC ? Thanks in advance to clarify.
