Daily Lockups - kswapd & Memory leaks
Wondering if any of you are experiencing the same? On a regular basis (on average once a day) ... my 3a grinds to an unusable state: swipes respond after delays of many seconds or even minutes. I usually just reboot rather than wait the 5 to 10 minutes or so it might take to resolve itself. Once when it happened, I had ssh running and was able to see loads of 45 in top. wa was really high ... kswapd was at the top of the list of processes. I recall seeing something about qtwebengine memory leak ... which would build up with my Morph use and its webapps...
Are you other daily users seeing the same?
Same here on OnePlus 3T
Have tried several options suggested like using Webber unfortunately to no avail.
I don't have a Pixel 3a but in my experience, apps and especially webviews gets killed easily after some time. I don't get lockups though. System remains responsive and smooth.
Keneda last edited by Keneda
I don't have this on MX4 if it is relevant to you.
I only use morph, no webbapp, and close it most time when finished using it, but even when i don't it never happens, although sometime morph hangs, with last OTA i believe, but when i have this problem, i only restart the app and it's fine.
... so this is still happening for me (dev). Seems I'm using all swap space and kswapd is consuming a ton of resources. If I list memory usage with
ps -o pid,user,%mem,command ax | sort -b -k3 -r
... a lot of QT-WebEngine processes lead the usage ... so I will try this and see if anything improves:
sudo ubports-qa install PR_qtwebengine_5
The QA update did not solve the problem. I backed out from Dev to RC ... hoping maybe OTA 19 might have fixed something, but it does not. I confirmed that the lockup/slowdown happens once SWAP hits 100% usage (and obviously, since we are using swap, memory use is already very high around ~95%). Shouldn't the system be freeing up swap/memory before it hits 100%?
Could we change the "swappiness" parameter ... or adjust the amount of SWAP (on my pixel 3a) SWAP is set to 1.57GB ... about 40% of system memory. I remember rule of thumb on servers back in the day is to give them the same amount of swap as they have memory. Could this be the issue?
Flohack last edited by
@standupmobile In general you dont want to use any swap at all on a devices backed by flash memory since it will wear out the flash card. So, swap should be as little as possible, and frankly, 4GB must be more than enough for a mobile device.
Memory leaks: I had such supicion but interestingly is this maybe device-dependent. It could be that our truncation of Android services lets a process continually eat through memory. I had same experience on the Nexus 6P, also a Google device. But so far on no other device.
Getting memleaks caught is one of the most challenging problems on a mobile device like a smartphone since every run of the OS will lead to different mem usage patterns.
Flohack last edited by
But basically I have the same problem on N6P, resulting in loss of modem connectivity. Mostly happens when I open the camera or other intense operations like starting GPS. Its never a problem after iniital boot, only after a day or 2.
@fredldotme could it be that there is still a process (maybe visible in logcat) that is unhappy and maybe just filling internal memory with error messages. My Nexus 6P for example has one process thats a bit spammy in logcat.
I've been using a Pixel 3a for the past days and so far I haven't encountered any slow downs. In fact, this js the best experience I have with UT. I can open a lot of apps simultaneously and I haven't seen app closed due to lack of memory. I also don't encounter my problem with crashing webviews. It's a fresh install and I didn't restore any data or app data from other UT device. I only use TELEports, Morph and Pesbuk most of the time. I've been trying out Waydroid in the past days too and even with that, I don't see slow downs. The only thing that made it really unresponsive was Genshing Impact I'm on devel BTW.
I use a lot of webapps created with Webber (News sites) ... Quick test shows they jump memory/swap up in increments by 4-8% (just opening them) ... From a fresh boot, opening 5 news webapps, Morph, Telegram and Axotl brought it up to ~ 85% Swap; ... then I switched to closing them (upswipe) to see if swap and memory got freed up, which it did - but not all the way back down. I suspect (and will test) that if I had left them open and come back to them later to keep reading, I would max out and lockup.
None of my apps get closed at any point due to the lack of memory ... The phone just becomes very slow (eg: swipe responds after 30 seconds) ... If I leave it for 5 minutes, It resolves itself (at least for a little while), but I typical reboot to get back to the snappy smooth 3a experience.
I would happily rid it of the swap partition if you felt it would be better ... I gather the installer just set it up that way as a standard.
Let me know if I can test out and report anything to help ID and pin down what might be the cause!
Same with N6P leading to hard lock up or it reboots. Have not noticed anything on Pixel2 XL.
I have had exactly the same issues for quite some time using OnePlus 3T, even before using Webber app. As it seems it's not limited to a specific device or brand. The 'lock-ups' occur like 3-4 times a day.
I can't figure out what causes it either, not with 100% certainty anyways.
If I would only know how to capture it I would be happy to contribute to get this matter solved.
@3t_ed If you enable USB debugging, next time it happens, you can plug into a computer (with adb tools) and run adb shell. This will bring up a terminal: run "Top" then hit "m" twice to see memory and swap % use.
One further update:
sudo cat /proc/sys/vm/swappiness returns 100 ...
Is this parameter used in UT? If so, that doesn't make sense.
The default value of vm.swappiness is 60 and represents the percentage of the free memory before activating swap. The lower the value, the less swapping is used and the more memory pages are kept in physical memory.
I set mine to 20 ... and my test of opening a ton of webapps starts using swap much later.... and I haven't hit lockup yet. Not sure how to persist it though...
sudo mount -o remount,rw /
sudo sysctl vm.swappiness=20
A bit too quick to declare success ... I locked up with 100% swap usage again today while my swappiness was set to 20. Still think it is a beneficial change for the life of the emmc ... however, issue persists.
@domubpkm ... Not sure yet how to persist this across reboots ... going to reach out to porters to see where this setting is established.
So turns out the swap showing on our Pixel3's is ZRam (Fake swap on memory that is compressed). The 100 setting is appropriate for this setup. Will keep digging around for some way to prevent 100% use ... (which is the source of my lockups).
So I built a routine that allowed me to reproduce the lockups (where the swap use shown in top (zram) hits 100%, and memory is 95+% used) - it basically involves opening a bunch of webapps / apps that I use; but all at once in fast sequence. The camera is one app that needs a bunch of memory ... and can easily push the device over the edge).
I then tweaked memory control parameters as per various suggestions found online and repeated the test. I was able to get a lot further before lockup with the settings below. (The phone uses less zram as I progress through my test, I guess ultimately pushing back the time to lockup). I haven't noticed any performance hit (purely subjective), and I can run many days now without lockup. On the one occasion where I did lockup, recovery was faster (rather than 1-3 mins, it was ~10 seconds)l; however I did notice one of my open apps was killed (white page with green triangle saying refresh). Haven't figured out how to set this across boots - although my reboots are a lot less frequent!
Disclaimer: This is for the pixel3a - that has 4GB of RAM and ~ 1.6GB zram setup. Based on my reading, I think some of these settings are very specific towards hardware (memory, cpu, swap vs. zram) on the device... also, I am a hack wrt linux and have NO other experience with these settings:)
sudo sysctl vm.min_free_kbytes=29620 &&
sudo sysctl vm.vfs_cache_pressure=200 &&
sudo sysctl vm.dirty_background_ratio=3 &&
sudo sysctl vm.dirty_ratio=40 &&
sudo sysctl vm.admin_reserve_kbytes=16384 &&
sudo sysctl vm.swappiness=80
Hope this helps ...