The MMS lost story

jezek · 4 Apr 2021, 20:09

Failed MMS redownload progress report:

Updated PR's:

nutium - Rebased to latest xenial + commits to make recovery of messages with unexpected errors during handling (phone/services crashes) more robust. The nuntium-inject-push tool was updated so it can simulate various errors. I've tried to test all possible error situations that came to my mind, found and fixed some flaws and I hope I didn't introduce any more errors. I will make more tests after I clear the last unsolved TODO (excluding testing & documentation TODOs) which was added by me.
messaging-app - Rebased to latest xenial.
telephony-service - Rebased to latest xenial (jekins still can't builf for armhf due to some test error which I think/hope is not my fault).

jezek · 11 Apr 2021, 19:24

Failed MMS redownload progress report:

During cleanup I found a small issue. During nuntium initialization I start dbus listeners to listen to redownload or read/delete requests for messages in storage, but telepathy didn't spawn listeners for these. It's not a big issue, redownload still works, but read/delete requests from messaging-app don't come through to nuntium and that leads to polluted storage. I've tried to fix it, but today I broke something, so no new commits until I fix and test. I hope this is the last issue, that needs to be solved before reviews starts.

jezek · 18 Apr 2021, 19:39

Failed MMS redownload progress report:

I've fixed the issue I found last week, but haven't tested it fully yet (so no commits). I will continue the cleanup, to start review process and inform you here on any progress.

jezek · 25 Apr 2021, 20:04

Failed MMS redownload progress report:

This week only nuntium PR got updates. I've tested message handling and initialization and fixed some bugs I found. Then cleaned the code of my own TODO comments and debug logs. The documentation got updated to more reflect current state and some info that may be helpful was added.

The last thing (I hope) to do in nuntium PR is to update debian changelog, which brings me to a question about versioning.

The last version in changelog is 1.4+ubports1. This suggests, that the project has adopted semantic versioning system, which suggest to bump up major version if API backward compatibility is broken, bump up minor version if some new API is added, or bump up patch version otherwise. I've broken the API backward compatibility in the PR, so we should change the version to 2.0. But (there is always a but) I plan to make additional work on the nuntium project and there is a possibility, the API will change again. Therefore I suggest one of these:

Change version to 2.0 and if API changes in future bump to 3.0, etc.
Change version to 2.0, with some suffix that would suggest, that the API may change in near time (something like 2.0.0-alpha+ubportsX).
Rewrite changes that we can bump only the minor version to 1.5 (there is no way to not add to the API and preserve the minor version). This would take the most effort and even more to guarantee the same behavior for all API functions. I really don't want to walk down this path.
Don't bother with versioning, leave it as it is, nobody uses our nuntium packages.

I would vote for no. 2.

Note: I'm talking here about the nuntium go packages API (storage, telepathy, mms, ofono), The nuntium dbus API is backward compatible.

Side question: Where do you define the version?

Note: I'm really don't know who to ask these questions, so I add some names here, to attract attention. @lduboeuf @dobey @Flohack

flohack · 25 Apr 2021, 20:43

@jezek So if I understand it right ubports1 was added since we are/were not the upstream for that project. Technically speaking you normally would not do such changes on the fork, and ubports1 can only be incremented to ubports2, whenever we add cosmetic patches etc.

But since we are probably now the upstream already, as Canonical dropped everything, we could remove the ubportsxy suffix and just bump to 2.0 as you explained. But please ask the developers in the Developer Telegram group, I am not heading the transition to 20.04 and some new rules might be in effect already.

lduboeuf · 26 Apr 2021, 06:56

@jezek
I would start making the PR ready for review before the versioning question, The maintainer should know what to do there when it will be merged

dobey · 26 Apr 2021, 15:22

@jezek said in The MMS lost story:

The last thing (I hope) to do in nuntium PR is to update debian changelog, which brings me to a question about versioning.

I would say, don't worry about this for now. Our CI automatically includes timestamps and git revision when building, so when people are proposing changes to one of our repos, they do not need to mess with the debian changelog.

jezek · 26 Apr 2021, 15:42

@dobey said in The MMS lost story:

I would say, don't worry about this for now. Our CI automatically includes timestamps and git revision when building, so when people are proposing changes to one of our repos, they do not need to mess with the debian changelog.

And what about the semantic versioning?

@lduboeuf said in The MMS lost story:

I would start making the PR ready for review before the versioning question, The maintainer should know what to do there when it will be merged.

Yop, will do, after the accompanying PRs are review ready. Or I don't need to wait? I bet the nuntium review will demand some commit squashing and some explaining. I changed/added a lot of code compared to the other PRs.

dobey · 26 Apr 2021, 16:19

@jezek said in The MMS lost story:

And what about the semantic versioning?

If there's issues with versioning, in that respect, we'll deal with them when migrating the repos to GitLab for supporting 20.04 and having upstream releases (version tags in the repo, so distros can package from them).

Yop, will do, after the accompanying PRs are review ready. Or I don't need to wait? I bet the nuntium review will demand some commit squashing and some explaining. I changed/added a lot of code compared to the other PRs.

If there are other PRs in other repos also needed for this, it would be best to propose them all at the same time, so you can link in the description to the other needed PRs, and in what order they need to merge/build. If you have some messy commit history, it is generally best to clean up commit history before hand, indeed.

jezek · 2 May 2021, 19:47

Failed MMS redownload progress report:

This week folowing PRs got update:

telepathy-ofono - pushed changes I forgot to push last week , clean code from my debug logs.
history-service - clean code from my debug logs

Right now I'm writing some tests for telephony-service code I introduced for error message redownload. It's fun, cause I don't know anything about writing tests using qt/c++ and I'm learning new stuff while struggling.

jezek · 9 May 2021, 20:34

Failed MMS redownload progress report:

Ok, I admit defeat in the telephony-service testing battle. I've written some code, to test the redownload function, but I couldn't make the tests run. The crossbuilder don't run test. I've read, that I have to run crossbuilder shell to access the container and run tests, but I failed to do so. In the container, I've tried dh_auto_test command, which hangs. I've tried to run a test executable, but it said, that the executable can't be executed. As last resort I copied the project to my phone and tried to run the tests executables there. But I think, the tests need to be run in some special way, cause I didn't manage to run them, so they will pass. I don't know how long will it take me to figure out the test, so I've decided to create an issue and return to them later.

With the tests skipping, everything is ready for review. These are the PRs for this project:

mardy · 5 Oct 2021, 18:15

Hi Jezek,

Ok, I admit defeat in the telephony-service testing battle. I've written some code, to test the redownload function, but I couldn't make the tests run. The crossbuilder don't run test.

as a matter of fact, crossbuilder always runs the tests when not cross-building. In other words, if the target architecture is the same as the host architecture, tests will be run. I tried now with telephony-service in my PC and tests are run (and pass), in the xenial branch.

Just run

crossbuilder --architecture=amd64

on your branch and you should see the tests being executed.

jezek · 10 May 2021, 21:35

@mardy Thank you, will try. One defeat doesn't mean, I (we) can't win at the end.

lduboeuf · 5 Jun 2021, 21:40

Hi, as @UniSuperBox said on last Q&A , the feature is ready for testing:

You need to be on devel or RC ( since 2021/06/09 )
From a terminal, run that commands one by one:

sudo ubports-qa install PR_nuntium_8
sudo ubports-qa install telepathy-ofono 20
sudo ubports-qa install history-service 35
sudo ubports-qa install telephony-service 20
sudo ubports-qa install messaging-app 260

And reboot

Test Plan example:
Make sure it works as before, send/receive MMS, multiple ones
Disable cellular-data, receive a MMS and check if MMS notification is here with the download button
Change proxy settings in APN or put wrong settings and check for notification reception

domubpkm · 6 Jun 2021, 07:30

@lduboeuf Technically, there is nothing to prevent from doing it on the most recent RC if you don't want to switch (again) to dev ?

lduboeuf · 6 Jun 2021, 11:11

@domubpkm it should be good on RC too. Last builds are from one week or two.

lduboeuf · 12 Jun 2021, 08:08

@jezek

Did some test this morning, i have a use case where the message is staying in the "pending state" ( so we can't click redownload button anymore ) even after reboot. It is with bad signal and by putting wrong info in MMS settings
In history db the message is x-ubports-nuntium-mms-error-activate-context but the status is still in Pending state

nunitum logs:
2021/06/11 09:15:26 Trying to set Active property to true for context on true /ril_0/context2
2021/06/11 09:15:26 Cannot set Activate to true (try 1/3) interface on /ril_0/context2: org.ofono.Error.NotAttached: GPRS is not attached
2021/06/11 09:15:32 Failed to activate for /ril_0/context2 : failed to activate context
2021/06/11 09:15:32 Cannot activate ofono context: no context available to activate

jezek · 12 Jun 2021, 14:48

@lduboeuf It seems you found a bug. You should fill an issue. I have some idea, where the problem is and I will elaborate under the issue. This is a veeery long thread and I don't want to make it any longer. We should close it.

Is the bug a big bug? Can we deploy to stable with this bug, or we need to fix it before deploying? Because there is a bigger issue here. I don't have time now to do any work on ubuntu touch due to personal and technical issues. And it will not change (at least) this month.

lduboeuf · 12 Jun 2021, 15:53

@jezek I don't think that issue is a blocker as it may come in rare situations and moreover will end with the same situation than it is today ( just ask user to send it again). I wll look if i can work around that on messaging app.

Yes agree for this thread becoming too long. I would add it in a nuntium PR instead of an issue since it is not merge yet.

Hope you will recover soon :). Btw i bet this will not land on OTA-18. Not enough feedback from users and big code base

[Edit]: not possible to work around it on messaging-app

rocket2nfinity · 26 Jun 2021, 19:23

Not sure if this is still needed, but latest ubports email said to test.

Fixes MMS perfectly on Nexus 6P