The MMS lost story

lduboeuf

@lduboeuf said in The MMS lost story:

Sorry, don't have my logs now, but as it was a MMS sent by myself to myself, could it be the issue ?

Oh, I completely forgot. I can send message to myself. Thanks for making me remember. Will test it that way.

Yes sometimes you need to be quick switching cellular off

@lduboeuf said in The MMS lost story:

Ok, but yes, it is preferable to have "normalized error" to be able to translate them later. e.g "x-mms-error-no-context", something like that.

Ok, will do it that way.

Of course we still need to make it translatable on both messaging-app and telephony-service for the notification part.
Can i see somewhere all the possible error messages ?
I should ask also in UI forum for what to do

jezek

Failed MMS redownload progress report:

This week nuntium PR only got updates.

I've been working on the download error message reports.
@lduboeuf sorry for late reply about errors, I've been working on it. Here is the result.

The error message text in json now contains following fields:

Code [string] - contains one of following error code:
- x-ubports-nuntium-mms-error-unknown - unknown error, should not happen.
- x-ubports-nuntium-mms-error-activate-context - failed first contact with MMS service & context activation. Redownload is allowed if not expired.
- x-ubports-nuntium-mms-error-get-proxy - first contact was successful, but getting proxy failed. Very rare, should occur only with bad signal. Redownload is allowed if not expired.
- "x-ubports-nuntium-mms-error-download-content - context & proxy ok, but download failed. Very rare, should occur only with bad signal or maybe if message expires (needs investigation what happens if trying to redownload expired message). Redownload is allowed if not expired.
- x-ubports-nuntium-mms-error-storage - the downloaded message file can't be saved to storage. Should happen only if disk full, or bad permissions (almost never). Redownload is NOT allowed (maybe should be?).
- x-ubports-nuntium-mms-error-forward - everything went ok, but for some unexplained reason, can't forward the message to telepathy-ofono. But if that's the case, I'll be a miracle if the error message can be sent o telepathy-ofono, so should never happen. Redownload is NOT allowed.
Message [string] - raw error message caught in nuntium (just for debug purposes).
Expire [string, optional] - date-time in RFC3339 format, when the message expires in MMS service. If field not present, the Expire field was not sent by provider ~~(how do we handle this?)~~. Edit: However, if expiry time not provided by operator, we assume 7 days expiry time, so this field will not be empty ever (unless some bug).
Size [int, optional] - Size in bytes of message. If not present, size info was no sent by provider or size is 0.
MobileData [bool, optional] - if mobile data was enabled in the time of error handling. If not present, the mobile data property could not be determined.

Note: The PR checks failed. I found FAIL github.com/ubports/nuntium/mms [build failed] in jenkins logs, but localy the mms package can be build. I think it's because of go version jenkins is using (Setting up golang-1.6-src (1.6.2-0ubuntu5~16.04.4) ...), but I need at least 1.7 (or better 1.13). I don't know how to force required version. Should I rewrite code to be able to compile with 1.6? I'd rather use latest (possible) golang version.
Edit: I tried to change the debian/control to use go-1.13. The package gets installed, but it fails with dh_auto_build: go install -v failed to to execute: No such file or directory
Edit: PR build is successful, thanks to @lduboeuf

lduboeuf

@jezek said in The MMS lost story:

Failed MMS redownload progress report:

This week nuntium PR only got updates.

I've been working on the download error message reports.
@lduboeuf sorry for late reply about errors, I've been working on it. Here is the result.

The error message text in json now contains following fields:

Code [string] - contains one of following error code:

x-ubports-nuntium-error-unknown - unknown error, should not happen.

x-ubports-nuntium-mms-activate-context-error - failed first contact with MMS service & context activation. Redownload is allowed if not expired.

x-ubports-nuntium-mms-get-proxy-error - first contact was successful, but getting proxy failed. Very rare, should occur only with bad signal. Redownload is allowed if not expired.

"x-ubports-nuntium-mms-download-content-error - context & proxy ok, but download failed. Very rare, should occur only with bad signal or maybe if message expires (needs investigation what happens if trying to redownload expired message). Redownload is allowed if not expired.

x-ubports-nuntium-mms-storage-error - the downloaded message file can't be saved to storage. Should happen only if disk full, or bad permissions (almost never). Redownload is NOT allowed (maybe should be?).

x-ubports-nuntium-mms-forward-error - everything went ok, but for some unexplained reason, can't forward the message to telepathy-ofono. But if that's the case, I'll be a miracle if the error message can be sent o telepathy-ofono, so should never happen. Redownload is NOT allowed.

if possible, i would suggest having the same prefix for all error status, x-ubports-nuntium-mms-error-...

Message [string] - raw error message caught in nuntium (just for debug purposes).

Expire [string, optional] - date-time in RFC3339 format, when the message expires in MMS service. If field not present, the Expire field was not sent by provider (how do we handle this?).

not present is ok for me, don't see what is your question.

Size [int, optional] - Size in bytes of message. If not present, size info was no sent by provider or size is 0.

MobileData [bool, optional] - if mobile data was enabled in the time of error handling. If not present, the mobile data property could not be determined.

Note: The PR checks failed. I found FAIL github.com/ubports/nuntium/mms [build failed] in jenkins logs, but localy the mms package can be build. I think it's because of go version jenkins is using (Setting up golang-1.6-src (1.6.2-0ubuntu5~16.04.4) ...), but I need at least 1.7 (or better 1.13). I don't know how to force required version. Should I rewrite code to be able to compile with 1.6? I'd rather use latest (possible) golang version.
Edit: I tried to change the debian/control to use go-1.13. The package gets installed, but it fails with dh_auto_build: go install -v failed to to execute: No such file or directory

~~humm, changes in debian control file is not enough ?~~
[Edit]: Did some tries with debian control, so even by setting the PATH to /usr/lib/go-1.13/bin in rules (export PATH := /usr/lib/go-1.13/bin:$(PATH) ), i had no success, it didn't find go files ( but no more "no such files or directory" ), though. But i'm really a noob in that debian packaging thing.

[EDIT2]:
So finally got it working... but really this is just some copy/paste from everywhere....
For the last part, see https://bugs.launchpad.net/ubuntu/+source/dh-golang/+bug/1911911

in control->

...
golang-1.13-go,
...

in rules ->

...
export DH_GOLANG_INSTALL_ALL := 1
#https://bugs.launchpad.net/ubuntu/+source/dh-golang/+bug/1911911
export GOCACHE := ${CURDIR}/_build/go-build
export GO111MODULE = off
export GOPATH := $(CURDIR)/obj-$(DEB_BUILD_GNU_TYPE)/src/$(DH_GOPKG)
export PATH := /usr/lib/go-1.13/bin:$(PATH)

jezek

@lduboeuf said in The MMS lost story:

if possible, i would suggest having the same prefix for all error status, x-ubports-nuntium-mms-error-...

ok, will fix & notice you (hopefully today).

not present is ok for me, don't see what is your question.

if there is no expire information from operator, can we expect the message never expires, or expires after some default time (eg. 7days)?

[Edit]: Did some tries with debian control, so even by setting the PATH to /usr/lib/go-1.13/bin in rules (export PATH := /usr/lib/go-1.13/bin:$(PATH) ), i had no success, it didn't find go files ( but no more "no such files or directory" ), though. But i'm really a noob in that debian packaging thing.

I'm also virgin in the debian packaging thing. For me it is read internet + trial/error cycles for now. Will experiment. But our approach (with the path) could be a good one. If the go install command runs, but didn't find go files, the setting GOROOT and/or GOPATH may help. Will try (also can you ). If we can't solve it today, tomorrow I will comment out the incriminating code (it's just some testing thing), to be able to compile at least.
Edit: The moment I posted this, you edited your comment wit solution to the packaging problem. Thank you. (I've read the same bug on launchpad, but you were quicker by implementing )

lduboeuf

if there is no expire information from operator, can we expect the message never expires, or expires after some default time (eg. 7days)?

Yes, lets say 7 days

I'm also virgin in the debian packaging thing. For me it is read internet + trial/error cycles for now. Will experiment. But our approach (with the path) could be a good one. If the go install command runs, but didn't find go files, the setting GOROOT and/or GOPATH may help. Will try (also can you ). If we can't solve it today, tomorrow I will comment out the incriminating code (it's just some testing thing), to be able to compile at least.

Yes, see above, i've edited my comment and finally got it built...

jezek

@lduboeuf I've pushed commits to set default expiry time (7days), unify download error code and changes you suggested to be able to build a dep package (thank you again). I've updated the previous post to reflect current state.

lduboeuf

@jezek said in The MMS lost story:

@lduboeuf I've pushed commits to set default expiry time (7days),

sorry, need confirmation, but i see 15 days seems the default duration

unify download error code

also maybe @UniSuperBox @Flohack or someone from core teams can have a look on them

and changes you suggested to be able to build a dep package (thank you again).

glad it helped , thanks also for keeping that alive

lduboeuf

@jezek Is there anything still to be done in backend side ?

jezek

@lduboeuf said in The MMS lost story:

@jezek Is there anything still to be done in backend side ?

Yes, I have a bunch TODOs in nuntium (and other apps along the way to messaging-app), but the error message part will stay the same. Yesterday I trimmed some TODOs (loccaly, not pushed yet) and looked at what has to be done and how it affects the current error message format (and error codes) and I've come to conclusion, that it will be not affected (unless you want to change something).

Why do you ask?

lduboeuf

@jezek ok,nice. Was just asking if i need to do the UI part now
BTW we don't need to have all errors message handling at first.
And if you see there is too much work remaining, we can cut some parts to be able to deliver a solution quicker.

This is so awesome that we can redownload afterward without asking the user

jezek

@lduboeuf said in The MMS lost story:

@jezek ok,nice. Was just asking if i need to do the UI part now

Yes, you can start (if you want).

BTW we don't need to have all errors message handling at first.

That, I leave for you, sir.

And if you see there is too much work remaining, we can cut some parts to be able to deliver a solution quicker.

I have a plan to move some TODOs to github issue tracker to speed up release, but some need to be done (in my opinion).

This is so awesome that we can redownload afterward without asking the user

Yes, that is awesome. And in my opinion it should be switchable (on/off). Eg.: I have a prepaid card and I wouldn't like it if it would automatically download some MMS and use some credit.

jezek

Failed MMS redownload progress report:

This week nuntium PR only got these updates:

Some forgoten and obsolete TODO comments got deleted
MNotificationInd.Expiry type and decoding got some rewrites and more tests
Expired incoming undownloaded messages handling (deleting from storage & notifying telepathy-ofono of deletion) upon nuntium start (resp. modem initialization)

I will continue to solve my own TODOs in the stack, see you next week.

lduboeuf

@jezek So on my side, did some tweaks for the UI part.
Now message when cellular-data is off is displayed like that:

For other cases, no changes

I wonder if we can simplify on telephony service for the notification part the message to "New MMS message received" instead of the "Oops...." and let user discover any errors within the messaging app. It will be easier to manage has we will not have to parse the msg, and check for different cases there.

One thing more, but will need to add some logic, in history service, can we take the received date (timestamp) from the previous message before deletion so that the MMS will not appear at the bottom of the list but as a replacement of the error message ?

domubpkm

Hello.
Just a little sidebar: can you tell me if MMS currently works for Android 9 devices in RC 8 or dev ?
Specifically for me, MMS don't work in RC 8 for Volla (cellular data ok, internet ok, sms ok, APN mms configured).

lduboeuf

@domubpkm idk, isn't that related ?: https://github.com/ubports/ubuntu-touch/issues/1657

Please move that question to support or issues on ubuntu touch or Volla directly ?

domubpkm

@lduboeuf ok. Maybe.
I signaled and asked.
Thank you

jezek

@lduboeuf said in The MMS lost story:

So on my side, did some tweaks for the UI part...

Thanks, will test.

I wonder if we can simplify on telephony service for the notification part the message to "New MMS message received" instead of the "Oops...." and let user discover any errors within the messaging app. It will be easier to manage has we will not have to parse the msg, and check for different cases there.

That's a nice idea, I'm not against. I will change the text.

One thing more, but will need to add some logic, in history service, can we take the received date (timestamp) from the previous message before deletion so that the MMS will not appear at the bottom of the list but as a replacement of the error message ?

Nice idea too, will implement.

jezek

Failed MMS redownload progress report:

This week these PR's got updated:

nuntium - send Received (uint - unix time) message parameter to telepathy-ofono, to ensure right place in message history. And Error (bool) and ErrorMessage (string) parameter got merged into one Error (string).
telepathy-ofono - on added message signal, fill message-received header with Received parameter (if found). Check and fill error message text from Error parameter.
history-service - no changes needed.
messaging-app - no changes, just rebased to xenial which changed upstream.
telephony-service - Just the MMS error text got truncated. Note: The PR wont build only for arm64 due to test fail. Seems not my fault, but needs to be fixed.

That's all this week. This projects is nearing it's end. If nothing goes wrong I'd say 2 weeks (without . documentation, tests and changelogs changes) and we are ready to test on dev, not only through ubports-qa.

Have a nice day.

jezek

Failed MMS redownload progress report:

This week I got rid of some TODOs in nuntium PR only.

@lduboeuf Can I ask you one more thing? Could you check (in messaging-app) if an error message is expired and if yes, don't allow to redownolad (hide the button and maybe change text). I'ts because I plan to delete and not listen to expired undownloaded messages in nuntium. And if the message is not listening in nuntium, then after clicking redownload in messaging-app, the message would be pending forever (until user deletes it). Thank you.

lduboeuf

@jezek Thanks for your weekly review of progress. I'm waiting it impatiently every Sunday evening

Good point for the expiry date, it needs to be done, will do it ASAP