The MMS lost story

[jezek]

@lduboeuf said in The MMS lost story:

@jezek btw, can you add an empty commit or any piece of code to trigger build on telepathy-ofono and history-service ? They are still marked as failed and thus not accessible with ubports-qa

Done

[lduboeuf]

@jezek Can you sync with xenial branch for those 2 repos, JenkinsFile have been updated and should fix the build issue

[jezek]

@lduboeuf said in The MMS lost story:

@jezek Can you sync with xenial branch for those 2 repos, JenkinsFile have been updated and should fix the build issue

Done

[lduboeuf]

@jezek oups, still messaging-app to be built again...
So i am almost ready to test it
sudo ubports-qa list:

PR_nuntium_8 PR_telepathy-ofono_20 PR_telephony-service_20 PR_history-service_35 PR_messaging-app_260

[lduboeuf]

@jezek so fetch your messaging-app to test. It seems to work well ;-).

Do you differentiate a download Error with a "Cellular-data off" state ?
For the "data-off", it could be a simple message like on other platforms with a "Download" button.

Awesome work

[jezek]

@lduboeuf said in The MMS lost story:

@jezek oups, still messaging-app to be built again...

messaging-app updated

@lduboeuf said in The MMS lost story:

Do you differentiate a download Error with a "Cellular-data off" state ?
For the "data-off", it could be a simple message like on other platforms with a "Download" button.

In the code nuntium returns an error on first connection attempt (there are more stages), but I really don't know if different kind of trouble will cause different error type. I can assume, that if an error happens after first connection attempt, it is not cause of "Celular-data off" state. And I really don't have an idea, how to cause different cause of errors. The plan is to send the error text and/or type to messaging-app via the error message. Maybe then, we will spot the pattern. Meanwhile you can investigate (if you want) and try to cause & get some non "Cellurae-data off" error and then see the log (code producing download errors is here). Maybe, if we can't differentiate by error message, nuntium could ask someone on dbus, or see the config, what the state of cellular switch is and make a note about it in the error message.

Btw, I don't know how are other platforms are handling mms download errors.

[lduboeuf]

@jezek
for download error on other platforms, idk but the current error message is good enough i think.

For the "cellular-data" off case, it is displayed as a normal message
e.g:

But in that case, if cellular data is off, the click on the download should check for data on or prevent user...

In order to know if cellular data is off, you can query via dbus, example with command line:

dbus-send --session --print-reply \
    --dest=com.ubuntu.connectivity1 \
    /com/ubuntu/connectivity1/Private \
    org.freedesktop.DBus.Properties.Get \
string:com.ubuntu.connectivity1.Private \
	string:'MobileDataEnabled'

[jezek]

Failed MMS redownload progress report:

This week only nuntium PR got updated.

I've been working on message initialization. To keep nuntium message store (located in ~/.local/share/nuntium/store) tidy, some stored messages are checked with history-service and if they are marked as read or are deleted in history-service, then there is no need to store them in nuntium storage and will be deleted.

It's not a big change, cause I've been struggling with DBus calls to history-service (and with sending/receiving in general). But after some comments from @lduboeuf (with a dbus example) and reading some documentations and blog I have now better uderstanding of dbus. Thank you.

See you next week with new updates.

PS: If you wan to test, you have to install all PR's for this project using ubports-qa script. I'm trying to push only tested commits by myself and everything should be functioning.

[lduboeuf]

@jezek tests are fine ;-), awesome, still need some UI polishing, but works fine.

I wonder for the "cellular-data" dbus call if an interface marked as "private“ in the namespace is something that can/should be queried in numtium or elsewhere ?

[jezek]

Failed MMS redownload progress report:

This week some work on propagating error details from nuntium to messaging-app was done, 4PR's were updated:

• nuntium - Upon decoding mNotificationInd handle Expiry token (absolute/relative) and fill Expiry field as unix time. Propagate error description, expiry date and message size encoded as json to telepathy-ofono.
• telepathy-ofono - forward the error message details in headers to history service.
• history-service - save the error message status to messageStatus and details (encoded as json) to messageText field in db.
• messaging-app - change error message detection, cause now the message can have a non empty text (with error details).
  Note: The branch can't be compiled, I've tracked the error and created issue. After it's fixed (fix is trivial) I will rebase the branch it should pass the CI checks (I hope, localy with crossbuilder it worked).
  Note: The expiry date could be wrongly encoded, haven't tested on real incoming messages. Some sources say the expiry date is encoded as Size + token + data, some Size + token + size + data. Majority and OpenMediaAssociation(?) documentation said it was the first case (s+t+d), so I've done it that way, but only tests in real world conditions will show. So keep an eye on the Expire field.

@lduboeuf Now is the time to ask for some help. Could you help with the messaging-app's UI part? Error details are saved in text field, encoded as json. Currently there are 3 fields:

• "Description" (string) - just a text with error description.
• "Expire" (string) - time in RFC3339 format, when the message expires at MMS provider.
• "Size" (int) - size in bytes (?) of the message

I'm planing (as you suggested) to add a "CellularDataOn" (bool) field (have a better name) which could indicate if the cellular data was on/off at download time.

If any other error detail, that could be propagated from nuntium, comes to your mind, say it, I will try to add.

[AppLee]

@jezek said in The MMS lost story:

"CellularDataOn" (bool) field (have a better name)

Usually for boolean I use "isMyState" or "hasMyProperty", in this case I would have used hasCellularData or hasMobileData (the first one is my favorite as the toggle switch has the same label in the settings.

[lduboeuf]

@jezek said in The MMS lost story:

Failed MMS redownload progress report:

Note: The branch can't be compiled, I've tracked the error and created issue. After it's fixed (fix is trivial) I will rebase the branch it should pass the CI checks (I hope, localy with crossbuilder it worked).

fixed, you can rebase now

Note: The expiry date could be wrongly encoded, haven't tested on real incoming messages. Some sources say the expiry date is encoded as Size + token + data, some Size + token + size + data. Majority and OpenMediaAssociation(?) documentation said it was the first case (s+t+d), so I've done it that way, but only tests in real world conditions will show. So keep an eye on the Expire field.

the one defined here is not working ?: https://github.com/ubports/nuntium/blob/38940464d532a145412c47d8901c1a74fb662eff/mms/decoder.go#L405

@lduboeuf Now is the time to ask for some help. Could you help with the messaging-app's UI part? Error details are saved in text field, encoded as json. Currently there are 3 fields:

• "Description" (string) - just a text with error description.
• "Expire" (string) - time in RFC3339 format, when the message expires at MMS provider.
• "Size" (int) - size in bytes (?) of the message

Yes for sure , how i can commit to your PR ?

Edit: For the expiry date, i just had a case where Expire field is empty in json

[lduboeuf]

@jezek said in The MMS lost story:

"Description" (string) - just a text with error description.

Humm, where does come from error messages, are they normalized somewhere ?

[jezek]

@lduboeuf said in The MMS lost story:

fixed, you can rebase now

heh, no need to rebase, you push right to my fork's branch, just merged my local branch with origin. Honestly, I didn't do this kind of collaboration until this time. How do you push to another's PR?

@lduboeuf said in The MMS lost story:

the one defined here is not working ?: https://github.com/ubports/nuntium/blob/38940464d532a145412c47d8901c1a74fb662eff/mms/decoder.go#L405

There is a TODO near. The Expiry can come in 2 formats, depending on the token. If token means "absolute" then the following bytes are unix date. If token means "relative" the following bytes are # of seconds. The previous code always handled the Expiry as relative (but it didn't matter, cause it was nowhere used). If we want to use Expiry field, it has to be correctly encoded. So the token has to come in account. I had some doubts, but I recently browsed some other code and pages and now I know I have a bug in decoding absolute token. Will fix.

@lduboeuf said in The MMS lost story:

Edit: For the expiry date, i just had a case where Expire field is empty in json

If I recall correctly, the Expire should be empty only when no Expiry field comes in the mNotifiationInd (message notification) from provider, but could be a bug. Can you prowide nuntium logs for that event please?

[jezek]

@lduboeuf said in The MMS lost story:

@jezek said in The MMS lost story:

"Description" (string) - just a text with error description.

Humm, where does come from error messages, are they normalized somewhere ?

The error is just a string, that comes from error, that is passed to handleMRetrieveConfDownloadError function in nuntium. They are not translated. I don't know what you mean under normalized. Maybe I should send some error type.

[lduboeuf]

@jezek said in The MMS lost story:

Honestly, I didn't do this kind of collaboration until this time. How do you push to another's PR?

If you are the maintainer of a repo ( my case for messaging-app ), you can add commits to the PR since users allow to edit the PR ( there is a checkbox for that when opening a PR). So for that case, if i checkout your repo, the PR branch is editable by the maintainer.

@lduboeuf said in The MMS lost story:

the one defined here is not working ?: https://github.com/ubports/nuntium/blob/38940464d532a145412c47d8901c1a74fb662eff/mms/decoder.go#L405

There is a TODO near. The Expiry can come in 2 formats, depending on the token. If token means "absolute" then the following bytes are unix date. If token means "relative" the following bytes are # of seconds. The previous code always handled the Expiry as relative (but it didn't matter, cause it was nowhere used). If we want to use Expiry field, it has to be correctly encoded. So the token has to come in account. I had some doubts, but I recently browsed some other code and pages and now I know I have a bug in decoding absolute token. Will fix.

@lduboeuf said in The MMS lost story:

Edit: For the expiry date, i just had a case where Expire field is empty in json

Thanks for the explanation

If I recall correctly, the Expire should be empty only when no Expiry field comes in the mNotifiationInd (message notification) from provider, but could be a bug. Can you prowide nuntium logs for that event please?

Sorry, don't have my logs now, but as it was a MMS sent by myself to myself, could it be the issue ?

[lduboeuf]

@jezek said in The MMS lost story:

@lduboeuf said in The MMS lost story:

Humm, where does come from error messages, are they normalized somewhere ?

The error is just a string, that comes from error, that is passed to handleMRetrieveConfDownloadError function in nuntium. They are not translated. I don't know what you mean under normalized. Maybe I should send some error type.

Ok, but yes, it is preferable to have "normalized error" to be able to translate them later. e.g "x-mms-error-no-context", something like that.

[jezek]

@lduboeuf said in The MMS lost story:

Sorry, don't have my logs now, but as it was a MMS sent by myself to myself, could it be the issue ?

Oh, I completely forgot. I can send message to myself. Thanks for making me remember. Will test it that way.

@lduboeuf said in The MMS lost story:

Ok, but yes, it is preferable to have "normalized error" to be able to translate them later. e.g "x-mms-error-no-context", something like that.

Ok, will do it that way.

[lduboeuf]

@jezek said in The MMS lost story:

@lduboeuf said in The MMS lost story:

Sorry, don't have my logs now, but as it was a MMS sent by myself to myself, could it be the issue ?

Oh, I completely forgot. I can send message to myself. Thanks for making me remember. Will test it that way.

Yes sometimes you need to be quick switching cellular off

@lduboeuf said in The MMS lost story:

Ok, but yes, it is preferable to have "normalized error" to be able to translate them later. e.g "x-mms-error-no-context", something like that.

Ok, will do it that way.

Of course we still need to make it translatable on both messaging-app and telephony-service for the notification part.
Can i see somewhere all the possible error messages ?
I should ask also in UI forum for what to do

[jezek]

Failed MMS redownload progress report:

This week nuntium PR only got updates.

I've been working on the download error message reports.
@lduboeuf sorry for late reply about errors, I've been working on it. Here is the result.

The error message text in json now contains following fields:

• Code [string] - contains one of following error code:
  • x-ubports-nuntium-mms-error-unknown - unknown error, should not happen.
  • x-ubports-nuntium-mms-error-activate-context - failed first contact with MMS service & context activation. Redownload is allowed if not expired.
  • x-ubports-nuntium-mms-error-get-proxy - first contact was successful, but getting proxy failed. Very rare, should occur only with bad signal. Redownload is allowed if not expired.
  • "x-ubports-nuntium-mms-error-download-content - context & proxy ok, but download failed. Very rare, should occur only with bad signal or maybe if message expires (needs investigation what happens if trying to redownload expired message). Redownload is allowed if not expired.
  • x-ubports-nuntium-mms-error-storage - the downloaded message file can't be saved to storage. Should happen only if disk full, or bad permissions (almost never). Redownload is NOT allowed (maybe should be?).
  • x-ubports-nuntium-mms-error-forward - everything went ok, but for some unexplained reason, can't forward the message to telepathy-ofono. But if that's the case, I'll be a miracle if the error message can be sent o telepathy-ofono, so should never happen. Redownload is NOT allowed.
• Message [string] - raw error message caught in nuntium (just for debug purposes).
• Expire [string, optional] - date-time in RFC3339 format, when the message expires in MMS service. If field not present, the Expire field was not sent by provider (how do we handle this?). Edit: However, if expiry time not provided by operator, we assume 7 days expiry time, so this field will not be empty ever (unless some bug).
• Size [int, optional] - Size in bytes of message. If not present, size info was no sent by provider or size is 0.
• MobileData [bool, optional] - if mobile data was enabled in the time of error handling. If not present, the mobile data property could not be determined.

Note: The PR checks failed. I found FAIL github.com/ubports/nuntium/mms [build failed] in jenkins logs, but localy the mms package can be build. I think it's because of go version jenkins is using (Setting up golang-1.6-src (1.6.2-0ubuntu5~16.04.4) ...), but I need at least 1.7 (or better 1.13). I don't know how to force required version. Should I rewrite code to be able to compile with 1.6? I'd rather use latest (possible) golang version.
Edit: I tried to change the debian/control to use go-1.13. The package gets installed, but it fails with dh_auto_build: go install -v failed to to execute: No such file or directory
Edit: PR build is successful, thanks to @lduboeuf