Backup and restore using ubports-installer

Lakotaubp

Nexus 7

SWAP.img        audio           data            nfc             system-data
adb             backup          drm             property        system.img
android-data    bootchart       local           qcks            tombstones
app             bugreports      lost+found      resource-cache  ubuntu.img
app-asec        cam_socket1     media           security        user
app-lib         cam_socket2     mediadrm        ss              user-data
app-private     dalvik-cache    misc            system

AlainW94

@Lakotaubp Many thanks ! it seems that the backup tool will work for at least those devices (Nexus 7, One Plus, Bq M10, Meizu Pro 5)

Lakotaubp

@AlainW94 Nexus 5

~ # ls /data
SWAP.img      android-data  system-data   system.img    ubuntu.img    user-data
~ #

Tried MX4 twice but got device off line message also OP3 no connection but guess that is to be expected at present.

matteo

@AlainW94 that is a very awesome functionality for the UT phone! Thank you for your work Alain! I also own a Pro5 so looking forward to use such a useful utility. I feel the day my UT phone will become my daily driver again is always closer and closer, thank to committed Community people as you are

flohack

Hello, some answers:

the 2 folders /data/user-data and /data/system-data are available on all devices and are the only ones you need to back up. They do not need mounting, but are real folders. You need to be careful with the symlinks though, as they are invalid if the normal OS is not active. So you might get backup errors during this process
The feature is great and much anticipated. Just want to know if the target could be either PC or sd card. Some users might want to have the backup on the internal SD card (and later on the UT recovery could utilize this also to backup/restore, just would need baking it into the code there)
Please get in contact with @NeoTheThird if you didnt do it already, he is the maintainer for the installer and you should coordinate any big features like this with him, to get aligned with his plans for the installer.

Thanks and happy coding!

NeoTheThird

Hey @AlainW94,

maintainer of the installer here

Kudos on how far you've come already, that's really cool! This is very similar to what we had planned in this issue. The commands there should work pretty much on every device with enough storage. Could you maybe open a PR soon, so we can coordinate the code to not conflict with the other things we have planned in the installer?

Some notes:

You might want to avoid streaming output of adb commands to tar directly to the disk of your machine, but rather tar on the device. Even though it'll take up some space on the device, it'll be worth the added stability. Adb is very unreliable when it comes to exit codes and what buffer to write to. You don't want it to silently write "waiting for devices" to your precious backup file and corrupting it without you knowing.
You'll need to implement adb pull in the promise-android-tools module. It'd be great if you could open a pr for that rather sooner than later.
This is not a small undertaking, so in try to aim to get your changes in early wherever possible to avoid merge conflicts going forward. You might for example want to get your pull animation in now already.
Avoid device-specific hacks and workarounds in the installer code. You'll have heard about the config file format introduced in 0.3.x-beta, which aims to pave the way to make the installer code device- and os-agnostic. I could imagine adding another field like actions[], which holds steps like this, to the entry in operating_systems. We might even want to introduce some braking changes to the file strucutre to do this really right.

Hope i did not overwhelm you, just excited for community contributions to my pet project XD

If you need any help, hints or pointers, please let me know.

Thank you!

AlainW94

Hi @Flohack: My reply to your answers :

Please try on a Meizu Pro5, if you have the same recovery.img than me, you will see that thoses folders aren't mounted by default, the /data folder is empty. But ok for the other devices.

2.Thanks ! For now I only write the code to backup on the computer and restore from the computer. Again because I only have a Pro5 as ubuntu phone and while in recovery, the sd card isn't mounted. But this can be done in the code for other devices.

Ok perfect, thanks for the contact ! I'll do that.

thanks to you for your hint @Flohack

AlainW94

Hello @NeoTheThird ! Thanks for being in the game now !
I would like to open a PR but I have some difficulties to do so, can you give me some hint please ?

1- I have to admit it, that's a good point, for now I didn't have any issue while doing the backup like this (almost 8 times) but you right, we don't know what could happens during the backup or restore operation (for example with a defective usb cable).

2- yes, this is how I did, I tried as much as possible to respect your software architecture.

3 - Yes of course, but need some help witch github, I cannot create a new branch to commit my changes.

4 - Oh, I didn't know about that config file, I will take a look, thanks!

I really want to help the community but not by doing wrong things so any hint are welcome.

Thanks to you !

flohack

@AlainW94 for working with our repos you typically just fork the project, then push your branch you want to submit and create a pull request from that branch to our base. Thats kinda "workaround" for not being allowed write access to our repos.

flohack

@AlainW94 regarding the unmounted data partition, how could the installer work on that device then @NeoTheThird ? Do we mount data specifically? I dont have a Pro 5, in fact there are not so many people who own one.

AlainW94

@Flohack Many thanks ! this is what I did after reading on it on google.

AlainW94

@Flohack own 4 of them and can confirm that is it like that on the 4. have we access to the recovery;img for the pro5 ? if yes I can modify it to have the partitions mounted at startup in the recovery.

flohack

@AlainW94 No the recovery is the old one from Canonical times, Pro 5, MX 4 and all BQ devices do not have an open tree and we cannot change anything there...

AlainW94

@Flohack Ok so we will have to deal with it in the installer

NeoTheThird

Awesome, saw you opened a PR here, i'll give you a code review with some pointers. Really appreciate the work you put into this, i'm confident we can get it done fairly soon

flohack

@AlainW94 It should be dealt already with, otherwise how would the installer install anything into /data??

AlainW94

@Flohack In my case, I did a check of the mounted partition in the installer, then if it is not mounted, I mount it. (/dev/block/sda44 -> /data)

flohack

@AlainW94 @NeoTheThird ??

AlainW94

@NeoTheThird Hi! It was a try. Now I need to do the same with promise-android otherwise it will not work

NeoTheThird

Re: Git workflow

Here's something i wrote up for another installer user the other day, maybe that's helpful to you

you'll want to keep your master completely in sync with our master, so when you navigate to it on your machine, the first thing you'll do is git checkout master to make sure you're on your master, then git pull upstream master --rebase to fetch the latest version from our master, git push origin master to push our master to your master on github. Now if you want to implement a new feature, you will create a branch by going git checkout -b nameofbranch. Then you make your changes. If you want to create a commit, you git add path/to/files for all your files you want to commit and then go git commit -m "My Commit Message". As soon as you have all your commits created, go ahead and git push origin nameofbranch to push your branch to your own fork on github. Then go to the webinterface to create the pull request.