Backup and restore using ubports-installer

kugiigi · 4 Dec 2019, 07:42

Does this work while the phone is booted normally or in recovery/bootloader mode?
Looks cool, has it been merged already?

AlainW94 · 4 Dec 2019, 18:03

@kugiigi It will work only on recovery, because it need to read or write system data. It is not merged yet, I will do it asap.

kugiigi · 5 Dec 2019, 05:42

@AlainW94 It writes? may I know for what?
Anyway, I just thought I can use it for backing up/restoring my MultiROM UT installs
Good luck with this

AppLee · 5 Dec 2019, 12:49

@kugiigi Backup AND RESTORE so yes, write

AlainW94 · 5 Dec 2019, 17:36

@kugiigi Yes it write if you restore a backup ! The backup tool will backup the content of 2 folders : /data/user-data and /data/system-data.
If you open a shell on the recovery, and type

ls /data

And if you can see in there the previouly named folders, then the backup tool will be able to backup this file on your multirom device.

kugiigi · 5 Dec 2019, 17:52

Of course, restoring will write data....what a stupid question from me
I'll check this functionality once it's merged in the app.
Thanks.

Lakotaubp · 5 Dec 2019, 18:34

@AlainW94 Like this ?
SWAP.img android-data system-data system.img ubuntu.img user-data

AlainW94 · 5 Dec 2019, 22:55

@Lakotaubp Yes perfect ! what device is this ?

Lakotaubp · 6 Dec 2019, 05:14

@AlainW94 OnePlus one

Lakotaubp · 6 Dec 2019, 13:29

SWAP.img      lost+found    system.img    user-data
android-data  system-data   ubuntu.img

Bq M10fhd

Lakotaubp · 6 Dec 2019, 16:06

Nexus 7

SWAP.img        audio           data            nfc             system-data
adb             backup          drm             property        system.img
android-data    bootchart       local           qcks            tombstones
app             bugreports      lost+found      resource-cache  ubuntu.img
app-asec        cam_socket1     media           security        user
app-lib         cam_socket2     mediadrm        ss              user-data
app-private     dalvik-cache    misc            system

AlainW94 · 6 Dec 2019, 17:12

@Lakotaubp Many thanks ! it seems that the backup tool will work for at least those devices (Nexus 7, One Plus, Bq M10, Meizu Pro 5)

Lakotaubp · 6 Dec 2019, 17:37

@AlainW94 Nexus 5

~ # ls /data
SWAP.img      android-data  system-data   system.img    ubuntu.img    user-data
~ #

Tried MX4 twice but got device off line message also OP3 no connection but guess that is to be expected at present.

matteo · 7 Dec 2019, 08:57

@AlainW94 that is a very awesome functionality for the UT phone! Thank you for your work Alain! I also own a Pro5 so looking forward to use such a useful utility. I feel the day my UT phone will become my daily driver again is always closer and closer, thank to committed Community people as you are

flohack · 8 Dec 2019, 09:30

Hello, some answers:

the 2 folders /data/user-data and /data/system-data are available on all devices and are the only ones you need to back up. They do not need mounting, but are real folders. You need to be careful with the symlinks though, as they are invalid if the normal OS is not active. So you might get backup errors during this process
The feature is great and much anticipated. Just want to know if the target could be either PC or sd card. Some users might want to have the backup on the internal SD card (and later on the UT recovery could utilize this also to backup/restore, just would need baking it into the code there)
Please get in contact with @NeoTheThird if you didnt do it already, he is the maintainer for the installer and you should coordinate any big features like this with him, to get aligned with his plans for the installer.

Thanks and happy coding!

NeoTheThird · 8 Dec 2019, 10:15

Hey @AlainW94,

maintainer of the installer here

Kudos on how far you've come already, that's really cool! This is very similar to what we had planned in this issue. The commands there should work pretty much on every device with enough storage. Could you maybe open a PR soon, so we can coordinate the code to not conflict with the other things we have planned in the installer?

Some notes:

You might want to avoid streaming output of adb commands to tar directly to the disk of your machine, but rather tar on the device. Even though it'll take up some space on the device, it'll be worth the added stability. Adb is very unreliable when it comes to exit codes and what buffer to write to. You don't want it to silently write "waiting for devices" to your precious backup file and corrupting it without you knowing.
You'll need to implement adb pull in the promise-android-tools module. It'd be great if you could open a pr for that rather sooner than later.
This is not a small undertaking, so in try to aim to get your changes in early wherever possible to avoid merge conflicts going forward. You might for example want to get your pull animation in now already.
Avoid device-specific hacks and workarounds in the installer code. You'll have heard about the config file format introduced in 0.3.x-beta, which aims to pave the way to make the installer code device- and os-agnostic. I could imagine adding another field like actions[], which holds steps like this, to the entry in operating_systems. We might even want to introduce some braking changes to the file strucutre to do this really right.

Hope i did not overwhelm you, just excited for community contributions to my pet project XD

If you need any help, hints or pointers, please let me know.

Thank you!

AlainW94 · 8 Dec 2019, 16:05

Hi @Flohack: My reply to your answers :

Please try on a Meizu Pro5, if you have the same recovery.img than me, you will see that thoses folders aren't mounted by default, the /data folder is empty. But ok for the other devices.

2.Thanks ! For now I only write the code to backup on the computer and restore from the computer. Again because I only have a Pro5 as ubuntu phone and while in recovery, the sd card isn't mounted. But this can be done in the code for other devices.

Ok perfect, thanks for the contact ! I'll do that.

thanks to you for your hint @Flohack

AlainW94 · 8 Dec 2019, 16:20

Hello @NeoTheThird ! Thanks for being in the game now !
I would like to open a PR but I have some difficulties to do so, can you give me some hint please ?

1- I have to admit it, that's a good point, for now I didn't have any issue while doing the backup like this (almost 8 times) but you right, we don't know what could happens during the backup or restore operation (for example with a defective usb cable).

2- yes, this is how I did, I tried as much as possible to respect your software architecture.

3 - Yes of course, but need some help witch github, I cannot create a new branch to commit my changes.

4 - Oh, I didn't know about that config file, I will take a look, thanks!

I really want to help the community but not by doing wrong things so any hint are welcome.

Thanks to you !

flohack · 8 Dec 2019, 17:37

@AlainW94 for working with our repos you typically just fork the project, then push your branch you want to submit and create a pull request from that branch to our base. Thats kinda "workaround" for not being allowed write access to our repos.

flohack · 8 Dec 2019, 17:38

@AlainW94 regarding the unmounted data partition, how could the installer work on that device then @NeoTheThird ? Do we mount data specifically? I dont have a Pro 5, in fact there are not so many people who own one.