Stuck early in the setup process: am I missing a key piece of the puzzle?

PhAndersson

Hello,
I've received the new phone and I'm trying to follow @magnargj suggested procedure to set it up (as per this post: https://forums.ubports.com/topic/6187/ut-installation-failure/5).

The phone currently runs Android 8.1.0. I've enabled dev mode, USB debugging and allowed OEM unlock. I then restarted the phone in "Fastboot" mode and was able to successfully unlock the bootloader:

$ fastboot oem unlock

('unlock_critical', as was suggested elsewhere, was not recognized)
This reverted the phone to factory settings, so I reconfigured it and re-enabled dev mode and USB debug, and now each time it boots, the phone dutifully warns me that the bootloader is unlocked (there is also an "unlocked" status at the bottom of the splash screen).

I'm now trying to boot on the TWRP recovery image, but that doesn't work. The "fastboot devices" command does list the phone:

$ fastboot devices -l
dbXXXX                 fastboot usb:1-1

But attempting boot yields the following error:

$ fastboot boot ./twrp-3.5.2_9-0-jasmine_sprout.img
Downloading 'boot.img'
FAILED (remote: unknown command)
Finished. Total time: 0.110s

I've also tried using fastboot to query various phone-side variables (as this should be a pretty basic feature), but all variables I tried (version, version-bootlader, serialno, product, secure, is-userspace) return either "unknown command" or "variable not found"). Just to make sure, I've also tried this with different USB cables.

So I feel I'm still missing a key piece of the puzzle.

An older (2014) page about ADB and fastboot utilities claimed that 'fastboot' could do little unless the Android phone was rooted. It also said that on Nexus devices, the bootloader unlock also rooted the phone. So I installed the so-called "Root Checker" app on the Xiaomi to query its status, and it confirmed the phone is currently NOT rooted.

Is rooting the phone necessary to proceed with the Android upgrade and OTA-19 installation?
Is there something else I might have overlooked?

TIA
Ph. A.

PhAndersson

@phandersson
Further experimental results. I just watched a tutorial video on how to root the phone (just in case), and it starts with early steps that are very similar to the ones I followed, so I tried those as well:

enable dev mode + USB debug (already done, obviously)
connect phone to PC
type "adb devices" => device is listed:

$ adb devices -l
List of devices attached
dbXXXX                 device usb:1-1 product:jasmine model:Mi_A2 device:jasmine_sprout transport_id:2

type "adb reboot bootloader" to reach fastboot: the phone does reboot and reaches the fastboot splash screen
use fastboot to boot on TWRP image -- this time, the message was different (although it still doesn't work):

$ fastboot boot ./twrp-3.5.2_9-0-jasmine_sprout.img
Downloading 'boot.img'
FAILED (data write failure (Cannot send after transport endpoint shutdown))
Finished. Total time: 0.850s

And indeed, the fastboot environment appears to have crashed: the fastboot splash screen is gone, and the phone displays a short text message: "press any key to shutdown" (pressing power then reboots the phone).

As the message could hint at a data transmission problem, I tried the same procedure with the other USB-C cable, and this time the output for the last step was:

$ fastboot boot ./twrp-3.5.2_9-0-jasmine_sprout.img
Downloading 'boot.img'
FAILED (remote: unknown command)
Finished. Total time: 0.012s

So, similar to the behaviour described in my initial post above. But I'm a bit concerned by this lack of repeatability...

MrT10001

@phandersson You do not need to root the phone. You need to be on Android 9 and not 8 to flash Ubuntu Touch. You have to follow the steps precisely with Xiaomi phones to unlock the bootloader, which involves:
1)having a working SIM card inserted
2)creating a MI account and linking the phone to it
3) setting up developer mode
4) in developer mode setting OEM unlock to on and enabling USB debugging
5)unlock the phone using the Mi unlock tool and not fastboot OEM unlock.

Then be patient and go back into Android and re-enable USB debugging and power the phone off. Then power back on and enter flashboot mode by holding the power on and volume down button. In flashboot mode, flash the latest version of TWRP for that phone and wipe and format the partitions to EXT 4. Reboot to fastboot mode and Ubuntu Touch should install.

PhAndersson

@mrt10001 Hello MrT10001,
Many thanks for your reply. I already read about the Mi-specific unlock and wondered whether it was equivalent or not to the OEM unlock. Seeing as the OEM unlock actually did something, I assumed it was, but now I know better

From what I read, the Xiaomi unlock process involves running a Windows .EXE file to unlock the bootloader -- which would be problematic for me. Do you know whether an equivalent Linux-based approach exists?

Also, will I need to "OEM relock" before I proceed with the Xiaomi-specific approach?

TIA
Ph. A.

MrT10001

@phandersson Check this out.

magnargj

@mrt10001 said in Stuck early in the setup process: am I missing a key piece of the puzzle?:

@phandersson You do not need to root the phone. You need to be on Android 9 and not 8 to flash Ubuntu Touch. You have to follow the steps precisely with Xiaomi phones to unlock the bootloader, which involves:
1)having a working SIM card inserted
2)creating a MI account and linking the phone to it
3) setting up developer mode
4) in developer mode setting OEM unlock to on and enabling USB debugging
5)unlock the phone using the Mi unlock tool and not fastboot OEM unlock.

Then be patient and go back into Android and re-enable USB debugging and power the phone off. Then power back on and enter flashboot mode by holding the power on and volume down button. In flashboot mode, flash the latest version of TWRP for that phone and wipe and format the partitions to EXT 4. Reboot to fastboot mode and Ubuntu Touch should install.

This was not necessary for me (I suppose there might be country specific differences, or versions of the phone ?). I never had the sim card nor was the phone connected to internet via wifi. After managing to get android 9 (image suggested by the installer), the install was staight forward. The challenge here seems to be getting the phone to boot the twrp image, which is puzzling...

magnargj

@phandersson said in Stuck early in the setup process: am I missing a key piece of the puzzle?:

Hello,
I've received the new phone and I'm trying to follow @magnargj suggested procedure to set it up (as per this post: https://forums.ubports.com/topic/6187/ut-installation-failure/5).

The phone currently runs Android 8.1.0. I've enabled dev mode, USB debugging and allowed OEM unlock. I then restarted the phone in "Fastboot" mode and was able to successfully unlock the bootloader:
$ fastboot oem unlock
('unlock_critical', as was suggested elsewhere, was not recognized)
This reverted the phone to factory settings, so I reconfigured it and re-enabled dev mode and USB debug, and now each time it boots, the phone dutifully warns me that the bootloader is unlocked (there is also an "unlocked" status at the bottom of the splash screen).

I'm now trying to boot on the TWRP recovery image, but that doesn't work. The "fastboot devices" command does list the phone:
$ fastboot devices -l
dbXXXX                 fastboot usb:1-1
But attempting boot yields the following error:
$ fastboot boot ./twrp-3.5.2_9-0-jasmine_sprout.img
Downloading 'boot.img'
FAILED (remote: unknown command)
Finished. Total time: 0.110s
I've also tried using fastboot to query various phone-side variables (as this should be a pretty basic feature), but all variables I tried (version, version-bootlader, serialno, product, secure, is-userspace) return either "unknown command" or "variable not found"). Just to make sure, I've also tried this with different USB cables.

So I feel I'm still missing a key piece of the puzzle.

An older (2014) page about ADB and fastboot utilities claimed that 'fastboot' could do little unless the Android phone was rooted. It also said that on Nexus devices, the bootloader unlock also rooted the phone. So I installed the so-called "Root Checker" app on the Xiaomi to query its status, and it confirmed the phone is currently NOT rooted.

Is rooting the phone necessary to proceed with the Android upgrade and OTA-19 installation?
Is there something else I might have overlooked?

TIA
Ph. A.

Just to be sure, you are in fastboot mode (volume down + power) when you try to boot with the twrp image file, right?

PhAndersson

@magnargj Yes, that's correct. Either 'volume down' + 'power' or "adb reboot bootloader". Same result in both cases.

Following the suggestion of a nice troubleshooting page about the "unknown command" issue, I also replaced the 'android-tools' RPM I was using (I'm on OpenSUSE Leap 15.2) by the official Google SDK ZIP file.

After doing this, the "fastboot boot twrp.img" command issued a different message ("sending twrp.img...") and just hung.

I'm also confused about this Xiaomi-specific unlock process. I found their FAQ (here: https://c.mi.com/thread-2262302-1-0.html) and in there they say: "For Android One users, please check out the unlocking tutorial in your device subforum. You don't have to go through the procedures mentioned here." They also say (further down): "How to check whether the bootloader is locked/unlocked? [...] An "Unlocked" word will be shown when booting." which is already the case on my phone.

messayisto

@phandersson It looks like the phone is already unlocked (because of the unlooked message on startup).

TWPR is not absolutely necessary, at least not for me: https://forums.ubports.com/topic/6349/installation-succes-at-the-first-try/7

Maybe the only important thing is to upgrade to android 9 before you try again.

PhAndersson

@messayisto said in Stuck early in the setup process: am I missing a key piece of the puzzle?:

@phandersson It looks like the phone is already unlocked (because of the unlooked message on startup).

TWPR is not absolutely necessary, at least not for me: https://forums.ubports.com/topic/6349/installation-succes-at-the-first-try/7

Maybe the only important thing is to upgrade to android 9 before you try again.

Hello @messayisto. Yes, I agree with you that the TWRP boot issue may not be an immediate blocker, but I'm still concerned that it highlights an underlying problem that will bite me later on. Call me finicky, but I always prefer to build on strong foundations

In parallel, I'll dig and/or enquire on the Xiaomi forums to try and sort out the current status of my bootloader. I'll report back when I know more.

PhAndersson

@mrt10001 said in Stuck early in the setup process: am I missing a key piece of the puzzle?:

@phandersson Check this out.

Excellent! Many thanks. I'll certainly give it a try if it turns out to be needed. This could be a life saver!

PhAndersson

@phandersson said in Stuck early in the setup process: am I missing a key piece of the puzzle?:

In parallel, I'll dig and/or enquire on the Xiaomi forums to try and sort out the current status of my bootloader. I'll report back when I know more.

Well, it didn't take long
[Tips] [ifelixit] Unlocking the Bootloader of Mi A2/Lite (Feb. 2020)
https://c.mi.com/thread-2853583-1-0.html?t=1583843866434

Turns out that on the MI A2 and Mi A2 Lite, 'fastboot oem unlock' IS the official procedure to unlock the bootloader. Great! So at least that step looks sorted out.

PhAndersson

@phandersson OK, yet more troubleshooting results.

1./ getvar issue
I read somewhere that it was also possible to query 'getvar all', so I tried that and it worked. Here is the (slightly doctored) output, in case it reveals something:

$ ./fastboot getvar all
(bootloader) crc:1
(bootloader) anti:2
(bootloader) token:[removed]
(bootloader) unlocked:yes
(bootloader) off-mode-charge:0
(bootloader) charger-screen-enabled:0
(bootloader) battery-soc-ok:yes
(bootloader) battery-voltage:4337
(bootloader) version-baseband:
(bootloader) version-bootloader:
(bootloader) variant:SDM EMMC
(bootloader) partition-type:cache:ext4
(bootloader) partition-size:cache: 0x10000000
(bootloader) partition-type:userdata:ext4
(bootloader) partition-size:userdata: 0xBB77F7E00
(bootloader) partition-type:system_a:ext4
(bootloader) partition-size:system_a: 0xC0000000
(bootloader) has-slot:modem:yes
(bootloader) has-slot:system:yes
(bootloader) current-slot:a
(bootloader) has-slot:boot:yes
(bootloader) slot-retry-count:b:7
(bootloader) slot-unbootable:b:yes
(bootloader) slot-successful:b:no
(bootloader) slot-retry-count:a:6
(bootloader) slot-unbootable:a:no
(bootloader) slot-successful:a:yes
(bootloader) slot-count:2
(bootloader) secure:yes
(bootloader) serialno:dbXXXX
(bootloader) product:jasmine
(bootloader) max-download-size:536870912
(bootloader) kernel:uefi
all: 
Finished. Total time: 0.004s

What is really weird is that this command only works once per session. If I re-execute it, I get the "unknown variable" error. If I try to query specifically for any of the defined variables, I stlll get either "unknown command" or "unknown variable".

2./ USB devices
Going through the 'dmesg' output, I noticed that the phone identified itself as two different USB devices:

idVendor=05c6, idProduct=9091 Manufacturer: Xiaomi
idVendor=18d1, idProduct=d00d Manufacturer: Google
(but both with same SerialNumber property).

So I checked inside the '51-android.rules' file installed by the 'android-udev-rules' package, and noticed two problems:

the idVendor 05c6 was assigned to Qualcomm, not Xiaomi
idProduct 9091 (= Mi A2) was not defined at all

I then hacked the file to make sure it matched what was reported by 'dmesg', but it turned out to be in vain (at least as pertains to my current problem), because I later discovered that these 2 devices are in fact never seen by the PC at the same time:

the Qualcomm/Xiaomi device appears when the phone runs Android
the Google device is only seen when the phone is rebooted in fastboot mode

It's also not clear to me that 'adb' and 'fastboot' commands really rely on these udev rules to work, as I was able to interact (at least minimally) with the phone even while running Android (e.g. 'adb reboot bootloader') before I created a rule for the Mi A2.

PhAndersson

@messayisto said in Stuck early in the setup process: am I missing a key piece of the puzzle?:

Maybe the only important thing is to upgrade to android 9 before you try again.

I checked on the phone (still running Android 8.1.0): there is a pending update identified as "November 2018 security update", but it refuses to apply ("Couldn't update / Installation problem"). I've already retried several times applying it, always with the same result. It's not a problem of space as there's no data whatsoever on the phone (50 GB free space), the wireless connection is at full strength and the Internet connection is stable. Anyway, it's not the download phase that fails, but the installation.

I'll try again after removing the 2 small apps I installed for earlier tests, and after leaving dev. mode just to see whether it makes a difference.

PhAndersson

@phandersson Trying to understand where 'fastboot boot twrp.img' gets suck, I ran it through 'strace'. Here below, the relevant parts.

1./ fastboot appears to sequentially probe attached USB devices until it finds one where a specific string is found, reads the device S/N off it, checks that it offers the fastboot interface and if so claims it:

openat(AT_FDCWD, "/dev/bus/usb/001/018", O_RDWR) = 4
read(4, "\22\1\20\2\0\0\0@\321\30\r\320\0\1\1\2\3\1\t\2 \0\1\1\0\200P\t\4\0\0\2"..., 1024) = 50
openat(AT_FDCWD, "/sys/bus/usb/devices/1-1/serial", O_RDONLY) = 5
read(5, "dbXXXX\n", 255)                = 7
close(5)                                = 0
openat(AT_FDCWD, "/sys/bus/usb/devices/1-1/1-1:1.0/interface", O_RDONLY|O_NOFOLLOW|O_CLOEXEC) = 5
fstat(5, {st_mode=S_IFREG|0444, st_size=4096, ...}) = 0
read(5, "fastboot\n", 8192)             = 9
read(5, "", 8192)                       = 0
close(5)                                = 0
ioctl(4, USBDEVFS_CLAIMINTERFACE, 0x7ffc6192745c) = 0

(as mentioned in an earlier post, I don't see any reliance on udev rules here)

2./ fastboot then reads the entire TWRP image into a memory-mapped area (I left those system calls out) and proceeds to send it on the USB device through 'ioctl()' calls:

write(2, "Sending 'boot.img' (31560 KB)   "..., 51Sending 'boot.img' (31560 KB)                      ) = 51
ioctl(4, USBDEVFS_BULK, 0x7ffc61927b80) = 17
ioctl(4, USBDEVFS_BULK, 0x7ffc61927930) = 12
ioctl(4, USBDEVFS_BULK, 0x7ffc61927bc0) = 16384
[...]
ioctl(4, USBDEVFS_BULK, 0x7ffc61927bc0) = 16384
ioctl(4, USBDEVFS_BULK, 0x7ffc61927bc0) = 8192
ioctl(4, USBDEVFS_BULK, 0x7ffc619279b0) = -1 ESHUTDOWN (Cannot send after transport endpoint shutdown)

Notice how two short strings (17 and 12 bytes) are sent before the image.
If I add the bytes for all 16384 + the last 8192 packets, I come up to the exact size of the TWRP image, to the byte.
But then (last line of the above extract), it tries to send yet another string from a different memory address, and that hangs forever. I would need to dig into the fastboot code to know what that last string is, of course.
The ESHUTDOWN error is triggered when I unplug the phone.

PhAndersson

@phandersson said in Stuck early in the setup process: am I missing a key piece of the puzzle?:

I would need to dig into the fastboot code to know what that last string is, of course.

I found the source code yesterday for an older version of 'fastboot'. C++ is definitely not my forte, but the way I understand it, the hanging ioctl() call is not an attempt to write more data, but rather fastboot blocking on a read operation (ioctl() calls can read or write depending on a flag part of the struct passed as 3rd param), trying to retrieve from the phone the status resulting from it receiving the TWRP image.

If I'm right, that would be on this line inside FastBootDriver::HandleResponse() method:

int r = transport_->Read(status, FB_RESPONSE_SZ);

'fastboot' expects a status of 'OKAY', but never gets anything. If it got something else, it would process it and error out with a different error message.

In any case, since the problem seems to be on the phone side, I'll try with older versions of the TWRP image, see whether that makes any difference.

PhAndersson

@phandersson said in Stuck early in the setup process: am I missing a key piece of the puzzle?:

In any case, since the problem seems to be on the phone side, I'll try with older versions of the TWRP image, see whether that makes any difference.

Just tried with 'twrp-3.3.1-0-jasmine_sprout.img' (released in June 2019). Exact same behaviour.

PhAndersson

@phandersson Success at last!
Tried from my home desktop (I was using my work laptop until now), and there it works perfectly. Don't ask. openSUSE in both cases (15.1 vs 15.2). Custom built vs Lenovo P53. Same version of the platform-tools, obviously (Google's latest).

Anyway:

$ ./fastboot boot ./twrp-3.5.2_9-0-jasmine_sprout.img 
Sending 'boot.img' (31560 KB)                      OKAY [  0.996s]
Booting                                            OKAY [ 10.273s]
Finished. Total time: 11.492s

I can now try to proceed with flashing Android 9. What a relief!

magnargj

@phandersson Hurray! Good job, but very strange...