Sony Xperia XZ (F8331 / Kagura) Porting

joe · 29 Jan 2019, 00:20

[EDIT] See this post: https://forums.ubports.com/post/38767 below. Thank you, Konrad!

I was inspired to try out the porting guide during the live porting session this past weekend:

I was able to make halium-boot and systemimage once, but the device didn't boot in the end.

There are a number of challenges with this device:

There is no official lineageos port: I found this possibly working port https://github.com/djselbeck/android_device_sony_kagura/
This is a "tone" device. It might need the SW_binaries_for_Xperia_AOSP_N_MR1_5.7_r1_v08_tone.img from (https://developer.sony.com/develop/open-devices/guides/kernel-compilation-guides/how-to-build-and-flash-a-linux-kernel-for-aosp-supported-devices)
Manifest is a hackjob assuming that since the Lineageos port borrows from the "sumire" device, that copying those parts might work:

<manifest>
    <project path="device/sony/kagura" name="android_device_sony_kagura" remote="djselbeck" />
    <project path="device/sony/common" name="android_device_sony_common" remote="los" />
    <project path="hardware/sony/timekeep" name="android_hardware_sony_timekeep" remote="los" />
    <project path="kernel/sony/msm8994" name="ubports/android_kernel_sony_msm8994" remote="hal" />
    <project path="vendor/sony" name="proprietary_vendor_sony" remote="them" />
</manifest>

Adding the remote as well, assuming that it is useful:

<remote name="djselbeck"
        fetch="https://github.com/djselbeck"
        revision="cm-14.1" />

There was no vendor file in https://github.com/TheMuppets/proprietary_vendor_sony
So, assumed that this might get somewhere:
copy vendor/sony/sumire to vendor/sony/kagura
rename sumire-vendor.mk to kagura-vendor.mk
Not sure about this one. It complained the first time, but not in later attempts
edit device/sony/kagura/BoardConfig.mk to reflect kernel m8994 instead of m8996
There is no kernel config, so... (again maybe a better kernel config somewhere)
copy sumire kernel config to kernel/sony/msm8994/arch/arm64/config/tone_kanguro_defconfig
When Buiding system image i had to use

LANG=C make systemimage

However, later attempts didn't make it far. I mistakenly used the "lunch" command the first time, but probably should use "breakfast". Attempts have been unsuccessful.

Just thought I'd share that and hope that somebody else is interested and has a better idea what to do.

The device, however, is not very friendly to android roms in general, so expectations are pretty low.

ressan627 · 11 Apr 2020, 12:20

Greetings joe,
I'm trying to build something similar for my F8331 but I don't know where to start as there is no official lineageos for it ?
don't know if I can build lineage OS first then port it to ubuntu ...... thanks for sharing .

joe · 11 Apr 2020, 15:44

@ressan627

I was thinking the same thing about first building a lineage port.
This is a working lineageos rom https://forum.xda-developers.com/xperia-xz/development/8-1-lineage-15-1-xperia-xz-t4072083

However, I haven't been able to find the source. I don't know if you can deconstruct the rom itself.

I would start here:
http://docs.ubports.com/en/latest/porting/introduction.html

And maybe something can be built off of the sumire device if nothign for kagura. I could have missed something in my attempt.

ressan627 · 11 Apr 2020, 18:54

@joe
I managed to make the manifist from the but it gives me

# breakfast kagura
including vendor/lineage/vendorsetup.sh
build/core/product_config.mk:243: *** _nic.PRODUCTS.[[device/sony/kagura/lineage.mk]]: "device/sony/tone/platform.mk" does not exist.  Stop.
Device kagura not found. Attempting to retrieve device repository from LineageOS Github (http://github.com/LineageOS).
Repository for kagura not found in the LineageOS Github repository list. If this is in error, you may need to manually add it to your local_manifests/roomservice.xml.
build/core/product_config.mk:243: *** _nic.PRODUCTS.[[device/sony/kagura/lineage.mk]]: "device/sony/tone/platform.mk" does not exist.  Stop.
build/core/product_config.mk:243: *** _nic.PRODUCTS.[[device/sony/kagura/lineage.mk]]: "device/sony/tone/platform.mk" does not exist.  Stop.

** Don't have a product spec for: 'lineage_kagura'
** Do you have the right repo manifest?

here is my xml file

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
    <remote name="beidl" fetch="git://github.com/fredldotme" />
    <remote name="tone" fetch="git://github.com/Tone-keep-on-4-4" />
    <remote name="sony" fetch="git://github.com/sonyxperiadev" />
    <remote name="nxp" fetch="git://github.com/NXPNFCProject/" />
    <project path="device/sony/kagura" name="device-sony-kagura" revision="lineage-15.1" remote="tone" />
    <project path="device/sony/common" name="device-sony-common" revision="lineage-15.1" remote="tone" />
    <project path="device/sony/common-init" name="device-sony-common-init" revision="n-mr1-ubports" remote="beidl" />

    <project path="external/gpg" name="android_external_gpg" revision="halium-7.1" remote="beidl" />
    <project path="external/libnfc-nci" name="platform/external/libnfc-nci" groups="pdk" remote="aosp" />

    <project path="external/toybox" name="android_external_toybox" revision="halium-7.1" remote="beidl" />

    <project path="hardware/qcom/audio" name="android_hardware_qcom_audio_aosp" groups="qcom,qcom_audio" revision="halium-7.1" remote="beidl" />


    <project path="kernel/sony/kagura" name="Kernel" remote="tone" />



    <project path="vendor/nxp/" name="vendor-nxp" groups="device" remote="sony" revision="master" />
    <project path="vendor/nxp/NXPNFCC_FW" name="NXPNFCC_FW" groups="device" remote="nxp" revision="master" />
    <project path="vendor/broadcom/bt-fm" name="vendor-broadcom-bt-fm" groups="device" remote="sony" revision="master" />
    <project path="vendor/broadcom/wlan" name="vendor-broadcom-wlan" groups="device" remote="sony" revision="master" />
    <project path="vendor/qcom/opensource/fm" name="vendor-qcom-opensource-fm" groups="device" remote="sony" revision="master" />
    <project path="vendor/qcom/opensource/location" name="vendor-qcom-opensource-location" groups="device" remote="sony" revision="master" />
    <project path="vendor/qcom/opensource/time-services" name="vendor-qcom-opensource-time-services" groups="device" remote="sony" revision="master" />
    <project path="vendor/qcom/opensource/wlan" name="hardware-qcom-wlan" groups="device" remote="sony" revision="master" />

    <project path="vendor/oss/fingerprint"  name="vendor-sony-oss-fingerprint" groups="device" remote="sony" revision="android-7.1.1_r55" />
    <project path="vendor/oss/macaddrsetup" name="macaddrsetup" groups="device" remote="sony" revision="master" />
    <project path="vendor/oss/thermanager" name="thermanager" groups="device" remote="sony" revision="master" />
    <project path="vendor/oss/timekeep" name="timekeep" groups="device" remote="sony" revision="master" />
    <project path="vendor/oss/transpower" name="transpower" groups="device" remote="sony" revision="android-7.1.1_r55" />
    <project path="vendor/oss/json-c" name="json-c" groups="device" remote="sony" revision="master" />
</manifest>

hope this helps I'll continue tomorrow cuz now it's kind of late have a nice day

joe · 11 Apr 2020, 23:17

@ressan627 Thanks for working on this port.

It seems that maybe the repository for the device might not be complete:
git://github.com/Tone-keep-on-4-4 may be missing some files.

This was the most complete one that I could find:
https://github.com/djselbeck/android_device_sony_kagura/

sjll · 25 May 2020, 08:08

Hi,
Don't use the tree on Tone-keep-on-4-4. It is too complex. I already bring up a ubuntu touch tree for Xperia X Performance.
You only need to rewrite the device tree for kagura. The other trees could use the same as dora.

joe · 25 May 2020, 14:15

@sjll Thanks for your help. Do links to the tree for Xperia X Performance?

joe · 27 May 2020, 15:05

@sjll I just realized that I think you made this as well:

https://forum.xda-developers.com/xperia-xz/development/8-1-lineage-15-1-xperia-xz-t4072083

This tree would be helpful to make the Ubuntu Touch port I think, but I couldn't find a github link.

Thank you!

joe · 29 Jun 2020, 00:53

I wonder if you can treblize the XZ:
https://forum.xda-developers.com/xperia-xz/development/f8331-2-project-treble-xperia-xz-t4029421

Then you might be able to install the Ubuntu Touch GSI:
https://t.me/ErfanGSI

joe · 6 Aug 2020, 20:46

Looks like Konrad has pretty much completed this port!
https://github.com/konradybcio/device-sony-kagura

The manifest has been requested to be merged into halium.

Hoping to build the images and see how it goes...

ressan627 · 17 Aug 2020, 15:55

Keep me updated as I was away for a while doing some other activities..

joe · 17 Aug 2020, 16:42

@ressan627 I was able to get it running on Kagura thanks to Konrad's help.

Issues:

No camera. Even with gstreamer
Graphical performance is a bit sluggish
No fingerprint reader
Wifi won't automatically connect after reboot
Phone reboots when plugged into usb attached to computer
Using suru dark system theme causes text fields in browser to be all black with black text (i.e. unreadable)
No flashlight

This device may show up in the installer some time, but here is what I did to get it running. Pieced together from docs.ubports.com , docs.halium.org and some documents from the telegram group: @ubports_porting

Building and Installing

If you don't want to build the images yourself, then contact me and I can find a way to send them to you.

Porting Environment

http://docs.halium.org/en/latest/porting/first-steps.html

sudo dpkg --add-architecture i386

sudo apt update

sudo apt install git gnupg flex bison gperf build-essential \
  zip bzr curl libc6-dev libncurses5-dev:i386 x11proto-core-dev \
  libx11-dev:i386 libreadline6-dev:i386 libgl1-mesa-glx:i386 \
  libgl1-mesa-dev g++-multilib mingw-w64-i686-dev tofrodos \
  python-markdown libxml2-utils xsltproc zlib1g-dev:i386 schedtool \
  repo liblz4-tool bc lzop imagemagick libncurses5 rsync

The problem with 20.04 is that repo doesn't exist in the repositories. You will have to install the package and dependency from eoan.

Download packages:
downloaded the packages manually:

python-kerberos from https://packages.ubuntu.com/eoan/python-kerberos
repo from https://packages.ubuntu.com/eoan/repo

I then installed them using sudo dpkg -i <filename>, python-kerberos first.

It may complain about python version, but it will work.

Get Halium Source

This takes a long time and a lot of space (~15GB).

mkdir halium && cd halium

repo init -u https://github.com/Halium/android -b halium-7.1 --depth=1

repo sync -c -j 16

Manifest

For Kagura the manifest is now part of halium source. So you don't need to do anything here.

But for reference, this is what needs to be done for devices not included.
// You need to make sure the correct manifest for the devices is in:
// /halium/halium/devices/manifests

// If you don't have a manifest, then you will have to create it for your device:
// http://docs.halium.org/en/latest/porting/get-sources.html

Get Devices Source

First get devices source (pulled from info in manifest)
./halium/devices/setup kagura --force-sync
// had to use force-sync or no kernel folder would show up

Fix Mounts

Not sure. Seems not applicable to kagura

Kernel Check

Run Kernel check for Ubuntu Touch
./halium/halium-boot/check-kernel-config kernel/sony/kagura/arch/arm64/configs/aosp_tone_kagura_defconfig -w

Also ran on :
./halium/halium-boot/check-kernel-config kernel/sony/kagura/arch/arm64/configs/defconfig -w

It will fix a few things. Run it again until everything is fixed.

BUILD IMAGE

Build the image

Once halium-boot is in place, you can build it quite simply. You will also need to rebuild system.img due to our changes.

cd to your Halium BUILDDIR
source build/envsetup.sh
Run `lunch` and select the correct device
`mka halium-boot`
`export USE_HOST_LEX=yes` // fixes an error 
`mka systemimage`

Anothe 15GB or so are needed to build these things.

Installing Ubuntu Touch

Get Halium-Install

Clone this repository

https://gitlab.com/JBBgameich/halium-install

Install dependencies: sudo apt install [those packages below]
qemu-user-static
qemu-system-arm
e2fsprogs
simg2img

Download rootfs:
https://ci.ubports.com/job/xenial-hybris-edge-rootfs-arm64/

Install Halium:
./path/to/halium-install -p ut path/to/rootfs.tar.gz path/to/system.img // using the correct paths and changed root password

After install. Reboot.

Do some fixes and updates

NOTE: I'm not sure if all this needs to be done before flashing the tone file (See last step. That might actually work as is) but if not.

Device should boot, show a purple indicator light briefly, and then hang at the Sony logo. It is actually booted at the moment.
There should be a new wired connection that shows up on the connected computer. When that shows "connected" you can ssh to the phone.

SSH:

ssh phablet@10.15.19.82

// sometimes requires a reboot first

Connect to wifi:

nmtui

Make device writable:

sudo mount -o remount,rw /

Edit and update:

sudo -i

Add Udev rules:

cat /var/lib/lxc/android/rootfs/ueventd*.rc|grep ^/dev|sed -e 's/^\/dev\///'|awk '{printf "ACTION==\"add\", KERNEL==\"%s\", OWNER=\"%s\", GROUP=\"%s\", MODE=\"%s\"\n",$1,$3,$4,$2}' | sed -e 's/\r//' >/usr/lib/lxc-android-config/70-kagura.rules

Install xenial arm 64 stuff:

ubports-qa install xenial_-_arm64

apt update && apt upgrade

apt install mir-platform-graphics-android-caf15

Change Lib path

nano /etc/environment
edited to LD_LIBRARY_PATH=/vendor/lib64:/system/lib64

Platform Apis

NOTE: I think NOT doing this allows the light sensor to work. Not sure which option is better.

comment out the if and fi in /usr/share/upstart/sessions/unity8.conf

Example:
# if [ ! -f /system/lib/libubuntu_application_api.so ]; then
initctl set-env --global UBUNTU_PLATFORM_API_BACKEND=test.so.3
initctl set-env --global UBUNTU_PLATFORM_API_SENSOR_TEST=/etc/ubuntusensors/test.sensors
# fi

Flash the Tone file

This is a key step to actually boot into Ubuntu Touch

Download:
https://developer.sony.com/file/download/software-binaries-for-aosp-nougat-android-7-1-kernel-4-4-tone/

Boot to bootloader

fastboot flash oem [filename]

joe · 9 Jul 2020, 03:52

This post is deleted!

joe · 7 Sept 2020, 04:52

It's in the installer now. Maybe this thread can be closed?

Keneda · 7 Sept 2020, 22:12

@joe said in Sony Xperia XZ (F8331 / Kagura) Porting:

It's in the installer now. Maybe this thread can be closed?

Or maybe you can let it open for support purposes ^^

joe · 8 Sept 2020, 02:24

@Keneda Good thinking!

Here's the updated video:
https://youtu.be/_aocnYkQcnc