building for Nexus 7 deb

doniks · 3 Jun 2017, 11:22

@doniks said in building for Nexus 7 deb:

@Flohack said in building for Nexus 7 deb:

@doniks You can safely apply all changes of this script - in fact they are all necessary, so dont bother with trying one by one

Ok, let me try.

Yup, that was easy enough. Applied all changes. Rebuilt. Flashed recovery. Rebooted. Works just as well until here!

flohack · 3 Jun 2017, 11:23

@doniks In general if you got a working CM kernel no real kernel patches are needed except apparmor3 => But you can try stealing that as a whole subdir from a kernel that has these patches applied. Should be more like copy/paste.

BR

doniks · 3 Jun 2017, 19:00

one step further. next road block. I've cloned the rootstock-ng thingy, downloaded the vivid....tar.gz, ran the rootstock thingy. It did it's thing, the tablet rebooted but is stuck at the Google logo screen. adb is not working. nothing seems to be happening. Not sure how to proceed, don't have any ideas

doniks · 4 Jun 2017, 13:16

I realize that there is a deb image in the rc channel

ubuntu-device-flash --verbose --server=https://system-image.ubports.com/ touch --device=deb --channel=ubuntu-touch/rc-proposed

that installs and starts just fine. Poking around in that image, it seems to me as if the initrd is basically the same as my self built one, but the kernel config and the android system.img look quite different. Though, I still don't know what exactly to do ...

Andreas Pokorny · 4 Jun 2017, 19:32

@doniks You might want to try the google_msm kernel in ubports. It already contains the apparmor and other backports:
https://github.com/ubports/android_kernel_google_msm/tree/ubp-5.1

but it does not contain a reasonable kernel config for flo.

You could also verify that the user space binaries and firmware files are being picked up get copied into the out/target/product/...../system directory.

If none of that gets you any further - use adb while in recovery and go to /data/system-data and examine the var/log files for clues..

Andreas Pokorny · 4 Jun 2017, 20:15

I meant deb .. not flo

flohack · 5 Jun 2017, 12:52

@Andreas-Pokorny You can edit posts here xD

doniks · 6 Jun 2017, 05:47

@Andreas-Pokorny said in building for Nexus 7 deb:

Thanks for your tips.

You could also verify that the user space binaries and firmware files are being picked up get copied into the out/target/product/...../system directory.

I think I'm good here:

cd vendor/qcom/deb/proprietary
for b in * ; do find ../../../../out/target/product/deb/system/ -name $b ; done
../../../../out/target/product/deb/system/vendor/firmware/a300_pfp.fw
../../../../out/target/product/deb/system/vendor/firmware/a300_pm4.fw
../../../../out/target/product/deb/system/bin/ATFWD-daemon
../../../../out/target/product/deb/system/bin/bridgemgrd
../../../../out/target/product/deb/system/bin/btnvtool
../../../../out/target/product/deb/system/bin/diag_klog
../../../../out/target/product/deb/system/bin/diag_mdlog
../../../../out/target/product/deb/system/bin/ds_fmc_appd
../../../../out/target/product/deb/system/vendor/lib/egl/eglsubAndroid.so
[...]

doniks · 6 Jun 2017, 05:49

@Andreas-Pokorny said in building for Nexus 7 deb:

At first I thought about looking into this:

If none of that gets you any further - use adb while in recovery and go to /data/system-data and examine the var/log files for clues..
But really, I think the problem with that build was that the root partition never got mounted, I'm not even sure whether much of anything of the initrd got mounted/executed

However, I think i must have messed up something earlier, maybe I did still end up using the old kernel config, I don't know, at some point (without having knowingly done something different) the screen came up (and adb as well). But then the display was all garbled up. I can recognize that it's trying to display the welcome screen, but the colors are all distorted, looking pink and green and it seems as if the characters shown are all way too big and don't really fit on the screen.

At that point, I thought, ok, lets switch to that other kernel,

@doniks You might want to try the google_msm kernel in ubports. It already contains the apparmor and other backports:
https://github.com/ubports/android_kernel_google_msm/tree/ubp-5.1

but it does not contain a reasonable kernel config for flo.

so I change the manifest to

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
  <remote       name="cm"       fetch="http://github.com/CyanogenMod"       revision="refs/heads/cm-12.1" />
  <project       path="device/asus/deb"       name="android_device_asus_deb"       remote="cm"       revision="cm-12.1" />  
  <project       path="device/asus/flo"       name="android_device_asus_flo"       remote="cm"       revision="cm-12.1" />

  <!--
  <project       path="kernel/google/msm"       name="android_kernel_google_msm"       remote="cm"       revision="cm-12.1" />
  <remote       name="ubp"      fetch="http://github.com/ubports"      review="http://review.ubports.com"      revision="refs/heads/ubp-5.1" />
  -->

   <project       path="kernel/google/msm"       name="ubports/android_kernel_google_msm"       remote="ubp"        revision="ubp-5.1" />  
</manifest>

and copied the cyanogen_flo_defconfig file back. Ie, the one from cm deb repo with all the check-config adjustments. Rebuilt and

fastboot boot boot.img

but that produces the same result actually, adb comes up, screen comes up, display is all garbled up. Not sure, what's next.

doniks · 6 Jun 2017, 05:55

Do you guys have a git repository of the kernel trees that Canonical had used?

flohack · 6 Jun 2017, 09:17

@doniks said in building for Nexus 7 deb:

e kernel trees that Canonical had used?

No I dont think so. Maybe @mariogrip knows that?

BR

doniks · 12 Jun 2017, 10:32

Canonical says:

You can find all the needed Android specific git repositories at code-review.phablet.ubuntu.com . This is a working gerrit server with everything needed to build the Android images used by Ubuntu Touch. The reference tree in there is based on AOSP (4.4.2 specifically), so make sure your device specific repositories are compatible with AOSP at least.

For any Android related project at our git server, you'll find a branch named phablet-4.4.2_r1. This branch contains a static known git HEAD and the required changes needed for Ubuntu, including our custom Androidmanifest ( https://code-review.phablet.ubuntu.com/gitweb?p=aosp%2Fplatform%2Fmanifest.git;a=shortlog;h=refs%2Fheads%2Fphablet-4.4.2_r1 ).

This custom manifest does contain:

  <project path="device/asus/deb" name="aosp/device/asus/deb" groups="device,flo" />
  <project path="device/asus/flo" name="aosp/device/asus/flo" groups="device,flo" />

And poking around on that website I see stuff where something deb related has happened:
https://code-review.phablet.ubuntu.com/#/q/aosp+deb,n,z

However,

I can't find a decent overview of all the repositories there, e.g, where is kernel/google/msm (or should it be called android_kernel_google_msm?! can't find it either)
I can't figure out what to change in the local manifest that I put in the directory where I did the initial checkout of ubports version (in my first post)

flohack · 12 Jun 2017, 17:53

@doniks BTW The gerrit server may go down any time at phablet.ubuntu.com

doniks · 12 Jun 2017, 18:05

@Flohack said in building for Nexus 7 deb:

@doniks BTW The gerrit server may go down any time at phablet.ubuntu.com

Do you know how to get an overview of the repositories there? Something like how I see an overview here: https://github.com/ubports

doniks · 17 Jun 2017, 13:05

I managed to download some sources from canonical, but in this double-digit-gigabyte-"source"-download, I seem to only have gotten a binary kernel so, I haven't gotten a lot wiser.

phablet-dev-bootstrap phablet
. build/envsetup.sh
lunch
# ... pick deb
m

Among others that spits out:

including ubuntu/platform-api/android/hybris/Android.mk ...
including ubuntu/upstart-property-watcher/Android.mk ...
No private recovery resources for TARGET_DEVICE deb
if [ -f kernel/asus/deb/Makefile ]; then \
        mkdir -p /media/peter/share2/data/canonical/phablet/out/target/product/deb/obj/KERNEL_OBJ; \
        make -C kernel/asus/deb O=/media/peter/share2/data/canonical/phablet/out/target/product/deb/obj/KERNEL_OBJ ARCH=arm  headers_install; \
        fi
Downloading linux-image-flo version 3.4.0.4.5
Downloading linux-image-3.4.0-4-flo version 3.4.0-4.16
'/media/peter/share2/data/canonical/phablet/out/target/product/deb/ubuntu/kernel/boot/vmlinuz-3.4.0-4-flo' -> '/media/peter/share2/data/canonical/phablet/out/target/product/deb/ubuntu/kernel/vmlinuz'

And I can't see a "kernel" directory either:

ls
abi  bionic  bootable  build  development  device  external  frameworks  hardware  Makefile  out  prebuilts  sdk  system  ubuntu

I can fastboot boot boot.img and that comes up fine. It doesn't have a /proc/config.gz though

FWIW, diff tells me that

the out/target/product/deb/kernel file and
the zImage inside the boot image inside the device image in the rc-proposed channel on the ubports servers
are identical according to diff.

doniks · 23 Jun 2017, 08:21

@doniks said in building for Nexus 7 deb:

Rebuilt and
fastboot boot boot.img
but that produces the same result actually, adb comes up, screen comes up, display is all garbled up. Not sure, what's next.

When booted this way with my self compiled kernel, I see loads of mdp4 messages in syslog which are not there when I boot the old image:

$ grep -C 5 -i mdp4 bad.syslog.004 
Jun 23 09:56:54 ubuntu-phablet kernel: [    0.980865] Registered led device: lcd-backlight
Jun 23 09:56:54 ubuntu-phablet kernel: [    0.980926] mipi_JDI_lcd_probe-
Jun 23 09:56:54 ubuntu-phablet kernel: [    0.981018] mipi_JDI_1080P_pt_init-
Jun 23 09:56:54 ubuntu-phablet kernel: [    0.981903] setting pdata->panel_info.fb_num to 1. type: 7
Jun 23 09:56:54 ubuntu-phablet kernel: [    0.982757] FrameBuffer[1] 1920x1080 size=0 bytes is registered successfully!
Jun 23 09:56:54 ubuntu-phablet kernel: [    0.983032] mdp4_dtv_vsync_init: ndx=0
Jun 23 09:56:54 ubuntu-phablet kernel: [    0.983184] hdmi_msm hdmi_msm.1: external_common_state_create: sysfs group c1f43e08
Jun 23 09:56:54 ubuntu-phablet kernel: [    0.983306] EXT_COMMON: external_common_state_create: kobject_uevent(KOBJ_ADD)
Jun 23 09:56:54 ubuntu-phablet kernel: [    0.983306] hdmi_msm hdmi_msm.1: hdmi_msm_config_hdcp_feature: HDCP Feature: Enabled
Jun 23 09:56:54 ubuntu-phablet kernel: [    0.984283] 
Jun 23 09:56:54 ubuntu-phablet kernel: [    0.984283]  msm_vidc_enc: Inside vid_enc_init()
--
Jun 23 09:57:03 ubuntu-phablet kernel: [   22.510498] audit_printk_skb: 234 callbacks suppressed
Jun 23 09:57:03 ubuntu-phablet kernel: [   22.510498] type=1400 audit(1498204623.097:90): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/bin/lxc-start" pid=1818 comm="apparmor_parser"
Jun 23 09:57:03 ubuntu-phablet kernel: [   22.561981] mipi_dsi_panel_power-
Jun 23 09:57:03 ubuntu-phablet kernel: [   22.567596] mipi_JDI_lcd_on+
Jun 23 09:57:03 ubuntu-phablet kernel: [   22.567596] mipi_JDI_lcd_on-, booting
Jun 23 09:57:03 ubuntu-phablet kernel: [   22.570373] mdp4_calc_req_mdp_clk: src_h is zero!
Jun 23 09:57:03 ubuntu-phablet kernel: [   22.571594] EXT_COMMON: hdmi_common_wta_hpd: '0' (unchanged)
Jun 23 09:57:03 ubuntu-phablet dbus[836]: [system] Successfully activated service 'org.freedesktop.UPower'
Jun 23 09:57:03 ubuntu-phablet kernel: [   22.648132] pil pil1: wcnss: Brought out of reset
Jun 23 09:57:03 ubuntu-phablet kernel: [   22.654937] type=1400 audit(1498204623.248:91): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="lxc-container-default" pid=1840 comm="apparmor_parser"
Jun 23 09:57:03 ubuntu-phablet kernel: [   22.654968] type=1400 audit(1498204623.248:92): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="lxc-container-default-with-mounting" pid=1840 comm="apparmor_parser"
--
Jun 23 09:57:03 ubuntu-phablet kernel: [   23.053497] wcnss: schedule dnld work for riva
Jun 23 09:57:03 ubuntu-phablet kernel: [   23.056915] wcnss: NV bin size: 29772, total_fragments: 10
Jun 23 09:57:03 ubuntu-phablet kernel: [   23.056945] wcnss: no space available for smd frame
Jun 23 09:57:03 ubuntu-phablet kernel: [   23.082000] wcnss: no space available for smd frame
Jun 23 09:57:03 ubuntu-phablet kernel: [   23.111999] wcnss: no space available for smd frame
Jun 23 09:57:03 ubuntu-phablet kernel: [   23.134521] mdp4_overlay_req2pipe: mdp4_overlay_format2type!
Jun 23 09:57:03 ubuntu-phablet kernel: [   23.134582] mdp4_overlay_set: mdp4_overlay_req2pipe, ret=-34
Jun 23 09:57:03 ubuntu-phablet kernel: [   23.134674] msmfb_overlay_set: ioctl failed, rc=-34
Jun 23 09:57:03 ubuntu-phablet kernel: [   23.135070] mdp4_overlay_mdp_perf_upd mdp clk is changed [1] from 0 to 266667000
Jun 23 09:57:03 ubuntu-phablet kernel: [   23.144042] wcnss: no space available for smd frame
Jun 23 09:57:03 ubuntu-phablet kernel: [   23.175872] wcnss: received WCNSS_NVBIN_DNLD_RSP from ccpu
Jun 23 09:57:04 ubuntu-phablet dbus[836]: [system] Activating service name='com.canonical.PropertyService' (using servicehelper)
Jun 23 09:57:04 ubuntu-phablet kernel: [   23.742034] mdm_modem_ioctl: normal boot of mdm id 0 done
Jun 23 09:57:04 ubuntu-phablet kernel: [   23.748870] mdm_pblrdy_change: mdm id 0: pbl ready:0
Jun 23 09:57:04 ubuntu-phablet dbus[836]: [system] Successfully activated service 'com.canonical.PropertyService'
Jun 23 09:57:04 ubuntu-phablet kernel: [   24.180572] mdp4_overlay_req2pipe: mdp4_overlay_format2type!
Jun 23 09:57:04 ubuntu-phablet kernel: [   24.180725] mdp4_overlay_set: mdp4_overlay_req2pipe, ret=-34

(I've only pasted the top part above, the two mdp4 lines about mdp4_overlay_format2type! and mdp4_overlay_req2pipe are repeated forever and ever)

In the old image it looks like this:

$ grep -C 5 -i mdp4 good.syslog 
Jun 22 23:33:59 ubuntu-phablet kernel: [    1.681640] Registered led device: lcd-backlight                                                                
Jun 22 23:33:59 ubuntu-phablet kernel: [    1.681793] mipi_JDI_lcd_probe-                                                                                 
Jun 22 23:33:59 ubuntu-phablet kernel: [    1.681915] mipi_JDI_1080P_pt_init-                                                                             
Jun 22 23:33:59 ubuntu-phablet kernel: [    1.683410] setting pdata->panel_info.fb_num to 1. type: 7                                                      
Jun 22 23:33:59 ubuntu-phablet kernel: [    1.686706] FrameBuffer[1] 1920x1080 size=0 bytes is registered successfully!                                   
Jun 22 23:33:59 ubuntu-phablet kernel: [    1.687286] mdp4_dtv_vsync_init: ndx=0
Jun 22 23:33:59 ubuntu-phablet kernel: [    1.687530] hdmi_msm hdmi_msm.1: external_common_state_create: sysfs group eddedc08                             
Jun 22 23:33:59 ubuntu-phablet kernel: [    1.687774] EXT_COMMON: external_common_state_create: kobject_uevent(KOBJ_ADD)                                  
Jun 22 23:33:59 ubuntu-phablet kernel: [    1.687805] hdmi_msm hdmi_msm.1: hdmi_msm_config_hdcp_feature: HDCP Feature: Enabled                            
Jun 22 23:33:59 ubuntu-phablet kernel: [    1.689575]                                                                                                     
Jun 22 23:33:59 ubuntu-phablet kernel: [    1.689605]  msm_vidc_enc: Inside vid_enc_init()
--
Jun 22 23:34:07 ubuntu-phablet NetworkManager[1588]: <warn>  [1498167247.3918] error poking ModemManager: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.ModemManager1 was not provided by any .service files
Jun 22 23:34:07 ubuntu-phablet NetworkManager[1588]: <info>  [1498167247.3930] ofono is now available
Jun 22 23:34:07 ubuntu-phablet kernel: [   16.623474] pil pil1: wcnss: Brought out of reset
Jun 22 23:34:07 ubuntu-phablet kernel: [   16.687835] EXT_COMMON: hdmi_common_wta_hpd: '1'
Jun 22 23:34:07 ubuntu-phablet /03mmsproxy: Cannot add proxy route, empty parameters
Jun 22 23:34:07 ubuntu-phablet kernel: [   16.841949] mdp4_overlay_mdp_perf_upd mdp clk is changed [1] from 0 to 200000000
Jun 22 23:34:07 ubuntu-phablet kernel: [   16.859924] audit_printk_skb: 291 callbacks suppressed
Jun 22 23:34:07 ubuntu-phablet kernel: [   16.859954] type=1400 audit(1498167247.631:109): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="com.ubuntu.weather_weather_3.4.248" pid=1807 comm="apparmor_parser"
Jun 22 23:34:07 ubuntu-phablet cron[1798]: (CRON) INFO (pidfile fd = 3)
Jun 22 23:34:07 ubuntu-phablet kernel: [   16.873504] type=1400 audit(1498167247.641:110): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="com.ubuntu.music_music_2.5.1019" pid=1807 comm="apparmor_parser"
Jun 22 23:34:07 ubuntu-phablet kernel: [   16.887664] type=1400 audit(1498167247.651:111): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="com.ubuntu.clock_clock_3.8.487" pid=1807 comm="apparmor_parser"

(these are the only two hits for mpd4)

Does that give anybody a clue as to what I could do to fix this?

flohack · 23 Jun 2017, 12:04

@doniks Are those message causing trouble, why do you think?

doniks · 26 Jun 2017, 08:25

@Flohack said in building for Nexus 7 deb:

@doniks Are those message causing trouble, why do you think?

The display is all garbled. It's not usable. The colors are off and the display scaling seems to be off - let me attach a foto:

I was just fishing in the dark for hints and stumbled over these log messages. I do not know whether they are related. The display is the actual problem.

flohack · 3 Mar 2019, 21:44

Finally some progress: We have fixed the system image for deb and are hoping now that it can be actually installed from our servers!