Proposal: The Edge channel

UniSuperBox · 12 Aug 2018, 21:51

This is a proposal to change the UBports release workflow by adding a new "edge" channel to our rootfs builds. I'd like to know the thoughts of the people who it will affect: mainly Halium and Ubuntu Touch developers.

What is currently not working?

Currently, images are promoted from devel to rc to stable. This creates a pipeline where a known good image on devel will eventually become the release image, which is overall good for setting user expectations. It also encourages developers to keep packages included into devel stable as they are candidates for promotion at any time.

We've found that this system is great when we are only introducing small features and fixing bugs, which realistically has been the majority of our work over the past year. However, we're starting to see the cracks. We are not able to introduce large changes to the devel channel for risk of breaking the experience for users there. Some projects we are going to undertake after the OTA-5 (not 4) release will require these large changes. These projects include:

Upgrading Libhybris
Upgrading Mir
Upgrading Unity8
(most importantly) Fixing Issue 404, "Ubuntu Touch does not run on unmodified Halium" and 494, "Mir servers fail to run on devices that ship with CAF's Android for MSM 7.1"

What will be changed?

I propose that we augment our current release channels, stable, rc, and devel, with a new channel called edge. This channel will be built daily for all devices using an edge rootfs.

The edge rootfs will be built from a branch extension of xenial, possibly xenial_-_edge. This will ensure that only specifically selected packages will diverge from the normal image.

edge images will not be candidates for promotion to any other channel. Instead, pull requests will be made from xenial_-_edge to xenial. After a PR has been accepted, the package will need to be removed from xenial_-_edge so that it will no longer diverge.

We will be very clear that the edge channel is expected to break at any time for any device. It is for developers, not users.

How is this different than using qa-scripts?

The edge channel is primarily useful for Halium developers. It is much more convenient for them to download a single image and install it than the currently required hacks and reinstalls of packages. Since the channel will also be available for all devices, it provides a good testing ground to ensure that changes that fix Halium devices do not break our officially supported devices.

arubislander · 13 Aug 2018, 09:50

@unisuperbox
I am probably not even among "the people who [this] will affect" but I do have a question about convention. If I understand this correctly the tl;dr of this propsal is:

Introduce an edge channel.
edge will be even more unstable than devel because of the bigger changes that will land there.
edge will not be (automatically) promoted to anything else. The changes will propagate to devel via pull requests.

So my question is: Does the description of what edge should become not sound awfully a lot like what is usually the role of the devel channel? In other words, should the roles of edge and devel not be switched with the introduction of the edge channel?

alan_g · 13 Aug 2018, 10:25

This sounds odd, because what you're proposing for edge is what devel has been described as (and is for a lot of projects). It would be reasonable to take the attitude people running devel were warned, and chose to accept the risk that devel is not guaranteed to be stable.

To create a new channel because you worry that some people expect something without good reason seems excessive.

However, convenience for developers is another story. Having an easy way to install experimental versions is important. Does a single channel cover this need? Would it be better to have the facility for multiple sub-channels?

devel/libhybris
devel/Mir
devel/Unity8
devel/caf
devel/halium

(From the sidelines as I'm content to run stable and await released versions.)

doniks · 13 Aug 2018, 11:53

Upon the first read it sounds like @UniSuperBox and @alan_g propose significantly different approaches. But after pondering this a little and if I read some fine print into the space between the lines, maybe it's basically the same:

devel will be what it always was.

for the sake of some bigger changes for which we expect that they will take some time to mature we want a different place to develop and test these features separate from devel.

for convenience this place should produce rootfs images.

@UniSuperBox foresees that right now one such feature branch is sufficient. Also there are a bunch of intertwined changes expected, so instead of giving this branch a descriptive name like libhybris12345, we just call it with some "other" name, like "edge".

I guess there is no reason not to go with this. If in a week(s) from now, we find out, we actually need multiple separate branches, we'll change that and have a "libhybris12345" and a "mirABC" and a "edge" and a "edgeedge" channel, two weeks later it turns out that "mirABC" is stale, because basically it's all sorted out and all focus is on integrating stuff in "edgeedge" .... sure why not.

So, with sufficient squinting, I feel like saying "Yes, good idea" to both of you @UniSuperBox and @alan_g

Did I miss anything?

alan_g · 13 Aug 2018, 12:19

@doniks said in Proposal: The Edge channel:

Did I miss anything?

Not really. I raise two typical developer questions:

is there a good reason for supporting this channel?
is one extra channel really sufficient?

(There are three numbers in computing 0, 1 and many.)

Marathon2422 · 13 Aug 2018, 15:13

@arubislander
Sounds like,the average user,should not have access to the development channel ,isn't stable,then RC,enough.
Then the devs,get do what they need,behind the scenes, and then trim the forum,so devs get to see all,and the average guys get to fault find on stable and RC,and see the info on the forum about those,and devel stuff,discussed/ updated on youtube channel It seems like its doubled up having 3 channels to choose from.

arubislander · 13 Aug 2018, 15:25

@marathon2422
Well, I can see the need for a channel that is accessible early in the development cycle, which is stable enough to encourage use, gets updated frequently, but is not yet rc grade stable.
You would want this channel to be tested as extensively as possible, but frequent breakage might scare off testers.

On the other hand, testers should know what they are getting into.

Stefano · 13 Aug 2018, 15:46

@unisuperbox Oh, yeah, bring it on. I'll happily test a new devices with such a channel.

mariogrip · 13 Aug 2018, 18:29

@alan_g The biggest major change is that this channel will be decouple from the ota release cycle and will only be cherry picked into devel (and then follow the release cycle) this is done due to our different types of devices where our older devices does not support newer libhybris just yet, and that way we can pull new libhybris and other major changes (unity8, mir, sensorfw, droidmedia, anbox and etc we want to experiment with) into this edge channel and keep it there to freely develop on without interfering with the normal OTA release cycle that happens at devel->rc->stable. Then the question is, why a channel/rootfs. This is because we want to support halium devices with this channel/rootfs and provide them with an option to download a rootfs that supports halium out of the box without needing to update libhybris etc to get it to boot with our rootfs, and for the channel this allows us to provide community channels using newer libhybris.

So keyword here is libhybris, we simply cannot bring libhybris in to devel->rc->stable cycle at this point, that would break some of the older devices.

mariogrip · 13 Aug 2018, 18:34

The idea of subchannel is something we got already, but using deb packages instead of full rootfs (we call them "branches"). We also got a tool to install/uninstall these branches. (http://docs.ubports.com/en/latest/about/process/ppa.html)

Stefano · 13 Aug 2018, 18:50

@mariogrip come Marius, we're waiting for Edge channel for Oneplus 3 and Oneplus 5. Let me know once is up and running, I'll test it straight away, same for Anbox.

flohack · 13 Aug 2018, 21:02

I sustain the edge channel @alan_g for a variety of reasons:

We are a small team, we cannot guarantee that things being pushed will not break the entire image. We lack auto and manual testing a lot, we cannot work with silos etc.
For some scenarios its very cumbersome to having install various packages from various branches at the same time to the phone
devel has become actually quite a "stable devel" and we want to keep it as described for the changes where we are sure that they will not break the whole image. People should not expect having to reflash their phones after a devel channel update going wrong.
This channel will not be on the phone for people to "just switch" to it.
I do not see much issues with yet another channel coming for devs, the low-level devs will use it, but middleware/Ui etc can stay on devel probably.

BR

domubpkm · 15 Aug 2018, 10:06

This post is deleted!

advocatux · 15 Aug 2018, 10:44

This post is deleted!

ThePossessor · 16 Aug 2018, 21:46

From what is being said I understand the following:

Differences are too big to just make a new "edge" channel of existing 16.04. It cannot be mixed with existing devices yet.
Also the way this would interconnect with normal 16.04 devel > rc > stable cycle is unclear

A different way to handle it would be make it independently coexist like 15.04 and 16.04 do. A third option 16.04.halium with its own devel > rc > stable cycle could be a way.

However since this is not only about halium itself but also other related sub projects this does not completely solve the problem. Maybe for this additionally using the "branches" mentioned by @mariogrip could be a way?

mardy · 17 Aug 2018, 20:49

This channel would be useless to me and to many other developers, but if it brings benefits to halium developers, then why not. But then, why not call it "halium", "porters" or some other name that makes it clear what its target audience is?

"edge" to me means "more ahead than devel", but there's again the assumption that packages from this channel will eventually end up in "devel", "rc" and "stable"; so, I would suggest a different, more specific name.