As we near the OTA-6 release, I would like to gather what I believe are the most important points from this post:
Everyone wants "release faster", without a doubt.
"Release faster" depends on having enough confidence that software being released to stable is well-tested, which we do not have. Some things which would translate into more release confidence include:
Automated tests on all system components
Integration tests between system components
Automated full-device testing, such as Canonical's fabled "Frankenstein device lab"
Formalized manual testing by users who can confirm stability
Even with full release confidence (but especially without), an automatic release model requires the ability to roll back to a previous version of apps and the full system image
Aside: I want to experiment with one of the new atomic release models like OSTree which allows agnostic packaging formats to be installed on top. From my quick foray, I see:
Benefits: Automatic and manual rollbacks, automatic diffing, and switchable roots
Drawbacks that make this a long-term or impossible idea for our existing devices
Probably requires newer kernel versions than 3.4 or 3.10
We don't have enough engineering power to concoct a solution which would allow converting a system-image system to OSTree in-flight, so we'd probably require users to manually switch.
I think that bringing up the edge channel as a place to do a large migrations out-of-band with the normal release cadence is going to have a huge impact on how quickly we can release in the future. Previously, @mariogrip's work on bringing us to upstream Libhybris would have had to wait until after OTA-6 lands, and then it would have delayed OTA-7 until we had proper confidence in it. With edge, we can have people help us test these huge changes with the ability to roll back and without disrupting current users.
To respond to my earlier hindsight: OTA-6 was not (yet ) taken away by TDS. We did not hit the original deadline set, but we will still be within our 6-8 week cycle when it releases. This includes our testing and release admin stage, which we are currently in. All of this while I'm not sweating bullets from pushing a release on ourselves before we're sure we're ready.
We have ended up in "Gotta release slower" in this cycle without a doubt. I hope to make the improvements we've identified here so "gotta release fast" can become a reality. I also hope to be able to take our release management up so that we hit 2-4 weeks development, 2 weeks testing. This means we'll get faster releases, but we still won't hit fast releases.
Important note for anyone interested in contributing to... basically anything I've said here: I'll be holding an OTA-7 development meeting at the start of the cycle where we'll discuss how we want to improve during the OTA-7 cycle and taking bugs from the tracker for the release. Only assigned tickets will be added to this milestone, so if you have a pet bug that you want fixed now is the time to get involved to help fix it! Subscribe to this GitLab issue for more information as the day draws nearer.
