How to extend the system UI and UX?

dobey

Firstly, what exactly is Springboard and how does one get to it on the Android build? Is it just a replacement launcher for Android? How does one get to their apps if so? Does it always force the OSK open, or must one first tap on the text entry?

The mock-up picture on the campaign page is quite small and difficult to understand as a whole concept.

PhoenixLandPirat

I think its also important to ask what your plans are with it?
are your plans for it to be a default thing installed on Vollaphones exclusive to your phones, or is it something that you'd want to publish to the open store.

becuase if you're doing it exclusively, then you tend to have the ability to break a few more rules than a standard app that you can install

dobey

@PhoenixLandPirat said in How to extend the system UI and UX?:

becuase if you're doing it exclusively, then you tend to have the ability to break a few more rules than a standard app that you can install

There's no such thing as exclusive, if it's going to be open source.
If it is meant to be exclusive, then Volla would need to fork lomiri I guess to include this widget in the background area, and change a few other behaviors to make it work, plus maintain their own build and image server, and updates. Which I think is not what they want.

marc_aurel

Thanks for your questions and comments. Let me try to answer your questions and explain the idea:

In my point of view Ubuntu Touch could be a platform for innovative products with custom system user interfaces. Our long term idea is to rethink the app centric paradigm of mobile devices. However we want to start with small, realistic steps.

We plan the Volla UX mainly for Volla devices, but we want to publish the code to involve the community. Thus a customized Ubuntu Touch could be an option. However we would prefer a modularization of Lomiri to avoid conflicts of customization. I could imagine a setting to select any launcher app as a replacement of the default one similar to Android. So developers could even provide such kind of apps in the UT app store and users have the choice to select the one they prefer.

Our Volla UX for Android is implemented as a launcher app. We already use Qt/QML to be prepared for easy migration and cross platform maintenance. Just the parts for the middle ware integration like information retrieval has to be specific.

Another approach is to use a finger print sensor. Is there already an API to receive finger print sensor events in an app? Or could we create such API? This would be very elegant without conflicts with the rest of the system. The API could even be useful for other use cases like an additional authentication for any encrypted content. The app itself wouldn't access to the finger print itself, only the event.

The so called super apps of Sailfish OS could also be an inspiration to enable some customization. Jolla call it partner space in the operating system. For updates there are no conflicts, just one or more registered repositories for an update.

AppLee

@marc_aurel
The actual solution would be to add the "volla launcher app" to the launcher (left bar).
This way no modification of Lomiri is required.

IMHO modularize Lomiri is a good idea, but I never looked under the hood.
From what people who have, this is not as simple as it could seem especially with convergence in mind.

What would be nice is a conception model so we can see the actual architecture of Lomiri.
Does someone has this kind of documentation or any idea how to make it ?
Retro engineering the documentation is not something easy but it might be very helpful to plan Lomiri's future.

dobey

@marc_aurel said in How to extend the system UI and UX?:

Another approach is to use a finger print sensor.

No, this would not be acceptable, as that is not what people use biometric interfaces for. Also, not all devices have fingerprint sensors.

kugiigi

I think the simplest and fastest approach is to have a customized version of Lomiri, at least for now, and change the function of the ubuntu button to open the sprongboard instead of the app drawer. App drawer can then still be be opened by a long left swipe.

potet

@dobey said in How to extend the system UI and UX?:

@marc_aurel said in How to extend the system UI and UX?:

Another approach is to use a finger print sensor.

No, this would not be acceptable, as that is not what people use biometric interfaces for. Also, not all devices have fingerprint sensors.

While I agree this needs to be optional, I do not at all get your point of it "not being acceptable". If I had this option I would absolutely take use it.

The sensor would only serve as a button, not read your fingerprint. One could use any finger to launch it. It's not about using your fingerprint to launch a service, it's about utilizing this button that is just hanging around at the back of phones doing nothing most of the time.

Furthermore, this is not unprecedented. In Android, there is an option to swipe the fingerprint sensor to pull down the notifications bar. It works like a charm except that it's absolutely worthless.

As for the fact that not every device has it: Hardware buttons could also be an option where available, and I would personally want the super key to open this functionality when a keyboard is connected. However, I fundamentally disagree that functionality should not be developed unless it would work on every piece of hardware out there. Following this logic development of convergence should never have taken place. Furthermore, this would not be a default feature in any case. Only having it available to install for users with compatible devices seems... normal.

If I missed your point on why it's unacceptable, please elaborate.

dobey

@potet said in How to extend the system UI and UX?:

The sensor would only serve as a button, not read your fingerprint. One could use any finger to launch it. It's not about using your fingerprint to launch a service, it's about utilizing this button that is just hanging around at the back of phones doing nothing most of the time.

It is not a button. It is a fingerprint sensor. If on some devices there is actually a button feature in conjuction with fingerprint sensor, the button can be handled separately and has nothing to do with fingerprint sensing.

Keneda

@dobey said in How to extend the system UI and UX?:

It is not a button. It is a fingerprint sensor.

My two cents : as i understood, his proposal is to use the sensor output in a binary way (touched = 1, not touched =0) to make it act like a button...

hummlbach

while i'm in favor of "make the app drawer replaceable" or lets have multiple app drawers side by side, we need to respect the concerns of lomiris maintainers. In the first place i understand their scepticism about who will carry the maintenance effort of a "pin app as drawer feature"
that said I would also appreciate and like to encourage the launcher modular devs to maybe help integrating some of lms features fixed into lomiri (although I know thats not quite their actual goal...)
My two cents on what most people actually want/miss in this whole home screen/launcher modular/pin apps discussion is a starting place, where news/recent data is aggregated and displayed as an overview and can then be further explored (i.e. apps are opened when their information abstract is clicked on.) @marc_aurel is springboard meant to be something like that?

dobey

@Keneda said in How to extend the system UI and UX?:

My two cents : as i understood, his proposal is to use the sensor output in a binary way (touched = 1, not touched =0) to make it act like a button...

But this is not how fingerprint sensors work. Yes, on some devices it is also combined with a button, but not on all. Also, not all devices that UT supports have a fingerprint sensor, nor any such button at all.

If Volla want to fork lomiri and build their own custom OS instead, that's fine, but we're talking about Ubuntu Touch here, and we need to keep in mind the goals of Ubuntu Touch and UBports when discussing development of Lomiri, and not only the goals of Volla or another third party that might want to use UT as merely a base to provide a different UX.

potet

@dobey said in How to extend the system UI and UX?:

[...] use UT as merely a base to provide a different UX.

I think this is an interesting point - is the idea to offer the springboard as an alternative UI altogether (similar to what Volla is doing with Android), or rather to provide it as an add-on/app that works on top of an otherwise normal Lomiri session?

As for the greater concern:
I think it could be fruitful to think about how extensions could allow users to change their UX without forking the entire project. A springboard could be more similar to adding a dock or Gnome Do to Gnome 2 than to change DE entirely. Of course this shouldn't distract from the aim and vision of UT/UBports, but some users will always want things differently. Allowing them to achieve this by installing or creating an extension would be great.

That being said, I'm ignorant to the technical details. If allowing for UX extensions would make developing UT significantly more difficult, of course it shouldn't be prioritized. And I doubt anyone would consider forking right now to be a good idea.

That being said, it Ubuntu Touch ends up being a popular daily driver in the Linux community, people will modify their user experience. I do believe there would be some benefit to allowing them to do so while remaining within the framework provided by UBports, rather than forking the system and coming up with creative hacks. Thinking about what this should look like sooner rather than later might be a good idea, and the possibility of a springboard is a good example.

marc_aurel

Thanks for your comments, concerns and recommendations.

I think about UT as a platform, because we think about further Volla devices including wearables like a UT watch of glasses. A similar approach has Sailfish OS with a proof of concept for a smart watch and feature phone by modifying Lipstick.

Back to the mid term plans:

@potet We want to introduce the Springboard to Volla devices and can confirm, that thy have a fingerprint sensor. I community members would install the Volla app on their they could open and use it the conventional way.

The idea came from someone at our Volla Community Days last week. I sounds very convenient and would avoid conflicts with system updates form UBports. We could focus on the development and maintenance of the Volla UX. So my question is: How could an app receive a fingerprint event? Is someone interested to work on a proof of concept? If you know someone, who could be interested, give me a note?

@AppLee How can a custom app become a default app in the sidebar if it is not running? I think, this was your recommendation, wasn't it?

Modifying Lomiri could be another option for sure, but I want to avoid conflicts with the default UT releases. I would prefer a kind of modularization for customization or plugin logic as a standard. Something like Super Apps and Partner Space (Sailfish OS) or Launcher or Kiosk Mode (Android). This could help to make UT attractive for OEMs and startups to make UT popular.

If someone is interested in a proof of concept. Let me know.

AppLee

@marc_aurel
Any app can be launched with the app drawer.
Once running, the app appears in the launcher (left bar).
Now a long press on the app icon opens a contextual menu allowing the user to "pin" the app to the launcher.

This way any user can customize the launcher and set its favorite/most used apps ready to start.

I guess reading your message that it's the easiest way. Springboard can be an app available in the open store and might probably be unconfined to have the necessary rights.

marc_aurel

@AppLee Ok. Thanks. Is it possible to do this programmatically?

The app wouldn't be available in the app store but exclusively available on our devices. Is it possible to add further repositories to the Store like F-Droid?

To implement a convenient UX we would need a one touch call of the springboard. Therefore I like the fingerprint idea.

Otherwise we would need to modify Lomiri but I would prefer standard customization optons.

dobey

@marc_aurel said in How to extend the system UI and UX?:

How could an app receive a fingerprint event? Is someone interested to work on a proof of concept?

Apps cannot use biometrics themselves, as Qt currently doesn't provide a Biometrics API in QML (even on Android).

@marc_aurel said in How to extend the system UI and UX?:

This could help to make UT attractive for OEMs and startups to make UT popular.

UT is not Android, and isn't trying to be like Android in this respect. Custom shell (ie, launchers as they are called on Android) is not doable without significant and very complex changes to the system. UT is about providing a whole OS with a specific design for convergence, not about providing a base platform for phone vendors to modify to their will like they might currently do with Android.

@marc_aurel said in How to extend the system UI and UX?:

Is it possible to add further repositories to the Store like F-Droid?

It is not.

kazord

@marc_aurel a think a good way to do it may be working on lomiri idle background and transform it to display a 'launcher' app content instead of a simple image.
As it's not a priority of ubports team, you should do the work and make a pull request. I already made some test and it's look like to be the more convenient way to extend the launcher.

marc_aurel

@dobey The app wouldn't need the biometrics information itself, just an event, that the sensor was touched. So the critical information would be protected.

For other use cases biometric confirmation, encapsuled in the middleware, could be an interesting feature, for example to unlock a protected data source or to confirm any transaction.

Where is the component to get this event? The middleware needs this event either?

@kazord Last year a developer has shown in our hackathon, that is s quite simple to exchange the system UI by modifying some QML files. He presented the nice grid style ofr open apps like Nemo Mobile and Sailfish OS instead of the current fan style in Ubuntu Touch.

My hope was, just to modify a QML for the sidebar with a Loader, that loads the launcher app instead of the current sidebar logic. To avoid conflicts with the default system this part could be dynamically controlled as a standard - with any setting or an installed app with specific privileges.

My idea in general is to provide customization of the system in a standard way. Sailfish OS could be a good example for this approach. OEMs like Intex and Jala have used the Standard Sailfish OS with some customization for their market. The idea of a Super App is quite smart. It is a app, that can be called with one swipe gesture. It can be controlled by a setting.

Anyway. If someone is interested to help us with a proof of concept. Let me know.

AppLee

@marc_aurel
That's a big topic because in people's mind (and mine first) it seems easy to just replace a GUI for another.

But Ubuntu Touch is build for both privacy and convergence.
For privacy reasons apps are confined and for convergence, all the system UI has to be thought for phones, tablets and desktop size screen.

In addition to that, Ubuntu Touch is made as a daily driver and is updated via an image.
The locked rootfs is there to warn people that updates will wipe out any modification made on the rootfs.

Hacking the system to replace a QML is easy, but providing updates and keeping this hack is a security breach.

About the fingerprint sensor, if I understood correctly what @dobey explained, I guess that depending on how the sensor is "seen" by the OS you might not be able to access the data coming from it.
If seen as a physical button maybe it could work, but seen as what is really is (a biometric sensor) it won't be usable.