Hello Everyone,

I'm starting this thread because I wanted to share my ideas about improving the OSK in UT.
Back in the Canonical days, I suggested to use the space key for controlling the text cursor.
It was a real joy when I learned that Canonical did implemented it which we can now enjoy with UBports.
And I want continue with that idea and improve or extend its functionality.

At the moment, I have two prototypes.

1. [Swipe to trigger cursor mover]
   Instead of the current "Press and Hold" gesture to trigger the cursor mover, I made it to work with just a horizontal swipe in the space key. For me, this gesture is more natural while typing unlike the current one which will some kind of disrupt your flow by long pressing before swiping. There's a catch though because there's more chance of accidental trigger especially for those who tend to drag fingers while typing. My experience so far is good though and did an accidental trigger just once.
   Demo Video

2. [Swipe to switch OSK language]
   This one adds a functionality to change OSK language via swipe in the space key. Some people needs multiple OSK layout/language enabled and it could be a bit hassle to switch between them especially if you have more than 2...remember emoji? yeah that counts I personally don't have this problem so I can''t attest if this one would be really helpful. And also, at the moment, emoji kinda breaks it because it has a very different layout which doesn't have the space key. I'm also planning to propose a new layout for emoji
   Demo Video

Now I don't have a concrete plan yet on how I will propose these. I'm thinking of adding a Gesture section in the settings app so users can customize the gestures and which functionality they want for them. For example, you can set "PressAndHold" for Cursor Mover then "Horizontal Swipe" for Language switch. But that could be too much settings? can't say yet.

EXPAND IT FURTHER!
Currently, I have another idea to expand this idea further and I'm trying to do a quick hack for it.
The idea is to also use the "cursor mover" feature for text selections/manipulations. My initial thought is to use double click/tap (without releasing) to start the text selection. I would also like to put Copy, paste, select all buttons above the OSK. I want to see and use it first in action before thinking of the best way to implement these gestures. Unfortunately, I can't even do the first step yet which is to enable text selection from the OSK and it seems like I have to modify the C++ part of maliit and recompile which I am not familiar enough.

Any thoughts, opinion, violent reaction, help maybe?

FUN FACT: When I got the idea of the "Space key cursor mover", it was a "Eureka" moment for me and I thought I was the first one in the world to come up with that idea. Then days after that, I saw the exact same functionality on my friend's LG G3. At least it was my original idea and it just happened someone got it first LOL

So the OSK theme landed a few OTAs back. Awesome right?!
And now I'm back with my OSK works....finally!....specifically with the advanced text functionalities in the OSK.
I've prototyped and demoed this almost half a year ago but now I want to make it "PR-ready".
So once I get satisfied, I will propose this officially.

List of changes and additional features

• Added audio and haptic feedback to the word ribbon (auto suggest bar)
• Temporary fix for the issue where cursor mover isn't working correctly in QtWebEngine text fields such as in the Morph Browser (some trade-offs to fix this temporarily)
• Horizontal movement in the cursor mover will now move vertically once the start/end of line is reached. I'm not sure how this really affects the experience but it was definitely implemented like that before for a reason. (a trade-off for the issue above)
• A new way to trigger the cursor mover mode (bottom swipe). This is available anywhere in the OSK so regardless of the layout or type, you can now trigger it. Press and hold in the space bar is still available.
• Implemented text manipulation features which are available in a top bar that shows up when in cursor mover mode.
  • Copy
  • Cut
  • Paste
  • Select All
  • Undo
  • Redo
• Implemented text selection mode (no timeout, needs manually exiting)
  • While in cursor mover mode, double tap/click to enter selection mode. You can also immediately select texts by double tapping then swiping immediately without lifting your finger.
  • Word Selection: Double tap while in cursor mover mode without swiping [Disabled for now to fix QtWebEngine issue]
• Added floating buttons in cursor mover mode
  • Move cursor to the start/end of line or document
  • They can also be used in conjunction with the selection mode!
  • Done button: Used to end selection mode and/or cursor mover mode
• Changed timeout of cursor mover mode to 5 seconds (experimental)

Know Issues:

• The new toolbar buttons and floating buttons sometimes retain their previous color when changing OSK theme.
  FIXED: Just needed to bind the colors

• Using the Start of line/document button from the floating buttons while in selection mode doesn't select texts. I'm not really sure why.
  FIXED: Stupid mistake by me....it worked in my first branch and just had a mistake with this new branch

• The bottom swipe trigger might be difficult to trigger in landscape mode especially for devices with protrusion on the sides/bezels especially those with cases.

Please feel free to try it out from the git branch linked below. You'd need to use crossbuilder to compile it and deploy to your device via ADB.
https://github.com/kugiigi/keyboard-component/tree/xenial_-_textfunctions

NOTE: BEWARE EDGE CHANNEL USERS. This won't work on edge because there's a separate branch for it which I don't have a modified fork yet. If there's plenty of interested then I might consider

Suggestions and feedback will also be greatly appreciated especially on the design

Some screenshots

Cursor mover mode

Selection Mode

@grenudi Because Ubuntu Touch is Lomiri. Without Lomiri, Ubuntu Touch loses its identity and loses its purpose. When the community/foundation picked up the project, it picked up the whole Ubuntu Touch project and continued its goal.
In an ideal world, we all want the same things but in reality we want different things and that's the reason we have so many DEs and distros. One of the main goal of Ubuntu Touch and Lomiri is convergence. I know other distros also kind of touches that concept too but it's pretty obvious that UT's direction is very much different. Most of us UT users obviously like Lomiri and that's why we continue to support it and contribute to it.

Morph can only open files from the Downloads folder. Try copying them there.

This guide isn't created by the porter/maintainer of the port. This is only based on my personal experience, specifically on S10 and hopefully it can be helpful for those that needs step by step and more detailed guide than the current ones. Please feel free to correct or add anything in this guide. Hopefully we'll get these devices in the installer soon.

You may also join the telegram group to ask questions/help and/or check the pinned messages.

My device

• S10 (beyond1lte)
• SM-G973F/DS

Starting point

• Locked bootloader
• Security update: 01/2022 with a pending update
• Android 12 - OneUI 4.0

Step by step

A. Unlock the Bootloader:
Source: https://forum.xda-developers.com/t/recovery-official-3-6-1-x-twrp-for-galaxy-s10-e-5g-exynos.4180287/

1. Make sure OEM Unlock is ON in developer settings (How To)
2. Turn off the phone
3. While pressing Vol Down + Bixby key, plug the device to a PC via USB cable to enter download mode;
4. Long press Vol Up then confirm you want to unlock bootloader (this is needed one time only)
   * Please be aware all your data will be erased including internal storage and phone will reboot!!
5. Let the phone boot normally. Complete Android setup and connect to internet (wireless or mobile data) in order to unlock RMM (Not sure what this is)
6. Make sure OEM Unlock is ON in developer settings
7. Turn off the phone
8. Repeat Step 3 to boot to Download mode then short press Vol Up to confirm you want to go to download mode
   * If by any chance you see RMM state=Prenormal or KG state=Prenormal, redo from step 5!

B. Flash TWRP Recovery:
(You may skip to flashing LOS recovery but I'm not sure how to use that to flash zip files and wiping partitions)

1. Download appropriate TWRP .img file according to your device.

2. Download avb disabled vbmeta .img

3. While your device is in Download mode, open your preferred terminal in linux and run the following (replace texts enclosed by <> with the path of downloaded img files):
   sudo heimdall flash --RECOVERY </path/to/twrp.img> --VBMETA </path/to/vbmeta.img>

   After pressing enter to run the command, grab the phone and press Vol Up + Bixby + Power to force it to boot straight to TWRP after flashing
   Otherwise, press the same buttons while the device is turned off to boot to recovery.

!! Only if coming from full stock firmware

• In TWRP home screen, press Wipe > Format data > yes
• Download and flash multidisabler-samsung-3.* zip to disable security and encryption (Install > Choose zip file > Swipe to start flashing)

C. Flash LOS Recovery:

1. Download recovery image from LineageOS
   Sample file name: lineage-18.1-20220311-recovery-beyond1lte.img
2. While your device is in Download mode, run the following command
   sudo heimdall flash --RECOVERY </path/to/LOS_recovery.img> --VBMETA </path/to/vbmeta.img>

D. _Flash Ubuntu Touch:
(This is using LOS recovery. I think you may flash with TWRP too but since updates are done by manually flashing, fastboot may be better and simpler instead of transferring files to the device for flashing via TWRP)

1. Download the latest artifacts.zip file by clicking on the action button on the right most side of the latest passed pipeline item. Select the appropriate item for your device.
   Example: devel-flashable-beyond<0/1/2>lte:archive
2. Download vendor image for your device.
3. Boot to recovery
4. Go to Advanced > Enter fastboot
5. Plug the device to your PC.
6. Run the following commands. Replace the texts enclosed by <> with the correct paths and file name.

   fastboot flash boot </path/to/boot.img>
   fastboot flash system </path/to/ubuntu.img>
   fastboot flash vendor </path/to/vendor-ut-beyond<0/1/2>lte.img>
   

7. Reboot and enjoy UT!

Hi,
I'm Kugi and I'm just mimicing this post
I'm posting this to introduce a new tool that I DON'T RECOMMEND to anyone but anyone is free to use No seriously, it can be dangerous to your system so it's important that you know what you are doing and know how to reflash in case of breakage.

Introducing the Jerk Installer!

What is Jerk?

No, it's not an offensive word. It's a physics term just like Snap, Crackle and Pop
Jerk is a script to easily install and uninstall customization on the system (rootfs). It's mostly only applicable on patches that involve text-based files like QML files. I haven't tested with binaries but maybe in the future?

What kind of customization?

Many things! Such as UI changes in Lomiri, new features in the keyboard, and Bluetooth configuration changes. Those are just the ones I came up with but there are more possibilities.

How do I use it?

I won't repeat myself here. You can go to the repo and read the....README
But basically, just download the whole repo, make the jerk script executable and run ./jerk --help. There are sample packages included in the repo.

Sample Packages

I've included many packages in the repo that you can try out. These are mostly the customization and experiments I've done in my devices. I highly recommend the package Lomiri_Ubuntu_22.04.tar.gz. 20.04 (focal) transition is taking a really long time to arrive so let's just skip and go straight to the latest Ubuntu
You can run ./jerk describe <packagename> to check what each package does.

It blocks OTA updates

Yup, it automatically blocks OTA updates until you uninstall all the packages you installed and/or intentionally unblocks it with ./jerk unblock-ota or reset all components with ./jerk reset all. I made this decision so users can avoid installing OTA updates that can possibly have conflicts with the installed packages. It's a good practice to remove all packages before updating then just reinstall after updating. Jerk will tell you if the package is still compatible or not.

I wanna create my own package!

Sure, why not? If you're like me who likes experimenting stuffs and want to share them to others, you can create your own package. Just check the...README
For the target component/path, it'll be better to make a PR to create new configuration if it's not present yet so the script is always aware of all the components with installed packages.

Is it perfect?

YES! No! Of course no
I'm not an expert with bash scripting and the script is a bit simplistic in terms of checking compatibility and no proper way to remember the installed packages. As always, PRs welcome

Can I trust you?

That's up to you

Can I see some screenshots of sample customization/patches?

NO!

This is kind of an update on this post. It's been a year and I've kind of improved and stabilized the tool (Jerk Installer and Jerk Packages) I created, or I'd like to think that Well, I do use it on basically all my devices so I guess it's proven now?

Most importantly, I've streamlined the hacks/packages that I've been developing and maintaining. I've combined them per component and just added settings so you can customize which things you want enabled or disabled. The main 2 ones are the following.

Malakiboard

• A collection of hacks for UT's onscreen keyboard.
• Notable new features and functions are custom keyboard height, floating mode, custom theme, and theme that follows system theme.

Lomiri Plus

• A collection of hacks for Lomiri, UT's desktop environment.
• Notable new features and functions are notch and rounded corners fix, manual rotation button, and so much more to mention and hard to explain too

IMPORTANT NOTES!

• DANGER!: This tool can be dangerous to your system. It can render your device unusable, though it has never happened to me yet and only during development . You can always reflash with the installer to reset the whole system and just make sure to NOT select the Wipe option.
• BE SAFE!: I haven't found a way to automatically create the packages so you'll have to create them manually. I highly discourage downloading and installing pre-created packages from anyone, even from me! although I do that in Telegram LOL. It's easy to manipulate the contents so it's better to be safe. As always, it's best to download from the original source.
• BE RESPONSIBLE!: To avoid creating unnecessary logged issues on the official UBports repos, make sure to reset your system first and reconfirm the bug you're encountering before filling them in Gitlab/Github. Resetting is easy. You can even reset all with the command jerk reset all
• HAVE FUN!: Enjoy all my crazy experiments, ideas, concepts, whatever they are. Let me know if you have comments or any feedback about them. The hope is to have some of them implemented officially

Yes, Dr. Strange knows what will happen
But seriously, apt isn't advisable to be used on UT so enabling a whole repository may cause even more havoc to the system.

I think it would be better to get the opinion of UBports people especially the design team before doing any serious effort on actually implementing these designs. I would be willing on helping on this

I think for fast typers UT's keyboard definitely would feel slower and I guess it also depends on the device and port. There are optimizations and improvements that can be done.
At the moment, autocorrect isn't recommended because it kinda suck You can still use word suggestions though which can be helpful.

@RJDan Oh okay. I guess you're on Noble and only the Essentials version is available for now The full version is available from the repo but it will be released in Ambot soon

It's recommended to install the correct Android version before flashing UT. This is to avoid unexpected issues.

In my experience on both FP4 and FP5, fingerprint reader may not work initially so you'll have to reinstall Android and reflash UT.

@RJDan You mean in Lomiri Plus?
It's under Convergence > External Display.
The settings can be accessed via the Power/System Indicator Panel.

@RJDan Interesting. I guess if the hardware supports this then it could work in UT. I wonder if Android phones could do this.

In UT however, single monitor might be the default behavior on phones and tablets. In Lomiri Plus via Ambot Installer, there's an option to use multi-display mode if you really want to try. I don't know if it'll work though and how well it'll work. The device screen will be a normal screen In this mode by the way, it won't be a touchpad.

@Alain This featute isn't in UT/Lomiri yet. Simplest way is to make something like a home screen app and there are some that exists already like Launcher Modular. Not sure they have app groups though. You can contribute to them if you don't want to make your own. However, they obviously act like a normal app. Best you can do is pin it to the Launcher for easy access.

Or you can contribute to Lomiri and implement it in the App drawer which definitely needs some love
There's even a bounty for it so there's money to get

@gpatel-fr That won't help in my case. My issue is that I'm used to flicking quickly when switching between two apps and that doesn't work well on the FP5

Unfortunately, this is not yet possible. Something like this was developed by Canonical but it wasn't finished so it was removed. So far now one has implemented it yet although there's a bounty for it now.

@fair FP4 actually has the one of the most responsive edge gestures in my expert. FP5 on the other hand is less responsive because it seems like it has an accidental touch algorithm that rejects quick short swipe from the edges. While on FP4, your swipe reacts to your swipe almost instantaneously. I wonder why you have a hard time. Perhaps you use a case which might make ot harder to reach the very edge or maybe the phone is just to big for one hand use?

In any case, I think a floating button isn't a bad idea for an accessibility feature. However, there are things you listed that might not be possible or at least not easily. For example, showing the keyboard cannot be a toggle because the onscreen keyboard shows up based on context such as a texbox being in focus. I guess there might be a workaround like utilizing the logic for hiding the keyboard when there's a physical keyboard connected.

Another thing is that back action concept doesn't exists in UT or in Linux systems in general. So I doubt you can have something that would work for everything. Even Plasma Mobile which mimics Android's bottom navigation bar doesn't have a back button. This one might be the trickiest thing to solve.

So in summary, it's possible to do it for some functions like opening the app drawer or app spread/switcher but other things wouldn't be simple.

If you're interested, you can try some of my hacks that makes UT easier to use one hand. Ambot Installer is available on OpenStore and you can use it to install my hacks. It's already available on Noble however my biggest hack called Lomiri Plus isn't available yet, at least via Ambot.

The hacks I have are mostly gestures though so you might not like it too
For example, you swipe down on the edges to pull down the whole shell so you can reach UI elements at the top. You can also do horizontal swipes at the bottom edge in apps to trigger the back button or even any if the page header actions. I also have what I call Quick Actions which can triggered via a swipe from the bottom of the left or right edge. Here you can have actions similar to what you mentioned and you can also add apps there.

This is awesome! Can't wait for the future of Lomiri