Hello Everyone,

I'm starting this thread because I wanted to share my ideas about improving the OSK in UT.
Back in the Canonical days, I suggested to use the space key for controlling the text cursor.
It was a real joy when I learned that Canonical did implemented it which we can now enjoy with UBports.
And I want continue with that idea and improve or extend its functionality.

At the moment, I have two prototypes.

1. [Swipe to trigger cursor mover]
   Instead of the current "Press and Hold" gesture to trigger the cursor mover, I made it to work with just a horizontal swipe in the space key. For me, this gesture is more natural while typing unlike the current one which will some kind of disrupt your flow by long pressing before swiping. There's a catch though because there's more chance of accidental trigger especially for those who tend to drag fingers while typing. My experience so far is good though and did an accidental trigger just once.
   Demo Video

2. [Swipe to switch OSK language]
   This one adds a functionality to change OSK language via swipe in the space key. Some people needs multiple OSK layout/language enabled and it could be a bit hassle to switch between them especially if you have more than 2...remember emoji? yeah that counts I personally don't have this problem so I can''t attest if this one would be really helpful. And also, at the moment, emoji kinda breaks it because it has a very different layout which doesn't have the space key. I'm also planning to propose a new layout for emoji
   Demo Video

Now I don't have a concrete plan yet on how I will propose these. I'm thinking of adding a Gesture section in the settings app so users can customize the gestures and which functionality they want for them. For example, you can set "PressAndHold" for Cursor Mover then "Horizontal Swipe" for Language switch. But that could be too much settings? can't say yet.

EXPAND IT FURTHER!
Currently, I have another idea to expand this idea further and I'm trying to do a quick hack for it.
The idea is to also use the "cursor mover" feature for text selections/manipulations. My initial thought is to use double click/tap (without releasing) to start the text selection. I would also like to put Copy, paste, select all buttons above the OSK. I want to see and use it first in action before thinking of the best way to implement these gestures. Unfortunately, I can't even do the first step yet which is to enable text selection from the OSK and it seems like I have to modify the C++ part of maliit and recompile which I am not familiar enough.

Any thoughts, opinion, violent reaction, help maybe?

FUN FACT: When I got the idea of the "Space key cursor mover", it was a "Eureka" moment for me and I thought I was the first one in the world to come up with that idea. Then days after that, I saw the exact same functionality on my friend's LG G3. At least it was my original idea and it just happened someone got it first LOL

So the OSK theme landed a few OTAs back. Awesome right?!
And now I'm back with my OSK works....finally!....specifically with the advanced text functionalities in the OSK.
I've prototyped and demoed this almost half a year ago but now I want to make it "PR-ready".
So once I get satisfied, I will propose this officially.

List of changes and additional features

• Added audio and haptic feedback to the word ribbon (auto suggest bar)
• Temporary fix for the issue where cursor mover isn't working correctly in QtWebEngine text fields such as in the Morph Browser (some trade-offs to fix this temporarily)
• Horizontal movement in the cursor mover will now move vertically once the start/end of line is reached. I'm not sure how this really affects the experience but it was definitely implemented like that before for a reason. (a trade-off for the issue above)
• A new way to trigger the cursor mover mode (bottom swipe). This is available anywhere in the OSK so regardless of the layout or type, you can now trigger it. Press and hold in the space bar is still available.
• Implemented text manipulation features which are available in a top bar that shows up when in cursor mover mode.
  • Copy
  • Cut
  • Paste
  • Select All
  • Undo
  • Redo
• Implemented text selection mode (no timeout, needs manually exiting)
  • While in cursor mover mode, double tap/click to enter selection mode. You can also immediately select texts by double tapping then swiping immediately without lifting your finger.
  • Word Selection: Double tap while in cursor mover mode without swiping [Disabled for now to fix QtWebEngine issue]
• Added floating buttons in cursor mover mode
  • Move cursor to the start/end of line or document
  • They can also be used in conjunction with the selection mode!
  • Done button: Used to end selection mode and/or cursor mover mode
• Changed timeout of cursor mover mode to 5 seconds (experimental)

Know Issues:

• The new toolbar buttons and floating buttons sometimes retain their previous color when changing OSK theme.
  FIXED: Just needed to bind the colors

• Using the Start of line/document button from the floating buttons while in selection mode doesn't select texts. I'm not really sure why.
  FIXED: Stupid mistake by me....it worked in my first branch and just had a mistake with this new branch

• The bottom swipe trigger might be difficult to trigger in landscape mode especially for devices with protrusion on the sides/bezels especially those with cases.

Please feel free to try it out from the git branch linked below. You'd need to use crossbuilder to compile it and deploy to your device via ADB.
https://github.com/kugiigi/keyboard-component/tree/xenial_-_textfunctions

NOTE: BEWARE EDGE CHANNEL USERS. This won't work on edge because there's a separate branch for it which I don't have a modified fork yet. If there's plenty of interested then I might consider

Suggestions and feedback will also be greatly appreciated especially on the design

Some screenshots

Cursor mover mode

Selection Mode

@grenudi Because Ubuntu Touch is Lomiri. Without Lomiri, Ubuntu Touch loses its identity and loses its purpose. When the community/foundation picked up the project, it picked up the whole Ubuntu Touch project and continued its goal.
In an ideal world, we all want the same things but in reality we want different things and that's the reason we have so many DEs and distros. One of the main goal of Ubuntu Touch and Lomiri is convergence. I know other distros also kind of touches that concept too but it's pretty obvious that UT's direction is very much different. Most of us UT users obviously like Lomiri and that's why we continue to support it and contribute to it.

This is kind of an update on this post. It's been a year and I've kind of improved and stabilized the tool (Jerk Installer and Jerk Packages) I created, or I'd like to think that Well, I do use it on basically all my devices so I guess it's proven now?

Most importantly, I've streamlined the hacks/packages that I've been developing and maintaining. I've combined them per component and just added settings so you can customize which things you want enabled or disabled. The main 2 ones are the following.

Malakiboard

• A collection of hacks for UT's onscreen keyboard.
• Notable new features and functions are custom keyboard height, floating mode, custom theme, and theme that follows system theme.

Lomiri Plus

• A collection of hacks for Lomiri, UT's desktop environment.
• Notable new features and functions are notch and rounded corners fix, manual rotation button, and so much more to mention and hard to explain too

IMPORTANT NOTES!

• DANGER!: This tool can be dangerous to your system. It can render your device unusable, though it has never happened to me yet and only during development . You can always reflash with the installer to reset the whole system and just make sure to NOT select the Wipe option.
• BE SAFE!: I haven't found a way to automatically create the packages so you'll have to create them manually. I highly discourage downloading and installing pre-created packages from anyone, even from me! although I do that in Telegram LOL. It's easy to manipulate the contents so it's better to be safe. As always, it's best to download from the original source.
• BE RESPONSIBLE!: To avoid creating unnecessary logged issues on the official UBports repos, make sure to reset your system first and reconfirm the bug you're encountering before filling them in Gitlab/Github. Resetting is easy. You can even reset all with the command jerk reset all
• HAVE FUN!: Enjoy all my crazy experiments, ideas, concepts, whatever they are. Let me know if you have comments or any feedback about them. The hope is to have some of them implemented officially

This guide isn't created by the porter/maintainer of the port. This is only based on my personal experience, specifically on S10 and hopefully it can be helpful for those that needs step by step and more detailed guide than the current ones. Please feel free to correct or add anything in this guide. Hopefully we'll get these devices in the installer soon.

You may also join the telegram group to ask questions/help and/or check the pinned messages.

My device

• S10 (beyond1lte)
• SM-G973F/DS

Starting point

• Locked bootloader
• Security update: 01/2022 with a pending update
• Android 12 - OneUI 4.0

Step by step

A. Unlock the Bootloader:
Source: https://forum.xda-developers.com/t/recovery-official-3-6-1-x-twrp-for-galaxy-s10-e-5g-exynos.4180287/

1. Make sure OEM Unlock is ON in developer settings (How To)
2. Turn off the phone
3. While pressing Vol Down + Bixby key, plug the device to a PC via USB cable to enter download mode;
4. Long press Vol Up then confirm you want to unlock bootloader (this is needed one time only)
   * Please be aware all your data will be erased including internal storage and phone will reboot!!
5. Let the phone boot normally. Complete Android setup and connect to internet (wireless or mobile data) in order to unlock RMM (Not sure what this is)
6. Make sure OEM Unlock is ON in developer settings
7. Turn off the phone
8. Repeat Step 3 to boot to Download mode then short press Vol Up to confirm you want to go to download mode
   * If by any chance you see RMM state=Prenormal or KG state=Prenormal, redo from step 5!

B. Flash TWRP Recovery:
(You may skip to flashing LOS recovery but I'm not sure how to use that to flash zip files and wiping partitions)

1. Download appropriate TWRP .img file according to your device.

2. Download avb disabled vbmeta .img

3. While your device is in Download mode, open your preferred terminal in linux and run the following (replace texts enclosed by <> with the path of downloaded img files):
   sudo heimdall flash --RECOVERY </path/to/twrp.img> --VBMETA </path/to/vbmeta.img>

   After pressing enter to run the command, grab the phone and press Vol Up + Bixby + Power to force it to boot straight to TWRP after flashing
   Otherwise, press the same buttons while the device is turned off to boot to recovery.

!! Only if coming from full stock firmware

• In TWRP home screen, press Wipe > Format data > yes
• Download and flash multidisabler-samsung-3.* zip to disable security and encryption (Install > Choose zip file > Swipe to start flashing)

C. Flash LOS Recovery:

1. Download recovery image from LineageOS
   Sample file name: lineage-18.1-20220311-recovery-beyond1lte.img
2. While your device is in Download mode, run the following command
   sudo heimdall flash --RECOVERY </path/to/LOS_recovery.img> --VBMETA </path/to/vbmeta.img>

D. _Flash Ubuntu Touch:
(This is using LOS recovery. I think you may flash with TWRP too but since updates are done by manually flashing, fastboot may be better and simpler instead of transferring files to the device for flashing via TWRP)

1. Download the latest artifacts.zip file by clicking on the action button on the right most side of the latest passed pipeline item. Select the appropriate item for your device.
   Example: devel-flashable-beyond<0/1/2>lte:archive
2. Download vendor image for your device.
3. Boot to recovery
4. Go to Advanced > Enter fastboot
5. Plug the device to your PC.
6. Run the following commands. Replace the texts enclosed by <> with the correct paths and file name.

   fastboot flash boot </path/to/boot.img>
   fastboot flash system </path/to/ubuntu.img>
   fastboot flash vendor </path/to/vendor-ut-beyond<0/1/2>lte.img>
   

7. Reboot and enjoy UT!

Morph can only open files from the Downloads folder. Try copying them there.

Hi,
I'm Kugi and I'm just mimicing this post
I'm posting this to introduce a new tool that I DON'T RECOMMEND to anyone but anyone is free to use No seriously, it can be dangerous to your system so it's important that you know what you are doing and know how to reflash in case of breakage.

Introducing the Jerk Installer!

What is Jerk?

No, it's not an offensive word. It's a physics term just like Snap, Crackle and Pop
Jerk is a script to easily install and uninstall customization on the system (rootfs). It's mostly only applicable on patches that involve text-based files like QML files. I haven't tested with binaries but maybe in the future?

What kind of customization?

Many things! Such as UI changes in Lomiri, new features in the keyboard, and Bluetooth configuration changes. Those are just the ones I came up with but there are more possibilities.

How do I use it?

I won't repeat myself here. You can go to the repo and read the....README
But basically, just download the whole repo, make the jerk script executable and run ./jerk --help. There are sample packages included in the repo.

Sample Packages

I've included many packages in the repo that you can try out. These are mostly the customization and experiments I've done in my devices. I highly recommend the package Lomiri_Ubuntu_22.04.tar.gz. 20.04 (focal) transition is taking a really long time to arrive so let's just skip and go straight to the latest Ubuntu
You can run ./jerk describe <packagename> to check what each package does.

It blocks OTA updates

Yup, it automatically blocks OTA updates until you uninstall all the packages you installed and/or intentionally unblocks it with ./jerk unblock-ota or reset all components with ./jerk reset all. I made this decision so users can avoid installing OTA updates that can possibly have conflicts with the installed packages. It's a good practice to remove all packages before updating then just reinstall after updating. Jerk will tell you if the package is still compatible or not.

I wanna create my own package!

Sure, why not? If you're like me who likes experimenting stuffs and want to share them to others, you can create your own package. Just check the...README
For the target component/path, it'll be better to make a PR to create new configuration if it's not present yet so the script is always aware of all the components with installed packages.

Is it perfect?

YES! No! Of course no
I'm not an expert with bash scripting and the script is a bit simplistic in terms of checking compatibility and no proper way to remember the installed packages. As always, PRs welcome

Can I trust you?

That's up to you

Can I see some screenshots of sample customization/patches?

NO!

Yes, Dr. Strange knows what will happen
But seriously, apt isn't advisable to be used on UT so enabling a whole repository may cause even more havoc to the system.

Some progress for the themes support hurray!

Here are the repositories I've uploaded which so far works quite well except the settings-app which I was not able to build myself but I've only changed QML and those worked fine when I tested directly on my phone
Keyboard component
Settings app

My initial plan was just create a dark mode for the OSK but since the foundation for the colors is already there so I experimented with different themes and these are the ones I've come up.
(I don't know how to shrink the photos here :P)

Ambiance (Default)

Suru Dark

Suru Black

Just White

Just Black

Just Grey

Bordered White

Bordered Black

Bordered Grey

Colored ones are also possible but I did not include any in the repo but here's a sample

I don't know if this will be accepted but if yes, @CiberSheep I would like get the opinion of the design team and you can check the colors.
They're just in json files in the repo and can be easily modified

And BTW, this completely removed the border images and replaced with rectangles with borders

@Yumi It can be done but unfortunately there are still issues that would make it not a good experience. One of it is mouse scrolling. It doesn't work properly in all browser tabs in the default browser. It's also a bit too fast in native apps. It's just finicky in general which ruins the experience.

Also, support for traditional desktop apps is not complete yet. They mostly run in Xwayland which isn't hardware accelerated and has a few things not working properly.

But if you just want to try it and maybe one day use it when it's actually usable, Fairphone 4 and Fairphone 5 are the popular ones that support external diaplay via the type-c port.

@Averiz Do we need to reinstall the daemon when there's a new app version or for example this latest version with battery optimizations?

@Averiz I have the first version since yesterday and to be fair the battery seems good. I didn't use that device much though

@Averiz I haven't updated my custom indicators for Noble yet so not sure how they work now but I think it is indeed a good thing to explore. You can have UI there to manage connections and such. And I believe you can also have something running all the time as replacement to the daemon. I am however not an expert with it so I might be wrong

@Averiz This is awesome. I'm gonna test this on my extra/test devices. Yes this is unconfined and has many security/privacy concerns but this is still a progress and hopefully, we can slowly implement things in better (confined) ways especially on the system's part

@uwahl I understand your concern but that is sadly not related to my post I'm talking about the virtual touchpad when connected to an external display.

Have you tried reinstalling the official images from Fairphone?
Note that you should install the Android 13 version.

@oldbutndy Just to set the expectation. Do not expect much from the external display experience. At the moment, it's kind of regressed compared to xenial and older versions. It has been better now in Noble but it's still have lingering issues and the most notable and experience breaking one is mouse scrolling. It doesn't work properly in the browser (only one tab or webview works) and scrolling in native apps is too "momentum"-ish
In any case, it's still fun to use from time to time. In fact, I'm using it right now to test some things

First and foremost, no I'm not implementing proper touchpad support in Lomiri. Mainly because I don't know how
I initially thought QML has a module or component for it but turns out there's none. I guess touchpads aren't as standardized as mouse?

Anyway, I'm actually experimenting with the virtual touchpad instead since it'll be much easier for my skill level. However, I'm not exactly a touchpad user (on my Steam Controller yes LOL). I rarely use a laptop and when I do, I use an external mouse. So I don't really know what's expected from it especially the modern ones.

Here's a video showcasing the gestures I experimented with.
https://youtu.be/xDXpPVnWaGQ?si=8qxr1PWXGkqINH8N

Basically, I tried to mimic the current touchscreen gestures in Lomiri but not sure if they would actually be useful. I'm thinking, it's nice to move and resize windows with a gesture but it might be better to be able to quickly open the drawer or switch between apps using the 3-finger gestures instead.

Unlike most of my experiments, I'm confident that I can actually propose this to upstream Lomiri since it doesn't really need skills that I don't have. So I would like to ask for people's opinion on this and maybe this would be basis once proper touchpad support were implemented.

One last thing, you may suggest pinch to zoom and such and although that's cool, Samsung DeX has it, but I don't know how to implement that LOL
I'm talking more on simple swipe and taps that

Thanks!

I don't understand why they even care where you bought it. Unlocking bootloader factory resets the device right?
It's not like you're using it to access data from a stolen device..And you'll need access on the device to even allow unlocking