Hello Everyone,

I'm starting this thread because I wanted to share my ideas about improving the OSK in UT.
Back in the Canonical days, I suggested to use the space key for controlling the text cursor.
It was a real joy when I learned that Canonical did implemented it which we can now enjoy with UBports.
And I want continue with that idea and improve or extend its functionality.

At the moment, I have two prototypes.

1. [Swipe to trigger cursor mover]
   Instead of the current "Press and Hold" gesture to trigger the cursor mover, I made it to work with just a horizontal swipe in the space key. For me, this gesture is more natural while typing unlike the current one which will some kind of disrupt your flow by long pressing before swiping. There's a catch though because there's more chance of accidental trigger especially for those who tend to drag fingers while typing. My experience so far is good though and did an accidental trigger just once.
   Demo Video

2. [Swipe to switch OSK language]
   This one adds a functionality to change OSK language via swipe in the space key. Some people needs multiple OSK layout/language enabled and it could be a bit hassle to switch between them especially if you have more than 2...remember emoji? yeah that counts I personally don't have this problem so I can''t attest if this one would be really helpful. And also, at the moment, emoji kinda breaks it because it has a very different layout which doesn't have the space key. I'm also planning to propose a new layout for emoji
   Demo Video

Now I don't have a concrete plan yet on how I will propose these. I'm thinking of adding a Gesture section in the settings app so users can customize the gestures and which functionality they want for them. For example, you can set "PressAndHold" for Cursor Mover then "Horizontal Swipe" for Language switch. But that could be too much settings? can't say yet.

EXPAND IT FURTHER!
Currently, I have another idea to expand this idea further and I'm trying to do a quick hack for it.
The idea is to also use the "cursor mover" feature for text selections/manipulations. My initial thought is to use double click/tap (without releasing) to start the text selection. I would also like to put Copy, paste, select all buttons above the OSK. I want to see and use it first in action before thinking of the best way to implement these gestures. Unfortunately, I can't even do the first step yet which is to enable text selection from the OSK and it seems like I have to modify the C++ part of maliit and recompile which I am not familiar enough.

Any thoughts, opinion, violent reaction, help maybe?

FUN FACT: When I got the idea of the "Space key cursor mover", it was a "Eureka" moment for me and I thought I was the first one in the world to come up with that idea. Then days after that, I saw the exact same functionality on my friend's LG G3. At least it was my original idea and it just happened someone got it first LOL

So the OSK theme landed a few OTAs back. Awesome right?!
And now I'm back with my OSK works....finally!....specifically with the advanced text functionalities in the OSK.
I've prototyped and demoed this almost half a year ago but now I want to make it "PR-ready".
So once I get satisfied, I will propose this officially.

List of changes and additional features

• Added audio and haptic feedback to the word ribbon (auto suggest bar)
• Temporary fix for the issue where cursor mover isn't working correctly in QtWebEngine text fields such as in the Morph Browser (some trade-offs to fix this temporarily)
• Horizontal movement in the cursor mover will now move vertically once the start/end of line is reached. I'm not sure how this really affects the experience but it was definitely implemented like that before for a reason. (a trade-off for the issue above)
• A new way to trigger the cursor mover mode (bottom swipe). This is available anywhere in the OSK so regardless of the layout or type, you can now trigger it. Press and hold in the space bar is still available.
• Implemented text manipulation features which are available in a top bar that shows up when in cursor mover mode.
  • Copy
  • Cut
  • Paste
  • Select All
  • Undo
  • Redo
• Implemented text selection mode (no timeout, needs manually exiting)
  • While in cursor mover mode, double tap/click to enter selection mode. You can also immediately select texts by double tapping then swiping immediately without lifting your finger.
  • Word Selection: Double tap while in cursor mover mode without swiping [Disabled for now to fix QtWebEngine issue]
• Added floating buttons in cursor mover mode
  • Move cursor to the start/end of line or document
  • They can also be used in conjunction with the selection mode!
  • Done button: Used to end selection mode and/or cursor mover mode
• Changed timeout of cursor mover mode to 5 seconds (experimental)

Know Issues:

• The new toolbar buttons and floating buttons sometimes retain their previous color when changing OSK theme.
  FIXED: Just needed to bind the colors

• Using the Start of line/document button from the floating buttons while in selection mode doesn't select texts. I'm not really sure why.
  FIXED: Stupid mistake by me....it worked in my first branch and just had a mistake with this new branch

• The bottom swipe trigger might be difficult to trigger in landscape mode especially for devices with protrusion on the sides/bezels especially those with cases.

Please feel free to try it out from the git branch linked below. You'd need to use crossbuilder to compile it and deploy to your device via ADB.
https://github.com/kugiigi/keyboard-component/tree/xenial_-_textfunctions

NOTE: BEWARE EDGE CHANNEL USERS. This won't work on edge because there's a separate branch for it which I don't have a modified fork yet. If there's plenty of interested then I might consider

Suggestions and feedback will also be greatly appreciated especially on the design

Some screenshots

Cursor mover mode

Selection Mode

This guide isn't created by the porter/maintainer of the port. This is only based on my personal experience, specifically on S10 and hopefully it can be helpful for those that needs step by step and more detailed guide than the current ones. Please feel free to correct or add anything in this guide. Hopefully we'll get these devices in the installer soon.

You may also join the telegram group to ask questions/help and/or check the pinned messages.

My device

• S10 (beyond1lte)
• SM-G973F/DS

Starting point

• Locked bootloader
• Security update: 01/2022 with a pending update
• Android 12 - OneUI 4.0

Step by step

A. Unlock the Bootloader:
Source: https://forum.xda-developers.com/t/recovery-official-3-6-1-x-twrp-for-galaxy-s10-e-5g-exynos.4180287/

1. Make sure OEM Unlock is ON in developer settings (How To)
2. Turn off the phone
3. While pressing Vol Down + Bixby key, plug the device to a PC via USB cable to enter download mode;
4. Long press Vol Up then confirm you want to unlock bootloader (this is needed one time only)
   * Please be aware all your data will be erased including internal storage and phone will reboot!!
5. Let the phone boot normally. Complete Android setup and connect to internet (wireless or mobile data) in order to unlock RMM (Not sure what this is)
6. Make sure OEM Unlock is ON in developer settings
7. Turn off the phone
8. Repeat Step 3 to boot to Download mode then short press Vol Up to confirm you want to go to download mode
   * If by any chance you see RMM state=Prenormal or KG state=Prenormal, redo from step 5!

B. Flash TWRP Recovery:
(You may skip to flashing LOS recovery but I'm not sure how to use that to flash zip files and wiping partitions)

1. Download appropriate TWRP .img file according to your device.

2. Download avb disabled vbmeta .img

3. While your device is in Download mode, open your preferred terminal in linux and run the following (replace texts enclosed by <> with the path of downloaded img files):
   sudo heimdall flash --RECOVERY </path/to/twrp.img> --VBMETA </path/to/vbmeta.img>

   After pressing enter to run the command, grab the phone and press Vol Up + Bixby + Power to force it to boot straight to TWRP after flashing
   Otherwise, press the same buttons while the device is turned off to boot to recovery.

!! Only if coming from full stock firmware

• In TWRP home screen, press Wipe > Format data > yes
• Download and flash multidisabler-samsung-3.* zip to disable security and encryption (Install > Choose zip file > Swipe to start flashing)

C. Flash LOS Recovery:

1. Download recovery image from LineageOS
   Sample file name: lineage-18.1-20220311-recovery-beyond1lte.img
2. While your device is in Download mode, run the following command
   sudo heimdall flash --RECOVERY </path/to/LOS_recovery.img> --VBMETA </path/to/vbmeta.img>

D. _Flash Ubuntu Touch:
(This is using LOS recovery. I think you may flash with TWRP too but since updates are done by manually flashing, fastboot may be better and simpler instead of transferring files to the device for flashing via TWRP)

1. Download the latest artifacts.zip file by clicking on the action button on the right most side of the latest passed pipeline item. Select the appropriate item for your device.
   Example: devel-flashable-beyond<0/1/2>lte:archive
2. Download vendor image for your device.
3. Boot to recovery
4. Go to Advanced > Enter fastboot
5. Plug the device to your PC.
6. Run the following commands. Replace the texts enclosed by <> with the correct paths and file name.

   fastboot flash boot </path/to/boot.img>
   fastboot flash system </path/to/ubuntu.img>
   fastboot flash vendor </path/to/vendor-ut-beyond<0/1/2>lte.img>
   

7. Reboot and enjoy UT!

Hi,
I'm Kugi and I'm just mimicing this post
I'm posting this to introduce a new tool that I DON'T RECOMMEND to anyone but anyone is free to use No seriously, it can be dangerous to your system so it's important that you know what you are doing and know how to reflash in case of breakage.

Introducing the Jerk Installer!

What is Jerk?

No, it's not an offensive word. It's a physics term just like Snap, Crackle and Pop
Jerk is a script to easily install and uninstall customization on the system (rootfs). It's mostly only applicable on patches that involve text-based files like QML files. I haven't tested with binaries but maybe in the future?

What kind of customization?

Many things! Such as UI changes in Lomiri, new features in the keyboard, and Bluetooth configuration changes. Those are just the ones I came up with but there are more possibilities.

How do I use it?

I won't repeat myself here. You can go to the repo and read the....README
But basically, just download the whole repo, make the jerk script executable and run ./jerk --help. There are sample packages included in the repo.

Sample Packages

I've included many packages in the repo that you can try out. These are mostly the customization and experiments I've done in my devices. I highly recommend the package Lomiri_Ubuntu_22.04.tar.gz. 20.04 (focal) transition is taking a really long time to arrive so let's just skip and go straight to the latest Ubuntu
You can run ./jerk describe <packagename> to check what each package does.

It blocks OTA updates

Yup, it automatically blocks OTA updates until you uninstall all the packages you installed and/or intentionally unblocks it with ./jerk unblock-ota or reset all components with ./jerk reset all. I made this decision so users can avoid installing OTA updates that can possibly have conflicts with the installed packages. It's a good practice to remove all packages before updating then just reinstall after updating. Jerk will tell you if the package is still compatible or not.

I wanna create my own package!

Sure, why not? If you're like me who likes experimenting stuffs and want to share them to others, you can create your own package. Just check the...README
For the target component/path, it'll be better to make a PR to create new configuration if it's not present yet so the script is always aware of all the components with installed packages.

Is it perfect?

YES! No! Of course no
I'm not an expert with bash scripting and the script is a bit simplistic in terms of checking compatibility and no proper way to remember the installed packages. As always, PRs welcome

Can I trust you?

That's up to you

Can I see some screenshots of sample customization/patches?

NO!

Morph can only open files from the Downloads folder. Try copying them there.

Hello Everyone,

I would just like to gather random ideas that we can maybe implement or fix in Ubuntu Touch.
I already have ~100 in my list of ideas but I thought it would also be cool to get other people's ideas. I base my ideas from my personal experience and preference so your ideas may give me wider perspective on things. Of course, I won't be promising anything and it will still depend if my skills would allow me to put your ideas into reality

So that you'll have an idea on what my skills can do, here's a list of some of my previous, current and planned works.

Previous works:

1. Advanced text manipulations in the OSK (selection mode, copy/paste, etc.)
2. Theme support in the OSK
3. New Downloads UX in Morph

Current works:

1. Check my github profile to see my open PRs (https://github.com/kugiigi)
   Most of my recent PRs are related or beneficial to tablets because I recently got one
2. Floating and one-handed mode in the OSK. It's still not an official PR but it's already working and here's a preview
   

Future plans:

1. Split mode and numbers row option in the OSK
2. Redesign language menu to something similar to Sailfish OS/2nd level characters
3. Try to implement tab navigation history save/restore feature in Morph

In summary, I'm quite knowledgeable with QML+JS and I can do a bit of C++ (templating from existing codes ).
Most of my ideas are not life changing and won't compare to the essential works from the core devs, rather, they improve the day to day experience with UT, well at least that's the goal

I've been a bum since early this year so I'm basically full time with Ubuntu Touch so I will have plenty of time working on the ideas I may pick. I'm dealing with something the past weeks though and maybe for a few more but I'm pretty sure I'll be back at full speed eventually

If you want to support me to stay being a bum while contributing to UT, I would greatly appreaciate any amount
http://www.paypal.me/Kugi
https://liberapay.com/kugi_eusebio/donate

I'm looking forward seeing your ideas

This is kind of an update on this post. It's been a year and I've kind of improved and stabilized the tool (Jerk Installer and Jerk Packages) I created, or I'd like to think that Well, I do use it on basically all my devices so I guess it's proven now?

Most importantly, I've streamlined the hacks/packages that I've been developing and maintaining. I've combined them per component and just added settings so you can customize which things you want enabled or disabled. The main 2 ones are the following.

Malakiboard

• A collection of hacks for UT's onscreen keyboard.
• Notable new features and functions are custom keyboard height, floating mode, custom theme, and theme that follows system theme.

Lomiri Plus

• A collection of hacks for Lomiri, UT's desktop environment.
• Notable new features and functions are notch and rounded corners fix, manual rotation button, and so much more to mention and hard to explain too

IMPORTANT NOTES!

• DANGER!: This tool can be dangerous to your system. It can render your device unusable, though it has never happened to me yet and only during development . You can always reflash with the installer to reset the whole system and just make sure to NOT select the Wipe option.
• BE SAFE!: I haven't found a way to automatically create the packages so you'll have to create them manually. I highly discourage downloading and installing pre-created packages from anyone, even from me! although I do that in Telegram LOL. It's easy to manipulate the contents so it's better to be safe. As always, it's best to download from the original source.
• BE RESPONSIBLE!: To avoid creating unnecessary logged issues on the official UBports repos, make sure to reset your system first and reconfirm the bug you're encountering before filling them in Gitlab/Github. Resetting is easy. You can even reset all with the command jerk reset all
• HAVE FUN!: Enjoy all my crazy experiments, ideas, concepts, whatever they are. Let me know if you have comments or any feedback about them. The hope is to have some of them implemented officially

Yes, Dr. Strange knows what will happen
But seriously, apt isn't advisable to be used on UT so enabling a whole repository may cause even more havoc to the system.

Some progress for the themes support hurray!

Here are the repositories I've uploaded which so far works quite well except the settings-app which I was not able to build myself but I've only changed QML and those worked fine when I tested directly on my phone
Keyboard component
Settings app

My initial plan was just create a dark mode for the OSK but since the foundation for the colors is already there so I experimented with different themes and these are the ones I've come up.
(I don't know how to shrink the photos here :P)

Ambiance (Default)

Suru Dark

Suru Black

Just White

Just Black

Just Grey

Bordered White

Bordered Black

Bordered Grey

Colored ones are also possible but I did not include any in the repo but here's a sample

I don't know if this will be accepted but if yes, @CiberSheep I would like get the opinion of the design team and you can check the colors.
They're just in json files in the repo and can be easily modified

And BTW, this completely removed the border images and replaced with rectangles with borders

@Martin5 Interesting. So it could indeed be related to eSim
Thanks for trying out reflashing Android again. I don't think I want to do that again on mine so I'm just waiting for the eSim management on UT
For now, my mobile data still sometimes don't work but I noticed that when I leave it for a bit, it'll start to work again. Sadly, it also sometimes happen the other way

@Charly Recording videos is also affected when playback is broken. You can try restarting media-hub (systemctl --user restart media-hub.service) but if it's still the same, only a reboot could fix it.

Broken video playback is quite a common occurrence on my FP4 which I observed started in recent OTA (7 or 8?) updates.

To be fair, this is a very valid use case. I've wanted this right from the start since if you use external keyboard and mouse, you won't need the virtual one so the screen will be useless. In Samsung DeX, you can do this although I noticed that some apps wakes it up when doing certain things.

In any case, there's no way to do this right now. It might need changes to the lower level components such as Mir. Also note that unlike Samsung DeX, UT is using the same UI or DE when connected to an external display so anything you do on the device would affect both displays.

@faveoled That's not currently possible in UT. Not sure if it's a hardware thing or can be fixed with software.

@bhdouglass Yeah, it's either we delay the release of the new site or we see what's causing it and perhaps and there's a quick/simple workaround in the new site.

@vily I don't think so. Probably something that should be implemented first in the system handling notification sound playback before the app.

@Anton Which app do you use? Podphoenix?
Unfortunately, the system doesn't support playback rate yet however there's already a work on this and we'll just wait for it to land and it'll be trivial to add this in apps.

@jojumaxx When did you check for updates. There are reported issues with the installer and channel switcher. Perhaps this is a network issue and OTA updates are affected as well.

@qnx_guy It should work but it's very finicky. Known dongles that work reliably on UT are the Microsoft ones. Other dongles would be a hit and miss.
But in any case, wireless display has noticeable input lag so it's really not ideal for using it like a computer with mouse/keyboard. Even on Android, it's only good for consuming contents or media.

@poVoq Yes it works