Here, in this thread we discuss

SPEECH RECOGNITION ENGINE on UBUNTU TOUCH

Yeah, it's real. I've made it so that it could run on UT. Locally. Without sending it to someone's server

This solution is called SOTY and is free software. It is a port of VOSK API wrapper for Kaldi - pretty neat speech recognition framework.

Well, back to the point. Post your test results here, be surprised how (in)accurate the results are, review the source code and propose changes, ask questions on adapting your application to this feature, and don't forget to ask again why this isn't working in background and how to use that. It is recommended you read all this long post before posting yourself.

What is it?

It is a speech recognition server, which means it receives input (raw audio data) from a client, processes it and then sends back a transcription of a data being recorded on a client.

The server itself is completely useless without a client, it doesn't even have an access to audio subsystem.
The server was made to be combined with other software that would utilize speech recognition, where it could be useful

So, right now this is more like a framework for developers, who might be interested in it.

Installation

Downloading the application from OpenStore will not be enough.
You also need to install models.

To install English language model you need to run these commands in terminal:

Models can be installed using the in-application installer, which is accessible through "gear" icon in top right corner of the app interface.
Now it supports transcribing in English. You can test your accent with...

List of applications that work with SOTY STT

UT Translator (recent update)
To enable SOTY integration run in terminal:

sed -i 's/enableSTT=false/enableSTT=true/g' /home/phablet/.config/ut-dictionary-frontend.ut-dictionary-frontend/ut-dictionary-frontend.ut-dictionary-frontend.conf

Then after installing and running SOTY properly

1. Open SOTY first and start server
2. Open UT Translator (WITHOUT CLOSING SOTY SERVER)
3. Choose English language. Microphone icon will appear on a top panel. Click on it to start recording audio.

I hope this list will grow bigger over time

(I would be more than happy to have it integrated in lomiri keyboard, and that would probably eliminate the need to integrate it with any other app, but I don't know if that's ever going to happen)

Quality

It is now possible to transcribe everything you say on a device locally, your smartphone that runs Ubuntu Touch could totally do that.
Too good to be true. There of course are limitations.
If we use small models, which is the current case, they won't cover all the words in language. And our small models are not good at transcribing previously unknown words and separate letters.

You could try using models that are much bigger and intended for use on servers, but they, however would require more RAM and more time to process your data. It is significally slower. You will not like it. Implementing VAD preprocessing might help a little. And might not.

Another fly in the ointment is that models currently in use are not helpful with spelling words. At all. You need to re-train them for this specific task.

Summary

I hope it has some potential. Will it evolve into an open-source voice assistant for your device of the future, or will it remain a funny conceptual toy, it's up to you, dear Community.

How can I help

Here's what you can do for this project:

• Design
• Code improvements
• Guides for other people
• If you are an app developer : think of ways it could be useful in your application
• Testing and reporting bugs
• (The most important)Improving models

Plans

Add models installer.
Make it configurable.
System OSK integration.

Improving models

Under construction

New version is out
Introducing protocol v2 - now client apps do not need to use the microphone, audio recording is performed on a server. There are some flaws for this decision, though.
CPU load has been reduced in this release
This release is also backwards compatible with protocol v1, where client sends recorded data to server, just in case someone wants to send audio data from other sources.

Also, my apologies, I forgot to link
Client library for v2 protocol
Client library for v1 protocol (requires Microphone permission)

This idea has some flaws: there are many good devs and their apps, with these restrictions probably some of them would be undeservedly not mentioned here, and some, despite putting a lot of work in this project, may be not nominated because we don't usually see them where we see other devs.

I would like to nominate the following apps:

• Waydroid because we all know why
• LogViewer since it makes debugging much less painful

Developers, who absolutely deserve mentioning here:

• Danfro
• fredldotme

New version is out
Changes:

• Fixed a few bugs
• Added models installer (works for many languages listed in the menu, other models will be uploaded later)
• The application UI can be translated.
• Now it comes with amd64 build

@Leroy_Linux
RC for 1.x has VoLTE support, and waydroid works fine

you might want to teach her to force shutdown, which is a little tricky for this model, just in case

@freddo
Thanks, that would be great to have a few mirrors, but currently, there's no integrity check mechanism implemented in OpenStore, and that is a requirement. I think I'll try doing something about it soon. That would also require cooperation with the OpenStore team

The Client library for v2 protocol is now a complete QML plugin, which can be easily added to your application and then used in your QML layout. The repository contains all steps to integrate speech recognition client in your project. No permissions needed, the only requirement is server application running in background locally.

@freddo
I see that files on system-image.ubports.com are stored with digital signatures, so, the situation is better here, but still requires implementing mechanism of choosing mirrors in installer software. I'll take a look at that soon. Thanks

@undrwater
Thanks for your interest. You can easily integrate TTS support in your application using espeak-ng. However, espeak data takes 20 MBytes of user storage space. If you want this functionality for Soty server, that would require changing communication protocol for both server and client. It also would not be too hard, but I personally think we should look for a more accurate solution for this task, that could be seamlessly integrated in system, such as speech-dispatcher.

@egregiousecan
There's some bug that causes random bootloop, forcing shutdown by holding all phone buttons simultaneously helps. I was also not able to boot into Ut 24.04 after upgrading from 20.04, I had to reflash the device with Android to do that.

@arubislander That's the correct answer, I missed the fact that these modules may not only be completely disabled in a specific kernel, but also be built-in. In which case the desktop solution would do nothing

@tidip said:

So if I correctly understood patching the kernel would be responsibility of the port maintainer(s).

It can be patched on a core system image level for most devices as well, and that would require way less work and prevent any further drama that is still going on.

Some of these series even came out before the mainline kernel patches so it took time to address the issue even for major Linux distros

@gpatel-fr
the second one looks like it successfully wrote something to su and tried to run it, but the su contents was misunderstood by OS
The mitigation should be the same as for desktop Linux: disabling affected modules completely in modprobe config:

install esp4 /bin/false
install esp6 /bin/false
install rxrpc /bin/false
install algif_aead /bin/false

I don't think USB tethering supports using smartphone for making calls on a PC. Seems like you need an actual modem for that.

@hacknorris
I found a way to run grafx2. I installed it in libertine, then, to allow touchscreen input, I had to enable windowed mode in UT Tweak Tool (for best experience also tick auto hide launcher option), and resize grafx2 window to fullscreen using the title bar button. However I did this on noble, so it may not work or require different steps on focal.

@sixwheeledbeast said:
To me this seems like something that should be included into a toggle on the Battery pull-down menu itself, much like most modern devices have a "low power mode".
How feasible would a toggle option to disable all these power hungry features and set the radio to the most energy efficient available in your location. Then toggling back would restore what you had previous. It seems like something the OS should provide and not additional apps.

Something like this is already implemented in the OS. It is called "Flight mode"
Humour aside, I don't know if it is really worth implementing switch for turning off all but 2G when it can be done in two or three clicks. But if you want to contribute to Ubuntu Touch this way, you can take a look at how it is implemented in lomiri-system-settings repository.

Patches that implement repository switching mechanism for system updates have been suggested a few months ago but gained no attention. Same with mirror hosting proposals.

@Kadafi said:
swipe up in the "calls" app does nothing. Keeps showing the number buttons to dial a number. Cannot find out where are the missed calls or made calls logs.

If you tried swiping up from the bottom of the screen while in application and the calls history did not appear, then this might be caused by using unstable version. The installer does not make it clear that only 24.04.1 stable and 24.04.1 Release Candidate are ready for daily use. This issue already caused a lot of confusion for users.

@developerbayman
Haven't tried that yet, but looks promising. Can you tell us a little bit more about it? Is that really related to Android app, or maybe it is better to pick some original name for that? Did you build original app's dependencies?