MyCroft (voice assistant) demo app

hummlbach · 6 Jul 2018, 23:49

Together with @jonius and @TimSueberkrueb, I had the great pleasure to build a small (about 250MB) MyCroft demo click package. Its at a very early stage: ssh server needs to be enabled on the device in order to use the app; its known to crash on FP2@16.04 and hammerhead@15.04 :-(but nonetheless you could try if its working on your FP2/nexus5) and is kind of working on turbo@15.04 and krillin@16.04 yippy Details on the status are found in the issue tracker. Everyones welcome to test it (obv)!

A little about how MyCroft works:

The device is listening for the wake word (atm) "Hey MyCroft". That is done locally on the device.
Everything said after the wake word is then sent to a speech to text web service. By default Googles is used.
The text returned is then mapped by the intent parser to a skill and depending on the skill different things will happen.
The skill returns text which is (locally) synthesized to speech again (also called text to speech).

A few sentences about privacy concerns and future plans:

To avoid having the microphone always on (beside privacy concerns it has an big impact on the battery too), it would be nice to wake MyCroft by pressing a hardware button. Whether this will be possible or not, depends on how it is integrated in/packaged for Ubuntu Touch.
Most probably it won't be possible to do the speech to text locally on the device. There will be a setting to select the service used (there are others than Google) but if you want to use it, you'll have to send your speech to some service at least or use a chat bot - @TimSueberkrueb is thinking about implementing this.
At the moment its funny in the first place. But my actual motivation having MyCroft, was for practical reasons: Being able to call mum while driving by car or adding milk to the buy list while baking... Therefor we have to interface with the apps (for example) and that of course should happen locally again, s.t. for the useful things only your verbal request (i.e. "call mum" or "add milk to the buy list") will be processed to text online and the rest will happen locally...

The issue tracker already also reflects these plans more or less. I'm very curious what you think about that and hope you enjoy testing.

kugiigi · 13 Jun 2018, 12:59

I would suggest you publish this in OpenStore so that you can get more tests from users

hummlbach · 13 Jun 2018, 15:34

@kugiigi Yes i would really like to do that. The very early tests have shown, that I need to do one or two improvements in advance. And I have to check with the openstore team whether they accept such a hacky - and its really hacky - click...

Bolly · 19 Jun 2018, 16:22

video demo?¿?¿

flohack · 25 Jun 2018, 19:16

Can you change the trigger word to Yumi? xD

flohack · 25 Jun 2018, 19:36

Actually I propose "Dear Yumi", its polite and friendly, and will for sure increase the fidelity of the results

And it will not confuse it by sounding too much like You or Me

hummlbach · 26 Jun 2018, 00:49

@Flohack nice idea!
@Bolly mhmm you don't see much actually - atm its just one switch to turn off and on... but i can give you an audio demo

advocatux · 26 Jun 2018, 05:51

@hummlbach there's an official Mycroft group on Telegram since May (https://t.me/mycroft_ai) maybe you can post that video there to get some extra-attention to the project

hummlbach · 26 Jun 2018, 10:56

@advocatux thanks for the hint! I'll join them.

Bolly · 26 Jun 2018, 21:23

@hummlbach thank you, impressive. Nice Yumi xD

PhoenixLandPirat · 23 Apr 2020, 21:30

I know this is 2 years old give or take, but I thought I'd give it a bump and see if anyone is interested in trying to revive this code, and bring it up to date, or even just to see if this is something people are still interested in a few years after its release.

I wasn't sure if mycroft was still a thing that had validity into days age, as in, I haven't seen if it had been updated, or if the goals had been the same, or where there interest is, but after seeing Mycroft being integrated into Plasma Bigscreen this year, I thought, perhaps this and having it on mobile might revive interest in open source AI, and if we have mycroft on our phones and tv's we might be able to do some interesting IOT, creative things?

"Dear Yumi, play the latest UBports Q&A on the TV please"

kind of commands?

Lakotaubp · 23 Apr 2020, 21:48

@PhoenixLandPirat perhaps best to create a new thread and use the web address of this one as a reference it is very old. We could then lock this one to avoid confusion.

PhoenixLandPirat · 23 Apr 2020, 22:05

@Lakotaubp Done

Lakotaubp · 24 Apr 2020, 06:28

I have locked this topic. A new one is here https://forums.ubports.com/topic/4149/mycroft-app