• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login
UBports Robot Logo UBports Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login

MyCroft (voice assistant) demo app

Scheduled Pinned Locked Moved Design
14 Posts 7 Posters 3.7k Views 5 Watching
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • H Offline
      hummlbach
      last edited by hummlbach 6 Jul 2018, 23:49 7 Jun 2018, 23:42

      Together with @jonius and @TimSueberkrueb, I had the great pleasure to build a small (about 250MB) MyCroft demo click package. Its at a very early stage: ssh server needs to be enabled on the device in order to use the app; its known to crash on FP2@16.04 and hammerhead@15.04 :-(but nonetheless you could try if its working on your FP2/nexus5) and is kind of working on turbo@15.04 and krillin@16.04 yippy πŸ™‚ Details on the status are found in the issue tracker. Everyones welcome to test it (obv)!

      A little about how MyCroft works:

      1. The device is listening for the wake word (atm) "Hey MyCroft". That is done locally on the device.
      2. Everything said after the wake word is then sent to a speech to text web service. By default Googles is used.
      3. The text returned is then mapped by the intent parser to a skill and depending on the skill different things will happen.
      4. The skill returns text which is (locally) synthesized to speech again (also called text to speech).

      A few sentences about privacy concerns and future plans:

      1. To avoid having the microphone always on (beside privacy concerns it has an big impact on the battery too), it would be nice to wake MyCroft by pressing a hardware button. Whether this will be possible or not, depends on how it is integrated in/packaged for Ubuntu Touch.
      2. Most probably it won't be possible to do the speech to text locally on the device. There will be a setting to select the service used (there are others than Google) but if you want to use it, you'll have to send your speech to some service at least or use a chat bot - @TimSueberkrueb is thinking about implementing this.
      3. At the moment its funny in the first place. But my actual motivation having MyCroft, was for practical reasons: Being able to call mum while driving by car or adding milk to the buy list while baking... Therefor we have to interface with the apps (for example) and that of course should happen locally again, s.t. for the useful things only your verbal request (i.e. "call mum" or "add milk to the buy list") will be processed to text online and the rest will happen locally...

      The issue tracker already also reflects these plans more or less. I'm very curious what you think about that and hope you enjoy testing.

      1 Reply Last reply Reply Quote 10
      • K Offline
        kugiigi
        last edited by 13 Jun 2018, 12:59

        I would suggest you publish this in OpenStore so that you can get more tests from users πŸ™‚

        H 1 Reply Last reply 13 Jun 2018, 15:34 Reply Quote 0
        • H Offline
          hummlbach @kugiigi
          last edited by 13 Jun 2018, 15:34

          @kugiigi Yes i would really like to do that. The very early tests have shown, that I need to do one or two improvements in advance. And I have to check with the openstore team whether they accept such a hacky - and its really hacky πŸ˜‰ - click...

          1 Reply Last reply Reply Quote 1
          • B Offline
            Bolly
            last edited by 19 Jun 2018, 16:22

            video demo?ΒΏ?ΒΏ πŸ˜›

            2015-Now (Daily use) : BQ Aquaris E4.5 Ubuntu edition
            2016-Now (Daily use) : BQ Aquaris M10 FHD Betatester

            2020-Now : PinePhone Braveheart & CE UBports
            2020-Now (Family/Daily use): Vollaphone Xenial
            2022-Now (Family/Daily use): Vollaphone 22 Focal

            1 Reply Last reply Reply Quote 1
            • F Offline
              flohack
              last edited by 25 Jun 2018, 19:16

              Can you change the trigger word to Yumi? xD

              My languages: πŸ‡¦πŸ‡Ή πŸ‡©πŸ‡ͺ πŸ‡¬πŸ‡§ πŸ‡ΊπŸ‡Έ

              1 Reply Last reply Reply Quote 2
              • F Offline
                flohack
                last edited by 25 Jun 2018, 19:36

                Actually I propose "Dear Yumi", its polite and friendly, and will for sure increase the fidelity of the results πŸ™‚

                • And it will not confuse it by sounding too much like You or Me

                My languages: πŸ‡¦πŸ‡Ή πŸ‡©πŸ‡ͺ πŸ‡¬πŸ‡§ πŸ‡ΊπŸ‡Έ

                1 Reply Last reply Reply Quote 0
                • H Offline
                  hummlbach
                  last edited by hummlbach 26 Jun 2018, 00:49

                  @Flohack nice idea!
                  @Bolly mhmm you don't see much actually - atm its just one switch to turn off and on... πŸ™‚ but i can give you an audio demo πŸ˜‰

                  A B 2 Replies Last reply 26 Jun 2018, 05:51 Reply Quote 2
                  • A Offline
                    advocatux @hummlbach
                    last edited by 26 Jun 2018, 05:51

                    @hummlbach there's an official Mycroft group on Telegram since May (https://t.me/mycroft_ai) maybe you can post that video there to get some extra-attention to the project πŸ™‚

                    1 Reply Last reply Reply Quote 2
                    • H Offline
                      hummlbach
                      last edited by 26 Jun 2018, 10:56

                      @advocatux thanks for the hint! I'll join them. πŸ™‚

                      1 Reply Last reply Reply Quote 2
                      • B Offline
                        Bolly @hummlbach
                        last edited by 26 Jun 2018, 21:23

                        @hummlbach thank you, impressive. Nice Yumi xD

                        2015-Now (Daily use) : BQ Aquaris E4.5 Ubuntu edition
                        2016-Now (Daily use) : BQ Aquaris M10 FHD Betatester

                        2020-Now : PinePhone Braveheart & CE UBports
                        2020-Now (Family/Daily use): Vollaphone Xenial
                        2022-Now (Family/Daily use): Vollaphone 22 Focal

                        1 Reply Last reply Reply Quote 0
                        • P Offline
                          PhoenixLandPirat
                          last edited by 23 Apr 2020, 21:30

                          I know this is 2 years old give or take, but I thought I'd give it a bump and see if anyone is interested in trying to revive this code, and bring it up to date, or even just to see if this is something people are still interested in a few years after its release.

                          I wasn't sure if mycroft was still a thing that had validity into days age, as in, I haven't seen if it had been updated, or if the goals had been the same, or where there interest is, but after seeing Mycroft being integrated into Plasma Bigscreen this year, I thought, perhaps this and having it on mobile might revive interest in open source AI, and if we have mycroft on our phones and tv's we might be able to do some interesting IOT, creative things?

                          "Dear Yumi, play the latest UBports Q&A on the TV please"

                          kind of commands?

                          L 1 Reply Last reply 23 Apr 2020, 21:48 Reply Quote 2
                          • L Offline
                            Lakotaubp @PhoenixLandPirat
                            last edited by 23 Apr 2020, 21:48

                            @PhoenixLandPirat perhaps best to create a new thread and use the web address of this one as a reference it is very old. We could then lock this one to avoid confusion.

                            P 1 Reply Last reply 23 Apr 2020, 22:05 Reply Quote 0
                            • P Offline
                              PhoenixLandPirat @Lakotaubp
                              last edited by 23 Apr 2020, 22:05

                              @Lakotaubp Done πŸ™‚

                              1 Reply Last reply Reply Quote 0
                              • L Offline
                                Lakotaubp
                                last edited by 24 Apr 2020, 06:28

                                I have locked this topic. A new one is here https://forums.ubports.com/topic/4149/mycroft-app

                                1 Reply Last reply Reply Quote 0
                                • First post
                                  Last post