Convergence via Deskreen app?

poVoq · 31 Mar 2021, 00:20

https://deskreen.com/
https://github.com/pavlobu/deskreen

Deskreen is a Linux app that allows using any device with e webbrowser as a second screen for a device. It works by streaming the screen via webRTC.

I think it would be pretty awesome to have a similar functionality build into UT for convergence mode. It should even work on phones without display-out and on a 5ghz wifi connection it is quite lag free.

Anyone interested in porting Deskreen (Electron app) or replicating the functionality directly in UT? Might be not too difficult with Morph if it supports screen sharing via webRTC?

kugiigi · 31 Mar 2021, 02:16

@povoq How's input lag and overall responsiveness?
I think the tricky part here is how to capture the screen since it may differ between X and wayland then UT mostly use mirclient. Screen capture is already possible on mirclient as it is being done by MirVNCServer but I'm not sure if it's good enough for a lag-free experience. Also, even if it works, it'll just mirror the screen, right? So scaling will still be the same as the phone unless maybe the program also creates a virtual screen with desktop scaling or automatically change the scaling of the built-in display.

poVoq · 31 Mar 2021, 08:09

@kugiigi lag remains to be seen, but in theory it can work quite well.

The screen can be anything. Deskreen usually uses a virtual secondary screen, but of course you can also mirror it. Scaling shouldn't be an issue.

poVoq · 4 Jan 2021, 16:22

bump

I wonder why there is not more interest in something like this. For me it seems like the ideal way of implementing ~~convergence~~ desktop mode with convergent apps: No extra hardware needed and you can set it up ad-hoc with nearly any device you happen upon.

alan_g · 1 Apr 2021, 10:36

@kugiigi said in Convergence via Deskreen app?:

think the tricky part here is how to capture the screen since it may differ between X and wayland then UT mostly use mirclient.

Yes. There's no future in pursuing mirclient support and I infer that this uses pipewire on Wayland. Support for the pipewire Wayland extension would need to be added to Mir (PRs welcome).

cliffcoggin · 4 Jan 2021, 10:43

@povoq Perhaps I am missing something here, but I can not see the point of trying to watch the content of 23 inch computer screen compressed on to a 5 inch phone screen.

Keneda · 4 Jan 2021, 10:43

@cliffcoggin
I think he talks about the opposite, watch the content of a 5" smartphone on a 23" or watever size PC screen.
All the point of the "convergence" thing... without the need of a compatible plug on smartphone.

alan_g · 1 Apr 2021, 10:55

@povoq said in Convergence via Deskreen app?:

convergence mode

Mentioning "convergence" here and in the title is confusing as the discussion proves.

The term originated as a way to describe what software needs to do to work effectively across a wide range of devices capabilities: screen size; physical, virtual or no keyboard, mouse or not, touchscreen or nor, etc.

Having "convergent" software is what allows Ubuntu Touch to dock with a fullsize screen, keyboard and mouse and offer a "desktop" mode.

It would be clearer to refer to "desktop mode" (although that name too is poor as it is frequently a laptop, TV or projector).

cliffcoggin · 1 Apr 2021, 13:52

@alan_g Thank you for the explanation. I don't believe I have fully grasped the concept of convergence, but then I only connect my phone to a computer to move a few photos so I probably have no need to.

poVoq · 4 Jan 2021, 16:21

Yeah, I guess I am referring to desktop mode with convergent apps.

Similar to how some phones supported by UT can use a display out cable to be converted into a more or less full desktop computer, this would allow doing it wireless to any device (and without a miracast dongle either).

Deskreen works by adding a virtual screen (that can be mirrored or not), so nothing would be compressed to a mobile resolution or anything like that. It would be just your regular 1920x1080 desktop resolution (or a bit lower via slower 2.4ghz wifi I guess).

dobey · 2 Apr 2021, 18:08

@alan_g I prefer to use the terms "windowed mode" and/or "external screen" depending on what specifically one is talking about. Since it is possible to use windowed mode even on small screens, or staged mode even on larger external displays, it helps to be specific.

dobey · 2 Apr 2021, 18:09

@povoq It sounds like it's just an app that does basically the same thing as VNC/RDP would do, but with yet another different protocol.

All it would take is for someone to do the work to make it function, and get it integrated into the system.

poVoq · 2 Apr 2021, 19:56

@dobey Yes similar, but with the big advantage that the device used as a screen just needs a webbrowser. This opens a lot of options as you don't need to install anything (on a company PCs for example) and you can also use plenty of devices like old tablets or smart TVs as a screen without any additional hardware.

dobey · 3 Apr 2021, 17:29

@povoq This only works for streaming the video though, no? You can't interact with the screen by dragging windows, or adjust focus and type, directly in the browser, right?

poVoq · 3 Apr 2021, 18:01

@dobey No, you are right. Screen sharing by stream only, no interactivity AFAIK.