Convergence via Deskreen app?

poVoq

https://deskreen.com/
https://github.com/pavlobu/deskreen

Deskreen is a Linux app that allows using any device with e webbrowser as a second screen for a device. It works by streaming the screen via webRTC.

I think it would be pretty awesome to have a similar functionality build into UT for convergence mode. It should even work on phones without display-out and on a 5ghz wifi connection it is quite lag free.

Anyone interested in porting Deskreen (Electron app) or replicating the functionality directly in UT? Might be not too difficult with Morph if it supports screen sharing via webRTC?

kugiigi

@povoq How's input lag and overall responsiveness?
I think the tricky part here is how to capture the screen since it may differ between X and wayland then UT mostly use mirclient. Screen capture is already possible on mirclient as it is being done by MirVNCServer but I'm not sure if it's good enough for a lag-free experience. Also, even if it works, it'll just mirror the screen, right? So scaling will still be the same as the phone unless maybe the program also creates a virtual screen with desktop scaling or automatically change the scaling of the built-in display.

poVoq

@kugiigi lag remains to be seen, but in theory it can work quite well.

The screen can be anything. Deskreen usually uses a virtual secondary screen, but of course you can also mirror it. Scaling shouldn't be an issue.

poVoq

bump

I wonder why there is not more interest in something like this. For me it seems like the ideal way of implementing ~~convergence~~ desktop mode with convergent apps: No extra hardware needed and you can set it up ad-hoc with nearly any device you happen upon.

alan_g

@kugiigi said in Convergence via Deskreen app?:

think the tricky part here is how to capture the screen since it may differ between X and wayland then UT mostly use mirclient.

Yes. There's no future in pursuing mirclient support and I infer that this uses pipewire on Wayland. Support for the pipewire Wayland extension would need to be added to Mir (PRs welcome).

cliffcoggin

@povoq Perhaps I am missing something here, but I can not see the point of trying to watch the content of 23 inch computer screen compressed on to a 5 inch phone screen.

Keneda

@cliffcoggin
I think he talks about the opposite, watch the content of a 5" smartphone on a 23" or watever size PC screen.
All the point of the "convergence" thing... without the need of a compatible plug on smartphone.

alan_g

@povoq said in Convergence via Deskreen app?:

convergence mode

Mentioning "convergence" here and in the title is confusing as the discussion proves.

The term originated as a way to describe what software needs to do to work effectively across a wide range of devices capabilities: screen size; physical, virtual or no keyboard, mouse or not, touchscreen or nor, etc.

Having "convergent" software is what allows Ubuntu Touch to dock with a fullsize screen, keyboard and mouse and offer a "desktop" mode.

It would be clearer to refer to "desktop mode" (although that name too is poor as it is frequently a laptop, TV or projector).

cliffcoggin

@alan_g Thank you for the explanation. I don't believe I have fully grasped the concept of convergence, but then I only connect my phone to a computer to move a few photos so I probably have no need to.

poVoq

Yeah, I guess I am referring to desktop mode with convergent apps.

Similar to how some phones supported by UT can use a display out cable to be converted into a more or less full desktop computer, this would allow doing it wireless to any device (and without a miracast dongle either).

Deskreen works by adding a virtual screen (that can be mirrored or not), so nothing would be compressed to a mobile resolution or anything like that. It would be just your regular 1920x1080 desktop resolution (or a bit lower via slower 2.4ghz wifi I guess).

dobey

@alan_g I prefer to use the terms "windowed mode" and/or "external screen" depending on what specifically one is talking about. Since it is possible to use windowed mode even on small screens, or staged mode even on larger external displays, it helps to be specific.

dobey

@povoq It sounds like it's just an app that does basically the same thing as VNC/RDP would do, but with yet another different protocol.

All it would take is for someone to do the work to make it function, and get it integrated into the system.

poVoq

@dobey Yes similar, but with the big advantage that the device used as a screen just needs a webbrowser. This opens a lot of options as you don't need to install anything (on a company PCs for example) and you can also use plenty of devices like old tablets or smart TVs as a screen without any additional hardware.

dobey

@povoq This only works for streaming the video though, no? You can't interact with the screen by dragging windows, or adjust focus and type, directly in the browser, right?

poVoq

@dobey No, you are right. Screen sharing by stream only, no interactivity AFAIK.