Idea: overlayfs for user terminal

Fuseteam

again, don't go the route of enabling apt. It simply not suited due to the way Ubuntu Touch is build. You will run into issues that we cannot help with. Even if you don't personally do, users of whatever you're cooking up will run into those issues. We cannot help with those issues not for lack of technical skills but for lack of time and people, there are other things we can focus our time and energy on. There is plenty we can fix to allow running packages you normally install with apt.

For people who want to install packages, you'd normally use apt for, i would like to suggest crackle instead. I don't want to push to hard on it, as it is something i wrote with the help of some in the community. I don't want to be like those companies praising their own products to sell it to you.

Crackle was born from the need to install packages but the lack of apt. It is the final result from experiments since 2018 which now 7 years and counting. The script itself started 4 years ago, and evolved quite a bit to get where we are now. At first it was wrapped around apt downloading packages and installing them into the home directory via various settings and environment variable, it worked fine for vim, git and even tailscale. But nowhere near the 80,000 packages ubuntu offers. Nobody, none of the people complaining about the lack of a way to install packages, even tried to help adapting more packages. Now 4 years later it uses nix and it works for all the packages i have tried. I even managed to install cargo and pipx with crackle, someone even managed to install flatpak— i have yet to find a package that plainly does not work. And there are over 120,000 packages to test.

I don't want to be over-confident but for this occasion i'll dare say that if a package installed with crackle doesn't work after installation, it would not have worked when installed with apt either— that is, is not an issue with crackle, but something we miss in UT— which is where can then focus our time and energy; improving UT's integration into the rest of the linux ecosystem

Once upon a time Ubuntu Touch used upstart, now we use systemd
Once upon a time Ubuntu Touch had its own display protocol, now we're moving towards Wayland
Once upon a time we had xmir, now we have xwayland
Once upon a time we had only libertine, now we have both snap and nix support
Once upon a time nix couldn't work on UT due to technical limitation, today it just works

Step by step integration work is done, to allow apps like firefox to work seamlessly

wait, did i just say nix just works, why did i then even mention crackle? am i a shill after all? well i can't deny i am biased, but one thing i noticed is that nobody talks about nix as a universal package manager and i think i know why, It is a completely different experience.
Since crackle was already a wrapper around apt, it was already close in experience to apt. So since i just swapped the "backend", it brings an apt-like experience to nix.

Yesterday i had a feeling i was forgetting something else you can without apt and without a writable rootfs. And today i know what it was: cargo! nvm! jekyll! all these package managers just work on UT! I completely forgot about it since i haven't touched it in a while but my personal website was made on UT

p.s. installing crackle is a one liner as shown in the readme, on UT that oneliner only works if your rootfs is NOT remounted as readwrite

pparent

@Fuseteam said in Idea: overlayfs for user terminal:

again, don't go the route of enabling ap

Am I actually free to go the route I want to go with my phone?

You will run into issues that cannot help with.

Can you share any fact that you base your analysis on ?

Everything shows the script I have shared cannot have any impact whatsoever on the rootfs for two distinct reasons, first because it is never remounted RW, so it remains read-only at all time, and second because it mounts an overlay over it, so the real "/" would not be modified even if it was read-write. In practice all the tests shows it actually does that and does not modify at all the phone system.

I'm the kind of person convinced by facts and proofs, if you're saying that what I'm doing is going to generate problems, then I want proof and detailed explanations of why.

Fuseteam

@pparent I base my analysis on 7 years of helping people in the community debug issues and the time we've wasted because they ignored all warnings given. And not just my experience, but of many in the community.

You are free to do what you want sure, but know that as a developer you're decisions impact users and the community at large. If problems, new and old, will you take personal responsibility for all of them? will you be the one to assist them 24/7? Because the community cannot given our current size and the work that still has to be done.

This path has been taken many times as can be seen in almost a decade of history in the over 50 telegram groups. I have take this path aswell crackle is something that is came to be after all those different attempts. Which again, i don't want to promote too hard— but i do encourage you to at least try it and see if it can fit the usecase you are trying to fill with apt. Why ignore the effort that has already been done, to solve the case of "installing packages via the terminal"? Sure it isn't apt but does that have to be? We now have access to over 120,000 packages, which includes everything apt has to offer. Wouldn't it be better to concentrate on finding packages that don't work and figure out why they don't?

Fuseteam

P.s. i have a feeling you haven't been reading my full posts because they are too long, so i again implore you to join either telegram or matrix, the more of the community can jump in to give all the proofs and data that you want, including why apt is not very suitable for general use. I'd explain the architecture of UT but that would explode my post you'd most likely not read the whole thing and as such not learn much from it.

given apt is such a core system component that UT uses in a very specific way— So if you absolutely insist on apt then you should at least first learn how updates work on UT. case in point: due to the architecture of Ubuntu Touch sudo apt upgrade has a very real chance of making your phone unbootable. I have seen this happen a couple of times already in the my past 7 years of assisting people in the community.

One good way to learn these things is by interacting with the people building the system, our main groups are at t.me/ubports and #ubports:matrix.org. I won't bore you with the tens of telegram groups and tens of matrix rooms i'm in— if you do join, the community will point you to the appropriate groups as necessary

pparent

@Fuseteam Well I don't want to fight, I have no energy for that, neither here or on telegram. I've not encouraged anybody to do anything, just shared the results of my experiments with all due warnings and disclaimers. Developers out there will be able to read the script an make their own opinion. I've shared all I had to share on that topic, and I don't think I will post anymore in this topic, except if I'm requested to do so. I'm honestly not sure why sharing an interesting idea and experiment triggers a so strong reaction, I did not demand that anything be integrated in the official system without proper validation.

Fuseteam

@pparent i'm not inviting you to fight, i'm inviting you to help you learn how it all works so you don't have to fight the system and run into issues we don't have the energy to help with.

many people don't read disclaimers, they see something someone publish, they use it. see open-store.io for a good example: clicking the bug icon tells them "don't do a bug report here, do it on click this button to go to the issue tracker. the amount of bug reports on the open store and missing on the actual issue tracker speaks for itself.

Fuseteam

@pparent the strong reaction around apt is because what you are trying has been tried, we've been around for almost a decade which has yielded results.

for this i invoke "Chesterston's fence", behind that fence is apt, you are looking for a way to tear down the fence. But the real question is, why hasn't this fence been removed in the past decade? Spoiler: it is not due to a lack of trying