Snap sucks. The way open source is straying further and further from its principles is highly annoying. The whole idea that you'd need a container like environment to install an application on a Unix system is very far from where we should be. What's the point of dynamic linking if we then end up shipping half an OS with an application just to get around dependency hell? Might as well ship a pre-linked binary that just does system calls.
And it's yet another way to do an end run around repositories, instead you will sooner or later get an app-store like environment that can be controlled by some entity. These large companies should stop fucking around with Linux, it was fine the way it was. Just fix the bugs and leave the rest to the application developers.
> The whole idea that you'd need a container like environment to install an application on a Unix system is very far from where we should be. What's the point of dynamic linking if we then end up shipping half an OS with an application just to get around dependency hell? Might as well ship a pre-linked binary that just does system calls.
There are two models.
The first model is the traditional distribution model. The distribution curates and integrates software, picking (generally) one version of everything and releasing it all together. Users do not get featureful software updates except when they upgrade to a new distribution release - all at once.
The downside of this model is that developers who want to ship their new software or feature updates without waiting for a new distribution release get stuck into dependency hell and have to operate outside the normal packaging system. Same for users who want to consume this. Third party apt repositories and similar efforts are all fundamentally hacks and generally all end up breaking users' systems sooner or later. Often this is discovered only on a subsequent distribution upgrade and users are unable to attribute distribution upgrade failures to the hacked third party software installation they did months or years ago.
The second model is the bundled model. Ship all the dependencies bundled in the software itself. That's what Snaps (and Flatpaks, and AppImage) do - same as iOS and Android. This allows one build to work on all distributions and distribution releases. They can be installed and removed without leaving cruft or a broken system behind. They allow security sandboxing.
All your objections seem to be criticizing the bundled model itself, rather than anything about Snaps themselves except that they use the bundled model.
If you don't like the bundled model, then you can carry on using Ubuntu 20.04 without Snaps just fine.
> If you don't like the bundled model, then you can carry on using Ubuntu 20.04 without Snaps just fine.
No, you can't. Ubuntu has started replacing apt packages with snaps. So if you want to install those packages (such as Chromium) you now have to use the snap.
You can install Chromium from a third party source just like you could before.
New distribution releases always add and remove packages. Chromium has been removed (as a deb). It's is no longer packaged as a deb because it's a rolling release upstream and it was too much work to backport featureful rolling release to debs.
The transition in Chromium packaging is understandable from a maintenance point of view. However, the general argument that you can just keep using the system without using snaps "just fine" doesn't really hold if the snaps begin to replace rather than supplement software that used to be packaged without it.
The same is also somewhat true if first-class tools such as Software (as in the "store" GUI) begin to push snap packages as the first thing, because then you will, by default, need to go through additional steps to find the packages that you want, and which used to be available as the first choice.
You can still do things, of course, but it might be that you need to start getting around the snap-centric design choices more often, and at that point it doesn't come at no usability cost to you anymore.
I don't actually use Ubuntu at the moment, so I don't know how much that is the case now, but if it is (or if seems like it's going that way), I understand the frustration.
> However, the general argument that you can just keep using the system without using snaps "just fine" doesn't really hold if the snaps begin to replace rather than supplement software that used to be packaged without it.
Well, if you don't like it, you can always fork it. That's the beauty of open source. /sarcasm
Seriously though, all out replacing stuff with Snaps doesn't seem like the right move.
>No, you can't. Ubuntu has started replacing apt packages with snaps. So if you want to install those packages (such as Chromium) you now have to use the snap.
You can almost always install via deb if you want instead.
I'd wager they will require deb packages to be signed with a certain Canonical key that they will use strictly for basic system packages. Everything else will be a sandboxed Snap, left to third-parties to maintain, distributed through a Canonical Appstore that enables payments.
Maybe they will give you an option to "root" the system, and if you use it you'll lose any right to support or updates from Canonical.
Snap is fundamentally a commercial play to reduce support costs and enable an appstore.
IF they do that I'm obviously leaving their ecosystem, though I'm just a consumer so no clue how much effect I would have.
I really don't see what benefit that would provide them over other distros though and I'm not sure why they would make that choice to close down their system?
Fundamentally Linux works so well because of the free software movement and I don't see any app maintainers choosing to charge a fee for their software if they aren't already.
Arguably the Linux desktop ecosystem does not “work so well”. Adoption rates are still minuscule when compared to Apple or Microsoft numbers, and there is precious little support from commercial developers. The “year of Linux on the desktop” never happened, even after Canonical made it really simple to run Linux, so commercial support is still lacking - which in turn keeps users away. The Snap play is their latest attempt to increase monetization rates on the platform.
The third model - rolling release. No need to test against many releases, no outdated dependencies and why so many people want to run latest browser on outdated system?
The forth model - multiple versions of dependencies live on the same system, environment constructed for each application. Deduplication works.
Bundled model has its values (just like vm) but it really shines with inadequate package management. If you don't like the bundled model switch to Arch Linux or NixOS.
I can't believe that in 2020 we're still discussing what is the best way to package and distribute software. Can anyone explain why it is taking the profession so long to get this fixed?
Because different solutions have different tradeoffs. It doesn't appear that there is a single solution which is best for all use cases.
So different systems switch between different methods as their prieceved value of the different usecases change which upsets people who see the values of the tradeoffs differently.
One issue on Linux in particular is the [relatively] tight bond between your kernel version and libc, which makes using software compiled with different version of libc problematic part of the time (a particular libc version still supports multiple kernels though).
I experienced this lately when my Rust-compiled binary used too modern a libc version for the aging Docker container environment we used for deployment, which forced me to use another Docker container for local development -- which obviously isn't ideal and removes the 100% reproduceability promise.
fundamental issue between "change everything once in a big step" and "many small changes".
First is pre internet, because updates cost money, and because pre internet security issues weren't common.
Second is now, change one thing every day, always run the latest code, automated testing to make the latest code always work. Also means don't need a security branch and a feature branch.
People hate change, and linux people have the ability to say no and do their own thing.
Python has easy_install, pip, anaconda and wheel; virtualenv to isolate packages. Node has npm and yarn (is it resolved?). Ruby gem defines version in code, bundler in Gemfile, Gemfile.lock; vendor/, rvm gemset, BUNDLE_PATH to isolate packages. Even developers can't find right answer.
Because it matters?
Package management is the main differentiator, I love pacman, I love PKGBUILD and makepkg.
I don't like the bundled model. I'm running Arch on my main machine currently. The AUR makes getting third-party packages easy. Unfortunately, I need VirtualBox for a class, and Arch's VirtualBox package won't run on the current Linux 5.8 kernel, and downgrading Linux on Arch is not easy. I switched to Windows for VirtualBox, but in retrospect I could've installed linux-lts.
Ubuntu is more stable in theory, but I've encountered broken packages (like hex editors with an incorrect hard-coded temp path, causing it to be unable to overwrite files).
It is not super simple but I lived with outdated kernel and Xorg for a couple of years (because of Intel Poulsbo [1]). Process described in wiki [2] and uses virtualbox-host-modules as an example.
$ pacman -Qi glibc
Depends On : linux-api-headers>=4.10 tzdata filesystem
1. This version is not present in Arch Linux Archive anymore [3], search for package page [4], View Changes, search for corresponding PKGBUILD revision [5], makepkg. It is much easier if package is not that old and can be found in /var/cache/pacman/pkg/ or archive.
2. install with `pacman -U linux-4.15.8-1-x86_64.pkg.tar.xz`
3. skip package from being upgraded with /etc/pacman.conf IgnorePkg [6]
I had problems both with Ubuntu and Arch Linux updates. At least I can fix Arch Linux issues, Ubuntu felt broken.
But then you can't run leading edge hardware... My motherboard's network interface, wireless interface and my gpu all require a kernel version that's newer than the 19.10 ubuntu (and even 20.04 support for stability), let alone the last LTS release.
Ukku worked well for me to get newer kernels without issue, but kvm/virtualbox were totally borked, just as my hardware support was stable.
> If you don't like the bundled model, then you can carry on using Ubuntu 20.04 without Snaps just fine.
While it's true you technically can, at some point you have to ask yourself why you're going against the grain instead of picking a different distro (or not upgrading to 20.04, at least). Ubuntu wants you to use snaps. You can sidestep this, but you should ask yourself if you shouldn't just switch distros.
I just recently upgraded from Ubuntu 16.04 to 18.04. I don't see the reason to upgrade to 20.04 as long as the software I need works and I keep getting security fixes. Once this LTS gets EOL'd, I'll see what the current deal is with Ubuntu and seriously consider switching to another distro.
Manjaro has 99% of what Ubuntu has, runs a much more modern kernel, and has packages for literally anything you can think of in the extended software repository (namely Arch Linux's AUR).
I regret not taking the plunge and installing Manjaro against the will of our IT during onboarding (they don't forbid it, they simply can't promise good support if you don't install Ubuntu). I am sure I could have found a way to install the 2-3 corporate VPN / spy / monitoring agents my employer requires on the machines they issue for employees.
But anything I've needed of Manjaro, I always got it. Granted that's an anecdotal evidence, obviously -- I haven't tried running games, for example.
> If you don't like the bundled model, then you can carry on using Ubuntu 20.04 without Snaps just fine.
Not really. The platform, Ubuntu in this case, has clearly signaled that they want to move from debs to snaps. And apart from the technical benefits of snaps (sandboxing) you also get the drawbacks (they are slow, updates are forced, and there is only one source for them). What you propose is fighting the platform you are staying on, and that's not a great place to be. Far better to move to platform more aligned with your choices as a user.
P.S.: If anyone is looking for quick and dirty suggestions, there's pop!_OS which is quite close to Ubuntu so there won't be many changes to the user experience. In my case it was Fedora. The linux ecosystem is diverse enough to offer many choices.
> The downside of this model is that developers who want to ship their new software or feature updates without waiting for a new distribution release get stuck into dependency hell and have to operate outside the normal packaging system.
When I was writing apps for distros, I had the opposite problem. Every single release of GNOME and Ubuntu would break PyGTK in some way so that my software that used zero new features in the OS, would require at least some modifications and force me to maintain many versions. Finally, Gtk changed something that broke something in a fundamental way (something in Gtk TreeView, I forget what exactly) and I simple gave up maintaining software rather than suffer through figuring out how rewrite everything again.
Isn't the sandboxing a good idea though? It feels that Linux got caught in the past and is actually one of the least secure OSes out there, and what keeps it safe is just its small desktop market share.
Disclaimer: I never used snap and don't use Ubuntu.
Flatpak gives you a sandbox, and has basically skirted around any shortcoming of Snap (faster, open, extensible, adopted by every other distro, doesn't pollute /proc/mounts)
At worst the "joke" sandbox value of flatpaks is the same as deb/rpm packages and snap "clasic" confinement. So as a user, you don't lose anything security wise by moving to flatpaks.
Flatpak, as well as snaps through clasic confinement, allows the developers to "escape" the sandbox because they know that they don't have all the permissions required to provide feature parity with deb/rpm packages. Another reason this is needed is that application developers are not writing their applications with flatpak compatibility in mind. However flatpak is going in the right direction.
Mobile operating systems have proven the value of sandboxing apps.
Fedora Silverblue would like to disagree. And in any case all parties know it's still in flux, but the stable parts are stable in my experience. I would not want something like Snap or otherwise immature forced down my throat.
Entirely voluntary in the sense that flatpak enforces sandboxes which the application tells it to enforce. If the app asks for full system access flatpak doesn't deny that. It kind of destroys the purpose of a system built to run third-party code. If I download a malicious app from flathub or some other repo which asks for full system access flatpak doesn't do anything in the name of security.
Note: I'm not making a value judgement about flatpak's sandboxing, merely describing it to the best of my knowledge.
Enforcing permissions and auditing third party code are two different things. The point of app permissions is that when you know an app should never do something, you constrain it from ever doing that. Then if it has a security vulnerability, it's limited in the damage from the resulting compromise.
The people configuring the sandbox should be packagers that you trust. The upstream developers might provide some recommendations to the packagers, but if it's obvious that an application shouldn't need a permission, it shouldn't have it.
But permissions can't save you from an actually malicious app. Constrain it from accessing the camera and it will still be using your device to host pornography. Constrain it from accessing the filesystem and it will still run up your electric bill mining bitcoin. You either need to trust the developer or you need to get the app through someone you trust to have audited it for you.
I don’t want the sandboxing to be mandatory! Some software inherently doesn’t work in a sandbox, or works less well. A good distribution format should still cover those use-cases, the user just needs to be informed about the capabilities of what they install. Especially on Linux, which is supposed to be about user-empowerment.
I guess just to provide a counterpoint, I can't remember having had any issues with flatpak on Pop!, including both open-source apps and proprietary ones like Spotify.
No issues for me--I've run Zoom pretty regularly, including participating in some fairly large meetings, plus screensharing, etc.
Occasionally I need to replug my headset in at the beginning of the meeting to get my voice audio working, but I'm not sure if that's actually an OS issue or not. Either way, it's never taken more than a couple seconds.
YMMV. Firefox is the golden standard, at least on Arch Linux. Never have I noticed it was running on a sandbox. I run Discord, Spotify, Thunderbird from Flatpak without any issue either.
Visual Studio Code is a regular package, just because its use case it not really suited for a sandbox (access to system binaries, libraries, etc.), but I honestly haven't even tested its Flatpak version.
VSCodium flatpak works fine for me. The caveat is that I use the operating system's terminal instead of the one from VSCodium (for things like make test, python environments, etc). VSCodium's terminal operates on a sandboxed environment and I haven't bothered to figure out how it relates to the OS environment that I get from the terminal app.
You set your shell as some wrapper which allow sandbox escape. It works for shell itself, but then you need to apply it to all external utilities extensions use as well and it gets annoying.
The problem with flatpack is that many apps don't really run in a sandbox. Or at least not in something with isolation features which you might expect from a sandbox.
To be honest, to me the selling point is not the sandbox, is the reproducible environment that's common for all distributions, which is a first in the Linux world, the sandbox is just the icing on the cake.
The other day I pushed an update to some flatpackaged app, and guess what, it's available to all Linux users. Packaging has become incredibly easier for third parties with these kind of technologies.
The problem is the combination of how updates are (often not) done (in time) + no sandbox.
I also am a strong believer that the future for sane desktop PC's is that every program (except the most fundamental core services) of a desktop OS should be sandbox by default. With basically no permission to access any local files or communicate with any other program/service.
It would need some MAJOR changes, slowly step by step. And I had hoped with snap & flatpack we would be slowly transitioning there. But it doesn't really look that way anymore to be honest.
(PS: And it can be done with reasonable UX experience without requiring the user to configure some magic access rules or anything, but it's tricky to get right and it not be fully backward compatible but often changes just need to go into the GUI toolkit (QT, GTK) so it should be possible.
Access control is a good idea, sandboxing on top of it is saying its too nuanced a problem so heres a kitchen sink of overhead.
Linux has had some of the best software security through MAC (SELinux, Apparmor) and cgroups. The problem is that there is no culture in free software to actually write specifications at the point of development, it generally falls on distros / maintainers to try to sort out MAC profiles or cgroups restrictions on a per-package basis.
That is why the packagers largely went with the Snap / Flatpak route of saying screw doing the grunt work heres a total sandbox with all the libraries built in.
It would be great if we could convince the whole ecosystem to start provisioning access specifications for libraries and binaries so upstream could start building apparmor / selinux profiles from provided files rather than having to do learning mode auditing that drove distro maintainers to not even bother.
Pinebook pro is using an arm64 cpu instead of an x86. That creates a big list of problems that you don't have on x86 because that's what all linux desktop developers have (mostly) been targeting.
As one example, I was surprised to find that the tor browser doesn't have an arm64 build :o
I've installed a few dozen AUR packages that only needed aarch64 added to the arch list. I think only one, syncterm, required any other modifications to build and even that was just setting a preprocessor define (__arm__).
Thus far, it's my favourite laptop purchase ever. Beats the feeling I got with the IBM X series or the Ti Book. It's light, zippy enough, great Linux hardware support, and totally silent.
Even if you did swap a symlink to upgrade to GLES3, all the apps will still call the GLES2 functions. No matter what, the devs would have to go back and rewrite their app to actually use the new features.
Many apps detect GL level and adapt renderers accordingly.
Still, it's not just graphics drivers: there's the oft-cited security patches, but also features for user-facing interfaces; Ie, an update to a URL parsing library might add additional codepage support transparently to an app that uses it.
Forcing Arch model on eveyone is a perfect way for Linux to forever stay minority platform. There's a reason why most people do not run rolling distros.
Might be nice for hardcore fans, but good luck supporting anything there.
I've observed a lot of recent Manjaro adoption lately (I don't know what the real numbers are). Anecdotal, but in my social circles I'm seeing people moving from Ubuntu to either Manjaro or Arch.
Don't be surprised when all the mainstream distros switch to sandboxed app model as the preferred one. Distro-based packaging of the entire world is hiting the not enough manpower problem today and eventually will be eventually relegated to super fans distro only, with similar status as Amiga has today.
As Arch Linux user I am completely confused. Looks like there is no official comparison page like [1]. Web search first entries [2], [3] gives another picture — more stable, graphical installer, graphical package manager, hardware detection, prepackaged desktop environments — that I can relate to.
Nothing about "based on the AUR". Because it is liability. It is insecure, have to check PKGBUILD each install and update.
I should have been more clear. In terms of package management Manjaro has access to the AUR, but it's unsupported. Manjaro does not have access to official Arch repositories. Sorry, I should have said "based on Arch" not "based on the AUR." https://wiki.manjaro.org/index.php/Arch_User_Repository
Ok, I would not recommend Arch as a first Linux distro, Arch based and easy to start is a plus. That said I've seen troubled Manjaro user on xmonad IRC, we tracked problem to AUR package, it worked fine on Arch.
Yeah, Arch Linux is definitely not for beginners. The install process alone is hands-on and requires users to be experienced with command line configuration. Arch is for power users who want a ports-based Linux with great package management (pacman).
Why would you expect to have a git snapshot shipped as the official freedesktop flatpak runtime? Could you not build your own freedesktop flatpak runtime using mesa git head?
Almost no apps are actually delivered via the Apple or Microsoft app store. Apps from outside app store apps aren't obliged to use the sandbox although mac apps can and they do have a line of defense against unsigned applications. Its harder but not impossible to end up pwned by signed apps on mac.
The number one and two attack vectors have always been tricking users into installing malware and attacking old insecure software.
Distributing virtually all software via app stores substantially solves acquiring safe software and ensuring it receives updates.
Defense in depth is virtuous but Linux is already more secure than windows in the ways that actually count and unlike MS they are actually positioned to in the future sandbox software because its all already mostly coming from app stores.
Containers, chroot jails and whathaveyous have existed long before snap came along. Like other have said, flatpak is a more sane alternative that doesn't impose idiotic requirements like systemd or x server.
> doesn’t impose idiotic requirements like ... x server
How are you going to sandbox graphical apps without knowing about (and having capabilities around) the system by which a containerized app would communicate with your OS’s graphics subsystem?
I mean, if you’re not going to run any X11-client graphical apps, it should probably be optional to have an X11 server installed; but either way, you’ll need the X11 wire-protocol libraries (“xorg-common” in most package repos) for the sandbox to link in.
> How are you going to sandbox graphical apps without knowing about (and having capabilities around) the system by which a containerized app would communicate with your OS’s graphics subsystem?
Wayland is the way forward.
> you’ll need the X11 wire-protocol libraries (“xorg-common” in most package repos) for the sandbox to link in.
Today, runtimes do contain client x11 libs. However, nothing in flatpak requires it and it is possible to phase them out in the future releases of runtimes.
> is actually one of the least secure OSes out there
Linux is one of the most secure platforms to run web applications on, however, because more man hours than I can comprehend were spent hardening that use case.
All of those hardening measures can transfer over to the Linux desktop use case.
For example, seccomp, cgroups and MAC can all be used to harden a Linux server, and they can also be used to harden the Linux desktop. It's just that no one has thrown the same billions of dollars at desktop Linux that were thrown at solving web application security.
If you really wanted to, you could run a lot of your software in unprivileged containers secured with seccomp.
Not if it slows down things as much as it does, especially launching. What are you doing on your machine that things like apparmor and selinux (for app isolation) aren't enough? Just use a VM. Not everything needs to be sandboxed if it ruins the zen of interacting with your machine
Sandboxing is great in theory, but last I checked the snap daemon opened Internet connections as root. I'll take my chances with traditional Unix permissions over that.
> Might as well ship a pre-linked binary that just does system calls.
I actually think that’s exactly what we should do. Containers are an over-engineered solution for a problem that never needed to exist.
Dynamic linking doesn’t work unless you can live inside a distro maintainer’s special bubble for all your software. If you can exist in that bubble, great—I really like Debian for certain use-cases—but if you can't, the benefits of dynamic linking everything are clearly outweighed by the drawbacks.
Dynamic linking doesn’t work unless you can live inside a distro maintainer’s special bubble for all your software. If you can exist in that bubble, great—I really like Debian for certain use-cases—but if you can't, the benefits of dynamic linking everything are clearly outweighed by the drawbacks.
Good luck patching that security vulnerability in all those static binaries without proper dependency tracking ;). Not that I am on a particular side of the fence, both have their downsides.
To me the problem are package managers from the '90ies that use a single global namespace, only allows UID 0 to install packages, and do not really provide reusable components.
Modern packaging systems like Nix and Guix allow users to install packages. Packages are non-conflicting, since they do not use a global namespace (so, you can have multiple versions or different build options). They provide a language and library that allows third-parties to define their own packages.
Not to say that they are the final say in packaging, but there is clearly a lot of room for innovation.
Snap and Flatpak are copying the packaging model of macOS, iOS, and Android. This is a perfectly legitimate approach (and IMO the execution of Flatpak is far better). But it is not for everyone -- e.g. if you prefer a more traditional Unix environment.
Rolling out shared library updates to resolve security vulnerabilities is not without its own issues.
The big one, which surprisingly places still manage to fumble due to poor process controls or simple mistakes, is that you have to restart all running processes that use the library after you update it.
I actually prefer to deploy static builds of critical services for this reason, because you already have to know that you're running version 1 build 5 everywhere -- and if everything is build 5, then they all have the fix. You don't also have to check if the process was started after May 5th.
Well, there is nothing that says you can't have proper dependency tracking just because something is statically linked. The infrastructure isn't currently there [x], but it definitely is something that languages and language package managers could with each other and provide.
[x] But can be built, now that more and more languages have access to language package managers with proper dependency tracking. One way would be to create a standard for how to query a binary for what it depends on. Then a computer could have a central database of the dependencies of static binaries that is installed.
Well, there is nothing that says you can't have proper dependency tracking just because something is statically linked.
Didn't say so. It is just easier with dynamic linking, because you can see what libraries (and versions) a binary is linked against.
But can be built, now that more and more languages have access to language package managers with proper dependency tracking.
Actually, approaches such as Nix' buildRustCrate (where every transitive crate dependency is represented as as a Nix derivation) + declarative package management offer this today.
But with curl | bash or traditional package managers, which are most widely used today, this is kind of dependency tracking hard/ad-hoc.
But can be built, now that more and more languages have access to language package managers with proper dependency tracking.
But then a static C library is used and nobody knows where it came from. Even if you look at the Rust ecosystem, which generally does things well when it comes to dependency handling, crates are all over the place when it comes to native libraries. I have seen everything from crates that use a system library (or something discoverable via pkg-config), via crates that have the library sources as a git submodule and build them as part of the build-script, to crates that download precompiled library from some shared Dropbox link.
Another fun example from another language ecosystem. numpy uses OpenBLAS. They compile their binary wheels on CI. However, OpenBLAS itself is retrieved as a precompiled binary from another project [1]. However, the rabbit hole goes deeper. In case OpenBLAS is built for macOS, a precompiled disk image is retrieved from yet another repository [2]. This disk image is added to that repository, but comes from yet another place.
This is all sort of the opposite the lessons to take from Reflections on Trusting Trust and the bootstrapping that the Guix folks try to do.
Anyway, with the mindset that most developers have, we will never have proper dependency tracking.
The 'horror story' people always mention, which is a special case of "having to wait for each application to update its own dependencies", is that if there's a security vulnerability in a much-used library, you have to wait for each application maintainer to update their application, rather than simply having the distribution maintainer update the shared library. I'm not sure I agree that this would be worse than the current situation...
I just honestly don't understand this. On Windows and Mac (the OSes that 99% of the planet uses on the desktop), this is exactly how things work. The OS provides a set of APIs. If an application author needs a library that isn't in the OS, they have to ship it themselves. If a vulnerability in that library comes to light, they have to fix it and ship an updated version of your application. If it's a commonly used library, a lot of application authors are going to have to ship updates.
Why couldn't this work for a Linux-based OS? Honest question.
On Windows and Mac, there is Microsoft / Apple who decides what is set of OS APIs, and everything outside is external library.
In Linux, there is no such authority and therefore no sharp line separating 'core OS' and 'external library'. It is just conglomerate of Linux kernel and independently developed tools and libraries (where each of them is more-or-less optional).
That way leads to high numbers of boxes with vulnerabilities, which may be "fine" for non-technical folks. That's not the audience linux serves however.
I don't think it's at all clear that it would lead to high numbers of boxes with vulnerabilities. Is it clear that a Mac is more vulnerable than a desktop Linux box, if you control for for the technical sophistication of the person maintaining it? I don't think that's at all clear.
While it's not guaranteed they'll be installed, the vast majority of linux desktops get security updates, including for all normally installed applications. That's a pretty big advantage over a manual update strategy.
It does seem like there could be a, well, canonical set of common libraries of specified version-ranges for a particular version of a particular distribution which are dynamically linked and with updates pushed by the distro maintainer, and if the application developer needed something else it would be statically linked.
If you have a BSD app that uses an LGPL library, congratulations, your app is now (L)GPL... let alone a differently licensed other library, and now your app can no longer be distributed/licensed.
> Just fix the bugs and leave the rest to the application developers.
Some dev flat out[0] refuse (and I am not debating if it's right or wrong) to package their app for every distro (even major ones: ubuntu, debian, arch, rhel/fedora) so it's up to distro maintainers to package them so users are always at the end of a line of other people packaging the apps (either through distro packages or sandboxed one click installer).
[0] Words are too strong and user CJefferson https://news.ycombinator.com/item?id=24384206 is right to call me out on that.
I agree, sorry about that. Poor choice of words. I had a very specific example in mind but there's obviously a whole gamut of reasons for not packaging. My position is that we can't nor should we expect dev to package their apps for the distro we use. Also it's no like app devs and distro/os devs/maintainers are living in hermetic boxes and their code/apps never interacting or evolving. Not editing out for context.
Yeah, there's more than a whiff of entitlement to that phrasing.
If you haven't seen it, I highly recommend looking at fpm for packaging. Unless you're doing something weird or need an obscure format, it is the tool you want.
> Yeah, there's more than a whiff of entitlement to that phrasing.
I have been thinking about that. I disagree. It's always nitpicking o'clock on HN and I specifically wrote "Some" and specifically wrote in brackets that I wasn't debating if it's either right or wrong. I agree that it should have been worded differently but the facts remains and that doesn't follow that there's “more than a whiff of entitlement“.
I'm not clear what you mean then, to be honest. Anyone who isn't packaging their software is, effectively, flat-out refusing to package it. I don't really see how I, or anyone else, could "more" flat-out refuse.
> What's the point of dynamic linking if we then end up shipping half an OS with an application just to get around dependency hell?
Not sure if you were commenting on the whole approach or just snap, but FTR, flatpak uses shared base layers which can be updated individually, so there's still an upside to dynamic linking.
Dynamic linking and containers aren’t necessarily incompatible, though nobody has combined them well yet.
Of course, half the point of containers is to “vendor” your dependencies — a container-image is the output of a release-management process. So the symbolic reference part of dynamic linking is an undesired goal here: the container is supposed to reference a specific version of its libraries.
But that reference can be just a reference. There’s nothing stopping container-images from being just the app, plus a deterministic formula for hard-linking together the rest of the environment that the app is going to run in, from a content-addressable store/cache that lives on the host.
With a design like this, you’d still only have one libimagemagick.so.6.0.1 on your system (or whatever), just hard-linked under a bunch of different chroots; and so all the containers that wanted to load that library at runtime, would be sharing their mmap(2) regions from the single on-disk copy of the file.
The primary issue with this approach is that if every program only sees its own version of the library anyway, there's no incentive to coordinate around library versions - you end up with tons of versions of everything anyway, maybe not one for every application but close to.
True, but it potentially works out anyway, for several reasons that end up covering most libraries:
• libraries that just doesn’t change very often, are going to be “coordinated on” by default.
• people building these container-images are the same people who actually tend to be running them in production, so they (unlike distro authors) actually feel the constraint of memory pressure; so they, at development time, have an incentive to push back on library authors to factor their libraries into fast-changing business layers wrapping slower-changing cores, where the business-layer library in turn dynamically links the core library. This is how huge libraries like browser runtimes tend to work: one glue layer that gets updates all the time, that dynamically links slower-moving targets like the media codec libraries, JavaScript runtime, etc. Those slower-moving libs can end up shared at runtime, even if the top-level library isn’t.
• on large container hosts, the most common libs are not app-layer libs, but rather base-layer libs, e.g. libc, libm, libresolv, ncurses, libpam, etc. These are going to be common to anything that uses the same base image (e.g. Ubuntu 20.04). Although these do receive bug-fix updates, those updates will end up as updates to the base-layer image, which will in turn cause the downstream container-images to be rebuilt under many container hosts.
• Homogenous workloads! Right now, due to software-design choices, many container orchestrators won’t ensure library-sharing even between multiple running instances of the same container-image. We could fix this issue without fixing the rest of this, but designing a container-orchestrator architecture around DLL-sharing generally, would also coincidentally solve this specific instance of it.
> The whole idea that you'd need a container like environment to install an application
Simple example. App A wants tensorflow 1.10, CUDA 8, and python 3.7. App B wants tensorflow 2.2, CUDA 10, and python 3.8. You want App A and B installed at the same time but the two versions of tensorflow are neither forward nor backward compatible. The two pythons will fight with each other for who gets to be "python3". How do you deal with this without containerization?
I don't think it violates the principles of open source at all, it's just making sure each application gets the exact versions of libraries it wants without messing up the rest of your system.
> obviously python 3.8 should be backwards compatible with 3.7
I think so too, but HN downvoted me to oblivion the last time I advocated for that. That's part of the problem, I guess, is that the dev community doesn't actually agree with 3.8 being backwards compatible with 3.7.
Would you argue the same if they were called Python v37.0 and v38.0? Let's imagine that they are and move on. The problem is to use the alias "python3" as if it were the executable name.
At the risk of getting kicked off HN for all these downvotes for trying to have a discussion ... (thanks free speech haters, enjoy your echo chamber after I'm kicked off)
My understanding of semantic versioning is that:
- (x+1).0 and (x).0 don't necessarily need to be able to run code writen for the other
- 3.(x-1) doesn't need to be able to run code written for 3.(x)
- 3.(x+1) should always run code written for 3.(x)
Hence, you should be able to point "python3" at the latest subversion of 3 that is available, continually upgrade from 3.6 to 3.7 to 3.8, and as long as you have a higher sub version of 3, you shouldn't break any code that is also written for an earlier subversion of 3. That's why it is supposed to be okay to have them all symlinked to "python3". If a package install candidate thinks the currently running "python3" isn't recent enough for the feature set it needs, it can request the dependency manager upgrade "python3" to the latest 3.(x+n) with the understanding of not breaking any other code on the machine.
Unfortunately that isn't true between 3.7 and 3.8. There are lots of cases where upgrading to 3.8 will break packages and that violates semantic versioning.
Python doesn't use semantic versioning, so you can't really expect them to follow it. As GP insinuated, if you just pretend that 3.7 is 37, and 3.8 is 38, you'll pretty much be able to apply semver thinking, though.
Right, so because Python doesn't coooperate, we end up needing containerization, which is what I was trying to explain in GGGGP. Because apt will upgrade 3.7 to 3.8 and unfortunately break anything that was written in 3.7 (and vice versa).
An app needs to be able to say "I'm ok with python3>=3.7" and be fine if it gets 3.8, 3.9, or 3.20, if we want to be able to run it without a container. (And likewise for all its other dependencies besides python)
If appA needs python3.6 then call it with `python3.6` not `python3`. It can exist in your /usr/bin in parallel with 3.7.
The standard python used by your distribution is python3. I think it's currently the way it's done.
One problem is, that those open source principles have not been codified somewhere. With free software we have its licenses guarding the principles. Like copyleft and the requirement to share modifications if distributing software etc. With many open source licenses we have almost nothing in the way of protecting the principles, because licenses like MIT basically say "I do not care.".
Open source will be exploited, until people learn, that they have to protect it.
> With many open source licenses we have almost nothing in the way of protecting the principles, because licenses like MIT basically say "I do not care.".
What's wrong with that? It's perfectly okay to say "I don't care".
I do release all my code under MIT because I care about attribution. I don't mind if people want to use it in commercial or closed source applications, nor if they want to modify it somehow.
I distribute code because that makes _me_ happy, not because I want to share an ideological statement about how others should distribute their code (or not).
It is not unethical to do what you do. I would say, that perhaps it is only a little short sighted. I don't mean this in a negative insulting way and I will explain why I think so.
The problem is rather, what it means in the long run. The point is, that this kind of only caring about for example attribution makes the ecosystem exploitable. It does not uphold ideals or enforce principles. Without upholding ideals and enforcing principles, how do we expect our principles to be followed in the future? If there is no legal obligation to do anything, which capitalistic (We need to maximize our profit! Ethical principles? Nah, come on ...) big company is willing to go the extra mile to respect the principles of some open source community, perhaps even at the expense of making more profit from a closed source solution? And I mean going the extra mile, without seeing it as an opportunity to use the very action as another means of promoting oneself. Simply going the extra mile, because it is the fair thing to do.
As I see it, as long as there is a chance to deviate from following the principles (no copyleft), someone somewhere will do so. Heck, even with such obligation to adhere to principles some people will deviate from the path. The tendency is always towards the wrong direction, if we do not enforce our principles of openness and such. It is an uphill battle. The whole ecosystem goes towards a not so open direction, by these initially small "missteps".
Especially when a a big company with a lot of developers takes stuff and makes proprietary stuff out of it, as its product, which usually initially has more functionality than its open source counterpart, users will quickly switch to the non-open proprietary version. They do so because they want that new shiny functionality immediately. The slightest inconvenience is sufficient for many users to drive them towards proprietary software. They do not know nor often do they care initially about using a non-open, non-free thing. Until they are sufficient users to create a bubble, in which the open source ideas are no longer existent. Then however, the network effects are already strong. "But all my friends use X. No on uses Y. I cannot convince them all to switch from X to Y!"
Example: There are loaaads of at least open source (and some also free software) messengers out there. All people need to do is to use them. But the network effect and features like integration with (a)social media are so convenient for them, that they give up on their freedoms and use things like Whatsapp or Facebook.
I care, and I want the world to be able to do whatever they want with code I gift to it.
MIT is beautiful is its concision, and reasonably reflects the "use however you want but don't blame me" legalese I used to custom craft before I found it.
> They do not know nor often do they care initially about using a non-open, non-free thing.
Man, I get this now esp with AWS services and everyone recommending how "easy" it is to x with y service, and why it we should use to too and it will magically solve all the problems… and I'm like: "no, i'm going to use this open source software that we'll have the code for and be able to tweak to do whatever we want to do and see how it all works (oh and is free to use), and unless you are going to be willing to hack around all the edge cases yourself that with y service without me getting involved at all" and then that usually works, though I suspect once I leave, the costs of running infrastructure are going to go through to roof and no one else is going to have any clue as to why that's so (but more likely, think that its impossible to have it any other way than paying to have y)…
Moments like these are great opportunities for folks that just don't accept "the non-open proprietary" by default, but its only an opportunity because most choose to accept "the non-open proprietary" by default… we all have to pay for the choices we make… some just want to pay more to not have to think about things… tradeoffs.
Honestly, it really depends on which stage of life your company is at, and the resources you can allocate to infrastructure work.
At the very beginning of my company, we did exactly what you mentioned here.
- Pay for a managed NAT gateway? No thanks I can do the NAT myself with iptables on a cheaper EC2 instance.
- Pay for a managed NFS? No thanks I can do it myself
- Pay for managed VPN? No thanks I can setup IPsec myself
- etc.
With time though, as the company starts to gain money, and the number of users increase, we switched back to more managed services. The key here is that you want to refocus your infrastructures efforts on more business centric issues.
Also, most of the time, the time spent to maintain a service is exponential with its scaling. NFS is a good example here. Setting up a number of NFS shares for 5/10 users is fine. Once you get 20+ NFS users, you just better focus on your real company product rather spend month and money on maintaining NFS yourself.
A small company in an EM country, I don't think infra spend will ever come close to those in the US to being within the budget without significantly affecting margins… and considering that a lot of companies in the US are funded with either massive amounts of debt and on E rounds of funding… I don't think most can afford it either…
Though saying this, even at a small corp level, are still affordable proprietary solutions out there (not necessarily in AWS), but most people trend towards whats trendy…
The work of open source community has been exploited to create closed source, walled gardens with superficially more convenience to attract users and make profit of them.
This pulls users from the open source projects and since contributions do not flow back to the open source project, it can quickly become obsolete in the eyes of the most users. The principles of open source will live in that open source project, which in the end (exaggerating) "no one uses" and will become pointless. Most users are not protected by these great ideas or principles any longer, because they will be sucked into the closed source swamp, because all their friends are there already.
"What's the point of dynamic linking if we then end up shipping half an OS with an application just to get around dependency hell? Might as well ship a pre-linked binary that just does system calls."
I'm curious - are there any Linux distributions that contain nothing but fully static, self-contained binary executables ? As in, ships with no libraries ?
It's about sandboxing your apps... though not every app needs to, or should be sandboxed, but plenty can and should be. I happen to like flatpak a bit more, just feels like a more open community, and appimage is okay as well.
It's about getting dependencies with the app. I can't tell you the number of times I borked my OS install because I wanted a version of a single application that had a feature newer than a year or two old version supported in my distro's repository.
This is the biggest downside for me. I understand why they want to use snaps of huge software applications especially for rolling releases. I just don't like one company being in charge of the method of distribution.
Yes indeed, I don't mind snaps for the really big third party stuff. But they were using it for literally everything. Even htop was snapped at one point. Seriously... :/
Clearly they are looking for a way to put some kind of proprietary dependency in Linux by propagating snap to other distros so they can then milk it for cash (e.g. charging a fee for access to the snap store to big publishers like Microsoft/Google). I don't think they realise the mainstream Linux users will hate it for exactly that reason.
There are many reasons to hate snap for. I have a comment that explains it in more detail, but seriously, this is the most anti-FOSS, anti-Linux crap I have ever encountered. Shame on Canonical.
Original Unix didn't support dynamic linking (despite having already been invented), and for some purists it was a mistake to add support for it. Plan 9 refused to add support for it. Dynamic linking is not some kind of holy principle.
> These large companies should stop fucking around with Linux
Canonical has 443 employees according to the wikipedia page. Is that large in this context? I don't really think so. Redhat (13k employees) is large. Canonical isn't large.
Thankfully both Arch Linux and Gentoo have native Chromium and don't require Snap. I can recommend them to everyone. Arch is more practical for everyday use though as compilations take time.
Canonical has been doing this sort of thing for a very long time, they way Ubuntu tough was designed should have been enough to make most people abandon them.
> The way open source is straying further and further from its principles is highly annoying.
I do think that's why the distinction between "free software" (copyleft) and merely "open-source" matters.
If you look historically, "open-source" became a thing as a reaction to free software where it preserves the most visible benefits, (source code in the open, modifyable by others), but treats these as purely a convenient workflow when working on code, whereas free software is more of a philosophy and so less likely to erode on principles.
I agree that something based strongly on principle rather than convenience is less likely to drift from them, for better and for worse.
Free software doesn't need to be copyleft, though. The MIT license, for example, is a free software license, even though it's not copyleft. Projects such as the various flavours of BSD can have pretty strong principles regarding their software distributions remaining free even though they don't prefer copyleft.
> flavours of BSD can have pretty strong principles regarding their software distributions remaining free even though they don't prefer copyleft
That is true, however speaking to many members of the FreeBSD community in particular, there seems to be a strong sentiment of this simply being a practical model of development, rather than a strong ideological stance. In fact a large portion seems to be rocking Macs, "cause it's BSD anyway", which to me does not seem particularly principled.
In fact they seem to take pride in completely closed systems being based on FreeBSD, like the PS4, Nintendo Switch etc.
Open source never had any principles. Open source is just a libertarian hijack of free software to ensure that it is accessible to companies without any legal issues.
In practice though, they don’t matter for a vast majority of users, and package managers are far less hassle. Maybe it’s time for the principles to change?
And it's yet another way to do an end run around repositories, instead you will sooner or later get an app-store like environment that can be controlled by some entity. These large companies should stop fucking around with Linux, it was fine the way it was. Just fix the bugs and leave the rest to the application developers.