Follow

TIL how “pip install torch” works: it downloads an 830 MiB “wheel” (zip file) containing .so files for PyTorch but also for some of its dependencies, including GCC’s libgomp and NVIDIA’s proprietary libcudart (is distribution legal?).

Trading user freedom and security for convenience.

@civodul I think most of per-language packages do something like that in any language. That's sad.

@civodul how would you distribute C stuff on systems where you can't assume a sane C env?
@civodul you can disable that btw https://pip.pypa.io/en/stable/cli/pip_install/#install-no-binary but I'm fairly sure most people just want pytorch or tensorflow or what have you to simply work without being greeted by compiler errors due to missing or outdated dependencies

@reto I would use of course, but more generally, I think we should all be clear that “getting random binaries from the Internet” is problematic.

And perhaps that’s the root issue: most of us users have come to dismiss or just be unaware of the implications of using these convenient tools.

@civodul @reto

> we should all be clear that “getting random binaries from the Internet” is problematic.

and what it means to depend on a system enforcing “getting random binaries from the internet”, looking at you simply-double-click-the-exe.
@samae @civodul I mean for most distos that's what you do.

There's only one source based distro I can name from the top of my head, gentoo.

The rest of the mainstream ones use binary packages and you're trusting whomever has access to the build farm or repositories. Better than a random binary on the internet but in the case of this specific example, pytorch is packaged by the upstream maintainers and most distros will just trust them not to mess up so you won't get much benefit by using the binary blob of the distros over the binary blob of PyPi

@reto @samae is a source distro, though it allows you to get pre-built binaries, if you wish.

However, the key property is that you can verify that pre-built binaries you get from a server are genuine, thanks to :
guix.gnu.org/en/blog/2015/repr

@civodul @samae reproducible builds is a common effort across many distros though and has nothing to do with guix per se. That has been ongoing even before guix existed or at least prior to me becoming (semi-) aware of guix

@reto @samae Sure, I’m well aware of that; projects such as and paved the way!

(I was at the 1st Reproducible Builds Summit where I had the chance to meet folks who’d been working on this: <reproducible-builds.org/events>.)

@reto @samae With or , you’re not trusting “whoever has access to the build farm or repositories”: builds are reproducible, which means anyone can verify binaries match the source, without trusting the server.

Guix has tools to facilitate verification, and it also makes it easy to pick another server for binaries, or several, or none.

@civodul @samae but you still are. You are just also hoping that someone ™ also validates the build on their own machine and hollers loud enough for the rest of the internet to notice in a reasonable timeframe.

However you can only validate a package once you have built it from source, negating any benefit you get from a binary distro in the first place.

And it assumes a package which actually is reproducible and we aren't quite there yet are we? What's the percentage nowdays? Didn't check for a while. But the last blog post of the Arch guy who has an interest in that (foxboron iirc) and that I remember wasn't all that good.

@civodul @bugaevc Wheels should be banned IMO. It pulls in binaries by random people without being asked to do so. What could possibly go wrong?!

Sign in to participate in the conversation
Mastodon (Aquilepouet)

The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!