Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"So that a truncated partial download doesn't end up executing half a script"

tailscale.com

Post reply on HN
Print view View original

71–80 of 87 posts

Re: "So that a truncated partial download doesn't end up executing half a script"

#71

Post by pjerem »

deanCommie wrote: post #34

A serious question for any Linux-heads here, no insult intended. How is it possible that there are ELEVEN different possible package managers that need to be supported by an installation script like this? I can understand that some divergences in philosophical or concrete requirements could lead to two, three, or four opinionated varieties, but ELEVEN? Does that mean that if I want to write an app that runs on Linux…

> edit: Thank you for the responses so far, but noone has yet answered the core question: WHY are there eleven of them?

Because Linux is "just" a kernel that happens to be used by different OS. Linux is just the program that runs binaries amongst your computer, it’s not a package manager or an OS.

There is no official cooperation between different Linux OS. People developing package management at Suse or Redhat (which are commercial companies) aren’t the same that are developing APT at Debian foundation. Odds are that they don’t even know each others.

Look, Android is based on Linux but they have their own package management because most of the existing ones weren’t compatible with what they wanted to achieve.

It’s the same with other vendors : while package managers looks like they are mostly doing the same thing, they all had their own requirements that justified their creation.

Anyway as a developer you mostly don’t have to package your app yourself. It’s the job of either the distribution developers themselves (in fact that’s most of the work in making a "distribution": they package and distribute software) or in some organizations like Suse or Arch, this job can also be done by the community which allows more up to date package.

Re: "So that a truncated partial download doesn't end up executing half a script"

#72

Post by wongarsu »

wiktor-k wrote: post #64
wongarsu wrote: post #28

Earlier quoted context omitted.

Windows checks the code signing certificate of the exe, and if it isn't present and the binary not widely used shows you a big scary warning to discourage you from running it. And if the exe is signed that at least tells you where to send the police after you were infected. Of course open source projects rarely sign their exes because those certificates are expensive ($300+/year).

> Windows checks the code signing certificate of the exe, and if it isn't present and the binary not widely used shows you a big scary warning to discourage you from running it. Actually even if the file is correctly signed but is new users will see the warning banners. (Unless using the more expensive EV Code Signing certs) > Of course open source projects rarely sign their exes because those certificates are expens…

That's great to see that they are so cheap now for open source work. I must have remembered the price of EV certificates (which are handy for completely getting rid of the warning screen and for getting Windows Defender off your back)

Re: "So that a truncated partial download doesn't end up executing half a script"

#73

Post by [deleted] »

kcb wrote: post #66
stavros wrote: post #48

Earlier quoted context omitted.

It's like anything else, it depends on how many people you want to get. Apt alone will get you 50%. Add pacman and that's another 30%. Yum is another 15%. Nix is another 2%. Foo is another 0.3%, bar 0.1%, and so on and so on. (Numbers are made up). You don't have to do anything, it's just about how convenient you want to make it.

Surely pacman is nowhere near as popular as yum/dnf.

[deleted]

Re: "So that a truncated partial download doesn't end up executing half a script"

#74

Post by thomastjeffery »

deanCommie wrote: post #34

A serious question for any Linux-heads here, no insult intended. How is it possible that there are ELEVEN different possible package managers that need to be supported by an installation script like this? I can understand that some divergences in philosophical or concrete requirements could lead to two, three, or four opinionated varieties, but ELEVEN? Does that mean that if I want to write an app that runs on Linux…

Why? Because there is. Every package repository has a community of maintainers who make sure a package is compatible with their OS/distribution. That's just the way each linux distro solves this problem. It's worked out pretty well so far, despite the sheer number of redundant packages.

Why not? This kind of script is a generally bad idea, because it's hoarding the responsibility of package maintenance. The better solution is to maintain a working package (better yet, convince someone to maintain it for you) for each distro's public repository, good documentation on their wiki, and working links to that documentation in your readme.

Why not not? Despite being a bad idea, it's not a hard idea. The implementation of this script is likely easier to manage than doing it the proper way.

--

Now if you really want to feel upset, start looking at build systems...

Re: "So that a truncated partial download doesn't end up executing half a script"

#75

Post by thomastjeffery »

LeoPanthera wrote: post #12
constantcrying wrote: post #7

Earlier quoted context omitted.

It is an awful habit of some open source projects to have the official way to install their software be to execute a shell script from the Internet. Nobody reads it, as they are usually quite complex and given the xz situation a well crafted shell script can seem harmless while being very dangerous.

How is this different to Windows users downloading a .exe file and running it?

Short answer: it's not, and that's the problem.

Long answer: Windows has a few conventions that make it "better", like a predictable place to install your files, a global authoritative "registry", and never having dynamically linked (and separately installed) dependencies. By sheer virtue of not having a good package manager, Windows has avoided dependency hell. That does, however, still leave it without the utility of a package manager.

Re: "So that a truncated partial download doesn't end up executing half a script"

#76

Post by kseifried » 

    #!/bin/bash
    SHA512="485fe3502978ad95e99f865756fd64c729820d15884fc735039b76de1b5459d32f8fadd050b66daf80d929d1082ad8729620925fb434bb09455304a639c9bc87"
    # This line and everything later gets SHA512'ed and put in the above line.
    # To generate the sha512 simply: tail -n +3 [SCRIPTNAME].sh | sha512sum
    check_sha512() {
        # Compute the SHA512 hash of the script excluding the first two lines
        local current_sha=$(tail -n +3 "$0" | sha512sum | awk '{print $1}')
    
        # Compare the computed SHA512 hash with the predefined one
        if [[ "$current_sha" != "$SHA512" ]]; then
            echo "Error: SHA512 hash does not match!"
            exit 1
        fi
    }
    
    # Call the function to perform the hash check
    check_sha512
    
    # Rest of your script starts here
    echo "Script execution continues..."
The idea is simple: if the first line get's mangled (#!/bin/bash) the script probably won't execute at all. If the second line gets mangled than obviously the SHA512 comparison won't work (variable name or value).

Finally if the rest of the script gets mangled or truncated it won't SDHA512 the same and it'll cause the function to exit.

For bonus points you can add a check if first line of script is exactly "#!/bin/bash" as well.

Re: "So that a truncated partial download doesn't end up executing half a script"

#77

Post by loeg »

wongarsu wrote: post #42
oneepic wrote: post #29

I read TFA. Why would a truncated partial download happen and still run the script?

A truncated download might happen for all sorts of reasons, like your internet connection dropping while you download the script. If you don't notice you might accidentally run an incomplete script and leave your system in some broken or at least confusing state. They wrapped everything in a main function to prevent that from happening

Browsers typically emit downloads to temporary files until they are complete, then rename them into the final location, to prevent this kind of issue.

Re: "So that a truncated partial download doesn't end up executing half a script"

#78

Post by o11c »

kseifried wrote: post #76

#!/bin/bash SHA512="485fe3502978ad95e99f865756fd64c729820d15884fc735039b76de1b5459d32f8fadd050b66daf80d929d1082ad8729620925fb434bb09455304a639c9bc87" # This line and everything later gets SHA512'ed and put in the above line. # To generate the sha512 simply: tail -n +3 [SCRIPTNAME].sh | sha512sum check_sha512() { # Compute the SHA512 hash of the script excluding the first two lines local current_sha=$(tail -n +3 "$0"…

If the file is truncated after the function's closing brace, it will succeed but do nothing.

If the file is truncated in the middle of the word `check_sha512` it will try to execute a hopefully-not-existing command.

Wrapping in simple { braces } should fix this - if the brace is missing, you get a syntax error, and if present, you can execute the full thing, regardless of whether a trailing newline is available. This is admittedly bash-specific, so won't work for the linked script, but (subshell) doesn't cause too many problems

Using a function and checking the SHA don't really add anything after these fixes.

Checking the shebang is hostile to environments that install bash elsewhere.

An almost-working possibility would be:

  exec some-interpreter -c 'commands' "$0" "$@" ""
which will fail if the second ' is missing. The child interpreter can then check for later truncation by checking that at least 2 arguments were passed and the last one is an empty string. However, this is still prone to truncation before the -c.

Re: "So that a truncated partial download doesn't end up executing half a script"

#79

Post by deanCommie »

pjerem wrote: post #71
deanCommie wrote: post #34

A serious question for any Linux-heads here, no insult intended. How is it possible that there are ELEVEN different possible package managers that need to be supported by an installation script like this? I can understand that some divergences in philosophical or concrete requirements could lead to two, three, or four opinionated varieties, but ELEVEN? Does that mean that if I want to write an app that runs on Linux…

> edit: Thank you for the responses so far, but noone has yet answered the core question: WHY are there eleven of them? Because Linux is "just" a kernel that happens to be used by different OS. Linux is just the program that runs binaries amongst your computer, it’s not a package manager or an OS. There is no official cooperation between different Linux OS. People developing package management at Suse or Redhat (whic…

It turns out my actual question is "Do we really need this many different Linux distributions" and I'm sure this will sound even more wrong but it really feels like the answer is no.

OK different ones for different hardware devices, sure.

But I asked ChatGPT of the difference between openSUSE and Debian (since you mentioned those) and everything listed seems like it could be just variants within the same OS not a fundamentally different OS

> Package Management: > openSUSE uses the Zypper package manager and supports the RPM package format. > Debian uses the APT (Advanced Package Tool) package manager and supports the DEB package format.

This comes back to my original question - they need different package managers because the OS's are different, the OS's aren't different because they wanted different package managers

> Release Model: > openSUSE typically follows a fixed release model with regular releases of a stable version, like openSUSE Leap, which has a predictable release cycle. > Debian follows a more flexible "when it's ready" release model. There are three main branches: stable, testing, and unstable.

you could have unstable, testing, stable, and then on top of that have a predictable stable release cycle. These aren't incompatible with each other.

> Philosophy: > openSUSE is known for its strong integration with the open-source community and its adherence to the principles of the Free Software Movement. It emphasizes stability and ease of use. > Debian is known for its commitment to free software principles, as outlined in the Debian Free Software Guidelines (DFSG). It prioritizes stability, security, and freedom.

This just feels like a ChatGPT hallucination. It sounds like both are focused on Free Software. That said, I suppose noone is stopped from creating a Linux OS extremely focused on FSM, a commercial one not-at-all interested in FSM, and one somewhere in between.

> Default Desktop Environment: > openSUSE offers various desktop environments, including KDE Plasma and GNOME, but its default desktop environment may vary depending on the edition (Leap or Tumbleweed). > Debian offers a wide range of desktop environments, including GNOME, KDE Plasma, Xfce, LXQt, and more. Its default desktop environment is GNOME.

Like the package manager, inverted cause and effect.

> Community and Support: > Both distributions have active and supportive communities, offering forums, mailing lists, documentation, and other resources for users seeking help or guidance.

> System Configuration: > openSUSE uses YaST (Yet Another Setup Tool), a comprehensive system configuration tool that allows users to manage various aspects of the system through a graphical interface. > Debian relies more on manual configuration files and command-line tools for system administration, although there are also some graphical tools available.

I wonder what YaST uses underneath, I bet it's a series of...configuration files :)

No reason why both couldn't work on the same OS.

Re: "So that a truncated partial download doesn't end up executing half a script"

#80

Post by kevincox »

noisy_boy wrote: post #53

A simple (not perfect) approach could be to have a comment containing a "unique" string on the last line and grep for it as the first check to ensure that the entire script has downloaded. #!/usr/bin/env bash set -u grep -wq '^# asfewdq42d3@asd$' $0 [ $? -ne 0 ] \ && echo "script is not complete - re-download" \ && exit 1 echo "script is complete" # asfewdq42d3@asd

This doesn't work for something like `curl ... | sh`
Post reply on HN