Bug 1202858 – Restarting squid results in deleting all files in hard-drive • Hacker News Board

Re: Bug 1202858 – Restarting squid results in deleting all files in hard-drive

#31

Post by thaumaturgy » Tue, Mar 24, 2015, 5:43 AM UTC

Hmm, there are two possible candidates in the init.d script for the RHEL 6 package of an older version of squid (doesn't look like bug submitter is using a current version).

In stop():

    rm -rf $SQUID_PIDFILE_DIR/*

and in restart():

    rm -rf $SQUID_PIDFILE_DIR/*

SQUID_PIDFILE_DIR is hardcoded to "/var/run/squid" at the top of my copy of the init script. But, neither of those rm commands check first to make sure that SQUID_PIDFILE_DIR isn't empty (or, better yet, is in /var and doesn't contain ".."), and either the submitter's copy of the script is mangled or something else somewhere is stomping on SQUID_PIDFILE_DIR in the shell environment.

...I should grep my init scripts for "rm".

Re: Bug 1202858 – Restarting squid results in deleting all files in hard-drive

#32

Post by lpsz » Tue, Mar 24, 2015, 5:45 AM UTC

kevan wrote: post #11

viraptor wrote: post #7

Earlier quoted context omitted.

I would bet on the issue being in the init script itself rather than squid. (I'm assuming squid doesn't run as root by default in rhel) If that's true then it's another point for more sane process managers (upstart/supervisord/systemd/...)

Agreed, I should've elaborated. All it takes is something like this in the init script without checking if the variable is empty: rm -rf "$STEAMROOT/"*

And this is why it is important to write something like

    set -eu

on top of your bash scripts -- execution will stop on errors (non-zero retvals) and on undefined variables.

Re: Bug 1202858 – Restarting squid results in deleting all files in hard-drive

#33

Post by DSMan195276 » Tue, Mar 24, 2015, 5:46 AM UTC

viraptor wrote: post #27

DSMan195276 wrote: post #24

Earlier quoted context omitted.

That's not completely true. At least with the GNU tools, 'rm' won't delete the root directory unless you specifically give it the '--no-preserve-root' flag. Since that flag has no use outside of deleting root, it's unlikely it has the flag on it. With that in mind the script must do some type of manual deleting for some reason.

I believe that "--preserve-root" applies only to / itself. That means `rm -rf /*` will expand to `rm -rf /bin /dev /etc /lib ...` and delete all anyway.

That's accurate. `rm -rf /* ` will still work to delete everything. But that said, `rm -rf "$STREAMROOT/"` can't ever expand to that, and more-over since the expansions in double-quotes it won't be subject to path expansion by bash. So even "/* /", which would normally expand into "/bin/ /dev/ /etc/ ..." won't. You can see what I mean yourself, just use echo:

    `echo /* `: /bin /dev /etc /lib ...

    `echo /*/`: /bin/ /dev/ /etc/ /lib/ ...

    `echo "/*"`: /*

    `echo "/*/"`: /*/

If you try it with `ls`, you'll find that `ls "/* "` results in `ls: "/* ": no such file or directory`.

Edit: Formatting.

Re: Bug 1202858 – Restarting squid results in deleting all files in hard-drive

#34

Post by kevan » Tue, Mar 24, 2015, 6:00 AM UTC

DSMan195276 wrote: post #24

kevan wrote: post #11

Earlier quoted context omitted.

Agreed, I should've elaborated. All it takes is something like this in the init script without checking if the variable is empty: rm -rf "$STEAMROOT/"*

That's not completely true. At least with the GNU tools, 'rm' won't delete the root directory unless you specifically give it the '--no-preserve-root' flag. Since that flag has no use outside of deleting root, it's unlikely it has the flag on it. With that in mind the script must do some type of manual deleting for some reason.

In the original Steam bug the asterisk is outside the double-quotes and path expansion definitely applies.

Re: Bug 1202858 – Restarting squid results in deleting all files in hard-drive

#35

Post by jiffytick » Tue, Mar 24, 2015, 6:17 AM UTC

lpsz wrote: post #32

kevan wrote: post #11

Earlier quoted context omitted.

Agreed, I should've elaborated. All it takes is something like this in the init script without checking if the variable is empty: rm -rf "$STEAMROOT/"*

And this is why it is important to write something like set -eu on top of your bash scripts -- execution will stop on errors (non-zero retvals) and on undefined variables.

or check the variable before using it, like any other programming language:

[[ "$VAR" ]] && rm -rf "$VAR/*"

I think most of these issues stem from the fact that most developers that write shell scripts don't actually understand what they're doing, treating the script as a necessary annoyance rather than a component of the software.

Re: Bug 1202858 – Restarting squid results in deleting all files in hard-drive

#36

Post by rachelbythebay » Tue, Mar 24, 2015, 6:18 AM UTC

thaumaturgy wrote: post #31

Hmm, there are two possible candidates in the init.d script for the RHEL 6 package of an older version of squid (doesn't look like bug submitter is using a current version). In stop(): rm -rf $SQUID_PIDFILE_DIR/* and in restart(): rm -rf $SQUID_PIDFILE_DIR/* SQUID_PIDFILE_DIR is hardcoded to "/var/run/squid" at the top of my copy of the init script. But, neither of those rm commands check first to make sure that SQUI…

squid-3.1.10-29.el6.src.rpm (from ftp.redhat.com, buried where they keep SRPMs) has squid.init within, and that file has no mention of SQUID_PIDFILE_DIR. A few other spot-checked versions are the same way.

https://github.com/mozilla-services/squid-rpm/blob/master/SO... ... however ... whatever that is, does.

Did they take this upstream init script somehow?

Re: Bug 1202858 – Restarting squid results in deleting all files in hard-drive

#37

Post by Animats » Tue, Mar 24, 2015, 6:27 AM UTC

raldi wrote: post #8

fermigier wrote: post #2

Scary. Even more scary is the fact that the bug has been open for a week, one person has confirmed 100% reproducibility, and no one seems to care at Red Hat. Isn't deleting peoples hard drives a big no no ?

And this is why every bug tracking system should have a triage SLA. Or, even if you don't want to declare a threshold, publish the current stats on its front page: "Over the last 30 days, our 99-percentile triage wait time was: XX hours." Similarly, open tickets with priority=urgent should never go 24 hours without a new comment from the owner.

"Or, even if you don't want to declare a threshold, publish the current stats on its front page: "Over the last 30 days, our 99-percentile triage wait time was: XX hours."

Now there's a good idea. Major open source projects should have software quality dashboards tracking things like that.

We're now seeing hospital emergency rooms displaying their current wait time in minutes on billboards.

Re: Bug 1202858 – Restarting squid results in deleting all files in hard-drive

#38

Post by cellularmitosis » Tue, Mar 24, 2015, 6:28 AM UTC

lpsz wrote: post #32

kevan wrote: post #11

Earlier quoted context omitted.

Agreed, I should've elaborated. All it takes is something like this in the init script without checking if the variable is empty: rm -rf "$STEAMROOT/"*

And this is why it is important to write something like set -eu on top of your bash scripts -- execution will stop on errors (non-zero retvals) and on undefined variables.

I also include set -o pipefail (exit if ANY command in a pipeline fails). Had to get bitten and waste an hour before that became a habit.

set -e and set -o pipefail really should have been the default, rather than an opt-in.

Re: Bug 1202858 – Restarting squid results in deleting all files in hard-drive

#39

Post by mirkules » Tue, Mar 24, 2015, 6:31 AM UTC

viraptor wrote: post #27

DSMan195276 wrote: post #24

Earlier quoted context omitted.

That's not completely true. At least with the GNU tools, 'rm' won't delete the root directory unless you specifically give it the '--no-preserve-root' flag. Since that flag has no use outside of deleting root, it's unlikely it has the flag on it. With that in mind the script must do some type of manual deleting for some reason.

I believe that "--preserve-root" applies only to / itself. That means `rm -rf /*` will expand to `rm -rf /bin /dev /etc /lib ...` and delete all anyway.

This just happened to my coworker today. I'm sitting behind him telling him which commands to type (he's new to Linux...) when suddenly he jumps the gun and pushes enter just as I say "slash". My heart nearly stopped. I didn't even know preserve-root existed (plus I always iterate not to log in as root). It was a snapshotted vm but we still would have lost the day's work.

Re: Bug 1202858 – Restarting squid results in deleting all files in hard-drive

#40

Post by cellularmitosis » Tue, Mar 24, 2015, 6:36 AM UTC

fermigier wrote: post #2

Scary. Even more scary is the fact that the bug has been open for a week, one person has confirmed 100% reproducibility, and no one seems to care at Red Hat. Isn't deleting peoples hard drives a big no no ?

Ahh Redhat, the distro which chose to symlink a bunch of binary lib files from Apache into the etc directory.

"Why is grepping /etc taking so long? Binary files in /etc?!? WTF?!?"

Coming from Debian, Redhat seems to make a lot of irk-worthy choices.